Replaced --no-upgrade switch with --upgrade-all and --upgrade-required

README.md

@@ -12,7 +12,7 @@ We're planning to release a growing amount of open source software, that is free
 
 | Name | Description | License | Current Version | Written in | Supported Distros | File |
 |------|-------------|---------|-----------------|------------|-------------------|------|
-| [Borgmatic Setup Tool](#borgmatic-setup-tool)   | If you plan to use borg as backup solution, you should also take a look at [borgmatic](https://torsion.org/borgmatic/). It's a Python wrapper for the award winning backup tool [borgbackup](https://borgbackup.readthedocs.io/en/stable/index.html) that simplifies creating secure and reliable backups even more. You can even store your configurations in files. This script will do the setup for you to.   | [MIT]([LICENSE](https://git.zion-networks.de/ZionNetworks/linux-bash-scripts/src/branch/main/LICENSE))  | v1.1.0 | Bash | Debian and derivates | [bormatic_setup.sh](https://git.zion-networks.de/ZionNetworks/linux-bash-scripts/src/branch/main/borgmatic_setup.sh)  |
+| [Borgmatic Setup Tool](#borgmatic-setup-tool)   | If you plan to use borg as backup solution, you should also take a look at [borgmatic](https://torsion.org/borgmatic/). It's a Python wrapper for the award winning backup tool [borgbackup](https://borgbackup.readthedocs.io/en/stable/index.html) that simplifies creating secure and reliable backups even more. You can even store your configurations in files. This script will do the setup for you to.   | [MIT]([LICENSE](https://git.zion-networks.de/ZionNetworks/linux-bash-scripts/src/branch/main/LICENSE))  | v1.2.0 | Bash | Debian and derivates | [bormatic_setup.sh](https://git.zion-networks.de/ZionNetworks/linux-bash-scripts/src/branch/main/borgmatic_setup.sh)  |
 
 #### Borgmatic Setup Tool
 
@@ -28,13 +28,18 @@ For beginners, the fully automated installation is recommended.
 
 BST offers several command-line options to automate the whole process and get your backup schedule ready to go as fast as possible.
 
-These options are available in version 1.1.0:
+These options are available in version **1.2.0**:
 
 ```
 -a, --auto: Enable automatic, non-interactive mode
 -n, --no-upgrade: Skip package upgrades
 -d, --debug: Enable debug mode
 -f, --first-backup: Run the first backup after setup
+-o, --override-crontab: Override the existing crontab file
+-c, --override-config: Override the existing borgmatic configuration file
+-r, --override-repository: Override the existing backup repository
+-s, --upgrade-required: Upgrade only required packages
+-u, --upgrade-all: Upgrade all installed packages
 -v, --version: Show script version
 -h, --help: Show help message
 
@@ -58,7 +63,7 @@ These options are available in version 1.1.0:
 
 To run a fully automated setup you need to set the `--auto` or `a` flag. This will cause the script to skip some prompts.
 
-If you don't want the script to also upgrade the packages on your system, set the `--no-upgrade` or `-n` flag. (this will be changed in the future, replacing the flag with `--do-upgrade` or `-u`)
+If you want the script to also upgrade the packages on your system, set the `--upgrade-all` or `--upgrade--required` flag, depending on your needs. If neither are given, the script won't upgrade any already installed packages.
 
 ###### Example: Backup local configurations and home directories
 
@@ -68,20 +73,19 @@ If you don't want the script to also upgrade the packages on your system, set th
 2. Set the execution flag using `chmod +x borgmatic_setup.sh`
 3. Run the script:
 
-`sudo ./borgmatic_setup.sh -a -n -f -R "/mnt/backups/my-host" -E "none" -D "/etc /home" -S "0 3 * * *" -H "my.backup.host" -U "backupuser" -W`
+`sudo ./borgmatic_setup.sh -a -f -R "/mnt/backups/my-host" -E "none" -D "/etc /home" -S "0 3 * * *" -H "my.backup.host" -U "backupuser" -W`
 
 Let's break up this script:
 
 1. `-a` enables the automatic mode, so the user will see no prompts unless it's required
-2. `-n` will prevent the script from upgrading packages via `apt-get upgrade`
+2. `-f` will cause the script to run a first backup when the setup is done
-3. `-f` will cause the script to run a first backup when the setup is done
+3. `-R` is the path to the backup directory on the _remote_ machine
-4. `-R` is the path to the backup directory on the _remote_ machine
+4. `-E` is the parameter for the backup repository encryption - we're using `none` here, so the repository won't be encrypted. You can also use `repokey` which requires the option `-P` to be set (either with value, like `-P "my_secret_key"` (not recommended!) or just `-P` to let the script prompt for the passphrase)
-5. `-E` is the parameter for the backup repository encryption - we're using `none` here, so the repository won't be encrypted. You can also use `repokey` which requires the option `-P` to be set (either with value, like `-P "my_secret_key"` (not recommended!) or just `-P` to let the script prompt for the passphrase)
+5. `-D` sets the _local_ directories that should be backed up
-6. `-D` sets the _local_ directories that should be backed up
+6. `-S` is the schedule in cron syntax - `0 3 * * *` will run the backup every day at 3 AM
-7. `-S` is the schedule in cron syntax - `0 3 * * *` will run the backup every day at 3 AM
+7. `-H` is the hostname or IP address of the machine, where the backups should be stored
-8. `-H` is the hostname or IP address of the machine, where the backups should be stored
+8. `-U` is the username to be used to connect to the backup host via ssh
-9. `-U` is the username to be used to connect to the backup host via ssh
+9.  `-W` will cause the script to prompt for the password of the user (you can also provide the password like `-W "my_ssh_password"`, but that is NOT recommended!)
-10. `-W` will cause the script to prompt for the password of the user (you can also provide the password like `-W "my_ssh_password"`, but that is NOT recommended!)
 
 This will cause the script to:
 1. Install all required packages to perform the backups on your local machine

borgmatic_setup.sh

@@ -28,9 +28,6 @@
 # - Pre-Backup commands
 # - Post-Backup commands
 
-# TODO:
-# - Invert the logic for upgrades (only upgrade if DO_UPGRADE is set) - this also includes upgrading single packages
-
 # SETTINGS - Make sure to adjust these settings to your needs
 DEBUG=0 # Set to 1 to enable debug mode
 
@@ -55,7 +52,8 @@ SSH_KEY_FILE="/root/.ssh/${SSH_KEY_NAME}_${SSH_KEY_TYPE}"
 
 # internal variables
 AUTO=0                  # Set to 1 to enable automatic, non-interactive mode
-NO_UPGRADE=0            # Set to 1 to skip package upgrades
+DO_UPGRADE_ALL=0        # Set to 1 to upgrade all installed packages
+DO_UPGRADE_REQUIRED=0   # Set to 1 to upgrade only required packages
 FIRST_BACKUP=0          # Set to 1 to run the first backup after setup
 IS_REMOTE_SYNOLGY=0     # Set to 1 if the remote end is a Synology NAS
 OVERRIDE_CRONTAB=0      # Set to 1 to override the existing crontab file
@@ -66,7 +64,7 @@ OVERRIDE_REPOSITORY=0   # Set to 1 to override the existing backup repository
 readonly SCRIPT_NAME="Borgmatic Backup Setup Tool"
 readonly SCRIPT_AUTHOR="Zion Networks at admin@zion-networks.de"
 readonly SCRIPT_SUPPORT="admin@zion-networks.de"
-readonly VERSION="1.1.0"
+readonly VERSION="1.2.0"
 
 # logging functions
 function inf {
@@ -237,6 +235,14 @@ function apt_upgrade {
     return $status
 }
 
+function apt_upgrade_selected {
+    exec 3>&1
+    status=$(apt-get install -y $* 2>&1 | while read -r line; do dbg "$line"; done 1>&3)
+    exec 3>&-
+
+    return $status
+}
+
 function apt_install {
     exec 3>&1
     status=$(apt-get install -y $1 2>&1 | while read -r line; do dbg "$line"; done 1>&3)
@@ -311,6 +317,21 @@ function run_command {
     return $status
 }
 
+function run_command_logged {
+    # run command and log output using inf function for each line
+
+    dbg "Running command: sudo $*"
+
+    exec 3>&1
+    sudo "$@" 2>&1 | while read -r line; do inf "$line"; done 1>&3
+    status=$?
+    exec 3>&-
+
+    dbg "Command exit status: $status"
+
+    return $status
+}
+
 function file_exists {
     if [ -f $1 ]; then
         return 0
@@ -361,11 +382,12 @@ function is_remote_synology {
 # Available arguments:
 # -a, --auto: Enable automatic, non-interactive mode
 # -d, --debug: Enable debug mode
-# -n, --no-upgrade: Skip package upgrades
 # -f, --first-backup: Run the first backup after setup
 # -o, --override-crontab: Override the existing crontab file
 # -c, --override-config: Override the existing borgmatic configuration file
 # -r, --override-repository: Override the existing backup repository
+# -s, --upgrade-required: Upgrade only required packages
+# -u, --upgrade-all: Upgrade all installed packages
 # -v, --version: Show script version
 # -h, --help: Show help message
 #
@@ -428,8 +450,12 @@ while [[ $# -gt 0 ]]; do
             echo "$SCRIPT_NAME v$VERSION"
             exit 0
             ;;
-        -n|--no-upgrade)
+        -s|--upgrade-required)
-            NO_UPGRADE=1
+            DO_UPGRADE_REQUIRED=1
+            shift
+            ;;
+        -u|--upgrade-all)
+            DO_UPGRADE_ALL=1
             shift
             ;;
         -f|--first-backup)
@@ -455,8 +481,10 @@ while [[ $# -gt 0 ]]; do
             inf "If no options are provided, the script will run in interactive mode."
             inf "All options are optional."
             inf ""
+            if [ "$EUID" -ne 0 ]; then
                 inf "\e[1m\e[31mImportant: The script must be run as root to work properly.\e[0m"
                 inf ""
+            fi
             inf "Options:"
             inf "  -a, --auto: Enable automatic, non-interactive mode"
             inf "  -n, --no-upgrade: Skip package upgrades"
@@ -465,6 +493,8 @@ while [[ $# -gt 0 ]]; do
             inf "  -o, --override-crontab: Override the existing crontab file"
             inf "  -c, --override-config: Override the existing borgmatic configuration file"
             inf "  -r, --override-repository: Override the existing backup repository"
+            inf "  -s, --upgrade-required: Upgrade only required packages"
+            inf "  -u, --upgrade-all: Upgrade all installed packages"
             inf "  -v, --version: Show script version"
             inf "  -h, --help: Show help message"
             inf ""
@@ -580,6 +610,10 @@ echo -e "╰──────────────────────
 
 inf "For support please contact $SCRIPT_AUTHOR"
 
+if [ $DO_UPGRADE_ALL -eq 1 ]; then
+    DO_UPGRADE_REQUIRED=1
+fi
+
 if [ $DEBUG -eq 1 ]; then
     dbg "Debug mode is enabled. This will print additional information."
 fi
@@ -600,23 +634,29 @@ wrn "This script will install \e[1mborgbackup\e[0m, \e[1mpipx\e[0m, python3-venv
 wrn "and \e[1msshpass\e[0m from the official repositories and \e[1mborgmatic\e[0m"
 wrn "from the official PyPI repository."
 
+inf ""
 inf "The following settings will be used for the backup setup:"
 inf "Backup repository: $BACKUP_REPO"
 inf "Backup encryption: $BACKUP_ENCRYPTION"
 inf "Backup directories: $BACKUP_DIRS"
 inf "Backup cron schedule: $BACKUP_CRON_SCHEDULE"
 inf "Backup server host: $SSH_USER@$SSH_HOST:$SSH_PORT"
+inf ""
 
-if [ -z $OVERRIDE_CRONTAB ]; then
+if [ $DO_UPGRADE_ALL -eq 1 ]; then
+    wrn "All installed packages will be upgraded!"
+fi
+
+if [ $OVERRIDE_CRONTAB -eq 1 ]; then
     wrn "Existing cron job at /etc/cron.d/borgmatic will be overridden!"
 fi
 
-if [ -z $OVERRIDE_CONFIG ]; then
+if [ $OVERRIDE_CONFIG -eq 1 ]; then
     wrn "Existing borgmatic configuration file at $BORGMATIC_CONFIG_FILE will be overridden!"
 fi
 
 # only ask for confirmation in interactive mode
-if [ -z $AUTO ] && ! wrnp "Do you want to continue? [y/N]"; then
+if [ $AUTO -eq 0 ] && ! wrnp "Do you want to continue? [y/N]"; then
     inf "Aborted."
     exit 1
 fi
@@ -654,14 +694,45 @@ if ! inf_follow "Checking if python3-venv is installed..." "\e[1;32mYes\e[0m" "\
     fi
 fi
 
-if [ -z $NO_UPGRADE ]; then
+if [ $DO_UPGRADE_ALL -eq 1 ]; then
-    if [ -z $AUTO ] && infp "Do you also want to upgrade all installed packages? [y/N]"; then
+    if [ $AUTO -eq 1 ] || wrnp "Do you want to upgrade all installed packages? [y/N]"; then
         if ! inf_follow "Upgrading packages..." "\e[1;32mOK\e[0m" "\e[1;31mFAILED\e[0m" apt_upgrade; then
             exit 1
         fi
     fi
 fi
 
+if [ $DO_UPGRADE_REQUIRED -eq 1 ]; then
+    if [ $AUTO -eq 1 ] || wrnp "Do you want to upgrade only required packages? [y/N]"; then
+        inf "Upgrading required packages..."
+
+        if ! inf_follow "Upgrading borgbackup..." "\e[1;32mOK\e[0m" "\e[1;31mFAILED\e[0m" apt_upgrade_selected borgbackup; then
+            err "Upgrade of borgbackup failed!"
+            exit 1
+        fi
+
+        if ! inf_follow "Upgrading pipx..." "\e[1;32mOK\e[0m" "\e[1;31mFAILED\e[0m" apt_upgrade_selected pipx; then
+            err "Upgrade of pipx failed!"
+            exit 1
+        fi
+
+        if ! inf_follow "Upgrading borgmatic..." "\e[1;32mOK\e[0m" "\e[1;31mFAILED\e[0m" apt_upgrade_selected borgmatic; then
+            err "Upgrade of borgmatic failed!"
+            exit 1
+        fi
+
+        if ! inf_follow "Upgrading python3-venv..." "\e[1;32mOK\e[0m" "\e[1;31mFAILED\e[0m" apt_upgrade_selected python3-venv; then
+            err "Upgrade of python3-venv failed!"
+            exit 1
+        fi
+
+        if ! inf_follow "Upgrading sshpass..." "\e[1;32mOK\e[0m" "\e[1;31mFAILED\e[0m" apt_upgrade_selected sshpass; then
+            err "Upgrade of sshpass failed!"
+            exit 1
+        fi
+    fi
+fi
+
 if ! inf_follow "Checking if borgbackup is installed..." "\e[1;32mYes\e[0m" "\e[1;31mNo\e[0m" apt_is_installed borgbackup; then
     if ! inf_follow "Installing borgbackup..." "\e[1;32mOK\e[0m" "\e[1;31mFAILED\e[0m" apt_install borgbackup; then
         err "Installation of borgbackup failed!"
@@ -687,10 +758,12 @@ if ! inf_follow "Checking if borgmatic is installed..." "\e[1;32mYes\e[0m" "\e[1
         exit 1
     fi
 else
-    if ! inf_follow "Upgrading borgmatic..." "\e[1;32mOK\e[0m" "\e[1;31mFAILED\e[0m" pipx_upgrade borgmatic; then
+    if [ $DO_UPGRADE_REQUIRED -eq 1 ]; then
+        if [ $AUTO -eq 0 ] && ! inf_follow "Upgrading borgmatic..." "\e[1;32mOK\e[0m" "\e[1;31mFAILED\e[0m" pipx_upgrade borgmatic; then
             err "Upgrade of borgmatic failed!"
             exit 1
         fi
+    fi
 fi
 
 if ! inf_follow "Checking for ~/.ssh directory..." "\e[1;32mYes\e[0m" "\e[1;31mNo\e[0m" dir_exists ~/.ssh; then
@@ -704,7 +777,7 @@ fi
 if ! inf_follow "Checking for default ssh key at $SSH_KEY_FILE..." "\e[1;32mYes\e[0m" "\e[1;31mNo\e[0m" file_exists $SSH_KEY_FILE; then
     inf "Generating a new ssh key pair..."
 
-    ssh-keygen -t $SSH_KEY_TYPE -C "borgmatic" -f $SSH_KEY_FILE -N "" -q > /dev/null 2>&1
+    ssh-keygen -t $SSH_KEY_TYPE -C "borgmatic_backup_$(hostname)" -f $SSH_KEY_FILE -N "" 2>&1 | while read -r line; do dbg "$line"; done
 
     if [ $? -eq 0 ]; then
         inf "Successfully generated a new ssh key pair."
@@ -867,7 +940,7 @@ fi
 # check for existing borg repository and abort if $OVERRIDE_REPOSITORY is not set
 if inf_follow "Checking for existing borgmatic repository..." "\e[1;32mYes\e[0m" "\e[1;31mNo\e[0m" run_command /root/.local/pipx/venvs/borgmatic/bin/borgmatic info; then
     if [ $OVERRIDE_REPOSITORY -ne 1 ]; then
-        if [ -z $AUTO ] && ! wrnp "A borgmatic repository already exists. Do you want to override it? [y/N]"; then
+        if [ $AUTO -eq 0 ] && ! wrnp "A borgmatic repository already exists. Do you want to override it? [y/N]"; then
             err "A borgmatic repository already exists."
             err "Please remove the existing repository or use the --override-repository option."
             exit 1
@@ -921,7 +994,7 @@ fi
 
 if [ $FIRST_BACKUP -eq 1 ]; then
     if [ $AUTO -eq 1 ] || infp "Do you want to run the first backup now? [Y/n]"; then
-        if ! inf_follow "Running first backup (this may take a while!)..." "\e[1;32mOK\e[0m" "\e[1;31mFAILED\e[0m" run_command /root/.local/pipx/venvs/borgmatic/bin/borgmatic create --verbosity 1 --list --stats; then
+        if ! inf_follow "Running first backup (this may take a while!)..." "\e[1;32mOK\e[0m" "\e[1;31mFAILED\e[0m" run_command_logged /root/.local/pipx/venvs/borgmatic/bin/borgmatic create --verbosity 1 --list --stats; then
             err "Failed to run first backup."
             err "If the error persists, please contact the support at $SCRIPT_SUPPORT."
             exit 1