diff --git a/README.md b/README.md index 060dccc..048a935 100644 --- a/README.md +++ b/README.md @@ -12,7 +12,7 @@ We're planning to release a growing amount of open source software, that is free | Name | Description | License | Current Version | Written in | Supported Distros | File | |------|-------------|---------|-----------------|------------|-------------------|------| -| [Borgmatic Setup Tool](#borgmatic-setup-tool) | If you plan to use borg as backup solution, you should also take a look at [borgmatic](https://torsion.org/borgmatic/). It's a Python wrapper for the award winning backup tool [borgbackup](https://borgbackup.readthedocs.io/en/stable/index.html) that simplifies creating secure and reliable backups even more. You can even store your configurations in files. This script will do the setup for you to. | [MIT]([LICENSE](https://git.zion-networks.de/ZionNetworks/linux-bash-scripts/src/branch/main/LICENSE)) | v1.1.0 | Bash | Debian and derivates | [bormatic_setup.sh](https://git.zion-networks.de/ZionNetworks/linux-bash-scripts/src/branch/main/borgmatic_setup.sh) | +| [Borgmatic Setup Tool](#borgmatic-setup-tool) | If you plan to use borg as backup solution, you should also take a look at [borgmatic](https://torsion.org/borgmatic/). It's a Python wrapper for the award winning backup tool [borgbackup](https://borgbackup.readthedocs.io/en/stable/index.html) that simplifies creating secure and reliable backups even more. You can even store your configurations in files. This script will do the setup for you to. | [MIT]([LICENSE](https://git.zion-networks.de/ZionNetworks/linux-bash-scripts/src/branch/main/LICENSE)) | v1.2.0 | Bash | Debian and derivates | [bormatic_setup.sh](https://git.zion-networks.de/ZionNetworks/linux-bash-scripts/src/branch/main/borgmatic_setup.sh) | #### Borgmatic Setup Tool @@ -28,13 +28,18 @@ For beginners, the fully automated installation is recommended. BST offers several command-line options to automate the whole process and get your backup schedule ready to go as fast as possible. -These options are available in version 1.1.0: +These options are available in version **1.2.0**: ``` -a, --auto: Enable automatic, non-interactive mode -n, --no-upgrade: Skip package upgrades -d, --debug: Enable debug mode -f, --first-backup: Run the first backup after setup +-o, --override-crontab: Override the existing crontab file +-c, --override-config: Override the existing borgmatic configuration file +-r, --override-repository: Override the existing backup repository +-s, --upgrade-required: Upgrade only required packages +-u, --upgrade-all: Upgrade all installed packages -v, --version: Show script version -h, --help: Show help message @@ -58,7 +63,7 @@ These options are available in version 1.1.0: To run a fully automated setup you need to set the `--auto` or `a` flag. This will cause the script to skip some prompts. -If you don't want the script to also upgrade the packages on your system, set the `--no-upgrade` or `-n` flag. (this will be changed in the future, replacing the flag with `--do-upgrade` or `-u`) +If you want the script to also upgrade the packages on your system, set the `--upgrade-all` or `--upgrade--required` flag, depending on your needs. If neither are given, the script won't upgrade any already installed packages. ###### Example: Backup local configurations and home directories @@ -68,20 +73,19 @@ If you don't want the script to also upgrade the packages on your system, set th 2. Set the execution flag using `chmod +x borgmatic_setup.sh` 3. Run the script: -`sudo ./borgmatic_setup.sh -a -n -f -R "/mnt/backups/my-host" -E "none" -D "/etc /home" -S "0 3 * * *" -H "my.backup.host" -U "backupuser" -W` +`sudo ./borgmatic_setup.sh -a -f -R "/mnt/backups/my-host" -E "none" -D "/etc /home" -S "0 3 * * *" -H "my.backup.host" -U "backupuser" -W` Let's break up this script: 1. `-a` enables the automatic mode, so the user will see no prompts unless it's required -2. `-n` will prevent the script from upgrading packages via `apt-get upgrade` -3. `-f` will cause the script to run a first backup when the setup is done -4. `-R` is the path to the backup directory on the _remote_ machine -5. `-E` is the parameter for the backup repository encryption - we're using `none` here, so the repository won't be encrypted. You can also use `repokey` which requires the option `-P` to be set (either with value, like `-P "my_secret_key"` (not recommended!) or just `-P` to let the script prompt for the passphrase) -6. `-D` sets the _local_ directories that should be backed up -7. `-S` is the schedule in cron syntax - `0 3 * * *` will run the backup every day at 3 AM -8. `-H` is the hostname or IP address of the machine, where the backups should be stored -9. `-U` is the username to be used to connect to the backup host via ssh -10. `-W` will cause the script to prompt for the password of the user (you can also provide the password like `-W "my_ssh_password"`, but that is NOT recommended!) +2. `-f` will cause the script to run a first backup when the setup is done +3. `-R` is the path to the backup directory on the _remote_ machine +4. `-E` is the parameter for the backup repository encryption - we're using `none` here, so the repository won't be encrypted. You can also use `repokey` which requires the option `-P` to be set (either with value, like `-P "my_secret_key"` (not recommended!) or just `-P` to let the script prompt for the passphrase) +5. `-D` sets the _local_ directories that should be backed up +6. `-S` is the schedule in cron syntax - `0 3 * * *` will run the backup every day at 3 AM +7. `-H` is the hostname or IP address of the machine, where the backups should be stored +8. `-U` is the username to be used to connect to the backup host via ssh +9. `-W` will cause the script to prompt for the password of the user (you can also provide the password like `-W "my_ssh_password"`, but that is NOT recommended!) This will cause the script to: 1. Install all required packages to perform the backups on your local machine diff --git a/borgmatic_setup.sh b/borgmatic_setup.sh index d2142c9..7bffbd5 100755 --- a/borgmatic_setup.sh +++ b/borgmatic_setup.sh @@ -28,9 +28,6 @@ # - Pre-Backup commands # - Post-Backup commands -# TODO: -# - Invert the logic for upgrades (only upgrade if DO_UPGRADE is set) - this also includes upgrading single packages - # SETTINGS - Make sure to adjust these settings to your needs DEBUG=0 # Set to 1 to enable debug mode @@ -55,7 +52,8 @@ SSH_KEY_FILE="/root/.ssh/${SSH_KEY_NAME}_${SSH_KEY_TYPE}" # internal variables AUTO=0 # Set to 1 to enable automatic, non-interactive mode -NO_UPGRADE=0 # Set to 1 to skip package upgrades +DO_UPGRADE_ALL=0 # Set to 1 to upgrade all installed packages +DO_UPGRADE_REQUIRED=0 # Set to 1 to upgrade only required packages FIRST_BACKUP=0 # Set to 1 to run the first backup after setup IS_REMOTE_SYNOLGY=0 # Set to 1 if the remote end is a Synology NAS OVERRIDE_CRONTAB=0 # Set to 1 to override the existing crontab file @@ -66,7 +64,7 @@ OVERRIDE_REPOSITORY=0 # Set to 1 to override the existing backup repository readonly SCRIPT_NAME="Borgmatic Backup Setup Tool" readonly SCRIPT_AUTHOR="Zion Networks at admin@zion-networks.de" readonly SCRIPT_SUPPORT="admin@zion-networks.de" -readonly VERSION="1.1.0" +readonly VERSION="1.2.0" # logging functions function inf { @@ -237,6 +235,14 @@ function apt_upgrade { return $status } +function apt_upgrade_selected { + exec 3>&1 + status=$(apt-get install -y $* 2>&1 | while read -r line; do dbg "$line"; done 1>&3) + exec 3>&- + + return $status +} + function apt_install { exec 3>&1 status=$(apt-get install -y $1 2>&1 | while read -r line; do dbg "$line"; done 1>&3) @@ -311,6 +317,21 @@ function run_command { return $status } +function run_command_logged { + # run command and log output using inf function for each line + + dbg "Running command: sudo $*" + + exec 3>&1 + sudo "$@" 2>&1 | while read -r line; do inf "$line"; done 1>&3 + status=$? + exec 3>&- + + dbg "Command exit status: $status" + + return $status +} + function file_exists { if [ -f $1 ]; then return 0 @@ -361,11 +382,12 @@ function is_remote_synology { # Available arguments: # -a, --auto: Enable automatic, non-interactive mode # -d, --debug: Enable debug mode -# -n, --no-upgrade: Skip package upgrades # -f, --first-backup: Run the first backup after setup # -o, --override-crontab: Override the existing crontab file # -c, --override-config: Override the existing borgmatic configuration file # -r, --override-repository: Override the existing backup repository +# -s, --upgrade-required: Upgrade only required packages +# -u, --upgrade-all: Upgrade all installed packages # -v, --version: Show script version # -h, --help: Show help message # @@ -428,8 +450,12 @@ while [[ $# -gt 0 ]]; do echo "$SCRIPT_NAME v$VERSION" exit 0 ;; - -n|--no-upgrade) - NO_UPGRADE=1 + -s|--upgrade-required) + DO_UPGRADE_REQUIRED=1 + shift + ;; + -u|--upgrade-all) + DO_UPGRADE_ALL=1 shift ;; -f|--first-backup) @@ -455,8 +481,10 @@ while [[ $# -gt 0 ]]; do inf "If no options are provided, the script will run in interactive mode." inf "All options are optional." inf "" - inf "\e[1m\e[31mImportant: The script must be run as root to work properly.\e[0m" - inf "" + if [ "$EUID" -ne 0 ]; then + inf "\e[1m\e[31mImportant: The script must be run as root to work properly.\e[0m" + inf "" + fi inf "Options:" inf " -a, --auto: Enable automatic, non-interactive mode" inf " -n, --no-upgrade: Skip package upgrades" @@ -465,6 +493,8 @@ while [[ $# -gt 0 ]]; do inf " -o, --override-crontab: Override the existing crontab file" inf " -c, --override-config: Override the existing borgmatic configuration file" inf " -r, --override-repository: Override the existing backup repository" + inf " -s, --upgrade-required: Upgrade only required packages" + inf " -u, --upgrade-all: Upgrade all installed packages" inf " -v, --version: Show script version" inf " -h, --help: Show help message" inf "" @@ -580,6 +610,10 @@ echo -e "╰────────────────────── inf "For support please contact $SCRIPT_AUTHOR" +if [ $DO_UPGRADE_ALL -eq 1 ]; then + DO_UPGRADE_REQUIRED=1 +fi + if [ $DEBUG -eq 1 ]; then dbg "Debug mode is enabled. This will print additional information." fi @@ -600,23 +634,29 @@ wrn "This script will install \e[1mborgbackup\e[0m, \e[1mpipx\e[0m, python3-venv wrn "and \e[1msshpass\e[0m from the official repositories and \e[1mborgmatic\e[0m" wrn "from the official PyPI repository." +inf "" inf "The following settings will be used for the backup setup:" inf "Backup repository: $BACKUP_REPO" inf "Backup encryption: $BACKUP_ENCRYPTION" inf "Backup directories: $BACKUP_DIRS" inf "Backup cron schedule: $BACKUP_CRON_SCHEDULE" inf "Backup server host: $SSH_USER@$SSH_HOST:$SSH_PORT" +inf "" -if [ -z $OVERRIDE_CRONTAB ]; then +if [ $DO_UPGRADE_ALL -eq 1 ]; then + wrn "All installed packages will be upgraded!" +fi + +if [ $OVERRIDE_CRONTAB -eq 1 ]; then wrn "Existing cron job at /etc/cron.d/borgmatic will be overridden!" fi -if [ -z $OVERRIDE_CONFIG ]; then +if [ $OVERRIDE_CONFIG -eq 1 ]; then wrn "Existing borgmatic configuration file at $BORGMATIC_CONFIG_FILE will be overridden!" fi # only ask for confirmation in interactive mode -if [ -z $AUTO ] && ! wrnp "Do you want to continue? [y/N]"; then +if [ $AUTO -eq 0 ] && ! wrnp "Do you want to continue? [y/N]"; then inf "Aborted." exit 1 fi @@ -654,14 +694,45 @@ if ! inf_follow "Checking if python3-venv is installed..." "\e[1;32mYes\e[0m" "\ fi fi -if [ -z $NO_UPGRADE ]; then - if [ -z $AUTO ] && infp "Do you also want to upgrade all installed packages? [y/N]"; then +if [ $DO_UPGRADE_ALL -eq 1 ]; then + if [ $AUTO -eq 1 ] || wrnp "Do you want to upgrade all installed packages? [y/N]"; then if ! inf_follow "Upgrading packages..." "\e[1;32mOK\e[0m" "\e[1;31mFAILED\e[0m" apt_upgrade; then exit 1 fi fi fi +if [ $DO_UPGRADE_REQUIRED -eq 1 ]; then + if [ $AUTO -eq 1 ] || wrnp "Do you want to upgrade only required packages? [y/N]"; then + inf "Upgrading required packages..." + + if ! inf_follow "Upgrading borgbackup..." "\e[1;32mOK\e[0m" "\e[1;31mFAILED\e[0m" apt_upgrade_selected borgbackup; then + err "Upgrade of borgbackup failed!" + exit 1 + fi + + if ! inf_follow "Upgrading pipx..." "\e[1;32mOK\e[0m" "\e[1;31mFAILED\e[0m" apt_upgrade_selected pipx; then + err "Upgrade of pipx failed!" + exit 1 + fi + + if ! inf_follow "Upgrading borgmatic..." "\e[1;32mOK\e[0m" "\e[1;31mFAILED\e[0m" apt_upgrade_selected borgmatic; then + err "Upgrade of borgmatic failed!" + exit 1 + fi + + if ! inf_follow "Upgrading python3-venv..." "\e[1;32mOK\e[0m" "\e[1;31mFAILED\e[0m" apt_upgrade_selected python3-venv; then + err "Upgrade of python3-venv failed!" + exit 1 + fi + + if ! inf_follow "Upgrading sshpass..." "\e[1;32mOK\e[0m" "\e[1;31mFAILED\e[0m" apt_upgrade_selected sshpass; then + err "Upgrade of sshpass failed!" + exit 1 + fi + fi +fi + if ! inf_follow "Checking if borgbackup is installed..." "\e[1;32mYes\e[0m" "\e[1;31mNo\e[0m" apt_is_installed borgbackup; then if ! inf_follow "Installing borgbackup..." "\e[1;32mOK\e[0m" "\e[1;31mFAILED\e[0m" apt_install borgbackup; then err "Installation of borgbackup failed!" @@ -687,9 +758,11 @@ if ! inf_follow "Checking if borgmatic is installed..." "\e[1;32mYes\e[0m" "\e[1 exit 1 fi else - if ! inf_follow "Upgrading borgmatic..." "\e[1;32mOK\e[0m" "\e[1;31mFAILED\e[0m" pipx_upgrade borgmatic; then - err "Upgrade of borgmatic failed!" - exit 1 + if [ $DO_UPGRADE_REQUIRED -eq 1 ]; then + if [ $AUTO -eq 0 ] && ! inf_follow "Upgrading borgmatic..." "\e[1;32mOK\e[0m" "\e[1;31mFAILED\e[0m" pipx_upgrade borgmatic; then + err "Upgrade of borgmatic failed!" + exit 1 + fi fi fi @@ -704,7 +777,7 @@ fi if ! inf_follow "Checking for default ssh key at $SSH_KEY_FILE..." "\e[1;32mYes\e[0m" "\e[1;31mNo\e[0m" file_exists $SSH_KEY_FILE; then inf "Generating a new ssh key pair..." - ssh-keygen -t $SSH_KEY_TYPE -C "borgmatic" -f $SSH_KEY_FILE -N "" -q > /dev/null 2>&1 + ssh-keygen -t $SSH_KEY_TYPE -C "borgmatic_backup_$(hostname)" -f $SSH_KEY_FILE -N "" 2>&1 | while read -r line; do dbg "$line"; done if [ $? -eq 0 ]; then inf "Successfully generated a new ssh key pair." @@ -867,7 +940,7 @@ fi # check for existing borg repository and abort if $OVERRIDE_REPOSITORY is not set if inf_follow "Checking for existing borgmatic repository..." "\e[1;32mYes\e[0m" "\e[1;31mNo\e[0m" run_command /root/.local/pipx/venvs/borgmatic/bin/borgmatic info; then if [ $OVERRIDE_REPOSITORY -ne 1 ]; then - if [ -z $AUTO ] && ! wrnp "A borgmatic repository already exists. Do you want to override it? [y/N]"; then + if [ $AUTO -eq 0 ] && ! wrnp "A borgmatic repository already exists. Do you want to override it? [y/N]"; then err "A borgmatic repository already exists." err "Please remove the existing repository or use the --override-repository option." exit 1 @@ -921,7 +994,7 @@ fi if [ $FIRST_BACKUP -eq 1 ]; then if [ $AUTO -eq 1 ] || infp "Do you want to run the first backup now? [Y/n]"; then - if ! inf_follow "Running first backup (this may take a while!)..." "\e[1;32mOK\e[0m" "\e[1;31mFAILED\e[0m" run_command /root/.local/pipx/venvs/borgmatic/bin/borgmatic create --verbosity 1 --list --stats; then + if ! inf_follow "Running first backup (this may take a while!)..." "\e[1;32mOK\e[0m" "\e[1;31mFAILED\e[0m" run_command_logged /root/.local/pipx/venvs/borgmatic/bin/borgmatic create --verbosity 1 --list --stats; then err "Failed to run first backup." err "If the error persists, please contact the support at $SCRIPT_SUPPORT." exit 1