2017-12-24 Borg Backup

The new laptop uses PureOS, a Debian variant, and thus will not use the Apple Timemachine backup disks. What else to use? I asked around on Mastodon:

What do you use to backup an entire GNU/Linux laptop to an external disk? Ideally it would be a bootable backup, of course, but that’s not mandatory. Déjà Dup Backup Tool seems to be designed for just user data. Duplicity still generates opaque files. Ideally, I’d use an encrypted external disk and just rsync every hour if the disk is mounted, and delete old backups when running out of space. Does this wrapper script already exist? Something like Time Machine for every Unix out there.

Déjà Dup Backup Tool

Duplicity

Time Machine for every Unix out there

I mean that rsync creates linked trees that look like complete sets of backups for every time period, and thus you can delete old link trees the actual file content will only get deleted when it is not referred to anymore. You basically want a clever use of the `--link-dest` parameter, as illustrated in the linked article. That also solves the problem with `--delete` deleting files in your backup.

Radomir suggested Borgbackup and that is what I went with. I still don’t like the opaque file format, but I have to start somewhere.

Borgbackup

Following the tutorial Automated backups to a local hard drive, I run only into a small number of problems.

Automated backups to a local hard drive

1. you have to `mkdir /mnt/backup/borg-backups` before creating the repo

2. you have to `chmod +x run.sh` before running it

3. if you create your repo using `borg init --encryption=repokey --progress /mnt/backup/borg-backups/backup.borg` you will type in a key and you then need to `export BORG_PASSPHRASE="*secret*"` in `run.sh` as indicated by the comment somewhere in the middle

When all that was done, it still wouldn’t run using `systemctl start --no-block automatic-backup`.

Here’s what it says when I check using `journalctl -fu automatic-backup`:

But I was able to run it manually using `/usr/bin/borg create --stats --one-file-system --compression lz4 --checkpoint-interval 86400 --exclude /root/.cache --exclude /var/cache --exclude /var/lib/docker/devicemapper /mnt/backup/borg-backups/backup.borg::2017-12-24-melanobombus-32376-system / /boot` and `/usr/bin/borg create --stats --one-file-system --compression lz4 --checkpoint-interval 86400 --exclude 'sh:/home/*/.cache' /mnt/backup/borg-backups/backup.borg::2017-12-24-melanobombus-32376-home /home/` so I’m not quite sure what the problem is.

Any ideas?

*I just came back* from the family event, ran `run.sh` as root, no problems. Then I started the backup service via `systemctl`, no problem. I guess it just works, for the moment?

In theory, plugging in the drive should now mount it automatically and once that happens, a new backup will be made.

This last part actually needs an explanation. I used the /Disks/ application to format the external disk and mount it.

Screenshot of the Disk app

This resulted in a change to `/etc/fstab`:

Thus, any disk labeled “Backup” will be mounted as `/mnt/backup`.

/etc/backups

Let me quickly copy and paste the content of the various files in `/etc/backups` just in case the original documentation changes.

40-backup.rules

You installed a symlink to this file using `ln -s /etc/backups/40-backup.rules /etc/udev/rules.d/40-backup.rules`.

autoeject-no

This is an empty file for me to rename. If the file `autoeject` exists, the disk will be ejected after the backup is made. I don’t know whether I will use this feature. This file serves as a reminder. See the end of `run.sh` for details.

(I eventually renamed it to `autoeject`.)

automatic-backup.service

You installed a symlink to this file using `ln -s /etc/backups/automatic-backup.service /etc/systemd/system/automatic-backup.service`.

backup.disks

This is the number derived from `lsblk -o+uuid,label`. It will differ from what you need to put here! This lists the disks that are actual backup disks. All the others will be ignored by `run.sh` even if they are mounted as `/mnt/backup`.

When I ran `lsblk -o+uuid,label` I also saw that the disk was `/dev/sdb`. To mount it for the first time: `mount /dev/sdb /mnt/backup`. Run `mkdir /mnt/backup/borg-backups` to create the directory. Run `borg init --encryption=repokey --progress /mnt/backup/borg-backups/backup.borg` to create the repo in that directory.

run.sh

Don’t forget to search for `BORG_PASSPHRASE` and change it to whatever you used when you ran `borg init --encryption=repokey --progress /mnt/backup/borg-backups/backup.borg`.

Run `sudo chmod 0700 run.sh*` to hide the passphrase from everybody else and to make it executable.

Examining the backups

Listing the archives:

Mounting an archive:

Looking good!

☯

Todo:

Handle exit codes? See Automatic backups.

Automatic backups

☯

Now that I have seen it all in action, perhaps Déjà Dup Backup Tool is close enough? After all, I still have opaque files, now. 😟 And Déjà Dup is well integrated in the system...

Déjà Dup Backup Tool

​#Backup ​#Borg

Comments

(Please contact me if you want to remove your comment.)

⁂

Moin Alex, nice post!

Some comments:

“you have to `mkdir /mnt/backup/borg-backups` before creating the repo”

borg creates the repo dir if it does not exist, but maybe it does not create missing parent dirs of it. Not sure if that would be an improvement if it did, esp. considering typos.

“`--checkpoint-interval 86400`”

be careful with a that long checkpoint interval. you will lose up to a day of “backup work” if the connection breaks down before finishing.

“`DATE=$(date --iso-8601)-$(hostname)`”

borg can expand `{utcnow}-{hostname}` internally.

“`borg --version`”

`borg create ... --show-version ...` (similar to `--show-rc`)

borg prune: maybe add `--stats` so it tells how much repo space it freed.

systemd issues: no idea

Cheers, Thomas

– Thomas Waldmann 2017-12-29 01:08 UTC

Thomas Waldmann

---

Thank you for the comments!

I read up on the various options and changed the script to now run as follows:

– Alex 2017-12-29 13:20 UTC

---

As a reminder to myself: what to do when you want to add another disk to your rotating disk schedule?

1. I used the disk utility to format and partition the backup disk. Partitioning: GUID Partition Table. Volume: Ext4.

2. `mkdir /mnt/backup/borg-backups/`

3. `borg init --encryption=repokey --progress /mnt/backup/borg-backups/backup.borg`

4. use the same password as I provided in `/etc/backups/run.sh`

5. use `lsblk --list --output=uuid,mountpoint` to find the new UUID

6. add this UUID to `/etc/backups/backup.disks`

Unmount the disk and unplug it, then plug it again and look at the output of `sudo journalctl -fu automatic-backup`.

– Alex Schroeder 2018-02-08 07:06 UTC

---

Make sure you check the journal! Today I ran `sudo journalctl -fu automatic-backup` and saw:

I was confused but finally decided to just try again, running `sudo umount /mnt/backup` and unplugging the disk, plugging it back in again, and then it worked:

Weird!

– Alex Schroeder 2018-02-16 13:24 UTC

---

Here’s something that happened when I plugged a very old backup disk into my laptop:

What does “Failed to create/acquire the lock /mnt/backup/borg-backups/backup.borg/lock.exclusive” mean? It means that there are lock files in the old backup directory. The solution is the “break-lock” subcommand.

– Alex 2021-05-08 10:45 UTC