Backup Paradise

forthebadge forthebadge Gem Version

This gem was last updated on the 21.05.2022 (dd.mm.yyyy notation), at 01:25:28 o'clock.

This project attempts to assist you - and me - in backing up data to an external USB device specifically.

The project is mostly catering to my own use cases, so it may not be very useful to other ruby users. I will try to make adjustments so that other ruby users can use it (see the file called config.yml), but this is an ongoing effort that may take some time.

The project is primarily just a fancy recursive copy operation, but it will do some minor additional notifications on the commandline, make use of colours (if the colours gem has been installed), automatically create some log files and even provide ruby-gtk3 bindings, as well as tk-bindings (although the latter ones are currently incomplete).

So it is a recursive copy script with some add-ons really, nothing really sophisticated aside from this on top.

The project has one major goal and a few minor goals.

The major goal is to be usable as a backup-assistant, on both linux, but also on windows systems.

The minor goals include:

  • Flexibility, to also allow us to back up only individual files or individual directories. The user should be able to specify what has to be backed up.

  • Notify the user about the status of the backup-operation and prior backup-operations.

  • Provide some simple GUI bindings to use this project, in particular for both ruby-gtk and ruby-tk. The latter so that we can use this on windows.

General usage of the BackupParadise project - backing up data via ruby

For the time being, I have aliased the main executable at bin/backup_paradise onto:

backup_paradise

To then use this, I can issue the following commands:

backup_paradise usb1
backup_paradise usb2
backup_paradise tousb1
backup_paradise tousb2
backup_paradise tousb3
backup_paradise tousb4

Depending on where the external USB device is mounted.

What does an argument, such as usb1, imply?

This used to refer to the hardcoded path at /Mount/USB1/ on my home system. usb2 would then point at /Mount/USB2/ and so forth.

These 'main aliases' can also be accessed via some shortcuts, such as "tousb1", without the quotes. See the examples listed above.

I adopted this scheme after having used GoboLinux. Note that this used to be valid from December 2018 up to May 2021.

In May 2021 this approach was changed. While I still use the above instructions, such as via backup_paradise usb1, a .yml file now decides the target directory. So, usb1, usb2 and so forth, are now simply aliases to whatever the .yml file specifies. The advantage of this approach is that other users can re-define the targets there - simply have a look at the config.yml file and change it if you would like to. (Of course overruling this via the commandline still works, so you don't really need to have to modify the config.yml file.)

Other mount points can also be used, although not with a convenience shortcut. Have a look at the option --backup-to=/opt/ documented elsewhere in this file, to specify another base-directory to use.

The configuration file called config.yml

The BackupParadise project comes with a small configuration file, called config.yml, in the yaml/ subdirectory.

This is an optional file, though - if it does not exist then the project will ignore this file completely and use internal defaults as-is. But if it is available, and it can be found by ruby, then it will be used as the default configuration.

Since as of September 2021* the **config.yml file has the following five main configuration settings:

use_this_program_to_rename_tabs: String
show_popup_notification:         Boolean
use_system_cp:                   Boolean
backup_these_directories:        Array
local_audio_directory:           String

The first three entries are not so important, but the fourth entry is quite important, because this Array specifies which directories we want to back up and thus copy. The fifth entry can be used to denote your local collection of audio files, if these are kept in a directory. I use /home/x/songs/ for this.

More options may be added in the future, but I will try to only add options that make sense, rather than overwhelm users with lots of configuration options.

The individual aliases, such as usb1, usb2, and so forth, are also specified now:

usb1: /Mount/USB1/
usb2: /Mount/USB2/
usb3: /Mount/USB3/
usb4: /Mount/USB4/
usb5: /Mount/USB5/

The rationale for these default to my home setup, where I mount devices to these directories (on Linux).

Log files

The BackupParadise project will try to log as to what has happened, since as of May 2018. Several different log files may be created.

If there is more than one log file stored at the target device then the current behaviour is to delete all but one log file.

Another log file that may commonly be used is the one that targets the file /Depot/Temp/backup.log. This will, however had, only work if the directory /Depot/Temp/ exists, which is the case on my home system. The whole gem here is heavily tailored to my own needs - I can adapt it to meet other people's wishes if necessary, of course.

The format of the log file going into backup.log is simply how long it took to make the backup, in seconds, and the exact date when that particular log file was created. This is equal to when the backup-operation happened, too.

Backing up individual directories

If you have a directory called FOOBAR/, then you can back it up simply by issuing:

backup_paradise --FOOBAR
backup_paradise FOOBAR

Whichever variant you prefer. Note that this requires that the external device has been mounted already. You can set where the device should reside via:

BackupParadise.set_mounted_path()

Simply pass the directory that you may wish to use for backup purpose there.

Note that class AdvancedBackup will assume a certain target device automatically, tailored to my own system. This is usually at /Mount/USB1 for the first mounted USB device on my system.

If you wish to overrule that and use another target, which you may most likely want to do, then you can use any of the following commandline flags for this:

backup_paradise --use-this-as-target-for-backup=/opt/
backup_paradise --use-this-as-target=/opt/
backup_paradise --backup-to=/opt/

In this case /opt/ would be the target that you use as source directory for the backup-operation.

If you wish to backup the /Programs/ hierarchy, should you keep an AppDir layout on your system, then this commandline invocation may be of help:

backup_paradise --programs-dir

To backup the /home/x/studium/ directory, the following commands are possible:

backup_paradise --studium-dir
backup_paradise --studium-directory
backup_paradise --studium

To backup, on my home system, the /AUTOGENERATED/ directory, I do:

backup_paradise --autogenerated

Miscellaneous comments

The BackupParadise project has been rewritten a few times already over the years.

In the past there was an instance variable in class AdvancedBackup called @last_backup_directory, which was pointing to an absolute path such as /Mount/USB1/last_backup-13.08.2018-21:11:15/.

It is no longer in use, but I keep it as a reference hint to myself - who knows what may be changed in the future.

Specific examples

This subsection details some invocation examples in a short way.

Some of these will only work on my home system for the time being, until I have sufficiently changed the project to allow more flexibility in this regard.

Backup all audio-files to /opt/:

backup_paradise --backup-to=/opt/ --audio-dir

Backup all relevant entries into the default chroot target (on my home system):

backup_paradise --chroot

How do I make use of this project

I make use of the backup_paradise project to back up relevant data on my home system. For example, all source-archives that are registered within the RBT project I also keep available locally. Then there is my ruby code, other files, and possibly books, papers and tutorials stored in .pdf format.

In September 2020, the regular backup size was 54G per backup-action, mostly owing to the source archives (krita alone occupies more than 167 MB, for example). This is way too much, in my opinion, because it takes about half an hour or so to back up on USB 2.0 already.

I will have to keep reducing file size and removing stuff that I no longer need; aka do some clean-up. Or perhaps hope that future USB variants will improve things here.

At any rate, this subsection shows just a bit how I may use the project.

My usual go-to alias is:

rbackup usb1

to backup to the mount point /Mount/USB1.

This is the command invocation I tend to use most frequently so.

class BackupParadise::Backup

This class is the main interface for the BackupParadise project.

For a list of documented options, pass in --help.

A few examples will be mentioned here as well, in a succinct manner:

rbackup --pwd # backup into the current working directory

BackupParadise.simple_backup

BackupParadise.simple_backup can be used as a simpler alternative to class BackupParadise::Backup. While the latter class is very feature-rich, sometimes you may want to use BackupParadise.simple_backup() instead.

Usage examples:

BackupParadise.simple_backup(:audio, '/Mount/HDD1/')
BackupParadise.simple_backup(:audio, :usb1)

Backup time (statistics)

This subsection is just a remainder for me how long backing up all my dataset (which is a LOT) takes. These times are gathered on Linux.

Ideally this time should be as small as possible, but evidently, the more time that has to be transferred, the longer it will take. This refers to about 100 GB or more right now. I am open for any suggestions to cut the time down.

The measurements happen when there is a low CPU load.

04.06.2021: 39.17 minutes

GUI components

Right now the GTK3 version (ruby-gtk3) is the most advanced GUI component of this project - but it is still rather limited.

Additionally I am working on ruby-tk code as well, just so that this project can be used on windows (addendum in August 2021: this has been superceded by ruby-libui, which is a LOT simpler to use on windows). The priority for ruby-tk is lower than that of the ruby-gtk3 code. (In the future, one day, we may try to use the same, unified code base for GUIs, but this is a minor, additional goal.)

You can try to invoke the GUI components from the commandline, possibly via any of the following flags:

backup_paradise --gui
backup_paradise --gtk # This variant now defaults to ruby-gtk3.

Or more specifically:

backup_paradise --gtk3

There used to be gtk2-bindings, but they currently (May 2021) do not work very well. In August 2021 I abandoned the ruby-gtk2 code completely - new GUI code will be written either in ruby-gtk3 and/or in ruby-libui. ruby-gtk2 is still nice, but ruby-gtk3 allows for more flexibility, in particular in regards to CSS.

In September 2021 I rewrote the old ruby-gtk3 code. The current iteration now looks like this:

The code base for that class is now largely a module, which I use for libui-bindings too. In the long run I would like to have ruby-tk and fxruby support as well as for the www (via sinatra and .cgi files). Stay tuned in this regard, but don't expect this to come too soon.

Licence

Since as of August 2021 the MIT licence is used for new releases of this gem. While the GPL is a fine licence, I simply do not have the time or willingness to want to enforce it for such a simple project like backup_paradise. MIT is much simpler to adhere to as well, so there we go.

Contact information

If your creative mind has ideas and specific suggestions to make this gem more useful in general, feel free to drop me an email at any time, via:

shevy@inbox.lt

Before that email I used an email account at Google gmail, but in 2021 I decided to slowly abandon gmail for various reasons. In part this is because the UI annoys me (on non-chrome browser loading takes too long), but also because of Google's attempt to establish mass surveillance via its federated cohorts sniffing (FLoC). I do not know what happened at Google, but enough is enough - there is only so much you can take while supporting greed. When it comes to data mining done by private groups, ultimately the user became the product.

Do keep in mind that responding to emails may take some time, depending on the amount of work I may have at that moment, due to reallife time constraints. I will, however had, read feedback eventually. Patches and code changes are welcome too, of course, as long as they are in the spirit of the project at hand, e. g. fitting to the general theme. For this I may make use of github as a discussion site, but this has a low priority right now.