Git for Synchronization and Backup

Git might be used as a back up system. There used to be an article on this topic on http://www.bytetrap.com/blog/2008/11/24/backup-and-synchronization-git/. Unfortunately it is not available any more. Here are the main headlines:

  • Why git, not rsync
  • What folders / files to exclude ( to be set in .gitignore )
  • How to do it
  • Pros and Cons

more resources:

gibak – Backup Your Home Folder

http://eigenclass.org/hiki/gibak-0.3.0

for drawbacks of the version scroll all the way down on http://eigenclass.org/hiki/gibak-backup-system-introduction gibak - a backup tool based on git

get it:

cd ~/Downloads
git clone http://eigenclass.org/repos/git/gibak/.git/

see how it works:

gedit gibak/gibak

installation

According to gibak/README.txt.

Compile gibak:

omake

Copy the following executables to a directory in your path:

find-git-files find-git-repos gibak ometastore

usage

According to gibak/README.txt.

Run gibak without any options to get a help message.

The normal workflow is: run once to initialize the backup system:

gibak init

edit to make sure you don't import unwanted files

vim .gitignore

edit .gitignore files in other subdirectories you can get a list of the files which will be saved using {{find-git-files}} or {{gibak ls-new-files}}

the first commit will be fairly slow (but the following ones will be very fast):

gibak commit

.... later ....

gibak commit

The backup will be placed in $HOME/.git. "Nested Git repositories" will be rsync'ed to $HOME/.git/git-repositories and they will be registered as submodules in the main Git repository (run git help submodule for more information on submodules). You might want to use a cronjob to save snapshots of the repositories in $HOME/.git/git-repositories.

After you gibak init, $HOME becomes a git repository, so you can use normal git commands. If you use gibak commit, however, new files will automatically be added to the repository if they are not ignored (as indicated in your .gitignore files), so you'll normally prefer it to git commit.

Where the backup is stored:
The backup data is placed in $HOME/.git. You can mount an external disk there if you want your backup to reside in a different physical disk for resiliency against disk crashes. You can also clone the repository (probably using a bare repository – without working tree – for smaller space usage) and rsync .git/git-repositories to remote machines for further protection.

proceeding with git

local repository in the homefolder on both: laptop and desktop remote repository on either the desktop or the vServer

commit often to your local repository

commit to remote server either using a cronjob or do it manually

if done using a cronjob: we nee a script with the following logic:

  • test if there is a modem connection active (online using UMTS)
  • yes? -> ask the user if he wants to proceed (rather not)
  • no? then proceed
  • test if the remote server is locally available
  • yes? -> do the commit / merge (because it's supposed to be fast then)
  • no? -> check if the server is available online
  • yes? -> do the commit / merge (can be slow)
  • no? -> write to a logfile and quit

etckeeper - backup /etc with git

http://kitenet.net/~joey/code/etckeeper/

etckeeper is a tool like gibak but with the speical purpose of keeping a git managed backup of the Linux configuration directory /etc.

Install it on Ubuntu 9.10:

sudo apt-get install etckeeper

The main configuration file is /etc/etckeeper/etckeeper.conf.

Read the fine documentation:

zless /usr/share/doc/etckeeper/README.gz

By default, etckeeper is set up to use the bzr VCS. We want to use git, so we do:

sudo etckeeper uninit
sudo gedit /etc/etckeeper/etckeeper.conf

and set VCS=git

Init the repository:

sudo etckeeper init
sudo etckeeper commit "initial commit."

now the setup to backup the configuration to the server:

sudo -i
ssh-keygen
ssh-copy-id git@reliable-ict.de
exit
ssh git@reliable-ict.de 'mkdir wolfdale_etc; cd wolfdale_etc; chmod 700 .; git --bare init'
sudo git remote add origin ssh://git@reliable-ict.de/~/wolfdale_etc
sudo git push origin master

and an entry to crontab:

sudo crontab -e

and set up a cron job like this:

19 * * * * cd /etc; git push origin master

In the standard configuration etckeeper automatically commits changes on a daily basis. For a more precise tracking of changes, it is recommended to commit your changes manually, together with a commit message, using:

sudo etckeeper commit "..Reason for configuration change.."

Comments