Linux Backup Final Project

*** This FP is performed on your virtual machine ***

*** All steps performed using the Command Line Interface ***

The goal of this final project is to familiarize you with the concept of backing up systems, something that is considered mission critical to ANY business.  Ubuntu Linux can be used to back up critical files and directories, and also can be used as a backup server for other machines using SFTP to maintain backups.  Types of backups and key concepts will be discussed.  This final project will also reinforce Command Line Interface (CLI) interaction learned throughout the semester in the Linux Labs.  

    1. You need a working Ubuntu Virtual Machine for this project.  It is written for Ubuntu 18.04LTS but has been tested and verified to work on Ubuntu 20.04LTS as well.
    2. You will take and paste screenshots into your 3 final project submission documents. These should be named “LastnameFirstnameFPBackups#.pdf” where the pound sign (#) will be replaces with the submission number.  Sections 1-4 will be #1, Sections 5-8 will be #2 and Sections 9-10 will be #3.  As an example I would submit 3 files titled (CantoreMatFPBackups1.pdf,  CantoreMatFPBackups2.pdf and CantoreMatFPBackups3.pdf)
    3. You will need to expand your Linux Terminal as large as you can vertically in an effort to capture the output as best as possible.
    4. You will have 1 or more screenshots for each individual component of the table of contents.
    5. You will need to label your screenshots with a heading that includes the Content number and letter.
    6. You need to follow the directions very carefully and any errors you encounter are input errors (spelling, case sensitivity, etc.) or improperly following the provided directions or a lack of required understanding of previous LL material.
    7. Should you not complete all components, you can obtain partial credit as long as you also complete and submit #16 (i.e. documents 1 & 2).

Should something go wrong, please retrace your steps and apply the principles introduced and used to date.  Note, you may need to re-perform your earlier LL readings and if you have done the necessary preparation (readings and rereadings), this FP will be straightforward. Lastly, please observe the coding practices rather than just blindly following the instructions as this tutorial will show you proper comment and output documentation.

 

Submission Document 1

    1. Preparing your system

In order for our system to be able to perform backups, we need some software.   First, we need to make sure our system is updated to the latest version of Ubuntu.  Issue the following commands to do this:

$ sudo apt-get update

$ sudo apt-get upgrade

Follow any prompts for the above commands, which you should have already encountered in your earlier system administration lab.

Screenshot #1 – pon completion of this take Screenshot #1 for Document 1.

    1. Installing and verifying Restic backup software

Next, we install the backup software we’re going to use.  As with many things Linux there is a wonderful open source tool called Restic that we will be using.  The install for Restic is very straightforward:

$ sudo apt-get install restic

Follow any prompts here as well (though there likely will be none).  

To verify Restic is installed and working, issue the following command:

$ sudo restic version

Mine looks like this:

restic 0.9.6 compiled with go1.12.12 on linux/amd64

Depending on your version of Linux this may be slightly different.

Upon completion of this, take Screenshot #2 for Document 1 showing your install and Restic version.

    1. Preparing Restic for use

In order for backups to really be meaningful they should be kept on a completely separate device from where they are taken.  In the business world there are often services that allow the secure transfer of backups to a completely separate location to be stored and recovered if needed.  Aside from a RAID storage solutions, backups can provide an extra layer of protection against data loss and are an integral part of a company’s disaster recovery plan.

Since we’re not large businesses we are going to use a USB device (thumb drive or external HDD) plugged into our machine that can be viewed by our Virtual Machine.  

Go ahead and plug in your USB device and give it a moment to be recognized.  Then, issue the command:

$ sudo lsblk

Depending on the number and type of devices present in your machine, your device  should be listed as one of the sdX devices.  Mine looks like this:

  sdb      8:16   0 931.5G  0 disk 

└─sdb1   8:17   0 931.5G  0 part

Pay careful attention to the letter and number after sd.  We will want to create a directory on our system and then mount this device there.  Then when Restic takes backups, it can store them directly into that directory, which is on the device.  The device can then be unmounted, safely removed, and kept safe somewhere in case we need it.

Let’s go ahead and prepare for this by creating a directory on our system to become the mount point for our device.   Please use your own name in place of where it says Lastname/Firstname below.

$ sudo mkdir /backupLastnameFirstname

Again my command looks like:

$ sudo mkdir /backupCantoreMat

Now we can mount our USB device to that directory for use using the sd information from above:

$ sudo mount /dev/sdXY /backupLastnameFirstname

My command looks like:

$ sudo mount /dev/sdb1 /backupCantoreMat

Verify it is mounted properly by using:

$ sudo df -h

Mine looks like this:

/dev/sdb1       932G  569G  364G  61% /backupCantoreMat

Upon completion of this, take Screenshot #3 for Document 1 showing your work creating the backup area and mounting and verifying the USB drive is connected to it.

    1. Create the Restic repository

We have one last task to complete before we can begin using Restic for backups.  We must initialize a repository and set a password for access.  I CANNOT EMPHASIZE THIS NEXT STATEMENT ENOUGH: When setting a password, make SURE you type it properly, and know exactly what it is.  You cannot recover it, and you cannot reset it.  If you forget or lose your password, your Restic repository is inaccessible along with all backups.  Pay special attention to upper and lower case letters, numbers and special symbols and make sure you know EXACTLY what you are typing.  You will NOT see it as you type it, but you will get a chance to verify what you want it to be.

Go ahead and create the repository and set the password now by typing:

$ sudo restic -r /backupLastnameFirstname init

Mine is:

$ sudo restic -r /backupCantoreMat init

And this results in the following:

$ sudo restic -r /backupCantoreMat init

enter password for new repository: 

enter password again: 

created restic repository 7fd38842bc at /backupCantoreMat

Please note that knowledge of your password is required to access the repository. Losing your password means that your data is irrecoverably lost.

$

Note the repository identifier listed above in my output.  This will be unique to each user AND repository, so if you were to create a new repository area and initialize it, it would have a separate password and a new unique identifier.

Upon completion of this, take Screenshot #4 for Document 1 showing your work initializing the restic repository area.

Submission Document 2

    1. Performing your first backup

There are two main types of backups we’re interested in.  The first type is called a full backup – every single directory and file is included.  The second type is called an incremental backup – only those directories and files that have been changed are included.  This would include any new directories, any new files, or any files whose contents have been modified since the last full or incremental backup has been performed.  We are going to perform both types to learn how they work.  It is industry practice to perform incremental backups very regularly and full backups less often, but periodically.  What those frequencies are can differ greatly from company to company, but the concepts are the same even if the frequencies are different.

Before we perform our first backup, we want to make sure there are some files we can work with.  To do this, we’re going to create some files in our home directory so we can practice.

Navigate to your home directory (if you are not already there – by now you should be well able to do this on your own).

Create a new directory in your home directory called backupTestLastnameFirstname.  Mine would be called backupTestCantoreMat.  Again you should be able to do this step easily.

Navigate to your new directory.

Using whatever method you prefer, create three files whose contents are each a different random sentence you write:

backupTestFileLastnameFirstname1.txt

backupTestFileLastnameFirstname2.txt

backupTestFileLastnameFirstname3.txt

My files would be called:

backupTestFileCantoreMat1.txt

backupTestFileCantoreMat2.txt

backupTestFileCantoreMat3.txt

Now we have some additional content that is standardized and we can work with.   Do a directory listing showing your 3 files in the directory you created.

Upon completion of this, take Screenshot #5 for Document 2 showing your listing above.

Let’s go ahead and perform our first full backup now.

    1. Performing our first full backup

This is incredibly simple with Restic.  So that we don’t have to spend lots of extra time waiting for backups to complete we’re going to work with the directory we just created.  We’ll talk about how we would expand this to include more on our system after we use the command to perform a full backp.  First, let’s back up our new directory:

$ sudo restic -r /backupLastnameFirstname backup /home/username/backupTestLastnameFirstname/

Note all the information you’ll need to change to make this work.  Mine looks like this:

$ sudo restic -r /backupCantoreMat backup /home/cantore/backupTestLastnameFirstname/

You will need to enter your password.  Your output should look similar to this:

enter password for repository: 

repository 7fd38842 opened successfully, password is correct

created new cache in /root/.cache/restic

Files:           3 new,     0 changed,     0 unmodified

Dirs:            2 new,     0 changed,     0 unmodified

Added to the repo: 1.179 KiB

processed 3 files, 99 B in 0:00

snapshot ac1769f5 saved

Let’s look at what we just did within restic by listing the snapshots available:

$ sudo restic -r /backupLastnameFirstname snapshots

My command looks like:

$ sudo restic -r /backupCantoreMat snapshots

My output looks like:

enter password for repository: 

repository 7fd38842 opened successfully, password is correct

ID        Time                 Host         Tags        Paths

——————————————————————————————

ac1769f5  2022-01-09 15:02:12  peachfields              /home/cantore/backupTestCantoreMat

——————————————————————————————

1 snapshots

You’ll note the ID in the snapshot listing corresponds with the ID in the backup we created above.

Upon completion of this, take Screenshot #6 for Document 2 showing your output above.

    1. Preparing for incremental backups

Before we do an incremental backup, let’s change some of the files around so we can see some differences in the third part of our project.

Navigate to your backupTestLastnameFirstname directory.  Complete the following actions:

    1. Delete backupTestFileLastnameFirstname3.txt
    2. Modify backupTestFileLastnameFirstname2.txt by adding another sentence to the contents.
    3. Create a new directory called newLastNameFirstname.
    4. Inside the newLastnameFirstname directory create backupTestFileLastnameFirstname4.txt and add another random sentence that you write.

Upon completion of this, take Screenshot #7 for Document 2 showing the listings of your new directory and file changes.  Make sure it shows the contents of backupTestLastnameFirstname as well as newLastnameFirstname.

Perform a cat backupTestFileLastnameFirstname2.txt to show the text you’ve added.

Add to Screenshot #7 by including the cat command output from above.

    1. Performing an incremental backup

Restic does the hard work for us.  All we have to do is issue the restic backup command again and it will sort the rest:

$ sudo restic -r /backupLastnameFirstname backup /home/username/backupTestLastnameFirstname

Again for reference mine looks like:

$ sudo restic -r /backupCantoreMat backup /home/cantore/backupTestCantoreMat

Your output should be similar to mine:

enter password for repository: 

repository 7fd38842 opened successfully, password is correct

Files:           1 new,     1 changed,     1 unmodified

Dirs:            0 new,     2 changed,     0 unmodified

Added to the repo: 1.168 KiB

processed 3 files, 118 B in 0:00

snapshot 44be4dc8 saved

Let’s verify our newest snapshot:

$ sudo restic -r /backupLastnameFirstname snapshots

Substitute accordingly above, by now hopefully you understand that.

My output looks like this:

enter password for repository: 

repository 7fd38842 opened successfully, password is correct

ID        Time                 Host         Tags        Paths

——————————————————————————————

ac1769f5  2022-01-09 15:02:12  peachfields              /home/cantore/backupTestCantoreMat

44be4dc8  2022-01-09 15:19:38  peachfields              /home/cantore/backupTestCantoreMat

——————————————————————————————

2 snapshots

I now have 2 snapshots.  The first (taken at 1:02pm) is my full backup and the second (taken at 1:19pm) is my incremental backup.

Upon completion of this, take Screenshot #8 for Document 2 showing the listing of the two snapshots for your backups.

Submission Document 3

    1. Restoring an entire backup

Let’s restore the full backup of our main directory.  This will undo the changes we made in section 7, as if we had never done them.  You’ll need the snapshot ID of your full backup in section 6 (mine is ac1769f5).  To restore the backup issue the command:

$ sudo restic -r /backupLastnameFirstname restore snapshotID –target /home/username/backupTestLastnameFirstname

My full command is:

  $ sudo restic -r /backupCantoreMat restore ac1769f5 –target /

The result of this command looks something like:

enter password for repository: 

repository 7fd38842 opened successfully, password is correct

restoring <Snapshot ac1769f5 of [/home/cantore/backupTestCantoreMat] at 2022-01-09 15:02:12.80991054 -0500 EST by root@peachfields> to /

Let’s go take a look at the directory.  Navigate to /home/username/backupTestLastnameFirstname now and perform a listing.  

Upon completion of this, take Screenshot #9 for Document 2 showing the listing of files 1,2 and 3.

You’ll note that it did not delete the directory that was created after the backup was taken.  Backups will not destroy data, only replace or overwrite it.  Explore the contents of files 1, 3 and 3 now to verify they were how they looked BEFORE we changed them in Section 7.  Specifically, cat the contents of file 2 to verify that the second sentence is gone.

Upon completion of this, add to Screenshot #9 for Document 2 showing the cat command and contents of file #2.

    1. Restoring specific files with Restic

Sometimes we aren’t sure if we need something from a backup or not, and we also may not be sure which backup contains what we need.  It is possible in those cases to restore the contents to a different directory, look through the contents, and then copy only the files we want.  Let’s say we want to restore the latest version of backupTestFileLastnameFirstname2.txt again.  We can do this by allowing Restic to restore inside our file structure and then navigating the backup that way.  We also can use Restic to restore the latest backup version without needing to get the snapshot ID (which can be challenging if there are a lot of them).  In this case, we’re going to restore the latest version of our backup (the incremental) into our existing directory in a way we can then go navigate and get what we want, but that won’t overwrite what’s already there.

To do this, we use the following command:

$ sudo restic -r /backupLastnameFirstname restore latest –path /home/username/backupTestFirstnameLastname –target /home/username/backupTestFirstnameLastname

There’s a lot there, so make sure to type everything properly.  This is what my command looks like:

$ sudo restic -r /backupCantoreMat/ restore latest –path /home/cantore/backupTestCantoreMat/ –target /home/cantore/backupTestCantoreMat/

Navigate to /home/username/backupTestCantoreMat and perform a directory listing.  Verify that there is now a /home directory present.  Inside that should be a directory with your username, and inside THAT should be your backupTestLastnameFirstname directory.  If you look inside there, and you cat the contents of backupTestFileLastnameFirstname2.txt you will see the second sentence has been restored again in this version.

Upon completion of this, take Screenshot #10 for Document 2 showing the cat command and contents of the restored new version of file #2.

Conclusion

It is possible to automate restic backups using scripting to take incremental backups on a regular basis.  It is also possible to take remote backups with Restic using the openssh-server with sftp package.  This would allow another Linux machine with restic installed to take backups and connect to this machine, store backups in a repository on it, restore from that repository, etc.

You now should have a good understanding of full and incremental backups, how they work, how the restore process works, and the concepts involved with all that.