Scenario 7: Backup using the Cloud plugin
In this scenario we will backup a Linux and/or Windows client from BWeb Management Suite using the “S3 Cloud” Storage daemon plugin. We will use Amazon’s Simple Storage Service (Amazon S3) object storage architecture to store the cloud volumes. Alternatively, you can also see the video: How To: Add Cloud Storage in Bacula Enterprise
For convenience and ease of testing the “S3 Cloud” plugin, the Trial VM has a “FakeS3” service already installed, and configured with a test bucket called “BaculaTrialS3Bucket”. An “S3 Cloud” resource, and an “S3 Autochanger” with two S3 devices has already been pre-configured on the Trial VM. Additionally, there are several jobs already configured to write to this local “FakeS3” cloud storage service.
For a Windows client, it will be necessary to have a Windows system (on physical hardware or a virtual machine) with the Bacula Enterprise Edition File Daemon installed as described in scenario 5 above.
For a Linux client, the one already installed for the Trial will be used.
To follow the examples in this scenario, you will need an Amazon Web Services (AWS) account to store your backups in the cloud.
If you do not have, or do not want to use your current cloud provider, we recommend that you create an Amazon Web Services (AWS) account and follow the AWS Best Practices to have a restricted user with credentials to be used with the Bacula Enterprise Trial only for the backup purposes.
Please go to https://aws.amazon.com/free/ to find more information and to create your AWS account.
Please find the AWS Best Practices guide here: http://docs.aws.amazon.com/IAM/latest/UserGuide/best-practices.html .
The Cloud plugin provides the AWS Command Line Interface (CLI) that will be used to perform the procedures explained below. The commands are available in a Linux shell. Please find more details about the AWS Command Line Interface (CLI) here: https://aws.amazon.com/documentation/cli/
The procedure below demonstrates how to have an IAM restricted user created with its own credentials with an attached policy to access a specific bucket for use with the Bacula Enterprise Trial. Please note you will need to create the bucket prior to following this procedure by using the AWS root credentials or any other IAM user with the necessary permissions.
Part 1 - Configure the AWS account access
All the commands in this part should be typed from the Linux shell on the Trial VM as the root user, password "bacula".
- Configure the AWS 'root' user configurations using the Linux root user's account:
[root@bacula-enterprise-trial ~]# aws configure
AWS Access Key ID [None]: ****************XBJQ
AWS Secret Access Key [None]: ************************************88wA
Default region name [None]: us-east-1
Default output format [None]:
- Notice the /root/.aws directory and config and credential files have been created:
[root@bacula-enterprise-trial ~]# ls -la /root/.aws
drwxr-xr-x 2 root root 39 Jul 12 10:02 .
dr-xr-x---. 6 root root 234 Jul 12 10:00 ..
-rw------- 1 root root 29 Jul 12 10:02 config
-rw------- 1 root root 116 Jul 12 10:02 credentials
- Create a new "limited" AWS user in our account for use with the Bacula Enterprise Trial:
[root@bacula-enterprise-trial ~]# aws iam create-user --region=us-east-1 --endpoint-url https://iam.amazonaws.com --user-name bacula_trial_user
- Create the new user's "Access Key ID" and "Secret Access Key":
[root@bacula-enterprise-trial ~]# aws iam create-access-key --region=us-east-1 --endpoint-url https://iam.amazonaws.com --user-name bacula_trial_user
- Create a JSON Policy file to attach to the limited user account, still using the Linux root user's account:
Note: Please see https://docs.aws.amazon.com/cli/latest/reference/iam/create-policy.html for information and examples on how to create this policy file.
Here is the same JSON policy file created locally:
[root@bacula-enterprise-trial ~]# cat /root/bacula_trial_user-policy
"Action": [ "s3:ListAllMyBuckets", "s3:CreateBucket" ],
- Create a user policy using the JSON file created in the previous step:
[root@bacula-enterprise-trial ~]# aws iam create-policy --region=us-east-1 --endpoint-url https://iam.amazonaws.com --policy-name bacula_trial_user-policy --policy-document file://bacula_trial_user-policy
- Attach the new user policy to the limited user adapting the "Arn" from the previous command:
[root@bacula-enterprise-trial ~]# aws iam attach-user-policy --region=us-east-1 --endpoint-url https://iam.amazonaws.com --user-name bacula_trial_user --policy-arn arn:aws:iam::********3064:policy/bacula_trial_user-policy
- Configure the limited user's profile in the ~root/.aws/config and credentials files:
[root@bacula-enterprise-trial ~]# aws configure --profile bacula_trial_user-profile
AWS Access Key ID [None]: ****************W6QU <== from "aws iam create-access-key" command above
AWS Secret Access Key [None]: ************************************R68r <== same --^^
Default region name [None]: us-east-1
Default output format [None]:
- Check the config and credentials files:
[root@bacula-enterprise-trial ~]# cat .aws/config
region = us-east-1
region = us-east-1[root@bacula-enterprise-trial ~]# cat .aws/credentials
aws_access_key_id = ****************XBJQ
aws_secret_access_key = ************************************88wA
aws_access_key_id = ****************W6QU
aws_secret_access_key = ************************************R68r
- Create the new "bacula-trial-bucket" bucket using the limited user account, or the bucket name specified in your json policy above:
[root@bacula-enterprise-trial ~]# aws s3 mb --region=us-east-1 s3://bacula-trial-bucket --profile bacula_trial_user-profile
- List the new bucket - it should be empty:
[root@bacula-enterprise-trial ~]# aws s3 ls --recursive --region us-east-1 s3:// --profile bacula_trial_user-profile
2019-07-12 10:43:56 bacula-trial-bucket <=== Just the bucket, nothing in it
Part 2 - Configure Bacula resources
Now we are ready to create the new Bacula resources needed for backups using a S3 compatible object storage solution.
To be able to use the Cloud plugin, we will create a new Cloud resource and a new Storage using BWeb Management Suite.
- In BWeb Management Suite, click Configuration --> Configure Bacula.
- Click on the Storage Daemons icon in the Bacula Enterprise Components box, then choose the Clouds tab in the top menu to create a Cloud resource.
- Click the "+" icon to add a new Cloud resource and fill in all the information as shown in the image below. Please note that the Region field at the bottom may also need to be modified to match your environment.
- Click on the Add button.
- Once the Cloud resource has been created, click the Devices menu item to create a new Device resource. Click the "+" to add a new Device resource and fill in all the information as shown in the image below.
Note: The Archive Device field is important in this page. The directory specified must exist, and there must be enough free space to store Cloud volumes in the "local cache". The Bacula Enterprise Trial has a 200GB disk mounted under /mnt/bacula where all backup jobs (file, dedup, S3 Cloud) are written to. It is safe to create a new directory (eg: /mnt/bacula/aws/s3) under here where Bacula can use as the local cache for testing your backups to a real S3 cloud provider. Please make sure the permissions to the folder created permit the user "bacula" to write to it.
- Click on the Add button.
- Next, we need to create a Storage resource in the Director that points to this new device called "AWS_Dev0". Click on the Storage/Autochangers tab in the main menu. Select an already made Storage (for example S3Autochanger), then copy it by clicking on the Copy button on the top right of the resource window. Rename it to "AWS_Dev0" then press 'OK'. Fill in all the information as in the image below.
Note: The Address will most likely be different here, or it may even point to a FQDN depending on your network environment. Whatever is pre-filled in for the Address should be correct, and should be left as-is.
- Click on the Save button. Now all the changes we have made are in the Workset, but they have not been committed to the production configuration, as shown in the image below.
- Click "Workset" in the top menu to see the current uncommitted changes.
- Click "Commit" to commit the change to the production configuration. Once the Workset is committed, the Director must be reloaded, and the Storage daemon must be restarted in order to use this new Cloud configuration.
You can check the status of the new Storage created. To do this, we first need to leave configuration mode by clicking on the Bacula Enterprise title on the top menu.
- Next, select Storage --> then Manage Device. A list of Storage devices appears. Select the new “AWS_Dev0” device from the list, then click on the Status button. You should have an output like:
Device Cloud (S3 Plugin): "AWS_Dev0" (/mnt/bacula/aws/s3) is not open.
Available Cache Space=200.1 GB
- You can also check the connection status between the new Storage created and a Client previously created.
To do this, select Storage from the main menu, then Storage Overview. Choose the new AWS_Dev0 storage from the drop-down list. Click on the Network test button. A new window is displayed. Select one client in the Client list and the new “AWS_Dev0” Storage device in the Storage list. Then click on the Run Test button. You should have an output like:
Part 3 - Backup data to the cloud
If both the Status and Network tests run as expected, the next step is to run a backup job to our new Cloud storage device.
You can use Scenario 1: “Backing up files from /usr folder via Bweb Management Suite” to run backup tests for a Linux host, and you can use Scenario 5: “Backup a Windows client with Bweb Management Suite” to run backup tests for a Windows host.
When testing restores, you can use Scenario 2: “Restoring files from your previous backup using BWeb Management Suite” to run restore tests for the Windows and Linux hosts.
In all cases, please note you will need to select the “AWS_Dev0” storage in order for your backup/restore job to use the Cloud device created in this scenario.
Congratulations! We hope that you enjoyed using Bacula Trial Edition. You still have the opportunity to further test some of your Bacula Trial installations’ other features. Please refer to our main manual, in particular the chapters "A Brief Tutorial" and "Getting Started with Bacula".
If you have any comments or suggestions please fill out the feedback form and let us know. We'll get back to you in a timely fashion, and would like to thank you in advance.
For more information about plugins, support options, features, training and pricing, please contact the sales team via our contact form.
Bacula Enterprise is used by a large number of high-profile customers, where each one exploits the special qualities of Bacula in a variety of different ways. Bacula Systems looks forward to bringing the same benefits to your organization; please contact us to find out how we can best help you to move forwards, modernize your backup and recovery strategy, and significantly reduce costs.
With our best regards,
The Bacula Systems Support Team
Additional notes: The Bacula Enterprise Trial VM will stop working after 30 days. If you would like to continue testing after this time, you can simply click the “Reset Bacula Enterprise Trial environment” icon on the Landing Page, or re-import the Trial ova.