- Using AWS Backup to create backups
- AWS Backup and its interaction with other AWS-based services
- Basic operations with Amazon S3 console
- Other methods of backing up your Amazon S3 bucket
- Enterprise-grade AWS S3 backup solutions with minimal restore costs.
- AWS S3 Backup with Bacula Enterprise
- Adding a new S3 storage in Bacula Enterprise
- Configuring your AWS S3 storage backup using Bacula Enterprise
- Finishing the S3 storage setup process
- Saving your new S3 backup settings
- Testing the AWS backup settings
Using AWS Backup to create backups
Backing up your information - data - is perhaps the most important part of protecting it from any harm as well as ensuring compliance. Even the most durable servers and storages are susceptible to bugs, human errors and other possible reasons for a disaster. But creating and managing all of your backup workflows might be a daunting task overall. Therefore, there are a variety of methods that you can use to simplify the entire process of creating a backup while using AWS S3.
A popular choice is Amazon’s own backup solution – AWS Backup. AWS backup can provide a way to manage your backups both in the AWS Cloud and on premise, as well as supporting a variety of other Amazon applications.
The backup process itself is fairly easy. A user would have to create a backup policy – their backup plan - specifying a number of parameters such as backup frequency, the amount of time that these backups should be kept, etc. As soon as the policy is set up – AWS Backup should start backing up your data automatically. After that you’ll be able to use AWS Backup’s console to view your backed up resources, have the option to restore a specific backup or just monitor your backup and restore activity.
AWS Backup and its interaction with other AWS-based services
There are plenty of different AWS services that can offer various useful features and work in tandem with AWS Backup service. For example, these services include, but not exclusive to:
- Amazon EBS (Elastic block store);
- Amazon RDS (Relational database service);
- Amazon DynamoDB backups;
- AWS storage gateway snapshots, etc.
Of course, you have to enable the specific service that you want to use in your backup process before using it in the first place. Trying to initiate or create the backup using specific resources from a service that you have yet to enable means you will likely receive an error message instead and won’t be able to perform the creation process.
To find the list of services that you can switch on or off, you have to follow a number of steps:
- Open the AWS backup console.
- Go into the “Settings” menu.
- Move to the “Service opt-in” page and click “Configure resources”.
This should get you to the page with a number of service names and toggles, and you can easily toggle each of the specific services on or off. Clicking “Confirm” after making the changes would save your operations.
AWS backup is capable of implementing a lot of existing AWS service capabilities in the process of creating a backup. One good example of that is the EBS snapshot capability that is used to create backups according to the backup plan you’ve created. Creating EBS snapshots, on the other hand, can be done using the EC2 API (Elastic compute cloud). This way you’ll be able to manage your backups from a centralized AWS backup console, monitor them, schedule different operations, and so on.
AWS backup can perform backup jobs on entire EC2 instances, allowing you to have less need to interact with the storage layer itself. The way it works is also quite simple – AWS backup takes a snapshot of the root EBS storage volume, as well as associated volumes and launch configurations. All of the data is stored in the specific image format called volume-backed AMI (Amazon machine image).
EC2 AMI backup files can also be encrypted in the backup process the same way AWS backup does this to EBS snapshots. You can either use the default KMS key if you don’t have one, or you can use your own one to apply it to the backup.
The restoration process for EC2 resources can be done in several different ways: AWS backup console, command line or just the API. Compared to the other two, the backup console has a lot of functionality limits for the restoration process and cannot restore several parameters like ipv6 addresses, some specific IDs, and so on. The other two methods, on the other hand, are capable of doing a full restore one way or another.
Basic operations with Amazon S3 console
Amazon’s own S3 toolkit allows for some basic operations when it comes to retrieving or storing a specific file from the bucket. It’s possible to point out four different operations that can be done using only Amazon S3 backup software – but we have to get to the Amazon S3 console, first.
It can be done by going to the AWS services page using this link, entering your login information should allow you to access the first screen of the AWS services toolkit. After that it’s possible to find the S3 console by either going to the “Services” menu and finding S3 there, or by typing in “S3” in the search bar located at the higher part of the page.
- Creating S3 Bucket.
- A bucket is a type of container that Amazon S3 uses to store your files. A bucket can be created in the AWS S3 interface by clicking the “Create Bucket” button in the title screen.
- It is worth noting that the web page would look differently depending on whether you’ve already created a bucket before within this account or not. If there are other buckets already in place, you’ll be met with a screen that allows you to manage said buckets, including renaming or outright deleting them.
- On the other hand, if it is your first time creating a bucket within this AWS S3 backup account, you’ll see a corresponding screen describing a way to create a bucket in the first place. If that happens, you can use either the “Create bucket” button or the “Get started” button, both of these should lead to the same place – the bucket creation screen.
- The first prompt of the screen should be about creating a name for your new bucket, and the field would also notify you if the name of the bucket does not meet some of Amazon’s own bucket name-related regulations. You’ll also have to choose an appropriate region for your future bucket. Once you’ve done that, click “Next” to continue.
- The second bucket creation screen allows you to enable one of the properties for your Amazon S3 backup bucket, such as Tags, Versioning, Encryption, Server-access logging and Object level logging. For the purpose of keeping this explanation simple, we are not enabling any of these properties. Click “Next” to proceed.
- The next screen allows for permission customization, including both system permissions and user permissions. You can also change your own permission levels and add specific people to have access to this bucket. Our example keeps all of the default permission levels – with the creator having access to anything inside of this bucket. Click “Next” when you’re done.
- The last part of the process is the confirmation screen, allowing you to review all of the settings you’ve previously configured. This includes permissions, properties and names. Clicking the “Create bucket” button after finishing the review process does exactly what it says – creates a bucket with your specific settings.
- Uploading a file.
- Uploading a file to your new AWS S3 bucket is also relatively easy, if you’re starting from the Amazon S3 console. Clicking on the name of your new AWS bucket would allow you to access said bucket and its contents.
- As soon as you’re in your bucket’s landing page, you can start the upload process by clicking the “Upload” button in the left part of the page.
- There are two ways of uploading a file in the following window – by either drag-and-dropping a file on the page, or by clicking the “Add files” button and selecting the file in question afterwards. After you’ve chosen the file to upload, you’re free to click “Next” to proceed.
- As with the backup S3 bucket creation process, you can change the file’s permissions before uploading it, including both your own permissions, accounts who have access to this file, and public permissions. Use the “Next” button to continue.
- The next page is more about specific properties for your file, such as storage class (Standard, Standard-IA and Reduced redundancy), encryption (None, S3 master key and KMS master key), and metadata. After you’re done choosing one of the options, you can continue through the “Next” button.
- The last screen of this sequence is all about confirming all of your changes before uploading. Your properties, permissions and the amount of files chosen are all displaced here. Clicking the “Upload” button after double-checking the details should begin the upload process.
- Retrieving a file.
- Downloading a file from your AWS S3 bucket can be done in two easy steps. First, you have to be on your bucket’s landing page, in which you’ll see all of the files that are being stored within said bucket. The first step you’ll have to do is click the checkmark field to the left of the file that you want to download
- Selecting at least one file on the list brings up a description pop-up window, which has two buttons: “Download” and “Copy path”. Use the “Download” button to receive the file in question.
- Deleting a file or a bucket.
- Deleting unneeded files or even buckets is not just easy, it’s also highly recommended by Amazon themselves to prevent excessive cluttering of your files. First of all, the file deletion process.
- As soon as you arrive at a bucket’s landing page, the first thing you have to do is click the checkbox field to the left of the file that you want to delete.
- After choosing the file or files for deletion, you can press the “More” button near the “Upload” and “Create Folder” buttons and choose the “Delete” option from the drop-down list.
- You’ll receive a confirmation screen that shows you which files would be deleted, and you have to click “Delete” once more to initiate the deletion process.
- The process of deleting an entire bucket is slightly different. First, you’ll need to back out of your bucket landing page and get back to the main Amazon S3 backup console that lists all of your buckets.
- Clicking on the blank space to the right of the bucket that you want to delete would select the bucket, and clicking the “Delete bucket” button would initiate the deletion process.
It is worth noting that all of these basic operations could be done with just Amazon’s own system and without the addition of any AWS backup solutions.
Other methods of backing up your Amazon S3 bucket
Using AWS Backup isn’t the only option when it comes to S3 backups, as well. There’s a variety of different options that can be both performed by an application within Amazon’s ecosystem as well as third-party solutions.
For example, here’s several more ways of creating an S3 backup without using AWS Backup application:
- Create backups using Amazon Glacier;
- Use AWS SDK to copy one S3 bucket to another;
- Copy information to the production server that is itself backed up;
- Use versioning as a backup service.
It is worth mentioning that most of these methods aren’t exactly fast or convenient. Amazon Glacier, for example, would be a good backup solution if it wasn’t a lot slower than your regular backup process, since Glacier is more about data archiving and less about ongoing data backups. On the other hand, using versioning as a backup solution might raise your storage costs a lot due to the amounts of data that need to be stored.
Speaking of third-party solutions, while there are a lot of different ones in the market, we’ll examine one of the most promising ones – the solution provided by Bacula Enterprise.
Enterprise-grade AWS S3 backup solutions with minimal restore costs.
Bacula delivers natively integrated AWS S3 backup solutions as part of its extensive enterprise cloud-based backup and recovery options. It delivers native integration with public and private clouds via the Amazon S3 interface, with transparent support for S3-IA. AWS S3 backup is available for Linux, Windows and other platforms. However, there is something else your organization should know about Amazon S3 backup with Bacula Enterprise: the ability to have unique control over your cloud backup - and at the same time bring significant cloud cost reduction for AWS backup solutions.
AWS S3 Backup with Bacula Enterprise
To begin the AWS backup process with Bacula, you need to enter configuration mode first. After that you will be able to see several new options available. You need the one titled “Add a New Storage Resource”.
Adding a new S3 storage in Bacula Enterprise
In this specific example we’re adding a new Amazon S3 storage to an existing storage daemon. We’ll also choose the “Cloud Virtual Disk Changer” under the “Device Type” – this device type allows for several simultaneous backups to the same cloud storage.
Since our storage daemon already exists – all of the information at step 2 (Configuring a new storage resource) can be taken from the previously created devices.
Configuring your AWS S3 storage backup using Bacula Enterprise
The next step of AWS backup process is the cloud storage information configuration. In this example we’ll be storing our backup volumes in the cloud cache, which is usually used as a small temporary area in-between loading a backup to a cloud, but it can still hold a week or more’s worth of the data to allow local backups for that period, and cloud backups if the time period is longer than a week. You can always contact Bacula support experts to know more about cloud cache’s storage size, cache retention policy and cloud upload behaviour.
Next we’ll choose a unique media type for our new storage device, to make it easier for Bacula to see this specific storage device’s files.
One more part of this step is choosing your AWS S3 cloud driver from a list of supported cloud drivers.
Next, we set up a list of arbitrary information like cloud hostname, account info, region and so on. You’ll also have a choice between choosing an existing bucket by connecting to your existing account or to enter a name in the corresponding line to confirm the creation of a new bucket.
Finishing the S3 storage setup process
There are two possible options left: cloud link status and “upload to the cloud during the job”. Cloud link status button allows you to immediately check your current system’s connection to a cloud of your choosing. “Upload to the cloud during the job” is an option that is chosen as a part of default settings to upload your backed up data to the cloud as soon as it’s ready (even in the process of a backup job), but you can also disable this option if you wish to upload after a job is finished or with some other schedule in mind.
The next step of this wizard is simply typing in your preferred storage name and optional description.
Saving your new S3 backup settings
After this step you can push the “Save” button to allow all of the previous changes to be committed to production. Keep in mind that in order to properly commit everything to production you’ll have to reload your storage daemon, meaning that any job that is running would fail in the process.
A logical step after this would be to set up new backup pools for this specific cloud storage and to properly configure jobs to write data to the new pools. You can address Bacula’s documentation, contact our support or watch our YouTube channel to get help in regards to these steps.
Testing the AWS backup settings
To test that everything we’ve just done works properly, we’ll run a small full Amazon S3 backup job manually directly to the new storage device. Usually this process is automated using a job schedule and/or other configurations.
After the job has run we will be able to see the entire process’s logs, and this specific section (on the screenshot below) shows us that everything uploaded correctly.
Bacula Enterprise is a sensible choice for reliably managing your AWS S3 and other cloud backups, including creating and configuring newer backup storages and setting up backup jobs to be performed automatically. Next we will list some of the AWS S3-specific features of Bacula Enterprise.