Home > Backup and Recovery Strategies Blog > Backup Virtual Machines with Bacula Enterprise – Technical Overview
1 Star2 Stars3 Stars4 Stars5 Stars
(39 votes, average: 5.00 out of 5)
Loading...

Backup Virtual Machines with Bacula Enterprise – Technical Overview

  • August 3, 2020, Rob Morrison

How to Backup Virtual Machines with Bacula Enterprise Edition?

backup virtual machine
Bacula Enterprise Edition provides tools to backup virtual machine images through their hypervisor backup APIs. Performing backups at this level rather than at the more ubiquitous file level provides many advantages, though in many cases a combined approach that uses both file and hypervisor backups will provide the best coverage in complex environments with virtual machines. This how-to will cover some of the advantages of adding hypervisor-level backup of virtual machines to your Bacula Enterprise installation, and demonstrate the basics of setting up these backups for a HyperV environment.

Some of the major advantages of performing backups at the hypervisor for virtual machines include:

  1. The hypervisor is a convenient central place to orchestrate all backup operations for your virtual environment.
  2. There is no need to install the Bacula client component (FileDaemon) on each virtual machine individually.
  3. New clients are automatically detected and added to the backup, reducing the risk of a machine being 'missed'.
  4. Single file restore is available as long as the hypervisor API allows us to access them, so there is no loss of functionality over file backups.

There are some potential pitfalls to look out for, such as backing up databases running on a virtual machine. In many cases, the virtualization software is able to quiesce the filesystem through the use of guest tools, and this often means that databases can be reliably backed up with a hypervisor initiated backup. However, it may be preferable to install a backup client and database-aware backup software on these machines for more granular backups of important databases and applications. Bacula Systems support can help determine which is the best approach for a given environment.

Bacula Enterprise Edition – Supported Virtualization Engines

Bacula Enterprise has plugins to backup virtual machines with these hypervisor vendors:

  • VMware
  • Hyper-V
  • KVM
  • Proxmox
  • Xen

Depending on the tools available from the hypervisor, the backup is either snapshot based or file based.

Where possible, such as with VMWare, image based backups allow for incremental backups and single file restores as well using tools like Changed Block Tracking (CBT) and the single file restore capabilities presented by VMWare's backup API.

Bacula Enterprise’s virtual machine backup plugins are all configured in a similar way. No matter which hypervisor you use, the process that follows in this how-to is always similar. We will configure Hyper-V backup as an example for backup of virtual machine with Bacula Enterprise Edition.

How-to backup a Hyper-V host with Bacula Enterprise Edition

On your Bacula Director you will have to add a Client resource for the Hyper-V host (in our case a Windows Server 2012R2 with Hyper-V activated), and a FileSet resource (if not previously created for a different backup client; in Bacula all resources can be reused) and finally a Job resource that defines all parameters of the backup job (when it is scheduled with which level for which fileset, etc.). Please find an example configuration below:

Excerpt from bacula-dir.conf

 

Job {
Name = "HyperV-job"
JobDefs = "DefaultJob"
Client = win2012-client
FileSet = "Hyper-V"
Accurate = "Yes"
}
FileSet {
Name = "Hyper-V"
Include {
Options {
signature = MD5
}
File = "C:/dummy.txt"
Plugin = "vss:/@HYPERV/"
}
}
Client {
Name = win2012-client
Address = 192.168.178.32
FDPort = 9102
Catalog = MyCatalog
Password = "windowspw"
File Retention = 60 days
Job Retention = 6 months
AutoPrune = yes
}

This is everything you need to get started. The plugin line in the fileset tells Bacula to back up all virtual machines found on this hypervisor. Please note that the FileSet contains a reference to a dummy text file that is necessary because the VSS snapshot of a Microsoft NTFS filesystem is triggered only if at least one file is specified. With that being said - if your Hyper-V data is spread across several drive letters you will need either a dummy text file on each of them (unless you also do additional file backups on that Windows Server anyway), or you will need to make use of the Alldrives Plugin that comes with the Bacula Enterprise client for Windows.

The backup Job above named "HyperV-job" was executed and finished successfully. Below is the joblog for this backup job:

Backup joblog

 

22-Jun 09:26 centos6-64-dir JobId 4: No prior Full backup Job record found.
22-Jun 09:26 centos6-64-dir JobId 4: No prior or suitable Full backup found in catalog. Doing FULL backup.
22-Jun 09:26 centos6-64-dir JobId 4: Start Backup JobId 4, Job=HyperV-job.2018-06-22_09.26.08_12
22-Jun 09:26 centos6-64-dir JobId 4: Using Device "FileChgr1-Dev2" to write.
22-Jun 09:26 centos6-64-sd JobId 4: Volume "Vol-0001" previously written, moving to end of data.
22-Jun 09:26 win-ud8s2399ti8-fd JobId 4: VSS plugin start backup Job.
22-Jun 09:26 win-ud8s2399ti8-fd JobId 4: Generate VSS snapshots. Driver="Win64 VSS"
22-Jun 09:26 win-ud8s2399ti8-fd JobId 4: Snapshot mount point: C:\
22-Jun 09:28 win-ud8s2399ti8-fd JobId 4: VSS Writer
(BackupComplete): "Task Scheduler Writer", State: 0x1 (VSS_WS_STABLE)
22-Jun 09:28 win-ud8s2399ti8-fd JobId 4: VSS Writer
(BackupComplete): "VSS Metadata Store Writer", State: 0x1 (VSS_WS_STABLE)
22-Jun 09:28 win-ud8s2399ti8-fd JobId 4: VSS Writer
(BackupComplete): "Performance Counters Writer", State: 0x1 (VSS_WS_STABLE)
22-Jun 09:28 win-ud8s2399ti8-fd JobId 4: VSS Writer
(BackupComplete): "System Writer", State: 0x1 (VSS_WS_STABLE)
22-Jun 09:28 win-ud8s2399ti8-fd JobId 4: VSS Writer
(BackupComplete): "Microsoft Hyper-V VSS Writer", State: 0x1 (VSS_WS_STABLE)
22-Jun 09:28 win-ud8s2399ti8-fd JobId 4: VSS Writer
(BackupComplete): "ASR Writer", State: 0x1 (VSS_WS_STABLE)
22-Jun 09:28 win-ud8s2399ti8-fd JobId 4: VSS Writer
(BackupComplete): "WMI Writer", State: 0x1 (VSS_WS_STABLE)
22-Jun 09:28 win-ud8s2399ti8-fd JobId 4: VSS Writer
(BackupComplete): "Registry Writer", State: 0x1 (VSS_WS_STABLE)
22-Jun 09:28 win-ud8s2399ti8-fd JobId 4: VSS Writer
(BackupComplete): "COM+ REGDB Writer", State: 0x1 (VSS_WS_STABLE)
22-Jun 09:28 win-ud8s2399ti8-fd JobId 4: VSS Writer
(BackupComplete): "Shadow Copy Optimization Writer", State: 0x1 (VSS_WS_STABLE)
22-Jun 09:28 centos6-64-sd JobId 4: Elapsed time=00:02:15, Transfer rate=22.30 M Bytes/second
22-Jun 09:28 centos6-64-sd JobId 4: Sending spooled attrs to the Director. Despooling 13,343 bytes ...
22-Jun 09:28 centos6-64-dir JobId 4: Bacula Enterprise centos6-64-dir 8.10.2
(25Apr18): Build OS: x86_64-redhat-linux-gnu-bacula-enterprise redhat JobId: 4 Job: HyperV-job.2018-06-22_09.26.08_12
Backup Level: Full (upgraded from Incremental) Client: "win2012-client" 8.10.2
(25Apr18) Microsoft Datacenter Edition (build 9200), 64-bit,Cross-compile,Win64 FileSet: "Hyper-V"
2018-06-22 09:22:15 Pool: "File" (From Job resource) Catalog: "MyCatalog" (From Client resource)
Storage: "File1" (From Job resource) Scheduled time: 22-Jun-2018 09:26:06
Start time: 22-Jun-2018 09:26:10 End time: 22-Jun-2018 09:28:25
Elapsed time: 2 mins 15 secs Priority: 10 FD Files Written: 36 SD Files Written: 36 FD
Bytes Written: 3,011,737,880 (3.011 GB) SD Bytes Written: 3,011,748,601 (3.011 GB) Rate: 22309.2 KB/s
Software Compression: None Comm Line Compression: 69.7% 3.3:1 Snapshot/VSS: yes
Encryption: no Accurate: yes Volume name(s): Vol-0001 Volume Session Id: 4 Volume Session Time: 1529652856
Last Volume Bytes: 3,021,011,807 (3.021 GB) Non-fatal FD errors: 0 SD Errors: 0
FD termination status: OK SD termination status: OK Termination: Backup OK
22-Jun 09:28 centos6-64-dir JobId 4: Begin pruning Jobs older than 6 months .
22-Jun 09:28 centos6-64-dir JobId 4: No Jobs found to prune.
22-Jun 09:28 centos6-64-dir JobId 4: Begin pruning Files.
22-Jun 09:28 centos6-64-dir JobId 4: No Files found to prune.
22-Jun 09:28 centos6-64-dir JobId 4: End auto prune.

As you can see the backup terminated with status "Backup OK" which, of course, is exactly what you want for all your backup jobs with Bacula Enterprise. In the joblog above the summary Bacula prints detailed information about what happens on the Client and Storage Daemon with timestamps. In particular you can see the VSS snapshot information with details about all the VSS writers in the above example joblog.

backup virtual machineThe restore of the contained virtual machines works like every other restore in Bacula. From Bweb or bconsole, navigate to the desired job, whether the most recent or a specific point in time. Then select the data that you want to have restored, either everything, a single file, or a specific VM. Finally, initiate the restore either to the original location if the VM has been removed, or perform a cross restore to a different host. Disaster recovery is simplified drastically with the ability to recover entire machines onto different hardware in a new location with the click of a button.

The Bacula configuration examples that we have shared above apply to Hyper-V virtual machine backup. Other hypervisors that we support work in a very similar manner, and each Bacula Enterprise plugin comes with extensive documentation about what resources need to be defined and about what options specific to each hypervisor are available in the FileSet. For instance we chose in the example above to back up all virtual machines on the Hyper-V host, but we could also have selected individual items for backup of virtual machines. If you split the backup job into smaller parts like that, you get a more fine-grained restore experience, your backups will be faster because the jobs are smaller, and you can run several jobs in parallel to speed up the overall process.

Backing up KVM with Bacula Enterprise

Bacula Enterprise helps to make the entire KVM backup/restore process as easy as possible. Individual clients have no need for additional software installation or scripting, and the users have complete control over the data that’s being backed up or recovered. There is also the auto detection capability, which allows users to locate VMs automatically after the initial plugin configuration.

Bacula Enterprise allows KVM backups at any point in time, no matter if the VM in question is active, on pause or shut off entirely. One of the key points of Bacula’s software is all about performing VM backups with no service interruptions and with a complete data consistency, since VM backups tend to take a lot of time and resources, slowing down all other processes within the system.

Saving both storage and bandwidth is also possible with Bacula’s Snapshot and Global Endpoint Deduplication features. Bacula’s general working principle is somewhat similar to how the backups are usually performed within KVM – via creating snapshots and dumping them in specific places. However Bacula, unlike the majority of regular backup scripts, has no need to be installed on the system in the first place and is working on a Hypervisor Level to begin with via agentless technology.

Backing up Xen with Bacula Enterprise

With Xen, Bacula Enterprise has a specific process that includes:

  • Cleaning up before starting the process (deleting older or stalled snapshots);
  • Creating a new snapshot of a guest VM in question;
  • Exporting the snapshot’s data to a Bacula storage daemon via XenServer’s vm-export command;
  • Deleting the snapshot afterwards.

Both halted and running states are fine for the backup process to work with. There’s also a specific title template for old snapshots that’ll be deleted in the backup process: BaculaSnapshot_<UUID>_JobID_<NR>. Of course, the Citrix Hypervisor module would notify you in the form of a log screen information about every guest VM backup process and its status, as well as about deleted snapshots. A backup process is automatically backing up every VM that it finds, unless specified otherwise via a number of specific plugin parameters, including:

  • vm=<name-label>
  • uuid=<uuid>
  • include=<name-label-regex>
  • exclude=<name-label-regex>
  • quiesce=<0|1>

Backing up VMware with Bacula Enterprise

Performing a VMware guest backup with Bacula is possible in two different ways: using vSphere plugin to create image backups and outfitting every guest VM with a Bacula client software.

After installing Bacula Enterprise File Daemon onto every VM you’ll have to remember to spread all of the backup jobs so that they won’t run at the same time and create a bottleneck effect. It’s more or less a regular backup process, other than that. Installing Bacula’s software on every VM allows you to take advantage of a number of features:

  • Job verification;
  • File level compression;
  • Individual file quick restore;
  • Checksum mismatch detection, and so on.

The other method is all about using the vSphere plugin to save client disks at the raw level. There’s no need for every guest VM to have Bacula software installed in this case, as well. vSphere plugin uses NBD or SAN access to read and save the contents of the VM in question. This method is fundamentally different from the previous one since this one has no need to use the file system to interact with files, so the ESXi infrastructure doesn’t need to delegate that much resources to the backup process every time. The downside of this process is that it also saves unnecessary system files, as well, like internet temporary files or the swap file.

Backing up Red Hat Virtualization with Bacula Enterprise

As with VMware, the RHV backup process can be performed in two different ways: internal and external.

The external method relies on Bacula Enterprise RHV module to perform a range of different backups by simply downloading disks via the API. Those methods aren’t dependable on the VM in the RHV environment, but they’re slower than the different one.

The external method is capable of taking advantage of both incremental and differential backup types, each with their own advantages and shortcomings. Incremental backup is quicker and saves a lot of storage space, but takes a long time to restore if there’s a large number of incremental backups after the last full one. Differential backup is somewhat slower than the incremental one and takes a bit more storage space, but has faster restoration speed and doesn’t suffer from the restoration speed slow down effect that incremental has.

There’s also the internal method, the one that is called proxyVM. The process itself consists of a special VM that is placed inside of the RHV environment, and that VM creates backups of all of the disks that belong to VMs. This method is capable of only working with full backup type, but it is much faster than the other ones in general. The process itself consists of binding the VM in question, creating a snapshot of all the data, and unbinding everything afterwards.

Backing up Proxmox with Bacula Enterprise

There’s three important steps that are a part of the Proxmox backup process, even though the nature of the process itself is somewhat similar to other ones.

  1. The configuration of a guest VM is saved
  2. A snapshot of the VM is created (while suspending or turning off the VM in question)
  3. The vzdump command is executed and the data is saved

The backup log would show all of the steps of the backup creation, from start to finish, as well as what old or outdated backups were deleted beforehand. Backups themselves are created in a .vma format (for QEMU guest VM) or .conf and .tar (for LXC guest VM). This job can also affect multiple VMs at once, each with distinctive file names and the appropriate file format.

Summary on how to backup virtual machines with Bacula

backup virtual machine
Backing up virtual machines through their hypervisor backup interfaces provides many advantages over traditional agent based backups, especially when simplicity of management and disaster recovery are paramount. Bacula Enterprise Edition gives you the ability to choose and combine virtualized and traditional approaches to provide a complete backup and recovery solution for an especially wide range of virtual environments, databases, applications and physical hardware - all from a single, scalable platform.

About the author

Rob Morrison

Rob Morrison is the marketing director at Bacula Systems. He started his IT marketing career with Silicon Graphics in Switzerland, performing strongly in various marketing management roles for almost 10 years. In the next 10 years Rob also held various marketing management positions in JBoss, Red Hat and Pentaho ensuring market share growth for these well-known companies. He is a graduate of Plymouth University and holds an Honours Digital Media and Communications degree, and completed an Overseas Studies Program.

2 Comments
  1. Alex
    Alex

    How do I back up an Oracle database using Bacula? When applying changes and saving the fileset, a schedule must be chosen from the available list of configured backup schedules, right?

Leave a comment

Your email address will not be published. Required fields are marked *

You may use these HTML tags and attributes: <a href="" title=""> <abbr title=""> <acronym title=""> <b> <blockquote cite=""> <cite> <code> <del datetime=""> <em> <i> <q cite=""> <s> <strike> <strong>