Home > Enterprise Backup Solutions > For HPC and Big Data

Data Backup and Recovery in HPC and Big Data

Bacula: Unbeatable in HPC and Super Computing Environments.

HPC centers are broadly working to modernize their IT infrastructure, embrace the need to correctly back up their data, and meet the quickly arriving needs of tomorrow. Their IT centers face an ongoing challenge to adapt and improve their IT operations to remain flexible and offer the latest performance capabilities. New and different approaches to security, efficiency and performance are needed to achieve these improvements. Bacula offers especially high levels of security and performance in HPC environments when compared to other vendors.

“Of those evaluated, Bacula Enterprise was the only product that worked with HPSS out-of-the-box without vendor development, provided multi-user access, had encryption compliant with Federal Information Processing Standards, did not have a capacity-based licensing model, and was available within budget”    NASA

In accordance with its Open Source pedigree, Bacula Enterprise perfectly supports Posix compliant filesystems, helping you to avoid vendor lock-in. In addition, every filesystem that can be mounted to a Linux or Windows host can be used, including parallel and clustered filesystems such as Lustre or Quobyte. POSIX file systems are the most common storage system in use today, providing a wide range of IO functions for applications to use, including byte-level access. However, with the large number of IO functions comes complexity, both for the application and the file system. Bacula helps HPC users to significantly reduce complexity by being file system-agnostic. Here is a non-exhaustive list of filesystems that Bacula Systems customers use:

ext 2|3|4
Hadoop *
Swift *

*requires a Bacula Enterprise module (plugin)

As a proven HPC backup and recovery solution of especially high performance, Bacula can handle vast volumes of data with ease. With the increasing need for HPC solutions and further improvements in technology, organizations are turning their attention to areas such as hybrid HPC solutions. Bacula anticipates that technology and innovation improvements in HPC space will increase, especially in specific areas such as Hybrid Cloud, edge compute, container technologies and security approaches. As IT teams work hard to create a balance between on-premises HPC solutions and cloud, Bacula provides a way to protect and recover these entire environments from a single platform.

Read the Bacula whitepaper that covers Backup and Recovery considerations for this sector:

Top 10 Whitepaper Highlights

  1. IT environment complexity in the research sector
  2. Technical & demanding IT environments
  3. Meeting RPO’s and RTO’s
  4. The need for especially high levels of security
  5. Bare metal recovery
  6. The need to de-risk implementation
  7. Hybrid cloud in the research sector
  8. Stand-alone capabilities and “air-gapping”
  9. Container technologies in the research sector
  10. How NASA benefits from Bacula
  11. Avoiding vendor lock-in

High Levels of Automation for HPC Users

Bacula offers an amazing degree of customization and automation, in practically every part of its broad feature-set. Contact Bacula for more details on Bacula high automation levels.

The Broadest Tape Compatibility of any backup vendor

For HPC environments that have petabytes of data to store, tape can still be the best option for long term archival and meeting RPO’s. It’s also very effective regarding retention requirements and media preservation. Today’s LTO-08 and LTO-9 tape drives have impressive specs including transfer rates up to 400 MB/sec. native, and storing up 18 TB of raw and up to 45 TB of compressed capacity. Bacula Enterprise is ideal for tape admins as we never license based on data volume and deliver unique features such as ACSLS support. Bacula supports tape libraries from all of the world’s top manufacturers and provides all tape library management operations. It also supports named user access to ACSLM, tape drive and volume locking in shared ACSLS environment, lock query and management, static tape drive location mapping and dynamic volume location mapping. Please note: Bacula is an industry leading backup and recovery solution for immutable backup.

Total Fit: Bacula’s Ultra-Broad HPC Compatibility

Bacula’s especially wide compatibility with HPC technologies is contributing to a positive change in technologies and tools to automate processes and to apply consistent governance across an HPC Federal/National laboratory or research organizations. For example, Bacula offers native integration with technologies such as HPSS,  Hadoop, Docker, Kubernetes, SAP Hana, GPFS, etc. Bacula supports over 33 different operating system versions!

As more HPC-driven research and government agencies shift to using Bacula, a pattern of needs clearly emerges from these organizations, and the reasons why the shift takes place. The Bacula Systems white paper ‘Data Backup and Recovery in Federal and National Research Agencies‘ discusses the considerations and advantages of using FIPS 140-2 compliant Bacula Enterprise as a central data backup and recovery system within research organizations, and how it can – and does – facilitate a specific, yet critical part in these organization’s enterprise-wide approach to digital modernization.

Large Backup File Split

This Bacula feature backs up large numbers of files with an optimal speed and efficiency splitting the workload in a number of workers (concurrent backup processes).  Bacula’s Large Backup File Split saves significant time in HPC environments when dealing with very large datasets. It can work with any kind of file system, and the backup is automatically managed by Bacula. This is applicable to any file-server or NAS use-case.

Another attraction of Bacula’s automation of this process is its ability to reduce the possibility of human error, as well as being fully dynamic, meaning no configuration update process is necessary regardless of any changes in the structure or size of the target dataset.

This tool can especially help where there are many users causing a complex folder structure as a result, and lots of different files. This kind of file structure may literally be being used by everybody (e.g. public/private, internal company users, shared data situations, etc.). More details are available in Bacula’s HPC white paper.

Contact us now to meet Bacula senior experts at booth# 1585 at SC23 in Denver, Co, Nov 12–17!