1. Autopruning

1.1 Autopruning and Manual Pruning

Bacula’s long-standing default behavior has been to perform pruning automatically at the end of a job run or when Bacula needs to recycle a volume. On many Bacula deployments this is usually fine and does not interfere with normal operations. However, with larger deployments with several millions of files, pruning can take a long time, and can possibly hold up other jobs.
Additionally, there have been some race conditions detected when volume pruning was being done while backup jobs were running. This is especially true in large environments, and even more so if truncation is being performed.
The recommended solution for medium and large environment (more than 50 clients and millions of files) is to disable automatic pruning everywhere, and perform pruning at a quiet time in the backup schedule.
There are three main resources, on Director side, where automatic prune is configured in Bacula:
• the Job{} resource
• the Client{} resource
• the Pool{} resource
For a Job{} resource, there are three directives used to configure automatic prune:
• Prune Jobs = (default is no)
• Prune Files = (default is no)
• Prune Volumes = (default is no)
For Client{} and Pool{} resources, there is only one directive used to configure automatic prune:
• AutoPrune = (default is yes)
If automatic prune is disabled in these resources (AutoPrune = no) and the directives Prune Jobs, Prune Files or Prune Volumes are set to y. .e.s in a Job{} resource, it will override the settings in the Client{} resource (for Jobs and Files) and in the Pool{} resource (for Volumes), .

1.2 Disabling Automatic Pruning

In order to disable automatic pruning of jobs and files, you should:
• In Job{} resources, avoid the usage of Prune Jobs = yes and Prune Files = yes.
• In all Client{} resources, configure AutoPrune = no. In order to disable automatic pruning of volumes, you should:
• In the Job{} resources, avoid the usage of Prune Volumes = yes.
• In all Pool{} resources, configure AutoPrune = no.
Once your Jobs, Clients and Pools have all been edited, you must tell the Director to reload its configuration:
Please run the following command to validate your modifications

/opt/bacula/bin/bacula-dir -t -u bacula -g bacula

Then reload the configuration from bconsole with:

reload

After disabling automatic prune for jobs, files and volumes in all the defined resources, Bacula will no longer perform any automatic pruning. As you might suspect, with no automatic pruning, the catalog database will grow forever, so we need to perform pruning manually — or in a more automated way by using an “Admin” job. Please contact Bacula Systems Supoort for manual ways to prune or automatic pruning jobs and scripts.
Should you wish to have volumes pruning for specific pools, you can do so by using separated prune commands for the different pools, such as the following:

echo “prune expired volume pool=Pool_1 yes” | /opt/bacula/bin/bconsole > /tmp/prunePool_1.log
echo “prune expired volume pool=Pool_2 yes” | /opt/bacula/bin/bconsole > /tmp/prunePool_2.log
echo “prune expired volume pool=Pool_3 yes” | /opt/bacula/bin/bconsole > /tmp/prunePool_3.log

Additional Notes
◾ When a volume is pruned, all the jobs and files associated to this volume are pruned as well. When a job is pruned, all the files associated to the job are also pruned. If you have your job and file retention greater than your volume retention, your jobs and files will be pruned as soon as the volume associated to it is pruned.
◾ The pruning process is closely related to retention period configuration. Bacula will only prune jobs, files or volumes where their retention time has expired. There is one exception: a volume can be pruned and reused by Bacula if there are no more jobs and files associated to it in the catalog, even if the volume retention time has not expired. Please check ’Schedule and Retentions’ section of this guide for retention period recommendations.

1.3 Notes

• When a volume is pruned, all the jobs and files associated to this volume are pruned as well.
When a job is pruned, all the files associated to the job are also pruned. If you have your job and file retention greater than your volume retention, your jobs and files will be pruned as soon as the volume associated to it is pruned.
• The pruning process is closely related to retention period configuration. Bacula will only prune jobs, files or volumes where their retention time has expired. There is one exception: a volume can be pruned and reused by Bacula if there are no more jobs and files associated to it in the catalog, even if the volume retention time has not expired. Please check “Schedule and Retentions” (cf. 3.3 on page 8) section of this guide for retention period recommendations.
• If a single Pool{} or Client{} resource has an Autoprune = yes set, auto- matic pruning is activated.

2. Tape Drives: update slots/label

2.1 Labelling tapes

If you want to use Tape Autochangers, please consider labelling them by barcode. Normally you can throw new Volumes into the “Scratch Pool” where other Pools with the same mediatype can collect them, when enabled as a directive. You can assign them directly to a given Pool if you prefer.
If you have a second drive that is idle during the label command, it should be chosen automatically. Assuming that your 1st Drive (drive=0) is busy with backups, you can, in this case, use your 2nd drive of your library . If not chosen automatically, you can select the 2nd drive by using “drive=1” in the command line. When changing tape cartridges in your Library by adding, removing Tapes or changing slots numbers you should issue an update slots command within bconsole.
Please be aware that some Libraries might unload a used cartridge to any other free slot, if the selected slot for unloading is currently occupied.

update slots barcode drive=1 storage=Autochanger-LTO

After that Bacula knows where the cartridges are, and the barcode sticker of each individual tape has been read. Now you must do the labelling with an additional command. Please use a non busy drive to label your barcodes.

2.2 Sample output

Assuming new tapes are in slots 30,29,28,16,14, you should choose a Pool, Storage and Slot(s) (drive), otherwise you will be prompted interactively.

label barcode storage=Autochanger-LTO pool=Scratch slots=30,29,28,16,14 drive=1

The bconsole status slots command gives you a list of your current slots as- signments.
2

2.3 Notes:

If using a Library without barcode support you can issue the commands without the “barcode” keyword, however scanning the tape media for a label will then take much longer.

6.3 How to configure an SD to SD copy

A copy job, as its name indicates, creates a copy of an already existing backup job. This can be used to change retention policy, storage media and/or where the data is physically stored. Bacula detects whether the reading and writing devices are in the same storage daemon and proceeds accordingly. In this case, the storage daemon currently holding the data to be copied acts as a client to the destination storage daemon.
Suppose these device definitions in bacula-dir.conf:

Autochanger {
Name = Local-Storage
Address = local-sd.bacula6.com SDPort = 9103
Password = “SomeFancyPassword” Device = Local-FileChgr1
Media Type = Local-MediaType Maximum Concurrent Jobs = 10 Autochanger = Local-Storage
}

Autochanger {
Name = Remote-Storage
Address = remote-sd.bacula6.com SDPort = 9903
Password = “SomeOtherFancyPassword” Device = Remote-FileChgr1
Media Type = Remote-MediaType Maximum Concurrent Jobs = 10 Autochanger = Remote-Storage
}

The job copies the jobs stored in Local-Storage to Remote-Storage:

Job {
Name = “CopyLocal2Remote” Type = “Copy”
JobDefs = “DefaultJob” NextPool = “Remote-Pool” Pool = “Local-Pool”
SelectionType = Pool Uncopied Jobs Storage = “Local-Storage”
WriteBootstrap = “/opt/bacula/bsr/%c_%n.bsr”
}

Be careful to be sure the pools use the right storages:

Pool {
Name = Local-Pool Pool Type = Backup
Storage = Local-Storage Recycle = yes
AutoPrune = no
Volume Retention = 1 year Maximum Volume Bytes = 50G Maximum Volumes = 100
Label Format = “Local-Vol-”
}
Pool {
Name = Remote-Pool Pool Type = Backup
Storage = Remote-Storage Recycle = yes
AutoPrune = no
Volume Retention = 2 years Maximum Volume Bytes = 50G Maximum Volumes = 100
Label Format = “Remote-Vol-”
}

The important definitions in local-sd.bacula6.com’s bacula-sd.conf:

Director {
Name = director.bacula6.com-dir Password = “SomeFancyPassword”

}
Autochanger {
Name = Local-FileChgr1
Device = Local-FileChgr1-Dev1, Local-FileChgr1-Dev2 Changer Command = “”
Changer Device = /dev/null
}
Device {
Name = Local-FileChgr1-Dev1 Media Type = Local-MediaType Archive Device = /backup/volumes
LabelMedia = yes
Random Access = yes AutomaticMount = yes RemovableMedia = no AlwaysOpen = no
Maximum Concurrent Jobs = 5
}
Device {
Name = Local-FileChgr1-Dev2 Media Type = Local-MediaType Archive Device = /backup/volumes
LabelMedia = yes
Random Access = yes AutomaticMount = yes RemovableMedia = no AlwaysOpen = no
Maximum Concurrent Jobs = 5
}

As well as the important definitions in the remote storage daemon’s configuration (remote-sd.bacula6.com):

Director {
Name = director.bacula6.com-dir Password = “SomeOtherFancyPassword”

}
Autochanger {
Name = Remote-FileChgr1
Device = Remote-FileChgr1-Dev1, Remote-FileChgr1-Dev2 Changer Command = “”
Changer Device = /dev/null
}
Device {
Name = Remote-FileChgr1-Dev1 Media Type = Remote-MediaType
Archive Device = /some/other/folder

LabelMedia = yes Random Access = Yes AutomaticMount = yes
Volume Poll Interval = 0 RemovableMedia = no AlwaysOpen = no
Maximum Concurrent Jobs = 5
}
Device {
Name = Remote-FileChgr1-Dev2 Media Type = Remote-MediaType
Archive Device = /some/other/folder LabelMedia = yes
Random Access = Yes AutomaticMount = yes RemovableMedia = no Volume Poll Interval = 0
AlwaysOpen = no
Maximum Concurrent Jobs = 5
}

The job output is:

08-Feb 10:15 director.bacula6.com-dir JobId 53: The following 1 JobId was chosen to be copied: 52
08-Feb 10:15 director.bacula6.com-dir JobId 53: Copying using JobId=52 Job=BackupClient1.2017-02-08_10.14.41_10
08-Feb 10:15 director.bacula6.com-dir JobId 53: Start Copying JobId 53, Job=CopyLocal2Remote.2017-02-08_10.15.14_11
08-Feb 10:15 director.bacula6.com-dir JobId 53: Using Device “Local-FileChgr1-Dev1″ to read.
08-Feb 10:15 director.bacula6.com-dir JobId 54: Created new Volume=”Remote-Vol-0006″, Pool=”Remote-Pool”, MediaType=”Remote-MediaType” in catalog.
08-Feb 10:15 director.bacula6.com-dir JobId 54: Using Device “Remote-FileChgr1-Dev1” to write.
08-Feb 10:15 remote-sd.bacula6.com JobId 54: Labeled new Volume “Remote-Vol-0006” on file device “Remote-FileChgr1-Dev1” (/some/other/folder).
08-Feb 10:15 remote-sd.bacula6.com JobId 54: Wrote label to prelabeled Volume “Remote-Vol-0006” on file device “Remote-FileChgr1-Dev1” (/some/other/folder) 08-Feb 10:15 sd.bacula6.com JobId 53: Ready to read from volume “Local-Vol-0005” on file device “Local-FileChgr1-Dev1” (/backup/volumes).
08-Feb 10:15 sd.bacula6.com JobId 53: Forward spacing Volume “Local-Vol-0005” to file:block 0:256.
08-Feb 10:15 sd.bacula6.com JobId 53: End of Volume at file 0 on device “Local-FileChgr1-Dev1” (/backup/volumes), Volume “Local-Vol-0005”
08-Feb 10:15 sd.bacula6.com JobId 53: Elapsed time=00:00:01, Transfer rate=19.16 M Bytes/second 08-Feb 10:15 remote-sd.bacula6.com JobId 54: Elapsed time=00:00:01,Transfer rate=19.16 M Bytes/second
08-Feb 10:15 remote-sd.bacula6.com JobId 54: Sending spooled attrs tothe Director. Despooling 6,114 bytes …
08-Feb 10:15 director.bacula6.com-dir JobId 53: Bacula Enterprise director.bacula6.com-dir 8.6.5 (10Aug16):
Build OS: x86_64-redhat-linux-gnu-bacula-enterprise redhat
Prev Backup JobId: 52
Prev Backup Job: BackupClient1.2017-02-08_10.14.41_10
New Backup JobId: 54
Current JobId: 53
Current Job: CopyLocal2Remote.2017-02-08_10.15.14_11
Backup Level: Full
Client: fd.bacula6.com
FileSet: “Full Set” 2015-03-03 17:12:31
Read Pool: “Local-Pool” (From Job resource)
Read Storage: “Local-Storage” (From Pool resource)
Write Pool: “Remote-Pool” (From Job resource)
Write Storage: “Remote-Storage” (From Job resource)
Catalog: “MyCatalog” (From Client resource)
Start time: 08-Feb-2017 10:15:16
End time: 08-Feb-2017 10:15:18
Elapsed time: 0 mins 1 secs
Priority: 10
SD Files Written: 22
SD Bytes Written: 19,167,642 (19.16 MB)
Rate: 19.16 MB/s
Volume name(s): Remote-Vol-0006
Volume Session Id: 3
Volume Session Time: 1486558493
Last Volume Bytes: 19,183,189 (19.18 MB)
SD Errors: 0
SD termination status: OK
Termination: Copying OK

Previous section Next section