1. Autopruning


1.1 Autopruning and Manual Pruning

Bacula’s long-standing default behavior has been to perform pruning automatically at the end of a job run or when Bacula needs to recycle a volume. On many Bacula deployments this is usually fine and does not interfere with normal operations.
However, with larger deployments with several millions of files, pruning can take a long time, and can possibly hold up other jobs.
Additionally, there have been some race conditions detected when volume pruning was being done while backup jobs were running. This is especially true in large environments with disk-based volumes, and even more so if truncation is being performed.
The recommended solution for medium and large environment (more than 50 clients with millions of files) is to disable automatic pruning everywhere, and perform pruning at a quiet time in the backup schedule.
There are three main resources, on Director side, where automatic prune is configured in Bacula:
◾ the Job resource
◾ the Client resource
◾ the Pool resource
For a Job resource, there are three directives used to configure automatic prune:
◾ Prune Jobs = (default is no)
◾ Prune Files (default is no)
◾ Prune Volumes (default is no)
For Client and Pool resources, there is only one directive used to configure automatic prune:
◾ AutoPrune = (default is yes)
If automatic prune is disabled in these resources (AutoPrune = no) and the directives Prune Jobs, Prune Files or Prune Volumes are set to yes in a Job resource, it will override the settings in the Client resource (for Jobs and Files) and in the Pool resource (for Volumes), .


1.2 Disabling Automatic Pruning

In order to disable automatic pruning of jobs and files, you should:
◾ In Job resources, configure Prune Jobs = no and Prune Files = no.
◾ In Client resources, configure AutoPrune = no.
In order to disable automatic pruning of volumes, you should:
◾ In the Job resources, configure Prune Volumes = no.
◾ In the Pool resources, configure AutoPrune = no.
Once your Jobs, Clients and Pools have all been edited, you must tell the Director to reload its configuration:
Please run the following command to validate your modifications

/opt/bacula/bin/bacula-dir -t -u bacula -g bacula

Then reload the configuration from bconsole with:

reload

After disabling automatic prune for jobs, files and volumes in all the defined resources, Bacula will no longer perform any automatic pruning. As you might suspect, with no automatic pruning, the catalog database will grow forever, so we need to perform pruning manually – or in a more automated way by using an “Admin” job. Please contact Bacula Systems Support for manual ways to prune or automatic pruning jobs and scripts.
Should you wish to have volumes pruning for specific pools, you can do so by using separated prune commands for the different pools, such as the following:

echo “prune expired volume pool=Pool_1 yes” | /opt/bacula/bin/bconsole > /tmp/prunePool_1.log
echo “prune expired volume pool=Pool_2 yes” | /opt/bacula/bin/bconsole > /tmp/prunePool_2.log
echo “prune expired volume pool=Pool_3 yes” | /opt/bacula/bin/bconsole > /tmp/prunePool_3.log

Additional Notes
◾ When a volume is pruned, all the jobs and files associated to this volume are pruned as well. When a job is pruned, all the files associated to the job are also pruned. If you have your job and file retention greater than your volume retention, your jobs and files will be pruned as soon as the volume associated to it is pruned.
◾ The pruning process is closely related to retention period configuration. Bacula will only prune jobs, files or volumes where their retention time has expired. There is one exception: a volume can be pruned and reused by Bacula if there are no more jobs and files associated to it in the catalog, even if the volume retention time has not expired. Please check ’Schedule and Retentions’ section of this guide for retention period recommendations.


2. Tape Drives: update slots/label


2.1 Labelling tapes

If you want to use Tape Autochangers, please consider labelling them by barcode.
Normally you can throw new Volumes into the “Scratch Pool” where other Pools with the same mediatype can collect them, when enabled as a directive. You can assign them directly to a given Pool if you prefer.
If you have a second drive that is idle during the label command, it should be chosen automatically. Assuming that your 1st Drive (drive=0) is busy with backups, you can, in this case, use your 2nd drive of your library . If not chosen automatically, you can select the 2nd drive by using “drive=1” in the command line. When changing tape cartridges in your Library by adding, removing Tapes or changing slots numbers you should issue a “update slots” command with bconsole. Please be aware that some Libraries might unload a used cartridge to any other free slot, if the selected slot for unloading is currently occupied.

update slots barcode drive=1 storage=Autochanger-LTO

After that Bacula knows where the cartridges are, and the barcode sticker of each individual tape has been read. Now you must do the labelling with an additional command. Please use a non busy drive to label your barcodes.


2.2 Sample output

Assuming new tapes are in slots 30,29,28,16,14, you should choose a Pool, Storage and Slot(s) (drive), otherwise you will be prompted interactively.

label barcode storage=Autochanger-LTO pool=Scratch slots=30,29,28,16,14 drive=1

The bconsole “status slots” command gives you a list of your current slots assignments.
2


2.3 Note:

If using a Library without barcode support you can issue the commands without the “barcode” keyword, however scanning the tape media for a label will then take much longer.


6.3 How to configure an SD to SD copy

A copy job, as its name indicates, creates a copy of an already existing backup job.
This can be used to change retention policy, storage media and/or where the data is physically stored. Bacula detects whether the reading and writing devices are in the same storage daemon and proceeds accordingly. In this case, the storage daemon currently holding the data to be copied acts as a client to the destination storage daemon.
Suppose these device definitions in bacula-dir.conf:

Autochanger {
Name = File1
Address = sd.bacula6.com
SDPort = 9103
Password = “SomeFancyPassword”
Device = FileChgr1
Media Type = File1
Maximum Concurrent Jobs = 10
Autochanger = File1
}
Autochanger {
Name = File1-sd1
Address = sd1.bacula6.com # N.B. Use a fully qualified name here
SDPort = 9103
Password = “SomeOtherFancyPassword”
Device = FileChgr1
Media Type = File1-sd1
Maximum Concurrent Jobs = 10
Autochanger = RemoteFile1
}

The job copies the jobs stored in File1-sd1 to File1:

Job {
Name = “CopyTest”
Type = “Copy”
JobDefs = “DefaultJob”
NextPool = “File”
Pool = “File-copy”
SelectionType = Pool Uncopied Jobs
Storage = “File1-sd1”
WriteBootstrap = “/opt/bacula/working/%c_%n.bsr”
}

Be careful to be sure the pool uses the right storage:

Pool {
Name = File-copy
Pool Type = Backup
Storage = File1-sd1
NextPool = File
Recycle = yes # Bacula can automatically recycle Volumes
AutoPrune = yes # Prune expired volumes
Volume Retention = 365 days # one year
Maximum Volume Bytes = 50G # Limit Volume size to something reasonable
Maximum Volumes = 100 # Limit number of Volumes in Pool
Label Format = “copy-” # Auto label
}

The important definitions in sd.bacula6.com’s bacula-sd.conf:

Director {
Name = director.bacula6.com-dir
Password = “SomeFancyPassword”

Autochanger {
Name = FileChgr1
Device = FileChgr1-Dev1, FileChgr1-Dev2
Changer Command = “”
Changer Device = /dev/null
}
Device {
Name = FileChgr1-Dev1
Media Type = File1
Archive Device = /storage/bacula
LabelMedia = yes; # lets Bacula label unlabeled media
Random Access = Yes;
AutomaticMount = yes; # when device opened, read it
RemovableMedia = no;
AlwaysOpen = no;
Maximum Concurrent Jobs = 5
}
Device {
Name = FileChgr1-Dev2
Media Type = File1
Archive Device = /storage/bacula
LabelMedia = yes; # lets Bacula label unlabeled media
Random Access = Yes;
AutomaticMount = yes; # when device opened, read it
RemovableMedia = no;
AlwaysOpen = no;
Maximum Concurrent Jobs = 5
}

As well as the important definitions in the remote storage daemon’s configuration (sd1.bacula6.com):

Director {
Name = director.bacula6.com-dir
Password = “SomeOtherFancyPassword”

Autochanger {
Name = FileChgr1
Device = FileChgr1-Dev1, FileChgr1-Dev2
Changer Command = “”
Changer Device = /dev/null
}
Device {
Name = FileChgr1-Dev1
Media Type = File1-sd1
Archive Device = /var/lib/bacula
LabelMedia = yes; # lets Bacula label unlabeled media
Random Access = Yes;
AutomaticMount = yes; # when device opened, read it
Volume Poll Interval = 0
RemovableMedia = no;
AlwaysOpen = no;
Maximum Concurrent Jobs = 5
}
Device {
Name = FileChgr1-Dev2
Media Type = File1-sd1
Archive Device = /var/lib/bacula
LabelMedia = yes; # lets Bacula label unlabeled media
Random Access = Yes;
AutomaticMount = yes; # when device opened, read it
RemovableMedia = no;
Volume Poll Interval = 0
AlwaysOpen = no;
Maximum Concurrent Jobs = 5
}

The job output is:

01-Dec 16:28 director.bacula6.com-dir JobId 43: The following 2 JobIds were chos
en to be copied: 29,30
01-Dec 16:28 director.bacula6.com-dir JobId 43: Job queued. JobId=44
01-Dec 16:28 director.bacula6.com-dir JobId 43: Copying JobId 44 started.
01-Dec 16:28 director.bacula6.com-dir JobId 43: Copying using JobId=30 Job=BackupClient1.2016-12-01_16.14.29_09
01-Dec 16:28 director.bacula6.com-dir JobId 43: Start Copying JobId 43, Job=CopyTest.2016-12-01_16.28.55_26
01-Dec 16:28 director.bacula6.com-dir JobId 43: Using Device “sd1-FileChgr1-Dev1” to read.
01-Dec 16:28 sd1.bacula6.com JobId 43: Ready to read from volume “copy-0003” on file device “sd1-FileChgr1-Dev1” (/var/lib/bacula).
01-Dec 16:28 sd1.bacula6.com JobId 43: Forward spacing Volume “copy-0003” to file:block 0:19183166.
01-Dec 16:28 sd1.bacula6.com JobId 43: End of Volume at file 0 on device “FileChgr1-Dev1” (/var/lib/bacula), Volume “copy-0003”
01-Dec 16:28 sd1.bacula6.com JobId 43: Elapsed time=00:00:01, Transfer rate=19.16 M Bytes/second
01-Dec 16:28 director.bacula6.com-dir JobId 43: Bacula Enterprise director.bacula6.com-dir 8.6.5 (10Aug16):
Build OS: x86_64-redhat-linux-gnu-bacula-enterprise redhat
Prev Backup JobId: 30
Prev Backup Job: BackupClient1.2016-12-01_16.14.29_09
New Backup JobId: 46
Current JobId: 43
Current Job: CopyTest.2016-12-01_16.28.55_26
Backup Level: Incremental
Client: fd.bacula6.com
FileSet: “Full Set” 2015-03-03 17:12:31
Read Pool: “File-copy” (From Job resource)
Read Storage: “File1-sd1” (From Pool resource)
Write Pool: “File” (From Job resource)
Write Storage: “File1” (From Job resource)
Catalog: “MyCatalog” (From Client resource)
Start time: 01-Dec-2016 16:28:58
End time: 01-Dec-2016 16:28:59
Elapsed time: 1 sec
Priority: 10
SD Files Written: 22
SD Bytes Written: 19,167,642 (19.16 MB)
Rate: 19167.6 KB/s
Volume name(s): Vol-0004
Volume Session Id: 2
Volume Session Time: 1480620520
Last Volume Bytes: 38,366,074 (38.36 MB)
SD Errors: 0
SD termination status: OK
Termination: Copying OK

Previous section Next section