Bacula

From Nix-Pro
Jump to: navigation, search

General information

Bacula is made up of the following five major components or services:

  • Director (bacula-dir) - The Bacula Director service is the program that supervises all the backup, restore, verify and archive operations. The system administrator uses the Bacula Director to schedule backups and to recover files. The Director runs as a daemon (or service) in the background.
  • Console - The Bacula Console service is the program that allows the administrator or user to communicate with the Bacula Director Currently, the Bacula Console is available in three versions: text-based console interface, QT-based interface, and a wxWidgets graphical interface. The first and simplest is to run the Console program in a shell window (i.e. TTY interface). Most system administrators will find this completely adequate. The second version is a GNOME GUI interface that is far from complete, but quite functional as it has most the capabilities of the shell Console. The third version is a wxWidgets GUI with an interactive file restore. It also has most of the capabilities of the shell console, allows command completion with tabulation, and gives you instant help about the command you are typing.
  • File (bacula-fd) - The Bacula File service (also known as the Client program) is the software program that is installed on the machine to be backed up. It is specific to the operating system on which it runs and is responsible for providing the file attributes and data when requested by the Director. The File services are also responsible for the file system dependent part of restoring the file attributes and data during a recovery operation.
  • Storage daemon (bacula-sd) - runs on a physical server, which is responsible for the data storage.

Monitor services Bacula components or services.png

Backup strategy

  • Incremental - A backup that includes all files changed since the last Full, Differential, or Incremental backup started. It is normally specified on the Level directive within the Job resource definition, or in a Schedule resource.
  • Differential - A backup that includes all files changed since the last Full save started. Note, other backup programs may define this differently.

Overview of Bacula configuration

Bacula-conf.png

Configuring Director

  • Here is the path to the Bacula director configuration file:

/etc/bacula/bacula-dir.conf

Bacula catalog

The Catalog is used to store summary information about the Jobs, Clients, and Files that were backed up and on what Volume or Volumes. The information saved in the Catalog permits the administrator or user to determine what jobs were run, their status as well as the important characteristics of each file that was backed up, and most importantly, it permits you to choose what files to restore. The Catalog is an online resource, but does not contain the data for the files backed up. Most of the information stored in the catalog is also stored on the backup volumes (i.e. tapes). Of course, the tapes will also have a copy of the file data in addition to the File Attributes.

Here is example of Catalog section in bacula-dir.conf file: 

Catalog {
Name = MyCatalog
dbname = bacula; DB Address = "10.10.0.1"; user = bacula; password = "some_password"
}

Bacula Schedule

Example of default schedule section in bacula-dir.conf file: 

Schedule {
  Name = "WeeklyCycle"
  Run = Full 1st sun at 23:05
  Run = Differential 2nd-5th sun at 23:05
  Run = Incremental mon-sat at 23:05
}

Здесь описано полное копирование каждое первое воскресенье месяца и далее дифференциальные и инкрементальные.

В неофициальном переводе официальной документации рекомендуется устанавливать период хранения томов в два раза больше интервала выполнения полных резервных копий. То есть, при использовании данного расписания хранить резервные копии следует не менее двух месяцев, а насколько дольше - определится политикой (секция Pool).

Differential бэкап обязательно 2е - 5е воскресенье, а не 4е, так как если в месяце будет 5 ВС, то не будет хватать одной копии.

Bacula FileSet

В данной секции конфигурационного файла bacula-dir.conf мы указываем список файлов, которые мы будем бэкапить и даем этому списку определенное имя. Имя FileSet будет использоваться позже в описании Job. Ниже пример секции FileSet: 

FileSet {
    Name = "server3"
    Include {
        Options {
            signature = MD5 #Для сверки используем MD5
            Compression=GZIP #Используем GZIP компрессию
        }
        File = /etc #Что именно бекапить
        File = /home/
        File = /var/www
    } 

    Exclude { #А что не бекапить, например логи
        File = /home/logs
        File = /var/www/logs
    }
}

Bacula Storage

В секции Storage указывается сервер на котором запущен bacula-sd сервис, и который будет доступен для использования директором. Обратите внимаение что в bacula-sd.conf мы описываем где именно хранить данные, имя директора, который может использовать данный SD сервер для бэкапов. Т.е. имя DIR и пароль SD в bacula-dir.conf должны быть такие же как и в bacula-sd.conf. Ниже примеры части конфигурационных файлов: bacula-dir.conf: 

...
Director {
  Name = local-director
  DIRport = 9101
  QueryFile = "/etc/bacula/scripts/query.sql"
  WorkingDirectory = "/var/lib/bacula"
  PidDirectory = "/var/run/bacula"
  Maximum Concurrent Jobs = 1
  Password = "director's-console-password"
  Messages = Daemon
  DirAddress = 127.0.0.1
}
....
Storage {
  Name = LocalStorage
  Address = 127.0.0.1
  SDPort = 9103
  Password = "storage-service-password"
  Device = LocalFileStorage
  Media Type = File
}

bacula-sd.conf: 

...
Device {
  Name = LocalFileStorage
  Media Type = File
  Archive Device = /backup/bacula
  LabelMedia = yes;
  Random Access = Yes;
  AutomaticMount = yes;
  RemovableMedia = no;
  AlwaysOpen = no;
}
Director {
  Name = local-director
  Password = "storage-service-password"
}
...
  • Name - имя storage ресурса, которое будет использоваться далее как Storage в описании Job ресурса.

Bacula Pool

Количество томов (volume)

В секции Pool по сути определяются скроки хранения бэкапов, параметры ротации сделанных в зависимости от schedule. Например у нас есть следующее расписание: 

Schedule {
    Name = "WeeklyCycle"
    Run = Level=Full 1st sun at 2:05
}

В данном случае будет делаться полный бэкап каждое первое воскресенье месяца. Если мы хотим хранить месячный бэкап за последние пол-года то секция Pool должна содержать примерно следующее 

Pool {
    Name = Full-Pool
    Pool Type = Backup
    Recycle = yes
    # automatically recycle Volumes
    AutoPrune = yes
    # Prune expired volumes
    Volume Retention = 6 months
    Maximum Volume Jobs = 1
    Label Format = Full-
    Maximum Volumes = 8
}

В данном примере мы говорим что Максимальное количество используемых в пуле томов равно 8.

NOTE Если хранится N томов, то нужно иметь в пуле как минимум на два тома больше. Один - на текущий бэкап, чтобы не затирать самый старый из предыдущих на случай ошибки бэкапа, и как минимум один - для свободы маневра в случае ошибок.

И самый старый бэкап будет удаляться каждые 6 месяцев.

Полный конфигурационный файл bacula-dir.conf

Director {
  Name = b1-dir
  DIRport = 9101
  QueryFile = "/etc/bacula/scripts/query.sql"
  WorkingDirectory = "/var/lib/bacula"
  PidDirectory = "/var/run/bacula"
  Maximum Concurrent Jobs = 1
  Password = "my-dir-pass"
  Messages = Daemon
  DirAddress = 127.0.0.1
}

FileSet {
  Name = "Full_Set"
  Include {
    Options {
      signature = MD5
    }
    File = /etc
    File = /home/smopa
  }

  Exclude {
    File = /var/lib/bacula
    File = /srv/backup
    File = /proc
    File = /tmp
    File = /.journal
    File = /.fsck
  }
}

Storage {
    Name = b1-storage
    Address = 127.0.0.1
    SDPort = 9103
    Password = "my-storage-pass"
    Device = LocalFileStorage
    Media Type = File
}


Schedule {
    Name = "WeeklyCycle"
    Run = Level=Full 1st sun at 2:05
    Run = Level=Differential 2nd-5th sun at 2:05
    Run = Level=Incremental mon-sat at 2:05
}

Job {
    Name = b1-backup
    Type = Backup
    Client = b1-fd
    FileSet = "Full_Set"
    Schedule = "WeeklyCycle"
    Storage = b1-storage
    Messages = Standard
    Pool = Default
    Full Backup Pool = Full-Pool
    Incremental Backup Pool = Inc-Pool
    Differential Backup Pool = Diff-Pool
    Write Bootstrap = "/var/lib/bacula/%n.bsr"
    Priority = 10
}

Job {
    Name = "RestoreFiles"
    Type = Restore
    Client=b1-fd
    FileSet="Full_Set"
    Storage = b1-storage
    Pool = Default
    Full Backup Pool = Full-Pool
    Incremental Backup Pool = Inc-Pool
    Differential Backup Pool = Diff-Pool
    Messages = Standard
    Where = /var/lib/bacula-restores
}

Pool {
    Name = Default
    Pool Type = Backup
    Recycle = yes                       # Bacula can automatically recycle Volumes
    AutoPrune = yes                     # Prune expired volumes
    Volume Retention = 365 days         # one year
}

Pool {
    Name = Full-Pool
    Pool Type = Backup
    Recycle = yes
    # automatically recycle Volumes
    AutoPrune = yes
    # Prune expired volumes
    Volume Retention = 6 months
    Maximum Volume Jobs = 1
    Label Format = Full-
    Maximum Volumes = 8
}

Pool {
    Name = Inc-Pool
    Pool Type = Backup
    Recycle = yes
    # automatically recycle Volumes
     AutoPrune = yes
    # Prune expired volumes
    Volume Retention = 10 days
    Maximum Volume Jobs = 1
    Label Format = Inc-
    Maximum Volumes = 7
}

Pool {
    Name = Diff-Pool
    Pool Type = Backup
    Recycle = yes
    AutoPrune = yes
    Volume Retention = 40 days
    Maximum Volume Jobs = 1
    Label Format = Diff-
    Maximum Volumes = 6
}

# Generic catalog service
Catalog {
  Name = MyCatalog
# Uncomment the following line if you want the dbi driver
# dbdriver = "dbi:sqlite3"; dbaddress = 127.0.0.1; dbport =
  dbname = "bacula"; dbuser = ""; dbpassword = ""
}

Messages {
  Name = Standard
  mailcommand = "/usr/lib/bacula/bsmtp -h localhost -f \"\(Bacula\) \<%r\>\" -s \"Bacula: %t %e of %c %l\" %r"
  operatorcommand = "/usr/lib/bacula/bsmtp -h localhost -f \"\(Bacula\) \<%r\>\" -s \"Bacula: Intervention needed for %j\" %r"
  mail = root@localhost = all, !skipped
  operator = my-email@domain.com = mount
  console = all, !skipped, !saved
  append = "/var/lib/bacula/log" = all, !skipped
  catalog = all
}

Messages {
  Name = Daemon
  mailcommand = "/usr/lib/bacula/bsmtp -h localhost -f \"\(Bacula\) \<%r\>\" -s \"Bacula daemon message\" %r"
  mail = my-email@domain.com = all, !skipped
  console = all, !skipped, !saved
  append = "/var/lib/bacula/log" = all, !skipped
}

# Client (File Services) to backup
Client {
  Name = b1-fd
  Address = localhost
  FDPort = 9102
  Catalog = MyCatalog
  Password = "my-fd-password"         # password for FileDaemon
  File Retention = 30 days            # 30 days
  Job Retention = 6 months            # six months
  AutoPrune = yes                     # Prune expired Jobs/Files
}

Bacula Storage Server (SD) configuration

В общем случае хранилищ может быть несколько и они могут быть физически разными устройствами. В данной ситуации хранилище будет создано на сервере S1.

Bacula поддерживает множество разных типов хранилищ, в качестве примера был выбран тип File, представляющий собой просто файл. Располагаться хранилище будет в директории /home/backup/bacula/: 

#
# Bacula Storage Daemon Configuration file

Storage {
  # Имя демона
  Name = S1-sd
  # Порт SD демона
  SDPort = 9103
  # рабочая директория
  WorkingDirectory = "/var/lib/bacula"
  # где создавать pid
  Pid Directory = "/var/run/bacula"
  # максимальное количество одновременных заданий
  Maximum Concurrent Jobs = 20
  SDAddress = 127.0.0.1
}

  # Director, который может работать со Storage Daemon
  Director {
  # имя демона Director
  Name = S1-dir
  # пароль, под которым Director будет общаться со Storage Daemon
  Password = "somepass"
}

  # Director, который может получать информацию о состоянии Storage Daemon
  Director {
  # имя демона Director
  Name = S1-mon
  # пароль, под которым Director будет общаться со Storage Daemon
  Password = "somepass"
  Monitor = yes
}

  # настройка хранилища
  Device {
  # имя хранилища
  Name = S1FileStorage
  # тип хранилища
  Media Type = File
  # физическое расположение хранилища
  Archive Device = /home/backup/bacula/
  # разрешать Bacula задавать лейбл хранилищу
  # без этой опции хранилище придется создавать вручную из консоли
  LabelMedia = yes;
  # хранилище поддерживает работу со случайными блоками
  Random Access = yes;
  # автоматически подключать хранилище по запросу
  AutomaticMount = yes;
  # хранилище не является сменным носителем
  RemovableMedia = no;
  # не держать хранилище все время открытым
  AlwaysOpen = no;
}

  # отправка сообщений
  Messages {
  # тип сообщения
  Name = Standard
  # Director, через который будут отправлены сообщения
  director = S1-dir = all
}

Bacula File Server (FD) configuration

File Daemon необходимо устанавливать на каждый сервер, данные с которого необходимо резервировать. 

# Director, который может работать со Storage Daemon
Director {
  # имя демона Director
  Name = S1-dir
  # пароль, под которым Director будет общаться со File Daemon
  Password = "somepass"
}

# Director, который может получать информацию о состоянии File Daemon
Director {
  # имя демона Director
  Name = S1-mon
  # пароль, под которым Director будет общаться со File Daemon
  Password = "somepass"
  Monitor = yes
}

# настройка File Daemon
FileDaemon {
  # имя File Daemon
  Name = S1-fd
  # порт
  FDport = 9102
  # рабочая директория
  WorkingDirectory = /var/lib/bacula
  # где создавать pid
  Pid Directory = /var/run
  # максимальное количество одновременных заданий
  Maximum Concurrent Jobs = 20
}

# отправка сообщений
Messages {
  # тип сообщений
  Name = Standard
  # Director, через который будут отправлены сообщения
  director = S1-dir = all, !skipped, !restored
}

Bconsole commands Quick Howto

  • list jobs that were/are running:
list jobs

here is example of output: 

+-------+---------------+---------------------+------+-------+----------+------------+-----------+
| JobId | Name          | StartTime           | Type | Level | JobFiles | JobBytes   | JobStatus |
+-------+---------------+---------------------+------+-------+----------+------------+-----------+
| 1     | BackupClient1 | 2013-04-20 20:24:31 | B    | F     | 19960    | 446288812  | T         |
| 2     | BackupClient1 | 2013-04-20 23:05:02 | B    | I     | 9        | 10237085   | T         |
| 3     | BackupCatalog | 2013-04-20 23:10:03 | B    | F     | 1        | 3633715    | T         |
| 4     | BackupClient1 | 2013-04-21 23:05:03 | B    | D     | 116      | 49319484   | T         |
+-------+---------------+---------------------+------+-------+----------+------------+-----------+

Type B means backup.
Level F means full, I - incremental, D differential
JobStatus T means terminated (terminated can be with OK status or with Fail).

  • Get a list of files that were backed up in particular job
list files jobid=<id>
  • Get information about Bacula Pools (including Volumes usage)
list pools
  • Get information about volumes
list volumes
  • Get status about Bacula
status
  • Run a particular job
run

then choose a needed job 

*run
Automatically selected Catalog: MyCatalog
Using Catalog "MyCatalog"
A job name must be specified.
The defined Job resources are:
     1: server-backup
     2: RestoreFiles
Select Job resource (1-2):
  • Restore files
restore

then you can choose for example to restore files from the needed Job: 

First you select one or more JobIds that contain files
to be restored. You will be presented several methods
of specifying the JobIds. Then you will be allowed to
select which files from those JobIds are to be restored.

To select the JobIds, you have the following choices:
     1: List last 20 Jobs run
     2: List Jobs where a given File is saved
     3: Enter list of comma separated JobIds to select
     4: Enter SQL list command
     5: Select the most recent backup for a client
     6: Select backup for a client before a specified time
     7: Enter a list of files to restore
     8: Enter a list of files to restore before a specified time
     9: Find the JobIds of the most recent backup for a client
    10: Find the JobIds for a backup for a client before a specified time
    11: Enter a list of directories to restore for found JobIds
    12: Select full restore to a specified Job date
    13: Cancel
Select item:  (1-13): 3
Enter JobId(s), comma separated, to restore: 186
You have selected the following JobId: 186

Building directory tree for JobId(s) 186 ...  +++++++++++++++++++++++++++++++++++++++++
20,042 files inserted into the tree.

You are now entering file selection mode where you add (mark) and
remove (unmark) files to be restored. No files are initially added, unless
you used the "all" keyword on the command line.
Enter "done" to leave this mode.

cwd is: /

Once this is done you need to specify files that should be restored. For example we want to restore /etc/hosts file: 

$ cd etc
cwd is: /etc/
$ add hosts
1 file marked.
$ done
Bootstrap records written to /var/lib/bacula/server1-dir.restore.1.bsr

The job will require the following
   Volume(s)                 Storage(s)                SD Device(s)
===========================================================================

    Full-0002                 server-storage                LocalFileStorage

Volumes marked with "*" are online.


1 file selected to be restored.

Defined Clients:
     1: server1-fd
     2: server2-fd
Select the Client (1-2): 1
Run Restore job
JobName:         RestoreFiles
Bootstrap:       /var/lib/bacula/server1-dir.restore.1.bsr
Where:           /var/lib/bacula-restores
Replace:         always
FileSet:         Full_Set
Backup Client:   server1-fd
Restore Client:  server1-fd
Storage:         server1-storage
When:            2013-08-20 21:49:22
Catalog:         MyCatalog
Priority:        10
Plugin Options:  *None*
OK to run? (yes/mod/no): yes
Job queued. JobId=187

Then we can check status of the Job: 

*status
Status available for:
     1: Director
     2: Storage
     3: Client
     4: All
Select daemon type for status (1-4): 1
b1-dir Version: 5.2.5 (26 January 2012) x86_64-pc-linux-gnu ubuntu 12.04
Daemon started 20-Aug-13 21:30. Jobs: run=2, running=0 mode=0,0
 Heap: heap=294,912 smbytes=81,644 max_bytes=3,496,788 bufs=242 max_bufs=270

Scheduled Jobs:
Level          Type     Pri  Scheduled          Name               Volume
===================================================================================
Incremental    Backup    10  21-Aug-13 03:00    server1-backup          *unknown*
====

Running Jobs:
Console connected at 20-Aug-13 21:46
Console connected at 20-Aug-13 21:49
No Jobs running.
====

Terminated Jobs:
 JobId  Level    Files      Bytes   Status   Finished        Name
====================================================================
   187                1       364   OK       20-Aug-13 21:49 RestoreFiles

====

From the example above we see that restoration was finished successfully

Additional information

The full list of List command forms can be found on http://www.bacula.org/manuals/en/console/console/Bacula_Console.html

Used Materials

Retrieved from "https://wiki.nix-pro.com/index.php?title=Bacula&oldid=1112"