Release 3.12.1

3.12.1 (2024-12-09)

Bugfixes

• Add isoformat fields for backup start and end times in json output

  This patch modifies the json output of the infofile object
  adding two new fields: begin_time_iso and end_time_iso.
  The new fields allow the use of a more standard and timezone aware
  time format, preserving compatibility with previous versions.
  It is worth noting that in the future the iso format for dates will be the
  standard used by barman for storing dates and will be used everywhere
  non human readable output is requested.

  As part of the work, this patch reverts BAR-316, which was introduced on Barman
  3.12.0.

  References: BAR-494.

Release 3.12.0

3.12.0 (2024-11-21)

Minor changes

• Add FIPS support to Barman

  The md5 hash algorithm is not FIPS compliant, so it is going to be replaced by
  sha256. sha256 is FIPS compliant, vastly used, and is considered secure for most
  practical purposes.
  Up until this release, Barman's WAL archive client used hashlib.md5 to generate
  checksums for tar files before they were sent to the Barman server. Here, a tar file is
  a file format used for bundling multiple files together with a MD5SUMS file that lists
  the checksums and their corresponding paths.
  In this release, the md5 hashing algorithm is replaced by sha256 as the default.
  As a result, checksums for the tar files will be calculated using sha256, and the
  MD5SUMS file will be named SHA256SUMS. Barman still has the ability to use the
  nondefault md5 algorithm and the MD5SUMS file from the client if there is a use
  case for it. The user just needs to add the --md5 flag to the barman-wal-archive
archive_command.

  References: BAR-155, CP-34954, CP-34391.

• Removed el7, debian10, and ubuntu1804 support; updated Debian and SLES.

  Support for el7, debian10, and ubuntu1804 has been removed. Additionally, version 12
  and version name "bookworm" has been added for Debian, addressing a previously
  missing entry. The SLES image version has also been updated from sp4 to sp5.

  References: BAR-389.

• Add support for Postgres Extended 17 (PGE) and Postgres Advanced Server 17 (EPAS)

  Tests were conducted on Postgres Extended 17 (PGE) and Postgres Advanced Server 17
  (EPAS), confirming full compatibility with the latest features in Barman. This
  validation ensures that users of the latest version of PGE and EPAS can leverage all the new
  capabilities of Barman with confidence.

  References: BAR-331.

• Improve WAL compression with zstd, lz4 and xz algorithms

  Introduced support for xz compression on WAL files. It can be enabled by specifying
  xz in the compression server parameter. WALs will be compressed when entering
  the Barman's WAL archive. For the cloud, it can be enabled by specifying --xz
  when running barman-cloud-wal-archive.

  Introduced support for zstandard compression on WAL files. It can be enabled by
  specifying zstd in the compression server parameter. WALs will be compressed
  when entering the Barman's WAL archive. For the cloud, it can be enabled by
  specifying --zstd when running barman-cloud-wal-archive.

  Introduced support for lz4 compression on WAL files. It can be enabled by
  specifying lz4 in the compression server parameter. WALs will be compressed
  when entering the Barman's WAL archive. For the cloud, it can be enabled by
  specifying --lz4 when running barman-cloud-wal-archive.

  References: BAR-265, BAR-423, BAR-264.

• Improve WAL upload performance on S3 buckets by avoiding multipart uploads

  Previously, WAL files were being uploaded to S3 buckets using multipart uploads
  provided by the boto3 library via the upload_fileobj method. It was noticed that
  multipart upload is slower when used for small files, such as WAL segments,
  compared to when uploading it in a single PUT request.
  This has been improved by avoiding multipart uploads for files smaller than 100MB.
  The average upload time of each WAL file is expected to be reduced by around 15%
  with this change.

  References: BAR-374.

• Modify behavior when enforcing retention policy for KEEP:STANDALONE full backups

  When enforcing the retention policy on full backups created with
  backup_method = postgres, Barman was previously marking all dependent (child)
  incremental backups as VALID, regardless of the KEEP target used. However, this
  approach is incorrect:

  • For backups labeled KEEP:STANDALONE, Barman only retains the WAL files needed to
    restore the server to the exact state of that backup. Because these backups are
    self-contained, any dependent child backups are no longer needed once the root
    backup is outside the retention policy.

  • In contrast, backups marked KEEP:FULL are intended for point-in-time recovery.
    To support this, Barman retains all WALs, as well as any child backups, to ensure
    the backup's consistency and allow recovery to the latest possible point.

  This distinction ensures that KEEP:STANDALONE backups serve as snapshots of a
  specific moment, while KEEP:FULL backups retain everything needed for full
  point-in-time recovery.

  References: BAR-366.

• Update documentation and user-facing features for Barman's recovery process.

  Barman docs and the tool itself used to use the terms "recover"/"recovery" both for
  referencing:

  • The Postgres recovery process;
  • The process of restoring a backup and preparing it for recovery.

  Both the code and documentation have been revised to accurately reflect the usage of
  the terms "restore" and "recover"/"recovery".

  Also, the barman recover command was renamed to barman restore. The old name is
  still kept as an alias for backward compatibility.

  References: BAR-337.

• Add --keep-compression flag to barman-wal-restore and get-wal

  A new --keep-compression option has been added to both barman-wal-restore and
  get-wal. This option controls whether compressed WAL files should be decompressed
  on the Barman server before being fetched. When specified with get-wal, default
  decompression is skipped, and the output is the WAL file content in its original
  state. When specified with barman-wal-restore, the WAL file is fetched as-is and,
  if compressed, decompressed on the client side.

  References: BAR-435.

• Ransomware protection - Add AWS Snapshot Lock Support

  Barman now supports AWS EBS Snapshot Lock, a new integrated feature to prevent
  accidental or malicious deletions of Amazon EBS snapshots. When a snapshot is
  locked, it can't be deleted by any user but remains fully accessible for use. This
  feature enables you to store snapshots in WORM (Write-Once-Read-Many) format for a
  specified duration, helping to meet regulatory requirements by keeping the data
  secure and tamper-proof until the lock expires.

  Special thanks to Rui Marinho, our community contributor who started this feature.

  References: BAR-242.

• Prevent orphan files from being left from a crash while deleting a backup

  This commit fixes an issue where backups could leave behind files if the system
  crashed during the deletion of a backup.

  Now, when a backup is deleted, it will get a "delete marker" at the start.
  If a crash happens while the backup is being deleted, the marker will help
  recognize incomplete backup removals when the server restarts.

  The Barman cron job has been updated to look for these deleted markers. If it finds
  a backup with a "delete marker", it will complete the process.

  References: BAR-244.

• Add support for using tags with snapshots

  Barman now supports tagging the snapshots when creating backups using the
  barman-cloud-backup script command. A new argument called --tags was added.

  Special thanks to Rui Marinho, our community contributor who started this feature.

  References: BAR-417.

• Use ISO format instead of ctime when producing JSON output of Barman cloud commands

  The ctime format has no information about the time zone associated with the timestamp.
  Besides that, that format is better suited for human consumption. For machine
  consumption the ISO format is better suited.

  References: BAR-316.

Bugfixes

• Fix barman check which returns wrong results for Replication Slot

  Previously, when using architectures which backup from a standby node and stream WALs
  from the primary, Barman would incorrectly use conninfo (pointing to a standby server)
  for replication checks, leading to errors such as:

  replication slot (WAL streaming): FAILED (replication slot 'barman' doesn't exist. Please execute 'barman receive-wal --create-slot pg17')

  This fixes the following issue
  #1024 by ensuring
  wal_conninfo is used for WAL replication checks if it's set.

  wal_conninfo takes precedence over wal_streaming_conninfo, when both are set.
  With this change, if only wal_conninfo is set, it will be used and will not fall
  back to conninfo.

  Also, in the documentation, changes were made so it is explicit that when conninfo
  points to a standby server, wal_conninfo must be set and used for accurate
  replication status checks.

  References: BAR-409.

• Fix missing options for barman keep

  The error message that the Barman CLI emitted when running barman keep
  without any options suggested there were shortcut aliases for status and
  release. These aliases, -s and -r, do not exist, so the error message was
  misleading.
  This fixes the issue by including these short options in the Barman CLI,
  aligning it with other tools like barman-cloud-backup-keep, where these
  shortcuts already exist.

  References: BAR-356.

• Lighten standby checks related to conninfo and primary_conninfo

  When backing up a standby server, Barman performs some checks to assert
  that conninfo is really pointing to a standby (in recovery mode) and
  that primary_conninfo is pointing to a primary (not in recovery).

  The problem, as reported in the issues #704 and #744, is that when a
  failover occurs, the conninfo will now be pointing to a primary
  instead and the checks will start failing, requiring the user to change
  Barman configs manually whenever a failover occurs.

  This fix solved the issue by making such checks non-critical, which
  means they will still fail but Barman will keep operating regardless.
  Essen...

release/3.11.1

Version 3.11.1 - 22 August 2024

Bug fixes

• Fix failures in barman-cloud-backup-delete. This command was failing when
  applying retention policies due to a bug introduced by the previous release.

release/3.11.0

Version 3.11.0 - 22 August 2024

Features

• Add support for Postgres 17+ incremental backups. This major feature is
  composed of several small changes:

  • Add --incremental command-line option to barman backup command. This is
    used to specify the parent backup when taking an incremental backup. The
    parent can be either a full backup or another incremental backup.

  • Add latest-full shortcut backup ID. Along with latest, this can be used
    as a shortcut to select the parent backup for an incremental backup. While
    latest takes the latest backup independently if it is full or incremental,
    latest-full takes the latest full backup.

  • barman keep command can only be applied to full backups when
    backup_method = postgres. If a full backup has incremental backups that
    depend on it, all of the incrementals are also kept by Barman.

  • When deleting a backup all the incremental backups depending on it, if any,
    are also removed.

  • Retention policies do not take incremental backups into consideration. As
    incremental backups cannot be recovered without having the complete chain of
    backups available up to the full backup, only full backups account for
    retention policies.

  • barman recover needs to combine the full backup with the chain of incremental
    backups when recovering. The new CLI option --local-staging-path, and the
    corresponding local_staging_path configuration option, are used to specify
    the path in the Barman host where the backups will be combined when recovering
    an incremental backup.

• Changes to barman show-backup output:

  • Add the “Estimated cluster size” field. It's useful to have an estimation
    of the data directory size of a cluster when restoring a backup. It’s
    particularly useful when recovering compressed backups or incremental
    backups, situations where the size of the backup doesn’t reflect the size of the
    data directory in Postgres. In JSON format, this is stored as
    cluster_size.

  • Add the “WAL summarizer” field. This field shows if summarize_wal was
    enabled in Postgres at the time the backup was taken. In JSON format, this
    is stored as server_information.summarize_wal. This field is omitted for
    Postgres 16 and older.

  • Add “Data checksums” field. This shows if data_checkums was enabled in
    Postgres at the time the backup was taken. In JSON format, this is stored as
    server_information.data_checksums.

  • Add the “Backup method” field. This shows the backup method used for this
    backup. In JSON format, this is stored as
    base_backup_information.backup_method.

  • Rename the field “Disk Usage” as “Backup Size”. The latter provides a more
    comprehensive name which represents the size of the backup in the Barman
    host. The JSON field under base_backup_information was also renamed from
    disk_usage to backup_size.

  • Add the “WAL size” field. This shows the size of the WALs required by the
    backup. In JSON format, this is stored as
    base_backup_information.wal_size.

  • Refactor the field “Incremental size”. It is now named “Resources saving”
    and it now shows an estimation of resources saved when taking incremental
    backups with rsync or pg_basebackup. It compares the backup size with
    the estimated cluster size to estimate the amount of disk and network
    resources that were saved by taking an incremental backup. In JSON format,
    the field was renamed from incremental_size to resource_savings under base_backup_information.

  • Add the system_id field to the JSON document. This field contains the
    system identifier of Postgres. It was present in console format, but was
    missing in JSON format.

  • Add fields related with Postgres incremental backups:

    • “Backup type”: indicates if the Postgres backup is full or incremental. In
      JSON format, this is stored as backup_type under base_backup_information.

    • “Root backup”: the ID of the full backup that is the root of a chain of
      one or more incremental backups. In JSON format, this is stored as
      catalog_information.root_backup_id.

    • “Parent backup”: the ID of the full or incremental backup from which this
      incremental backup was taken. In JSON format, this is stored as
      catalog_information.parent_backup_id.

    • “Children Backup(s)”: the IDs of the incremental backups that were taken
      with this backup as the parent. In JSON format, this is stored as
      catalog_information.children_backup_ids.

    • “Backup chain size”: the number of backups in the chain from this
      incremental backup up to the root backup. In JSON format, this is
      stored as catalog_information.chain_size.

• Changes to barman list-backup output:

  • It now includes the backup type in the JSON output, which can be either
    rsync for backups taken with rsync, full or incremental for backups
    taken with pg_basebackup, or snapshot for cloud snapshots. When printing
    to the console the backup type is represented by the corresponding labels
    R, F, I or S.

  • Remove tablespaces information from the output. That was bloating the
    output. Tablespaces information can still be found in the output of
    barman show-backup.

• Always set a timestamp with a time zone when configuring
  recovery_target_time through barman recover. Previously, if no time zone
  was explicitly set through --target-time, Barman would configure
  recovery_target_time without a time zone in Postgres. Without a time zone,
  Postgres would assume whatever is configured through timezone GUC in
  Postgres. From now on Barman will issue a warning and configure
  recovery_target_time with the time zone of the Barman host if no time zone
  is set by the user through --target-time option.

• When recovering a backup with the “no get wal” approach and --target-lsn is set,
  copy only the WAL files required to reach the configured target. Previously
  Barman would copy all the WAL files from its archive to Postgres.

• When recovering a backup with the “no get wal” approach and --target-immediate
  is set, copy only the WAL files required to reach the consistent point.
  Previously Barman would copy all the WAL files from its archive to Postgres.

• barman-wal-restore now moves WALs from the spool directory to pg_wal
  instead of copying them. This can improve performance if the spool directory
  and the pg_wal directory are in the same partition.

• barman check-backup now shows the reason why a backup was marked as FAILED
  in the output and logs. Previously for a user to know why the backup was
  marked as FAILED, they would need to run barman show-backup command.

• Add configuration option aws_await_snapshots_timeout and the corresponding
  --aws-await-snapshots-timeout command-line option on barman-cloud-backup.
  This specifies the timeout in seconds to wait for snapshot backups to reach
  the completed state.

• Add a keep-alive mechanism to rsync-based backups. Previously the Postgres
  session created by Barman to run pg_backup_start() and pg_backup_stop() would
  stay idle for as long as the base backup copy would take. That could lead to a
  firewall or router dropping the connection because it was idle for a long
  time. The keep-alive mechanism sends heartbeat queries to Postgres
  through that connection, thus reducing the likelihood of a connection
  getting dropped. The interval between heartbeats can be controlled through the new
  configuration option keepalive_interval and the corresponding CLI
  option --keepalive-interval of the barman backup command.

Bug fixes

• When recovering a backup with the “no get wal” approach and --target-time
  set, copy all WAL files. Previously Barman would attempt to “guess” the WAL
  files required by Postgres to reach the configured target time. However,
  the mechanism was not robust enough as it was based on the stats of the WAL
  file in the Barman host (more specifically the creation time). For example:
  if there were archiving or streaming lag between Postgres and Barman, that
  could be enough for recovery to fail because Barman would miss to copy all
  the required WAL files due to the weak check based on file stats.

• Pin python-snappy to 0.6.1 when running Barman through Python 3.6 or
  older. Newer versions of python-snappy require cramjam version 2.7.0 or
  newer, and these are only available for Python 3.7 or newer.

• barman receive-wal now exits with code 1 instead of 0 in the following
  cases:

  • Being unable to run with --reset flag because pg_receivewal is
    running.

  • Being unable to start pg_receivewal process because it is already
    running.

• Fix and improve information about Python in barman diagnose output:

  • The command now makes sure to use the same Python interpreter under which
    Barman is installed when outputting the Python version through
    python_ver JSON key. Previously, if an environment had multiple Python
    installations and/or virtual environments, the output could eventually be
    misleading, as it could be fetched from a different Python interpreter.

  • Added a python_executable key to the JSON output. That contains the path
    to the exact Python interpreter being used by Barman.

Barman 3.10.1

Version 3.10.1 - 12 June 2024

Bug fixes

• Make argcomplete optional to avoid installation issues on some
  platforms.
• Load barman.auto.conf only when the file exists.
• Emit a warning when the cfg_changes.queue file is malformed.
• Correct in documentation the postgresql version where
  pg_checkpoint is available.
• Add --no-partial option to barman-cloud-wal-restore.

release/3.10.0

Version 3.10.0 - 24 January 2024

Features

• Limit the average bandwidth used by barman-cloud-backup when backing
  up to either AWS S3 or Azure Blob Storage according to the value set by
  a new CLI option --max-bandwidth.
• Add the new configuration option lock_directory_cleanup
  That enables cron to automatically clean up the barman_lock_directory
  from unused lock files.
• Add support for a new type of configuration called model.
  The model acts as a set of overrides for configuration options
  for a given Barman server.
• Add a new barman command barman config-update that allows the creation
  and the update of configurations using JSON

Bug fixes:

• Fix a bug that caused --min-chunk-size to be ignored when using
  barman-cloud-backup as hook script in Barman.

Barman 3.9.0

Version 3.9.0 - 3 October 2023

Features

• Allow barman switch-wal --force to be run against PG>=14 if the
  user has the pg_checkpoint role (thanks to toydarian for this patch).

• Log the current check at info level when a check timeout occurs.

• The minimum size of an upload chunk when using barman-cloud-backup
  with either S3 or Azure Blob Storage can now be specified using the
  --min-chunk-size option.

• backup_compression = none is supported when using pg_basebackup.

• For PostgreSQL 15 and later: the allowed backup_compression_level
  values for zstd and lz4 have been updated to match those allowed by
  pg_basebackup.

• For PostgreSQL versions earlier than 15: backup_compression_level = 0
  can now be used with backup_compression = gzip.

Bug fixes:

• Fix barman recover on platforms where Multiprocessing uses spawn by
  default when starting new processes.

Barman 3.8.0

Version 3.8.0 - 31 August 2023

Features

• Clarify package installation. barman is packaged with default python version
  for each operating system.

• The minimum-redundancy option is added to barman-cloud-backup-delete.
  It allows to set the minimum number of backups that should always be available.

• Add a new primary_checkpoint_timeout configuration option. Allows define
  the amount of seconds that Barman will wait at the end of a backup if no
  new WAL files are produced, before forcing a checkpoint on the primary server.

Bug fixes:

• Fix race condition in barman retention policies application. Backup
  deletions will now raise a warning if another deletion is in progress
  for the requested backup.

• Fix barman-cloud-backup-show man page installation.

Barman 3.7.0

Version 3.7.0 - 25 July 2023

Features

• Support is added for snapshot backups on AWS using EBS volumes.

• The --profile option in the barman-cloud-* scripts is renamed
  --aws-profile. The old name is deprecated and will be removed in
  a future release.

• Backup manifests can now be generated automatically on completion
  of a backup made with backup_method = rsync. This is enabled by
  setting the autogenerate_manifest configuration variable and can
  be overridden using the --manifest and --no-manifest CLI options.

Bug fixes:

• The barman-cloud-* scripts now correctly use continuation
  tokens to page through objects in AWS S3-compatible object
  stores. This fixes a bug where barman-cloud-backup-delete
  would only delete the oldest 1000 eligible WALs after backup
  deletion.

• Minor documentation fixes.

Barman 3.6.0

Version 3.6.0 - 15 June 2023

Features

• PostgreSQL version 10 is no longer supported.

• Support is added for snapshot backups on Microsoft Azure using
  Managed Disks.

• The --snapshot-recovery-zone option is renamed --gcp-zone for
  consistency with other provider-specific options. The old name
  is deprecated and will be removed in a future release.

• The snapshot_zone option and --snapshot-zone argument are
  renamed gcp_zone and --gcp-zone respectively. The old names
  are deprecated and will be removed in a future release.

• The snapshot_gcp_project option and --snapshot-gcp-project
  argument are renamed to gcp_project and --gcp-project. The
  old names are deprecated and will be removed in a future release.

Bug fixes:

• Barman will no longer attempt to execute the replication-status
  command for a passive node.

• The backup_label is deleted from cloud storage when a
  snapshot backup is deleted with barman-cloud-backup-delete.

• Man pages for the generate-manifest and verify-backup
  commands are added.

• Minor documentation fixes.