Release Notes

Release Notes

Release Date: August 11, 2021

Tanzu Greenplum Backup and Restore utilities are released separately from Tanzu Greenplum Database (from Backup and Restore v1.13.0), and are updated independently of the core server.

Version 1.21.0

Tanzu Greenplum Backup and Restore version 1.21.0 is a minor release.

Platform Support

Tanzu Greenplum Backup and Restore is compatible with these Greenplum Database versions:

  • Tanzu Greenplum Database 4.3.22 and later
  • Tanzu Greenplum Database 5.5.0 and later
  • Tanzu Greenplum Database 6.0.0 and later

Software Component Versions

This release includes the following utilities:

See Data Domain Boost for Data Domain Boost support information.

See Known Issues for a list of issues known to exist in this release.

See Differences Compared to Open Source Greenplum Backup and Restore for the list of features that are only in Tanzu Greenplum Backup and Restore.

See Release Numbering Conventions for a description of the Tanzu Greenplum Backup and Restore release numbering scheme.

gpbackup v1.21.0

Resolved Issues

  • [178238581] Resolved an issue where exclusion of tables was failing when the table or schema name included a reserved word.
  • [178478743] Resolved an issue where gpbackup would attempt to acquire an access share lock on external and foreign tables when the --jobs option was specified, which led to the backup operation failing.

gpbackup_helper v1.21.0

No changes since Tanzu Greenplum Backup and Restore version 1.14.0.

gprestore v1.21.0

New Features

  • The gprestore inclusive filtering options --include-schema and --include-schema-file can now be used with the --redirect-schema filtering option.
  • The --on-error-continue flag now correctly reports that a gprestore step completed with failures.

Resolved Issues

  • [494] Resolved an issue where using the --jobs option while restoring multiple indexes could result in an error.
  • [177736931] The xmloption session GUC now defaults to content to ensure that XML document data is reliably restored with a gprestore operation.
  • [176646873] Resolved an issue where gprestore invoked with the --on-error-continue option on a single data file backup would very occasionally block directly after a table data restore error when restoring on Tanzu Greenplum Database 6.
  • [177605133] Resolved an incompatibility – introduced in gprestore 1.18.0 – between the --redirect-schema and --metadata-only options.
  • [178294679] Resolved an issue where gpprestore's exclusion filters failed to quote excluded tables.

gpbackup_manager v1.4.0

New Feature

  • In the output of the gpbackup_manager delete command, the date deleted column now provides additional information, including whether the deletion is in progress and whether the deletion failed.
  • The delete command has been improved to minimize the chances of partial delete states.

Resolved Issues

  • [31378] Resolved an issue where gpbackup_manager was terminating with a segmentation fault during replication.
  • [169686303] The backup delete procedure now checks whether a backup has already been deleted before proceding.
  • [178150185] Resolved an issue where when the SQL password encryption query failed the query containing the user-provided password was logged to the Greenplum Database log.

Backup/Restore Plugin API

No changes since Tanzu Greenplum Backup and Restore version 1.19.0.

gpbackup_ddboost_plugin v1.6.0

New Features

  • The DDBoost plugin now handles unknown configuration option names in plugin configuration files by issuing the error message, Plugin validation failed. Unknown option <option name> found in configuration file.
  • The DDBoost plugin now has enhanced replication thread handling to prevent users from running out of threads.
  • The DDBoost plugin now performs additional logging to better show runtime progression.
  • DDBoost errors are now logged separately in a new log file specific to the DDBoost plugin.
  • Before replicating DDBoost-based backups, gpbackup checks for preexisting data files with the same name.
  • The DDBoost plugin now uses DDBoost SDK version 7.4, which is compatible with DDOS versions 6.2, 7.0, 7.1, 7.2, 7.4, 7.5 and the next seven future releases of DDOS.
  • DDBoost error codes are now converted to human-readable messages.

Resolved Issues

  • [178117291] Resolved an issue where the DDBoost plugin was only decrypting Data Domain remote passwords if replication was set to on.
  • [178123364] Added proper error handling for the case where the DDBoost plugin was segment faulting if the local file path being backed up contained fewer than 3 forward slashes.
  • [178150185] Resolved an issue where, if a SQL password decryption query failed, both the query and the decryption key were printed to the Greenplum Database log.

gpbackup_s3_plugin v1.7.0

New Features

  • The S3 plugin now handles unknown configuration option names in plugin configuration files by issuing the error message, Plugin validation failed. Unknown option <option name> found in configuration file.
  • Cloudian HyperStore has been certified with the gpbackup S3 storage plugin.
  • The S3 plugin now supports connections to Amazon S3 over IPv6 networks.

Changed Features

  • The default value for backup_multipart_chunksize and restore_multipart_chunksize has been increased from 10MB to 500MB.

Resolved Issues

  • [31349] Resolved an issue where backups to S3 with multipart uploads were failing.

Data Domain Boost

This release of gpbackup and gprestore supports Data Domain Boost for backup on Red Hat Enterprise Linux. This table lists the supported versions of DDOS, Data Domain Boost SDK, and BoostFS.

Table 1. Data Domain Boost Compatibility
DDOS Data Domain Boost BoostFS
7.5 7.5 7.5
7.4 7.4 7.4
7.2 7.2 7.2
7.1 7.1 7.0
6.2 3.5 1.3
6.1 (all versions) 3.4 1.1
6.0 (all versions) 3.3 n/a
Note: In addition to the DDOS versions listed in the previous table, gpbackup and gprestore support all minor patch releases (fourth digit releases) later than the certified version.

Known Issues

  • When attempting to restore data from Greenplum 4.3.x/5.x/6.x (prior to Greenplum v6.14.1 and v5.28.6), views with anyarray typecasts are not restorable. From Tanzu Greenplum Backup and Restore v1.20.3, gpbackup generates an error when such views are detected.

    Workaround: On the original cluster, identify any views that exhibit this symptom, using a query similar to:

    SELECT relname AS anyarray_views 
                      FROM pg_class WHERE relkind = 'v' AND oid >= 16384 AND (pg_get_viewdef(oid) 
                      LIKE '%::anyarray%') IS TRUE; 

    Then select option 1 or 2:

    1. Using the backup set with the errors, run gprestore with the --on-error-continue flag. The affected views will not be re-created. After the restore completes, re-create each VIEW using your original VIEW definition.
    2. Drop each affected VIEW on the original Greenplum cluster. Take another backup of the older cluster and run gprestore on the new cluster. Finally, recreate the VIEW on the new cluster using the original VIEW definition.
  • When attempting to restore data from Greenplum 4.3.x/5.x into Greenplum 6.x, if the restore involved a table distributed by character(n) or char(n) using the legacy bpchar hash operator, the restore would fail. This issue has been resolved in Greenplum Database 6.12. Upgrade to the latest Greenplum release to avoid this issue.
  • gprestore does not support restoring a backup that contains partitioned tables where the table is created with a non-reserved keyword that is used as a partition name. For example, the non-reserved keyword at is used as a partition name in this SUBPARTITION TEMPLATE clause fragment:
    ... 
                  SUBPARTITION TEMPLATE
                  ( SUBPARTITION "at" VALUES ('usa'),
                  ...

    gpbackup backs up the partitioned table, but gprestore returns an error when attempting to restore the table.

    Before performing a backup with gpbackup, you must ensure that partitioned tables do not use any of these non-reserved keywords as a partition name:

    ADD, ALTER, ALWAYS, AT, ATTRIBUTE, CATALOG, COMMENTS, CONFIGURATION, CONFLICT, CONTINUE, CURRENT, DATA, DAY, DENY, DEPENDS, DICTIONARY, DISCARD, DOCUMENT, DXL, EVENT, EXTENSION, FAMILY, FILESPACE, FILTER, FULLSCAN, FUNCTIONS, HOUR, IDENTITY, IGNORE, IMPORT, INITPLAN, INLINE, LABEL, LEAKPROOF, LOCKED, LOGGED, MAPPING, MATERIALIZED, METHOD, MINUTE, MONTH, NOCREATEEXTTABLE, OFF, ORDERED, ORDINALITY, OVER, PARALLEL, PARSER, PASSING, PLANS, POLICY, PROGRAM, RANDOMLY, READABLE, READS, RECURSIVE, REF, REFRESH, REJECT, REPLICATED, ROOTPARTITION, SECOND, SEQUENCES, SERVER, SKIP, SNAPSHOT, SQL, STANDALONE, STRIP, TABLES, TEXT, TRANSFORM, TYPES, UNLOGGED, VALIDATION, VARYING, VIEWS, WEB, WHITESPACE, WITHIN, WITHOUT, WRAPPER, WRITABLE, XML, YEAR, YES

  • A gprestore operation using the --redirect-schema option fails if gprestore attempts to restore an index in a schema and the name of the index is the name of the schema followed by a '.' (period). For example, this CREATE INDEX command creates the index named test. on a table in the schema test. The index is in the schema test.

    CREATE INDEX "test." ON test.mytbl USING btree (i);
    If the index and table are backed up with gpbackup, restoring the backup with this gprestore command fails because gprestore fails to restore the test. index.
    gprestore --timestamp <timestamp> --redirect-schema foo2
  • Beginning with versions 4.3.33 and 5.19, Greenplum Database checks that the distribution key for a table is a prefix of any unique index key. This policy is not in place for Greenplum Database versions before 4.3.33 or 5.19, and it is possible to create a backup of a database from one of these earlier versions with unique indexes that do not comply with the policy. When you restore such a backup to a Greenplum Database version that does enforce the policy:
    • If the unique index is for a primary key constraint, Greenplum Database automatically modifies the table's distribution policy if the table has no data.
    • In other cases, creating a unique index with a key that does not begin with the table's distribution key fails.

    This issue affects restoring backups made from Greenplum Database 4.3.32.0, 5.18.0, or earlier to a Greenplum Database version 4.3.33.0, 5.19.0, 6.0, or later system. The issue affects the gprestore, gpdbrestore, and pg_restore utilities.

Differences Compared to Open Source Greenplum Backup and Restore

Tanzu Greenplum Backup and Restore includes all of the functionality in the open source Greenplum Backup github repository and S3 storage plugin repository and adds:

  • Greenplum backup plugin for DD Boost
  • Greenplum gpbackup_manager utility
  • Greenplum Backup and Restore installation file in gppkg format

Release Numbering Conventions

The Tanzu Greenplum Backup and Restore distribution release number indicates the type of the release.

  • The first number is the gpbackup/gprestore major release number. For example, given the release number, 1.16.0, the major release number is 1.
  • The second number is the minor release number. Given the release number 1.16.0, the minor release number is 16. This number increments when new features are added to the gpbackup/gprestore utilities.
  • The third number is the maintenance release number. This number increments when the gpbackup/gprestore utilities included in the distribution contain fixes without new features, or if one or more components included in the distribution, such as gpbackup_manager and the backup storage plugins, have been updated.

The release versions of the components included in the distribution, such as gpbackup_manager and the backup storage plugins, are separate from the distribution version, but follow the same numbering scheme as the distribution.