VMware Tanzu Greenplum 6.14 Release Notes

A newer version of this documentation is available. Use the version menu above to view the most up-to-date release of the Greenplum 6.x documentation.

VMware Tanzu Greenplum 6.14 Release Notes

This document contains pertinent release information about VMware Tanzu Greenplum Database 6.14 releases. For previous versions of the release notes for Greenplum Database, go to VMware Tanzu Greenplum Database Documentation. For information about Greenplum Database end of life, see VMware Tanzu Greenplum Database end of life policy.

VMware Tanzu Greenplum 6 software is available for download from the VMware Tanzu Greenplum page on VMware Tanzu Network.

VMware Tanzu Greenplum 6 is based on the open source Greenplum Database project code.

Release 6.14.1

Release Date: 2021-2-22

VMware Tanzu Greenplum 6.14.1 is a maintenance release that resolves several issues and includes related changes

Changed Features

Greenplum Database 6.14.1 includes this change:

  • PostGIS version 2.5.4+pivotal.4 is included, which resolves the segfault described in PostGIS ticket #4691.

Resolved Issues

VMware Tanzu Greenplum 6.14.1 resolves these issues:

31258 - Server
Resolves an issue where the array typecasts of operands in view definitions with operators were erroneously transformed into anyarray typecasts. This caused errors in the backup and restore of the view definitions.
31249 - Query Optimizer
Resolves an issue where Greenplum Database generated a PANIC during query execution when the Query Optimizer attempted to access the argument of a function (such as random() or timeofday()), but the query did not invoke the function with an argument.
31242 - Server
Optimized locking to resolve an issue with certain SELECT queries on the pg_partitions system view, which were waiting on locks taken by other operations.
31232 - Server
Resolves an issue where, after an upgrade from version 5.28 to 6.12, a query execution involving external tables resulted in a query PANIC and segment failover. This issue has been resolved by optimizing the query subplans.
31211 - gpfdist
When an external table was configured with a transform, gpfdist would sporadically return the error 404 Multiple reader to a pipe is forbidden. This issue is resolved.
176684985 - Query Optimizer
This release improves Greenplum Database’s performance for joins with multiple join predicates.

Release 6.14.0

Release Date: 2021-2-5

VMware Tanzu Greenplum 6.14.0 is a minor release that includes changed features and resolves several issues.

Features

Greenplum Database 6.14.0 includes these new and changed features:

  • CentOS/RHEL 8 and SUSE Linux Enterprise Server x86_64 12 (SLES 12) Clients packages are available with this Greenplum Database release; you can download them from the Release Download directory named Greenplum Clients on VMware Tanzu Network.
  • The PXF version 5.16.1 distribution is available with this release; you can download it from the Release Download directory named Greenplum Platform Extension Framework on VMware Tanzu Network.
  • The default value of the optimizer_join_order server configuration parameter is changed from exhaustive to exhaustive2. The Greenplum Database Query Optimizer (GPORCA) uses this configuration parameter to identify the join enumeration algorithm for a query. With this new default, GPORCA operates with an emphasis on generating join orders that are suitable for dynamic partition elimination. This often results in faster optimization times and/or better execution plans, especially when GPORCA evaluates large joins. The Faster Optimization of Join Queries in ORCA blog provides additional information about this feature.
  • The default cost model for the optimizer_cost_model server configuration parameter, calibrated, has been enhanced; GPORCA is now more likely to choose a faster bitmap index with nested loop joins rather than hash joins.
  • GPORCA boosts query execution performance by improving its partition selection algorithm to more often eliminate the default partition.
  • GPORCA now generates a plan alternative for a right outer join transform from a left outer join when equivalent. GPORCA's cost model determines if/when to pick this alternative; using such a plan can greatly improve query execution performance by introducing partition selectors that reduce the number of partitions scanned.
  • The output of the gprecoverseg -a -s command has been updated to show more verbose progress information. Users can now monitor the progress of the recovering segments in incremental mode.
  • The gpcheckperf command has been updated to support Internet Protocol version 6 (IPv6).

Resolved Issues

VMware Tanzu Greenplum 6.14.0 resolves these issues:

31195 - Server: Execution
Resolves an issue where Greenplum Database generated a PANIC when the pg_get_viewdef_name_ext() function was invoked with a non-view relation.
31094 - Server: Execution
Resolves an issue where a query terminated abormally with the error Context should be init first when gp_workfile_compression=on because Greenplum Database ignored a failing return value from a ZSTD initialization function.
31067 - Query Optimizer
Resolves a performance issue where GPORCA did not consistently eliminate the default partition when the filter condition in a query matched more than a single partition. GPORCA has improved its partition selection algorithm for predicates that contain only disjunctions of equal comparisons where one side is the partition key by categorizing these comparisions as equal filters.
31062 - Cluster Management
Resolves a documentation and --help output issue for the gprecoverseg, gpaddmirrors, gpmovemirrors, gpinitstandby utilities, where the --hba-hostnames command line flag details were missing.
31044 - Query Optimizer
Fixes a plan optimizer issue where the query would fail due to the planning time being dominated by the sort process of irrelevant indexes.
30974 - Server: Execution
Greenplum Database generated a PANIC when a query run in a utility mode connection invoked the gp_toolkit.gp_param_setting() function. This issue is resolved; Greenplum now ignores a function's EXECUTE ON options when in utility mode, and executes the function only on the local node.
30950 - Query Optimizer
Resolves an issue where GPORCA did not use dynamic partition elimination and spent a long time planning a query that included a mix of unions, outer joins, and subqueries. GPORCA now caches certain object pointers to avoid repeated metadata lookups, substantially decreasing planning time for such queries when optimizer_join_order is set to query or exhaustive2.
30947 - Query Optimizer
Resolves an issue where Greenplum Database returned the error no hash_seq_search scan for hash table "Dynamic Table Scan Pid Index" because GPORCA generated a query plan that incorrectly rescanned a partition selector during dynamic partition elimination. GPORCA now generates a plan that does not demand such a rescan.
11211 - Server
During the parallel recovery and rebalance of segment nodes after a failure, if an error occurred during segment resynchronization, the main recovery process would halt and wait indefinitely. This issue has been fixed.
11058 - Query Optimizer
Resolves an optimizer issue where CTE queries with a RETURNING clause would fail with the error INSERT/UPDATE/DELETE must be executed by a writer segworker group.
174873438 - Planner
Resolves an issue where an index scan generated for a query involving a system table and a replicated table could return incorrect results. Greenplum no longer generates the index scan in this situation.

Upgrading from Greenplum 6.x to Greenplum 6.14

Note: Greenplum 6 supports direct upgrades, using gpupgrade, from Greenplum 5.x releases to Greenplum 6.x. For more information, see the gpupgrade documentation. See also Migrating Data from Greenplum 4.3 or 5 for guidelines and considerations for migrating existing Greenplum data to Greenplum 6, using standard backup and restore procedures.

See Upgrading from an Earlier Greenplum 6 Release to upgrade your existing Greenplum 6.x software to Greenplum 6.14.

Deprecated Features

Deprecated features will be removed in a future major release of Greenplum Database. VMware Tanzu Greenplum 6.x deprecates:

  • The gpsys1 utility.
  • The analzyedb option --skip_root_stats (deprecated since 6.2).

    If the option is specified, a warning is issued stating that the option will be ignored.

  • The server configuration parameter gp_statistics_use_fkeys (deprecated since 6.2).
  • The server configuration parameter gp_ignore_error_table (deprecated since 6.0).

    To avoid a Greenplum Database syntax error, set the value of this parameter to true when you run applications that execute CREATE EXTERNAL TABLE or COPY commands that include the now removed Greenplum Database 4.3.x INTO ERROR TABLE clause.

  • Specifying => as an operator name in the CREATE OPERATOR command (deprecated since 6.0).
  • The Greenplum external table C API (deprecated since 6.0).

    Any developers using this API are encouraged to use the new Foreign Data Wrapper API in its place.

  • Commas placed between a SUBPARTITION TEMPLATE clause and its corresponding SUBPARTITION BY clause, and between consecutive SUBPARTITION BY clauses in a CREATE TABLE command (deprecated since 6.0).

    Using this undocumented syntax will generate a deprecation warning message.

  • The timestamp format YYYYMMDDHH24MISS (deprecated since 6.0).

    This format could not be parsed unambiguously in previous Greenplum Database releases, and is not supported in PostgreSQL 9.4.

  • The createlang and droplang utilities (deprecated since 6.0).
  • The pg_resqueue_status system view (deprecated since 6.0).

    Use the gp_toolkit.gp_resqueue_status view instead.

  • The GLOBAL and LOCAL modifiers when creating a temporary table with the CREATE TABLE and CREATE TABLE AS commands (deprecated since 6.0).

    These keywords are present for SQL standard compatibility, but have no effect in Greenplum Database.

  • Using WITH OIDS or oids=TRUE to assign an OID system column when creating or altering a table (deprecated since 6.0).
  • Allowing superusers to specify the SQL_ASCII encoding regardless of the locale settings (deprecated since 6.0).

    This choice may result in misbehavior of character-string functions when data that is not encoding-compatible with the locale is stored in the database.

  • The @@@ text search operator (deprecated since 6.0).

    This operator is currently a synonym for the @@ operator.

  • The unparenthesized syntax for option lists in the VACUUM command (deprecated since 6.0).

    This syntax requires that the options to the command be specified in a specific order.

  • The plain pgbouncer authentication type (auth_type = plain) (deprecated since 4.x).

Migrating Data to Greenplum 6

Note: Greenplum 6 supports direct upgrades, using gpupgrade, from Greenplum 5.x releases to Greenplum 6.x. For more information, see the gpupgrade documentation.

See Migrating Data from Greenplum 4.3 or 5 for guidelines and considerations for migrating existing Greenplum data to Greenplum 6, using standard backup and restore procedures.

Known Issues and Limitations

VMware Tanzu Greenplum 6 has these limitations:

  • Upgrading a Greenplum Database 4 or 5 release, or Greenplum 6 Beta release, to VMware Tanzu Greenplum 6 is not supported.
  • MADlib, GPText, and PostGIS are not yet provided for installation on Ubuntu systems.
  • Greenplum for Kubernetes is not yet provided with this release.

The following table lists key known issues in VMware Tanzu Greenplum 6.x.

Table 1. Key Known Issues in VMware Tanzu Greenplum 6.x
Issue Category Description
31010 Query Optimizer A view created in Greenplum Database 5.28.3 or older that specified an external table in the FROM clause, and that was migrated to Greenplum Database 6.x, always falls back to the Postgres Planner when queried.

Workaround: If you migrated a view from Greenplum Database 5.28.3 or earlier, and the view specified an external table in the FROM clause, you must drop and recreate the view in Greenplum 6.x to ensure that the Query Optimizer is exercised when you query the view.
N/A Backup/Restore Restoring the Greenplum Database backup for a table fails in Greenplum 6 versions earlier than version 6.10 when a replicated table has an inheritance relationship to/from another table that was assigned via an ALTER TABLE ... INHERIT statement after table creation.
Workaround: Use the following SQL commands to determine if Greenplum Database includes any replicated tables that inherit from a parent table, or if there are replicated tables that are inherited by a child table: 
SELECT inhrelid::regclass FROM pg_inherits,
  gp_distribution_policy dp
WHERE inhrelid=dp.localoid AND dp.policytype='r';
SELECT inhparent::regclass FROM pg_inherits,
  gp_distribution_policy dp
WHERE inhparent=dp.localoid AND dp.policytype='r';

If these queries return any tables, you may choose to run gprestore with the -–on-error-continue flag to not fail the entire restore when this issue is hit. Or, you can specify the list of tables returned by the queries to the -–exclude-table-file option to skip those tables during restore. You must recreate and repopulate the affected tables after restore.
N/A Spark Connector This version of Greenplum is not compatible with Greenplum-Spark Connector versions earlier than version 1.7.0, due to a change in how Greenplum handles distributed transaction IDs.
N/A PXF Starting in 6.x, Greenplum does not bundle cURL and instead loads the system-provided library. PXF requires cURL version 7.29.0 or newer. The officially-supported cURL for the CentOS 6.x and Red Hat Enterprise Linux 6.x operating systems is version 7.19.*. Greenplum Database 6 does not support running PXF on CentOS 6.x or RHEL 6.x due to this limitation.

Workaround: Upgrade the operating system of your Greenplum Database 6 hosts to CentOS 7+ or RHEL 7+, which provides a cURL version suitable to run PXF.
29703 Loading Data from External Tables Due to limitations in the Greenplum Database external table framework, Greenplum Database cannot log the following types of errors that it encounters while loading data:
  • data type parsing errors
  • unexpected value type errors
  • data type conversion errors
  • errors returned by native and user-defined functions
LOG ERRORS returns error information for data exceptions only. When it encounters a parsing error, Greenplum terminates the load job, but it cannot log and propagate the error back to the user via gp_read_error_log().

Workaround: Clean the input data before loading it into Greenplum Database.
30594 Resource Management Resource queue-related statistics may be inaccurate in certain cases. VMware recommends that you use the resource group resource management scheme that is available in Greenplum 6.
30522 Logging Greenplum Database may write a FATAL message to the standby master or mirror log stating that the database system is in recovery mode when the instance is synchronizing with the master and Greenplum attempts to contact it before the operation completes. Ignore these messages and use gpstate -f output to determine if the standby successfully synchronized with the Greenplum master; the command returns Sync state: sync if it is synchronized.
30537 Postgres Planner The Postgres Planner generates a very large query plan that causes out of memory issues for the following type of CTE (common table expression) query: the WITH clause of the CTE contains a partitioned table with a large number partitions, and the WITH reference is used in a subquery that joins another partitioned table.

Workaround: If possible, use the GPORCA query optimizer. With the server configuration parameter optimizer=on, Greenplum Database attempts to use GPORCA for query planning and optimization when possible and falls back to the Postgres Planner when GPORCA cannot be used. Also, the specified type of query might require a long time to complete.
170824967 gpfdists For Greenplum Database 6.x, a command that accesses an external table that uses the gpfdists protocol fails if the external table does not use an IP address when specifying a host system in the LOCATION clause of the external table definition.
n/a Materialized Views By default, certain gp_toolkit views do not display data for materialized views. If you want to include this information in gp_toolkit view output, you must redefine a gp_toolkit internal view as described in Including Data for Materialized Views.
168957894 PXF The PXF Hive Connector does not support using the Hive* profiles to access Hive transactional tables.

Workaround: Use the PXF JDBC Connector to access Hive.
168548176 gpbackup When using gpbackup to back up a Greenplum Database 5.7.1 or earlier 5.x release with resource groups enabled, gpbackup returns a column not found error for t6.value AS memoryauditor.
164791118 PL/R PL/R cannot be installed using the deprecated createlang utility, and displays the error:
createlang: language installation failed: ERROR:
no schema has been selected to create in
Workaround: Use CREATE EXTENSION to install PL/R, as described in the documentation.
N/A Greenplum Client/Load Tools on Windows The Greenplum Database client and load tools on Windows have not been tested with Active Directory Kerberos authentication.

Differences Compared to Open Source Greenplum Database

VMware Tanzu Greenplum 6.x includes all of the functionality in the open source Greenplum Database project and adds:
  • Product packaging and installation script
  • Support for QuickLZ compression. QuickLZ compression is not provided in the open source version of Greenplum Database due to licensing restrictions.
  • Support for data connectors:
    • Greenplum-Spark Connector
    • Greenplum-Informatica Connector
    • Greenplum-Kafka Integration
    • Greenplum Streaming Server
  • Data Direct ODBC/JDBC Drivers
  • gpcopy utility for copying or migrating objects between Greenplum systems
  • Support for managing Greenplum Database using VMware Tanzu Greenplum Command Center
  • Support for full text search and text analysis using VMware Tanzu GPText
  • Greenplum backup plugin for DD Boost
  • Backup/restore storage plugin API