VMware Tanzu Greenplum 6.18 Release Notes
VMware Tanzu Greenplum 6.18 Release Notes
This document contains pertinent release information about VMware Tanzu Greenplum Database 6.18 releases. For previous versions of the release notes for Greenplum Database, go to VMware Tanzu Greenplum Database Documentation. For information about Greenplum Database end of life, see VMware Tanzu Greenplum Database end of life policy.
VMware Tanzu Greenplum 6 software is available for download from the VMware Tanzu Greenplum page on VMware Tanzu Network.
VMware Tanzu Greenplum 6 is based on the open source Greenplum Database project code.
Release Date: 2021-10-8
VMware Tanzu Greenplum 6.18.0 is a minor release that includes feature changes and resolves several issues.
Greenplum Database 6.18.0 includes these new and changed features:
- The gpstate -e and gpstate -s commands now provide
more detailed output about the status of primary-mirror segment WAL synchronization.
Moreover, as part of these output changes:
- The gpstate -s output fields Change tracking data size, Estimated total data to synchronize , and Data synchronized output fields are now gone
- The Mirror status field in gpstate -s output for primary segments now has just two valid values: Synchronized or Not in Sync
- Greenplum 6.18.0 introduces a new Query Optimizer server configuration parameter, optimizer_xform_bind_threshold. You can use this parameter to reduce the optimization time and overall memory usage of queries that include deeply nested expressions by specifying the maximum number of bindings per transform that GPORCA produces per group expression.
- The new gp_autostats_allow_nonowner server configuration parameter enables you to configure Greenplum Database to trigger automatic statistics collection on a table when the table is updated by a non-owner. This parameter is off by default.
- Greenplum 6.18.0 introduces the new contrib module gp_legacy_string_agg. This module implements the single-argument string_agg( text ) function that is available in Greenplum 5; you may choose to use the module to aid migration to Greenplum 6.
- The license file for the Windows Client and Loader Tools Package was updated to the latest version.
- Greenplum 6.18.0 removes the ~/.gphostcache file; the management utilities now use an alternate mechanism to map hostnames to interfaces.
- To enhance the supportability of the product and to aid debugging efforts, Greenplum Database now reports both the reserved and maximum virtual memory allocation when it encounters an out of memory (OOM) condition.
VMware Tanzu Greenplum 6.18.0 resolves these issues:
- Postgres CVE fixes
- This release backports the following Postgres CVE fixes:
- 31736, 31727
- When the log_lock_waits GUC was enabled it resulted in spurious deadlock reports and orphaned wait queue states which, in turn, could lead to memory corruption of certain internal tables. This issue is resolved by disabling the log_lock_waitsGUC.The log_lock_waits GUC is not supported by Greenplum Database.
- 31736 - Resource Queues
- Due to improper error handling, Greenplum Database raised a duplicate portal identifier warning that was in some cases immediately followed by an out of shared memory error. This issue is resolved; Greenplum Database now raises distinct errors for duplicate portal identifier and out of shared memory.
- Greenplum was generating the error ERROR: interconnect error: A HTAB entry for motion node 77 already exists after creating the extension for the ltree module. This issue has been resolved.
- 31725 - Execution
- In some cases, Greenplum Database generated a PANIC when the user cancelled a query on an AO table due to a double free of a visimap object. This issue is resolved.
- 31708 - Catalog and Metadata
- Resolves an issue where a non-superuser who ran VACUUM FULL on a table that they had no permission to access could block further access to the table by currently running transactions. Greenplum Database now performs the permission check before it acquires a lock on the table.
- 31704 - Planner
- In some cases, Greenplum Database returned an incorrect plan when an UPDATE statement included a subquery. This issue is resolved.
- 31679 - Functions and Languages
- Resolves an issue where a function invocation failed with the error cannot create a unique ID for path type: 116 by disallowing the unique row id path when Greenplum Database encounters an index-only scan.
- 31654 - Storage
- Resolves an issue where Greenplum Database did not honor the temp_tablespaces setting when it generated temporary files during a sort operation on a large data set. Greenplum now places these temporary files in a tablespace specified in temp_tablespaces.
- 31617 - Resource Groups
- A database instance was failing to start up, with the message Command pg_ctl reports Master gdm instance active because a mirror segment couldn't be recovered. This was due to the incorrect type of lock being used when creating or altering a resource group. This issue is resolved.
- 31615 - Analyze
- Resolves an issue where Greenplum Database did not collect statistics on a table when the table was updated by a non-owner. Greenplum Database 6.18 introduces the new server configuration parameter gp_autostats_allow_nonowner (default is off) to enable automatic statistics collection when a non-owner updates a table.
Fixed a bug with function fixup_unknown_vars_in_setop that would cause Greenplum Database to PANIC.
- 31385 - Planner
- Resolves an issue where Greenplum Database returned wrong results for a query because it chose an incorrect motion type during plan tree iteration.
- Reinstates the Greenplum Database-specific --wrapper and --wrapper-args options to the pg_ctl command that were available in Greenplum Database 5.
- Fixes an environmental variable generation issue caused when symlinks were pointing to alternative locations than the ones expected.
- Resolves an issue where Greenplum Database, during node recovery, generated an
error when the field standby_mode=on was set in the
recovery.conf file. The error was similar to:
"FATAL","22023","recovery command file ""recovery.conf"" request for standby mode not specified",,,,,,,0,,"xlog.c",5465Greenplum Database now supports server recovery in a non-continuous mode using standby_mode=on.
- pg_relation_filenode() function did not provide proper output for Append Optimized (AO) auxiliary tables. This issue has been resolved.
- Resolves an issue where Greenplum Database threw an assertion failure when the size of an AO table exceeded 1GB.
- Resolves an issue where Greenplum Database failed to complete a query when the target relation of an UPDATE or DELETE operation was a partition table and Greenplum generated a unique row id plan.
- 12299 - gpexpand
- Resolves an issue where WAL-replication was using excessive external bandwith due to misuse of the hostname and address in gp_segment_configuration. Greenplum Database now uses the primary's address for WAL-replication in gpexpand.
- Greenplum Database would initialize pg_aocsseg table entries with frozen tuples to ensure these entries were implicitly visible even after a rollback. This strategy created issues with the roll back of Append Optimized Columnar (AOC) tables. This issue has now been resolved.
- 685, 9208, 9429
- In some cases, the value of a server configuration parameter could be inconsistent among the query dispatcher (QD) and/or query executors (QEs) when the parameter value was updated and then reset in the same session. This issue is resolved.
- Queries were crashing due to GPORCA prematurely terminating the motion node before interconnect was torn down. This issue is now resolved.
- Running queries with GPORCA enabled generated the error ERROR: could not open existing temporary file "base/pgsql_tmp/pgsql_tmp_SIRW_145011_97_0": No such file or directory, caused by temporary file name changes during cross-slice communication. This issue has been resolved.
- Resolved an issue with the minirepro utility where it generated incorrect input when inserting columns containing arrays.
- Resolved an issue where Greenplum processes crashed when trying to write an error message to the log due to a null pointer dereference.
- Resolved an issue that caused wrong results when running queries with GPORCA when having the IS DISTINCT FROM FALSE predicate inside a NOT EXIST subquery.
- Resolves an issue where Greenplum Database threw cosmetic errors during gpstate execution when a database of the same name as the $USER running the command did not exist.
- gpconfig would fail with error similar to ValueError: filedescriptor out of range in select() due to liveness checks performed on all hostnames associated with each database id. This issue has been resolved and liveness checks are now performend only on unique hostnames.
- 31558 - gpinitsystem
- If a gpinisystem operation failed before creating the necessary segment backout scripts, manual steps were required to clean up the directories after the failure. To resolve this problem gpinisystem now creates a single backout script earlier in the process; the script can be used to cleanup all segment directories even if a failure occurs later in the gpinisystem operation.
- The Greenplum utilities gprecoverseg, gpinitstandby, gpstart, and gpstop fail when there is a message banner in .bashrc. This issue has been resolved, and now banner messages are not parsed.
- 178936675 - gpstate
- Beginning in Greenplum 6.0, the gpstate utility no longer provided mirror synchronization status. This problem was resolved by adding the Mirror status field in gpstate -s output for primary segments.
- 178831426 - gpcheckperf
- When executing gpcheckperf via gpssh, the gpcheckperf utility could issue a pkill -f command that killed the gpssh process and disconnected the established SSH connections. This problem was resolved by removing the use of pkill -f in gpcheckperf.
- Removes unrelated output message regarding netperf test, that was shown when running gpcheckperf -r N.
- Fixes the issue of the missing --help option for gpcheckcat.
- Reverts a previously-committed gpload improvement that removed a left-join merge because it introduced a performance regression in certain situations.
- 31613, 12454
- Resolves an issue where a COPY command failed to read a file that included special characters at the end of a line because Greenplum Database did not recognize the EOL character.
Upgrading from Greenplum 6.x to Greenplum 6.18
See Upgrading from an Earlier Greenplum 6 Release to upgrade your existing Greenplum 6.x software to Greenplum 6.18.
Deprecated features will be removed in a future major release of Greenplum Database. VMware Tanzu Greenplum 6.x deprecates:
- The gpsys1 utility.
- The analzyedb option --skip_root_stats (deprecated
If the option is specified, a warning is issued stating that the option will be ignored.
- The server configuration parameter gp_statistics_use_fkeys (deprecated since 6.2).
- The server configuration parameter gp_ignore_error_table (deprecated
To avoid a Greenplum Database syntax error, set the value of this parameter to true when you run applications that execute CREATE EXTERNAL TABLE or COPY commands that include the now removed Greenplum Database 4.3.x INTO ERROR TABLE clause.
- Specifying => as an operator name in the CREATE OPERATOR command (deprecated since 6.0).
- The Greenplum external table C API (deprecated since 6.0).
Any developers using this API are encouraged to use the new Foreign Data Wrapper API in its place.
- Commas placed between a SUBPARTITION TEMPLATE clause and its
corresponding SUBPARTITION BY clause, and between consecutive
SUBPARTITION BY clauses in a CREATE TABLE command
(deprecated since 6.0).
Using this undocumented syntax will generate a deprecation warning message.
- The timestamp format YYYYMMDDHH24MISS (deprecated since 6.0).
This format could not be parsed unambiguously in previous Greenplum Database releases, and is not supported in PostgreSQL 9.4.
- The createlang and droplang utilities (deprecated since 6.0).
- The pg_resqueue_status system view (deprecated since 6.0).
Use the gp_toolkit.gp_resqueue_status view instead.
- The GLOBAL and LOCAL modifiers when creating a
temporary table with the CREATE TABLE and CREATE TABLE
AS commands (deprecated since 6.0).
These keywords are present for SQL standard compatibility, but have no effect in Greenplum Database.
- Using WITH OIDS or oids=TRUE to assign an OID system column when creating or altering a table (deprecated since 6.0).
- Allowing superusers to specify the SQL_ASCII encoding regardless of the
locale settings (deprecated since 6.0).
This choice may result in misbehavior of character-string functions when data that is not encoding-compatible with the locale is stored in the database.
- The @@@ text search operator (deprecated since 6.0).
This operator is currently a synonym for the @@ operator.
- The unparenthesized syntax for option lists in the VACUUM command
(deprecated since 6.0).
This syntax requires that the options to the command be specified in a specific order.
- The plain pgbouncer authentication type (auth_type = plain) (deprecated since 4.x).
Migrating Data to Greenplum 6
Greenplum 6 supports direct upgrades, using gpupgrade, from Greenplum 5.x releases to Greenplum 6.x. For more information, see the gpupgrade documentation. See also Migrating Data from Greenplum 4.3 or 5 for guidelines and considerations for migrating existing Greenplum data to Greenplum 6, using standard backup and restore procedures.
Known Issues and Limitations
VMware Tanzu Greenplum 6 has these limitations:
- Upgrading a Greenplum Database 4 release to VMware Tanzu Greenplum 6 is not supported. Upgrading a Greenplum Database 5.x release to VMware Tanzu Greenplum 6 is supported via gpupgrade. For more information, see the gpupgrade documentation.
- MADlib, GPText, and PostGIS are not yet provided for installation on Ubuntu systems.
- Greenplum for Kubernetes is not yet provided with this release.
The following table lists key known issues in VMware Tanzu Greenplum 6.x.
|31010||Query Optimizer||A view created in Greenplum Database 5.28.3 or older that specified an external
table in the FROM clause, and that was migrated to Greenplum
Database 6.x, always falls back to the Postgres Planner when queried.
Workaround: If you migrated a view from Greenplum Database 5.28.3 or earlier, and the view specified an external table in the FROM clause, you must drop and recreate the view in Greenplum 6.x to ensure that the Query Optimizer is exercised when you query the view.
|N/A||Backup/Restore||Restoring the Greenplum Database backup for a table fails in Greenplum 6
versions earlier than version 6.10 when a replicated table has an inheritance
relationship to/from another table that was assigned via an ALTER TABLE ...
INHERIT statement after table creation.
Workaround: Use the following SQL commands to determine if Greenplum Database includes any replicated tables that inherit from a parent table, or if there are replicated tables that are inherited by a child table:
SELECT inhrelid::regclass FROM pg_inherits, gp_distribution_policy dp WHERE inhrelid=dp.localoid AND dp.policytype='r'; SELECT inhparent::regclass FROM pg_inherits, gp_distribution_policy dp WHERE inhparent=dp.localoid AND dp.policytype='r';
If these queries return any tables, you may choose to run gprestore with the -–on-error-continue flag to not fail the entire restore when this issue is hit. Or, you can specify the list of tables returned by the queries to the -–exclude-table-file option to skip those tables during restore. You must recreate and repopulate the affected tables after restore.
|N/A||Spark Connector||This version of Greenplum is not compatible with Greenplum-Spark Connector versions earlier than version 1.7.0, due to a change in how Greenplum handles distributed transaction IDs.|
|N/A||PXF||Starting in 6.x, Greenplum does not bundle cURL and instead
loads the system-provided library. PXF requires cURL version 7.29.0
or newer. The officially-supported cURL for the CentOS 6.x and Red
Hat Enterprise Linux 6.x operating systems is version 7.19.*. Greenplum Database 6
does not support running PXF on CentOS 6.x or RHEL 6.x due to this limitation.
Workaround: Upgrade the operating system of your Greenplum Database 6 hosts to CentOS 7+ or RHEL 7+, which provides a cURL version suitable to run PXF.
|29703||Loading Data from External Tables||Due to limitations in the Greenplum Database external table framework,
Greenplum Database cannot log the following types of errors that it encounters while
Workaround: Clean the input data before loading it into Greenplum Database.
|30594||Resource Management||Resource queue-related statistics may be inaccurate in certain cases. VMware recommends that you use the resource group resource management scheme that is available in Greenplum 6.|
|30522||Logging||Greenplum Database may write a FATAL message to the standby master or mirror log stating that the database system is in recovery mode when the instance is synchronizing with the master and Greenplum attempts to contact it before the operation completes. Ignore these messages and use gpstate -f output to determine if the standby successfully synchronized with the Greenplum master; the command returns Sync state: sync if it is synchronized.|
|30537||Postgres Planner||The Postgres Planner generates a very large query plan that causes out of
memory issues for the following type of CTE (common table expression) query: the
WITH clause of the CTE contains a partitioned table with a large
number partitions, and the WITH reference is used in a subquery
that joins another partitioned table.
Workaround: If possible, use the GPORCA query optimizer. With the server configuration parameter optimizer=on, Greenplum Database attempts to use GPORCA for query planning and optimization when possible and falls back to the Postgres Planner when GPORCA cannot be used. Also, the specified type of query might require a long time to complete.
|170824967||gpfdists||For Greenplum Database 6.x, a command that accesses an external table that uses the gpfdists protocol fails if the external table does not use an IP address when specifying a host system in the LOCATION clause of the external table definition.|
|n/a||Materialized Views||By default, certain gp_toolkit views do not display data for materialized views. If you want to include this information in gp_toolkit view output, you must redefine a gp_toolkit internal view as described in Including Data for Materialized Views.|
|168957894||PXF||The PXF Hive Connector does not support using the Hive*
profiles to access Hive transactional tables.
Workaround: Use the PXF JDBC Connector to access Hive.
|168548176||gpbackup||When using gpbackup to back up a Greenplum Database 5.7.1 or earlier 5.x release with resource groups enabled, gpbackup returns a column not found error for t6.value AS memoryauditor.|
|164791118||PL/R||PL/R cannot be installed using the deprecated createlang
utility, and displays the
createlang: language installation failed: ERROR: no schema has been selected to create inWorkaround: Use CREATE EXTENSION to install PL/R, as described in the documentation.
|N/A||Greenplum Client/Load Tools on Windows||The Greenplum Database client and load tools on Windows have not been tested with Active Directory Kerberos authentication.|
Differences Compared to Open Source Greenplum Database
- Product packaging and installation script
- Support for QuickLZ compression. QuickLZ compression is not provided in the open source version of Greenplum Database due to licensing restrictions.
- Support for data connectors:
- Greenplum-Spark Connector
- Greenplum-Informatica Connector
- Greenplum-Kafka Integration
- Greenplum Streaming Server
- Data Direct ODBC/JDBC Drivers
- Greenplum PostGIS Extension
- gpcopy utility for copying or migrating objects between Greenplum systems
- Support for managing Greenplum Database using VMware Tanzu Greenplum Command Center
- Support for full text search and text analysis using VMware Tanzu GPText
- Greenplum backup plugin for DD Boost
- Backup/restore storage plugin API