Upgrading PXF When You Upgrade Greenplum 6
If you are using PXF in your current Greenplum Database 6.x installation, you must perform some PXF upgrade actions when you upgrade to a newer version of Greenplum Database 6.x.
The PXF upgrade procedure describes how to upgrade PXF in your Greenplum Database installation. This procedure uses PXF.from to refer to your currently-installed PXF version and PXF.to to refer to the PXF version installed when you upgrade to the new version of Greenplum Database.
Note: if you are planning to upgrade PXF but not as part of a Greenplum Database upgrade, refer to the PXF Documentation
The PXF upgrade procedure has two parts. You perform one procedure before, and one procedure after, you upgrade to a new version of Greenplum Database:
- Step 1: PXF Pre-Upgrade Actions
- Upgrade to a new Greenplum Database version
- Step 2: Upgrading PXF
Step 1: PXF Pre-Upgrade Actions
Perform this procedure before you upgrade to a new version of Greenplum Database:
Log in to the Greenplum Database master node. For example:
$ ssh gpadmin@<gpmaster>
Identify and note the PXF.from version number. For example:
gpadmin@gpmaster$ pxf version
Identify the file system location of the
$PXF_CONF
setting in your PXF 5.x PXF installation; you might need this later. If you are unsure of the location, you can find the value inpxf-env-default.sh
.Stop PXF on each segment host as described in Stopping PXF.
Upgrade to the new version of Greenplum Database and then continue your PXF upgrade with Step 2: Upgrading PXF.
Step 2: Registering or Upgrading PXF
After you upgrade to the new version of Greenplum Database, perform the following procedure to upgrade and configure the PXF.to software:
Log in to the Greenplum Database master node. For example:
$ ssh gpadmin@<gpmaster>
If you installed the PXF
rpm
on your Greenplum 6 hosts, you must register it to continue using PXF:Copy the PXF extension files from the PXF installation directory to the new Greenplum 6 install directory:
gpadmin@gpmaster pxf cluster register
Start PXF on each segment host as described in Starting PXF.
Skip the following steps and exit this procedure.
To continue using the PXF version included with the Greenplum Database installation, initialize PXF on each segment host as described in Initializing PXF. You may choose to use your existing
$PXF_CONF
for the initialization.If you are upgrading from Greenplum Database version 6.1.x or earlier and you have configured any JDBC servers that access Kerberos-secured Hive, you must now set the
hadoop.security.authentication
property to thejdbc-site.xml
file to explicitly identify use of the Kerberos authentication method. Perform the following for each of these server configs:- Navigate to the server configuration directory.
Open the
jdbc-site.xml
file in the editor of your choice and uncomment or add the following property block to the file:<property> <name>hadoop.security.authentication</name> <value>kerberos</value> </property>
Save the file and exit the editor.
If you are upgrading from Greenplum Database version 6.7.x or earlier: The PXF
Hive
andHiveRC
profiles now support column projection using column name-based mapping. If you have any existing PXF external tables that specify one of these profiles, and the external table relied on column index-based mapping, you may be required to drop and recreate the tables:- Identify all PXF external tables that you created that specify a
Hive
orHiveRC
profile. For each external table that you identify in step 1, examine the definitions of both the PXF external table and the referenced Hive table. If the column names of the PXF external table do not match the column names of the Hive table:
Drop the existing PXF external table. For example:
DROP EXTERNAL TABLE pxf_hive_table1;
Recreate the PXF external table using the Hive column names. For example:
CREATE EXTERNAL TABLE pxf_hive_table1( hivecolname int, hivecolname2 text ) LOCATION( 'pxf://default.hive_table_name?PROFILE=Hive') FORMAT 'custom' (FORMATTER='pxfwritable_import');
Review any SQL scripts that you may have created that reference the PXF external table, and update column names if required.
- Identify all PXF external tables that you created that specify a
Synchronize the PXF configuration from the master host to the standby master and each Greenplum Database segment host. For example:
gpadmin@gpmaster$ $GPHOME/pxf/bin/pxf cluster sync
Start PXF on each segment host as described in Starting PXF.