gpss.json

gpss.json

Greenplum Streaming Server configuration file.

Synopsis

{
    "ListenAddress": {
        "Host": "gpss_host",
        "Port": gpss_portnum [,
        "DebugPort": gpss_debug_portnum] [,
        "Certificate": {
            "CertFile": "certfile_path",
            "KeyFile": "keyfile_path",
            "CAFile": "CAfile_path",
            "MinTLSVersion": "min_version"
        }]
    },
    "Gpfdist": {
        "Host": "gpfdist_host",
        "Port": gpfdist_portnum [,
        "ReuseTables": bool_value ][,
        "Certificate": {
            "CertFile": "certfile_path",
            "KeyFile": "keyfile_path",
            "CAFile": "CAfile_path",
            "MinTLSVersion": "min_version"
        }] [,
        "BindAddress": bind_addr ]
    }[,
    "Shadow": {
        "Key": "passwd_shadow_key",
    }] [,
    "JobStore": {
        "File": {
            "Directory": "jobstore_dir"
        }
    }] [,
    "Logging": {
        "BackendLevel": "level",
        "FrontendLevel": "level"
    }] [,
    "Monitor": {
        "Prometheus": {
            "Listening": "prom_addr:prom_port"
        }
    }] [,
    "KeepAlive": {
        "MaxConnectionIdle": "duration",
        "Time": "duration",
        "Timeout": "duration",
        "MinTime": "duration",
        "PermitWithoutStream": bool_value
    }]
}

Description

You specify runtime configuration properties for a gpss service instance in a JSON-formatted configuration file. Run properties for GPSS include gpss and gpfdist service hosts, addresses, port numbers, and optional encryption certificates and key files. You can specify a directory in which GPSS stores job status information. addresses, port numbers, and optional encryption certificates and key files.

You can configure the log level of messages that GPSS commands write to stdout and to log files. You can also specify a shadow key that GPSS uses to encode and decode the Greenplum Database password.

This reference page uses the name gpss.json to refer to this file; you may choose your own name for the file.

Keywords and Values

GPSS ListenAddress Options
Host: gpss_host
The host name or IP address on which GPSS listens for client connections. The default host is 127.0.0.1.
Port: gpss_portnum
The port number on which the gpss service instance listens. The default port number is 5000.
DebugPort: gpss_debug_portnum
The port number on which gpss starts a debug server on the local host (127.0.0.1). When you specify this configuration option, gpss makes debug information including the call stack and performance statistics available from the debug server.
Certificates:
When you specify certificates, GPSS uses the SSL certificates to authenticate both the client connection and the connection to Greenplum Database.
CertFile: certfile_path
File system path to the server certificate.
KeyFile: keyfile_path
File system path to the server key file.
CAFile: CAfile_path
File system path to the Certificate Authority file. The CAfile_path must contain the entire Certificate Authority chain.
MinTLSVersion: min_version
The minimum transport layer security (TLS) version that GPSS requests on the connection. The default value is 1.0. GPSS supports minimum TLS versions of 1.0, 1.1, 1.2, and 1.3.
Gpfdist Options

GPSS implements the gpfdist protocol. The gpss.json file exposes configuration options that you can use to identify the service location and bind options.

Host: gpfdist_host
The gpfdist service host name or IP address that GPSS sets in the external table LOCATION clause. This hostname or IP address must be reachable from each Greenplum Database segment host. The default value is the fully qualified distinguished name of the host on which GPSS is running.
Port: gpfdist_portnum
The gpfdist service port number. The default port number is 8080.
ReuseTables: bool_value
A boolean that identifies whether or not GPSS should reuse an external table when the job associated with a load operation is restarted. The default value is true, reuse the external table.
If you choose not to reuse external tables, GPSS deletes the external table associated with a load operation when you stop the job, and creates a new external table when you restart the job.
Certificates:
When you specify gpfdist certificates, GPSS uses the SSL certificates and the gpfdists protocol to transfer encrypted data between itself and Greenplum Database.
CertFile: certfile_path
File system path to the server certificate.
KeyFile: keyfile_path
File system path to the server key file.
CAFile: CAfile_path
File system path to the Certificate Authority file. The CAfile_path must contain the entire Certificate Authority chain.
MinTLSVersion: min_version
The minimum transport layer security (TLS) version that GPSS requests on the connection. The default value is 1.0. GPSS supports minimum TLS versions of 1.0, 1.1, 1.2, and 1.3.
BindAddress: bind_addr
The address from which GPSS listens for connections from Greenplum Database segments. Set bind_addr to an IP address that is reachable from every segment host by resolving the gpfdist_host configuration value, or set it to 0.0.0.0 to listen for connections from any host. The default bind address is 0.0.0.0.
Shadow Option

The encode/decode key for the Greenplum Database password.

Shadow: passwd_shadow_key
The key that GPSS uses to encode and decode the Greenplum Database shadow password string. Keep this key secret.
JobStore Option

The GPSS job information store.

File: Directory: jobstore_dir
The file system directory on the local host in which GPSS maintains current job information and status.
Logging Options

The back-end (log file) and front-end (command line output) logging levels for GPSS commands. GPSS supports the following log levels (from most to least severe): fatal, error, warn or warning, info, and debug.

BackendLevel: level
The logging level for messages that gpss, gpsscli, and gpkafka write to log files. The default back-end logging level is debug.
FrontendLevel: level
The logging level for messages that gpss, gpsscli, and gpkafka write to stdout. The default front-end logging level is info.
Monitor Option

The address to which GPSS binds the Prometheus scrape target for the GPSS server instance.

Prometheus: Listening: prom_addr:prom_port
The host name or IP address and port number from which the GPSS service instance allows Prometheus to pull the server's metrics.
KeepAlive Options

These properties control the gRPC connection between the GPSS client and GPSS server. Refer to the gRPC keepalive package documentation for more information about these properties and their default values.

MaxConnectionIdle: duration
The amount of time after which an idle connection is closed. The default value is infinity.
Time: duration
The amount of time of no activity after which a server pings the client to see if the transport is still alive. The default value is 2h.
Timeout: duration
The amount of time of no activity after a keepalive ping that prompts closure of the connection. The default value is 20s.
MinTime: duration
The mininum amount of time a client should wait before sending a keepalive ping. The default value is 5m.
PermitWithoutStream: bool_value
Determines if the server allows keepalive pings when there are no active streams. If false, the server will close the connection when a client sends a ping with no active streams. The default value is false.

Notes

When you provide the --config gpss.json option to the gpsscli shadow command, GPSS uses the Shadow:Key specified in the file, or a default key, to encode a Greenplum Database password that you specify and generate a shadowed password string.

When you provide a variant of this file to the other gpsscli subcommands via the --config gpsscliconfig.json option, GPSS uses the information provided in the ListenAddress block of the file to identify the GPSS server instance to which to route the request, and/or to identify the client certificates when SSL is enabled between GPSS client and server.

When you provide the --config gpfdistconfig.json option to the gpkafka load command, GPSS uses the information provided in the Gpfdist block of the file to specify the gpfdist protocol instance, and, when SSL is enabled on the data channel to Greenplum Database, to identify the GPSS SSL certificates.

Examples

Start a Greenplum Streaming Server instance with properties as defined in a configuration file named gpss4ic.json located in the current directory:

$ gpss --config gpss4ic.json

Example gpss4ic.json configuration file:

{
    "ListenAddress": {
        "Host": "",
        "Port": 5019
    },
    "Gpfdist": {
        "Host": "",
        "Port": 8319,
        "ReuseTables": false
    },
    "Shadow": {
        "Key": "a_very_secret_key"
    },
    "Monitor": {
        "Prometheus": {
            "Listening": "0.0.0.0:5001"
    },
    "JobStore": {
        "File": {
            "Directory": "/home/gpadmin/jobstore"
        }
    }
}

See Also

gpss, gpsscli, gpkafka load