Tyk Pump Environment Variables

You can use environment variables to override the config file for the Tyk Pump. Environment variables are created from the dot notation versions of the JSON objects contained with the config files. To understand how the environment variables notation works, see Environment Variables.

All the Pump environment variables have the prefix TYK_PMP_. The environment variables will take precedence over the values in the configuration file.

purge_delay

ENV: TYK_PMP_PURGEDELAY
Type: int

The number of seconds the Pump waits between checking for analytics data and purge it from Redis.

purge_chunk

ENV: TYK_PMP_PURGECHUNK
Type: int64

The maximum number of records to pull from Redis at a time. If it’s unset or 0, all the analytics records in Redis are pulled. If it’s set, storage_expiration_time is used to reset the analytics record TTL.

storage_expiration_time

ENV: TYK_PMP_STORAGEEXPIRATIONTIME
Type: int64

The number of seconds for the analytics records TTL. It only works if purge_chunk is enabled. Defaults to 60 seconds.

dont_purge_uptime_data

ENV: TYK_PMP_DONTPURGEUPTIMEDATA
Type: bool

Setting this to false will create a pump that pushes uptime data to Uptime Pump, so the Dashboard can read it. Disable by setting to true.

uptime_pump_config

Example Uptime Pump configuration:

"uptime_pump_config": {
  "uptime_type": "mongo",
  "mongo_url": "mongodb://localhost:27017",
  "collection_name": "tyk_uptime_analytics"
},

### SQL Uptime Pump
*Supported in Tyk Pump v1.5.0+*

In `uptime_pump_config` you can configure a SQL uptime pump. To do that, you need to add the
field `uptime_type` with `sql` value. You can also use different types of SQL Uptime pumps,
like `postgres` or `sqlite` using the `type` field.

An example of a SQL Postgres uptime pump would be:
```{.json}
"uptime_pump_config": {
    "uptime_type": "sql",
    "type": "postgres",
    "connection_string": "host=sql_host port=sql_port user=sql_usr dbname=dbname password=sql_pw",
    "table_sharding": false
},

Take into account that you can also set log_level field into the uptime_pump_config to debug, info or warning. By default, the SQL logger verbosity is silent.

uptime_pump_config.EnvPrefix

ENV: TYK_PMP_UPTIMEPUMPCONFIG_ENVPREFIX
Type: string

The prefix for the environment variables that will be used to override the configuration. Defaults to TYK_PMP_PUMPS_SQL_META

uptime_pump_config.mongo_url

ENV: TYK_PMP_UPTIMEPUMPCONFIG_MONGOURL
Type: string

The full URL to your MongoDB instance, this can be a clustered instance if necessary and should include the database and username / password data.

uptime_pump_config.mongo_use_ssl

ENV: TYK_PMP_UPTIMEPUMPCONFIG_MONGOUSESSL
Type: bool

Set to true to enable Mongo SSL connection.

uptime_pump_config.mongo_ssl_insecure_skip_verify

ENV: TYK_PMP_UPTIMEPUMPCONFIG_MONGOSSLINSECURESKIPVERIFY
Type: bool

Allows the use of self-signed certificates when connecting to an encrypted MongoDB database.

uptime_pump_config.mongo_ssl_allow_invalid_hostnames

ENV: TYK_PMP_UPTIMEPUMPCONFIG_MONGOSSLALLOWINVALIDHOSTNAMES
Type: bool

Ignore hostname check when it differs from the original (for example with SSH tunneling). The rest of the TLS verification will still be performed.

uptime_pump_config.mongo_ssl_ca_file

ENV: TYK_PMP_UPTIMEPUMPCONFIG_MONGOSSLCAFILE
Type: string

Path to the PEM file with trusted root certificates

uptime_pump_config.mongo_ssl_pem_keyfile

ENV: TYK_PMP_UPTIMEPUMPCONFIG_MONGOSSLPEMKEYFILE
Type: string

Path to the PEM file which contains both client certificate and private key. This is required for Mutual TLS.

uptime_pump_config.mongo_db_type

ENV: TYK_PMP_UPTIMEPUMPCONFIG_MONGODBTYPE
Type: int

Specifies the mongo DB Type. If it’s 0, it means that you are using standard mongo db. If it’s 1 it means you are using AWS Document DB. If it’s 2, it means you are using CosmosDB. Defaults to Standard mongo (0).

uptime_pump_config.omit_index_creation

ENV: TYK_PMP_UPTIMEPUMPCONFIG_OMITINDEXCREATION
Type: bool

Set to true to disable the default tyk index creation.

uptime_pump_config.mongo_session_consistency

ENV: TYK_PMP_UPTIMEPUMPCONFIG_MONGOSESSIONCONSISTENCY
Type: string

Set the consistency mode for the session, it defaults to Strong. The valid values are: strong, monotonic, eventual.

uptime_pump_config.driver

ENV: TYK_PMP_UPTIMEPUMPCONFIG_MONGODRIVERTYPE
Type: string

MongoDriverType is the type of the driver (library) to use. The valid values are: “mongo-go” and “mgo”. Since v1.9, the default driver is “mongo-go”. Check out this guide to learn about MongoDB drivers supported by Tyk Pump.

uptime_pump_config.mongo_direct_connection

ENV: TYK_PMP_UPTIMEPUMPCONFIG_MONGODIRECTCONNECTION
Type: bool

MongoDirectConnection informs whether to establish connections only with the specified seed servers, or to obtain information for the whole cluster and establish connections with further servers too. If true, the client will only connect to the host provided in the ConnectionString and won’t attempt to discover other hosts in the cluster. Useful when network restrictions prevent discovery, such as with SSH tunneling. Default is false.

uptime_pump_config.collection_name

ENV: TYK_PMP_UPTIMEPUMPCONFIG_COLLECTIONNAME
Type: string

Specifies the mongo collection name.

uptime_pump_config.max_insert_batch_size_bytes

ENV: TYK_PMP_UPTIMEPUMPCONFIG_MAXINSERTBATCHSIZEBYTES
Type: int

Maximum insert batch size for mongo selective pump. If the batch we are writing surpasses this value, it will be sent in multiple batches. Defaults to 10Mb.

uptime_pump_config.max_document_size_bytes

ENV: TYK_PMP_UPTIMEPUMPCONFIG_MAXDOCUMENTSIZEBYTES
Type: int

Maximum document size. If the document exceed this value, it will be skipped. Defaults to 10Mb.

uptime_pump_config.collection_cap_max_size_bytes

ENV: TYK_PMP_UPTIMEPUMPCONFIG_COLLECTIONCAPMAXSIZEBYTES
Type: int

Amount of bytes of the capped collection in 64bits architectures. Defaults to 5GB.

uptime_pump_config.collection_cap_enable

ENV: TYK_PMP_UPTIMEPUMPCONFIG_COLLECTIONCAPENABLE
Type: bool

Enable collection capping. It’s used to set a maximum size of the collection.

uptime_pump_config.type

ENV: TYK_PMP_UPTIMEPUMPCONFIG_TYPE
Type: string

The supported and tested types are sqlite and postgres.

uptime_pump_config.connection_string

ENV: TYK_PMP_UPTIMEPUMPCONFIG_CONNECTIONSTRING
Type: string

Specifies the connection string to the database.

uptime_pump_config.postgres

ENV: TYK_PMP_UPTIMEPUMPCONFIG_POSTGRES
Type: PostgresConfig

Postgres configurations.

uptime_pump_config.postgres.prefer_simple_protocol

ENV: TYK_PMP_UPTIMEPUMPCONFIG_POSTGRES_PREFERSIMPLEPROTOCOL
Type: bool

Disables implicit prepared statement usage.

uptime_pump_config.mysql

ENV: TYK_PMP_UPTIMEPUMPCONFIG_MYSQL
Type: MysqlConfig

Mysql configurations.

uptime_pump_config.mysql.default_string_size

ENV: TYK_PMP_UPTIMEPUMPCONFIG_MYSQL_DEFAULTSTRINGSIZE
Type: uint

Default size for string fields. Defaults to 256.

uptime_pump_config.mysql.disable_datetime_precision

ENV: TYK_PMP_UPTIMEPUMPCONFIG_MYSQL_DISABLEDATETIMEPRECISION
Type: bool

Disable datetime precision, which not supported before MySQL 5.6.

uptime_pump_config.mysql.dont_support_rename_index

ENV: TYK_PMP_UPTIMEPUMPCONFIG_MYSQL_DONTSUPPORTRENAMEINDEX
Type: bool

Drop & create when rename index, rename index not supported before MySQL 5.7, MariaDB.

uptime_pump_config.mysql.dont_support_rename_column

ENV: TYK_PMP_UPTIMEPUMPCONFIG_MYSQL_DONTSUPPORTRENAMECOLUMN
Type: bool

change when rename column, rename column not supported before MySQL 8, MariaDB.

uptime_pump_config.mysql.skip_initialize_with_version

ENV: TYK_PMP_UPTIMEPUMPCONFIG_MYSQL_SKIPINITIALIZEWITHVERSION
Type: bool

Auto configure based on currently MySQL version.

uptime_pump_config.table_sharding

ENV: TYK_PMP_UPTIMEPUMPCONFIG_TABLESHARDING
Type: bool

Specifies if all the analytics records are going to be stored in one table or in multiple tables (one per day). By default, false. If false, all the records are going to be stored in tyk_aggregated table. Instead, if it’s true, all the records of the day are going to be stored in tyk_aggregated_YYYYMMDD table, where YYYYMMDD is going to change depending on the date.

uptime_pump_config.log_level

ENV: TYK_PMP_UPTIMEPUMPCONFIG_LOGLEVEL
Type: string

Specifies the SQL log verbosity. The possible values are: info,error and warning. By default, the value is silent, which means that it won’t log any SQL query.

uptime_pump_config.batch_size

ENV: TYK_PMP_UPTIMEPUMPCONFIG_BATCHSIZE
Type: int

Specifies the amount of records that are going to be written each batch. Type int. By default, it writes 1000 records max per batch.

uptime_pump_config.uptime_type

ENV: TYK_PMP_UPTIMEPUMPCONFIG_UPTIMETYPE
Type: string

Determines the uptime type. Options are mongo and sql. Defaults to mongo.

pumps

The default environment variable prefix for each pump follows this format: TYK_PMP_PUMPS_{PUMP-NAME}_, for example TYK_PMP_PUMPS_KAFKA_.

You can also set custom names for each pump specifying the pump type. For example, if you want a Kafka pump which is called PROD you need to create TYK_PMP_PUMPS_PROD_TYPE=kafka and configure it using the TYK_PMP_PUMPS_PROD_ prefix.

pumps.csv.name

ENV: TYK_PMP_PUMPS_CSV_NAME
Type: string

The name of the pump. This is used to identify the pump in the logs. Deprecated, use type instead.

pumps.csv.type

ENV: TYK_PMP_PUMPS_CSV_TYPE
Type: string

Sets the pump type. This is needed when the pump key does not equal to the pump name type. Current valid types are: mongo, mongo-pump-selective, mongo-pump-aggregate, csv, elasticsearch, influx, influx2, moesif, statsd, segment, graylog, splunk, hybrid, prometheus, logzio, dogstatsd, kafka, syslog, sql, sql_aggregate, stdout, timestream, mongo-graph, sql-graph, sql-graph-aggregate, resurfaceio.

pumps.csv.filters

This feature adds a new configuration field in each pump called filters and its structure is the following:

"filters":{
  "api_ids":[],
  "org_ids":[],
  "response_codes":[],
  "skip_api_ids":[],
  "skip_org_ids":[],
  "skip_response_codes":[]
}

The fields api_ids, org_ids and response_codes works as allow list (APIs and orgs where we want to send the analytics records) and the fields skip_api_ids, skip_org_ids and skip_response_codes works as block list.

The priority is always block list configurations over allow list.

An example of configuration would be:

"csv": {
 "type": "csv",
 "filters": {
   "org_ids": ["org1","org2"]
 },
 "meta": {
   "csv_dir": "./bar"
 }
}

pumps.csv.filters.org_ids

ENV: TYK_PMP_PUMPS_CSV_FILTERS_ORGSIDS
Type: []string

Filters pump data by an allow list of org_ids.

pumps.csv.filters.api_ids

ENV: TYK_PMP_PUMPS_CSV_FILTERS_APIIDS
Type: []string

Filters pump data by an allow list of api_ids.

pumps.csv.filters.response_codes

ENV: TYK_PMP_PUMPS_CSV_FILTERS_RESPONSECODES
Type: []int

Filters pump data by an allow list of response_codes.

pumps.csv.filters.skip_org_ids

ENV: TYK_PMP_PUMPS_CSV_FILTERS_SKIPPEDORGSIDS
Type: []string

Filters pump data by a block list of org_ids.

pumps.csv.filters.skip_api_ids

ENV: TYK_PMP_PUMPS_CSV_FILTERS_SKIPPEDAPIIDS
Type: []string

Filters pump data by a block list of api_ids.

pumps.csv.filters.skip_response_codes

ENV: TYK_PMP_PUMPS_CSV_FILTERS_SKIPPEDRESPONSECODES
Type: []int

Filters pump data by a block list of response_codes.

pumps.csv.timeout

ENV: TYK_PMP_PUMPS_CSV_TIMEOUT
Type: int

By default, a pump will wait forever for each write operation to complete; you can configure an optional timeout by setting the configuration option timeout. If you have deployed multiple pumps, then you can configure each timeout independently. The timeout is in seconds and defaults to 0.

The timeout is configured within the main pump config as shown here; note that this example would configure a 5 second timeout:

"pump_name": {
  ...
  "timeout":5,
  "meta": {...}
}

Tyk will inform you if the pump’s write operation is taking longer than the purging loop (configured via purge_delay) as this will mean that data is purged before being written to the target data sink.

If there is no timeout configured and pump’s write operation is taking longer than the purging loop, the following warning log will be generated: Pump {pump_name} is taking more time than the value configured of purge_delay. You should try to set a timeout for this pump.

If there is a timeout configured, but pump’s write operation is still taking longer than the purging loop, the following warning log will be generated: Pump {pump_name} is taking more time than the value configured of purge_delay. You should try lowering the timeout configured for this pump..

pumps.csv.omit_detailed_recording

ENV: TYK_PMP_PUMPS_CSV_OMITDETAILEDRECORDING
Type: bool

Setting this to true will avoid writing raw_request and raw_response fields for each request in pumps. Defaults to false.

pumps.csv.max_record_size

ENV: TYK_PMP_PUMPS_CSV_MAXRECORDSIZE
Type: int

Defines maximum size (in bytes) for Raw Request and Raw Response logs, this value defaults to 0. If it is not set then tyk-pump will not trim any data and will store the full information. This can also be set at a pump level. For example:

"csv": {
  "type": "csv",
  "max_record_size":1000,
  "meta": {
    "csv_dir": "./"
  }
}

pumps.csv.ignore_fields

ENV: TYK_PMP_PUMPS_CSV_IGNOREFIELDS
Type: []string

IgnoreFields defines a list of analytics fields that will be ignored when writing to the pump. This can be used to avoid writing sensitive information to the Database, or data that you don’t really need to have. The field names must be the same as the JSON tags of the analytics record fields. For example: ["api_key", "api_version"].

pumps.csv.meta.EnvPrefix

ENV: TYK_PMP_PUMPS_CSV_META_ENVPREFIX
Type: string

The prefix for the environment variables that will be used to override the configuration. Defaults to TYK_PMP_PUMPS_CSV_META

pumps.csv.meta.csv_dir

ENV: TYK_PMP_PUMPS_CSV_META_CSVDIR
Type: string

The directory and the filename where the CSV data will be stored.

pumps.dogstatsd.name

ENV: TYK_PMP_PUMPS_DOGSTATSD_NAME
Type: string

The name of the pump. This is used to identify the pump in the logs. Deprecated, use type instead.

pumps.dogstatsd.type

ENV: TYK_PMP_PUMPS_DOGSTATSD_TYPE
Type: string

Sets the pump type. This is needed when the pump key does not equal to the pump name type. Current valid types are: mongo, mongo-pump-selective, mongo-pump-aggregate, csv, elasticsearch, influx, influx2, moesif, statsd, segment, graylog, splunk, hybrid, prometheus, logzio, dogstatsd, kafka, syslog, sql, sql_aggregate, stdout, timestream, mongo-graph, sql-graph, sql-graph-aggregate, resurfaceio.

pumps.dogstatsd.filters

This feature adds a new configuration field in each pump called filters and its structure is the following:

"filters":{
  "api_ids":[],
  "org_ids":[],
  "response_codes":[],
  "skip_api_ids":[],
  "skip_org_ids":[],
  "skip_response_codes":[]
}

The fields api_ids, org_ids and response_codes works as allow list (APIs and orgs where we want to send the analytics records) and the fields skip_api_ids, skip_org_ids and skip_response_codes works as block list.

The priority is always block list configurations over allow list.

An example of configuration would be:

"csv": {
 "type": "csv",
 "filters": {
   "org_ids": ["org1","org2"]
 },
 "meta": {
   "csv_dir": "./bar"
 }
}

pumps.dogstatsd.filters.org_ids

ENV: TYK_PMP_PUMPS_DOGSTATSD_FILTERS_ORGSIDS
Type: []string

Filters pump data by an allow list of org_ids.

pumps.dogstatsd.filters.api_ids

ENV: TYK_PMP_PUMPS_DOGSTATSD_FILTERS_APIIDS
Type: []string

Filters pump data by an allow list of api_ids.

pumps.dogstatsd.filters.response_codes

ENV: TYK_PMP_PUMPS_DOGSTATSD_FILTERS_RESPONSECODES
Type: []int

Filters pump data by an allow list of response_codes.

pumps.dogstatsd.filters.skip_org_ids

ENV: TYK_PMP_PUMPS_DOGSTATSD_FILTERS_SKIPPEDORGSIDS
Type: []string

Filters pump data by a block list of org_ids.

pumps.dogstatsd.filters.skip_api_ids

ENV: TYK_PMP_PUMPS_DOGSTATSD_FILTERS_SKIPPEDAPIIDS
Type: []string

Filters pump data by a block list of api_ids.

pumps.dogstatsd.filters.skip_response_codes

ENV: TYK_PMP_PUMPS_DOGSTATSD_FILTERS_SKIPPEDRESPONSECODES
Type: []int

Filters pump data by a block list of response_codes.

pumps.dogstatsd.timeout

ENV: TYK_PMP_PUMPS_DOGSTATSD_TIMEOUT
Type: int

By default, a pump will wait forever for each write operation to complete; you can configure an optional timeout by setting the configuration option timeout. If you have deployed multiple pumps, then you can configure each timeout independently. The timeout is in seconds and defaults to 0.

The timeout is configured within the main pump config as shown here; note that this example would configure a 5 second timeout:

"pump_name": {
  ...
  "timeout":5,
  "meta": {...}
}

Tyk will inform you if the pump’s write operation is taking longer than the purging loop (configured via purge_delay) as this will mean that data is purged before being written to the target data sink.

If there is no timeout configured and pump’s write operation is taking longer than the purging loop, the following warning log will be generated: Pump {pump_name} is taking more time than the value configured of purge_delay. You should try to set a timeout for this pump.

If there is a timeout configured, but pump’s write operation is still taking longer than the purging loop, the following warning log will be generated: Pump {pump_name} is taking more time than the value configured of purge_delay. You should try lowering the timeout configured for this pump..

pumps.dogstatsd.omit_detailed_recording

ENV: TYK_PMP_PUMPS_DOGSTATSD_OMITDETAILEDRECORDING
Type: bool

Setting this to true will avoid writing raw_request and raw_response fields for each request in pumps. Defaults to false.

pumps.dogstatsd.max_record_size

ENV: TYK_PMP_PUMPS_DOGSTATSD_MAXRECORDSIZE
Type: int

Defines maximum size (in bytes) for Raw Request and Raw Response logs, this value defaults to 0. If it is not set then tyk-pump will not trim any data and will store the full information. This can also be set at a pump level. For example:

"csv": {
  "type": "csv",
  "max_record_size":1000,
  "meta": {
    "csv_dir": "./"
  }
}

pumps.dogstatsd.ignore_fields

ENV: TYK_PMP_PUMPS_DOGSTATSD_IGNOREFIELDS
Type: []string

IgnoreFields defines a list of analytics fields that will be ignored when writing to the pump. This can be used to avoid writing sensitive information to the Database, or data that you don’t really need to have. The field names must be the same as the JSON tags of the analytics record fields. For example: ["api_key", "api_version"].

pumps.dogstatsd.meta.EnvPrefix

ENV: TYK_PMP_PUMPS_DOGSTATSD_META_ENVPREFIX
Type: string

The prefix for the environment variables that will be used to override the configuration. Defaults to TYK_PMP_PUMPS_DOGSTATSD_META

pumps.dogstatsd.meta.namespace

ENV: TYK_PMP_PUMPS_DOGSTATSD_META_NAMESPACE
Type: string

Prefix for your metrics to datadog.

pumps.dogstatsd.meta.address

ENV: TYK_PMP_PUMPS_DOGSTATSD_META_ADDRESS
Type: string

Address of the datadog agent including host & port.

pumps.dogstatsd.meta.sample_rate

ENV: TYK_PMP_PUMPS_DOGSTATSD_META_SAMPLERATE
Type: float64

Defaults to 1 which equates to 100% of requests. To sample at 50%, set to 0.5.

pumps.dogstatsd.meta.async_uds

ENV: TYK_PMP_PUMPS_DOGSTATSD_META_ASYNCUDS
Type: bool

Enable async UDS over UDP https://github.com/Datadog/datadog-go#unix-domain-sockets-client.

pumps.dogstatsd.meta.async_uds_write_timeout_seconds

ENV: TYK_PMP_PUMPS_DOGSTATSD_META_ASYNCUDSWRITETIMEOUT
Type: int

Integer write timeout in seconds if async_uds: true.

pumps.dogstatsd.meta.buffered

ENV: TYK_PMP_PUMPS_DOGSTATSD_META_BUFFERED
Type: bool

Enable buffering of messages.

pumps.dogstatsd.meta.buffered_max_messages

ENV: TYK_PMP_PUMPS_DOGSTATSD_META_BUFFEREDMAXMESSAGES
Type: int

Max messages in single datagram if buffered: true. Default 16.

pumps.dogstatsd.meta.tags

ENV: TYK_PMP_PUMPS_DOGSTATSD_META_TAGS
Type: []string

List of tags to be added to the metric. The possible options are listed in the below example.

If no tag is specified the fallback behavior is to use the below tags:

• path
• method
• response_code
• api_version
• api_name
• api_id
• org_id
• tracked
• oauth_id

Note that this configuration can generate significant charges due to the unbound nature of the path tag.

"dogstatsd": {
  "type": "dogstatsd",
  "meta": {
    "address": "localhost:8125",
    "namespace": "pump",
    "async_uds": true,
    "async_uds_write_timeout_seconds": 2,
    "buffered": true,
    "buffered_max_messages": 32,
    "sample_rate": 0.5,
    "tags": [
      "method",
      "response_code",
      "api_version",
      "api_name",
      "api_id",
      "org_id",
      "tracked",
      "path",
      "oauth_id"
    ]
  }
},

On startup, you should see the loaded configs when initializing the dogstatsd pump

[May 10 15:23:44]  INFO dogstatsd: initializing pump
[May 10 15:23:44]  INFO dogstatsd: namespace: pump.
[May 10 15:23:44]  INFO dogstatsd: sample_rate: 50%
[May 10 15:23:44]  INFO dogstatsd: buffered: true, max_messages: 32
[May 10 15:23:44]  INFO dogstatsd: async_uds: true, write_timeout: 2s

pumps.elasticsearch.name

ENV: TYK_PMP_PUMPS_ELASTICSEARCH_NAME
Type: string

The name of the pump. This is used to identify the pump in the logs. Deprecated, use type instead.

pumps.elasticsearch.type

ENV: TYK_PMP_PUMPS_ELASTICSEARCH_TYPE
Type: string

Sets the pump type. This is needed when the pump key does not equal to the pump name type. Current valid types are: mongo, mongo-pump-selective, mongo-pump-aggregate, csv, elasticsearch, influx, influx2, moesif, statsd, segment, graylog, splunk, hybrid, prometheus, logzio, dogstatsd, kafka, syslog, sql, sql_aggregate, stdout, timestream, mongo-graph, sql-graph, sql-graph-aggregate, resurfaceio.

pumps.elasticsearch.filters

This feature adds a new configuration field in each pump called filters and its structure is the following:

"filters":{
  "api_ids":[],
  "org_ids":[],
  "response_codes":[],
  "skip_api_ids":[],
  "skip_org_ids":[],
  "skip_response_codes":[]
}

The fields api_ids, org_ids and response_codes works as allow list (APIs and orgs where we want to send the analytics records) and the fields skip_api_ids, skip_org_ids and skip_response_codes works as block list.

The priority is always block list configurations over allow list.

An example of configuration would be:

"csv": {
 "type": "csv",
 "filters": {
   "org_ids": ["org1","org2"]
 },
 "meta": {
   "csv_dir": "./bar"
 }
}

pumps.elasticsearch.filters.org_ids

ENV: TYK_PMP_PUMPS_ELASTICSEARCH_FILTERS_ORGSIDS
Type: []string

Filters pump data by an allow list of org_ids.

pumps.elasticsearch.filters.api_ids

ENV: TYK_PMP_PUMPS_ELASTICSEARCH_FILTERS_APIIDS
Type: []string

Filters pump data by an allow list of api_ids.

pumps.elasticsearch.filters.response_codes

ENV: TYK_PMP_PUMPS_ELASTICSEARCH_FILTERS_RESPONSECODES
Type: []int

Filters pump data by an allow list of response_codes.

pumps.elasticsearch.filters.skip_org_ids

ENV: TYK_PMP_PUMPS_ELASTICSEARCH_FILTERS_SKIPPEDORGSIDS
Type: []string

Filters pump data by a block list of org_ids.

pumps.elasticsearch.filters.skip_api_ids

ENV: TYK_PMP_PUMPS_ELASTICSEARCH_FILTERS_SKIPPEDAPIIDS
Type: []string

Filters pump data by a block list of api_ids.

pumps.elasticsearch.filters.skip_response_codes

ENV: TYK_PMP_PUMPS_ELASTICSEARCH_FILTERS_SKIPPEDRESPONSECODES
Type: []int

Filters pump data by a block list of response_codes.

pumps.elasticsearch.timeout

ENV: TYK_PMP_PUMPS_ELASTICSEARCH_TIMEOUT
Type: int

By default, a pump will wait forever for each write operation to complete; you can configure an optional timeout by setting the configuration option timeout. If you have deployed multiple pumps, then you can configure each timeout independently. The timeout is in seconds and defaults to 0.

The timeout is configured within the main pump config as shown here; note that this example would configure a 5 second timeout:

"pump_name": {
  ...
  "timeout":5,
  "meta": {...}
}

Tyk will inform you if the pump’s write operation is taking longer than the purging loop (configured via purge_delay) as this will mean that data is purged before being written to the target data sink.

If there is no timeout configured and pump’s write operation is taking longer than the purging loop, the following warning log will be generated: Pump {pump_name} is taking more time than the value configured of purge_delay. You should try to set a timeout for this pump.

If there is a timeout configured, but pump’s write operation is still taking longer than the purging loop, the following warning log will be generated: Pump {pump_name} is taking more time than the value configured of purge_delay. You should try lowering the timeout configured for this pump..

pumps.elasticsearch.omit_detailed_recording

ENV: TYK_PMP_PUMPS_ELASTICSEARCH_OMITDETAILEDRECORDING
Type: bool

Setting this to true will avoid writing raw_request and raw_response fields for each request in pumps. Defaults to false.

pumps.elasticsearch.max_record_size

ENV: TYK_PMP_PUMPS_ELASTICSEARCH_MAXRECORDSIZE
Type: int

Defines maximum size (in bytes) for Raw Request and Raw Response logs, this value defaults to 0. If it is not set then tyk-pump will not trim any data and will store the full information. This can also be set at a pump level. For example:

"csv": {
  "type": "csv",
  "max_record_size":1000,
  "meta": {
    "csv_dir": "./"
  }
}

pumps.elasticsearch.ignore_fields

ENV: TYK_PMP_PUMPS_ELASTICSEARCH_IGNOREFIELDS
Type: []string

IgnoreFields defines a list of analytics fields that will be ignored when writing to the pump. This can be used to avoid writing sensitive information to the Database, or data that you don’t really need to have. The field names must be the same as the JSON tags of the analytics record fields. For example: ["api_key", "api_version"].

pumps.elasticsearch.meta.EnvPrefix

ENV: TYK_PMP_PUMPS_ELASTICSEARCH_META_ENVPREFIX
Type: string

The prefix for the environment variables that will be used to override the configuration. Defaults to TYK_PMP_PUMPS_ELASTICSEARCH_META

pumps.elasticsearch.meta.index_name

ENV: TYK_PMP_PUMPS_ELASTICSEARCH_META_INDEXNAME
Type: string

The name of the index that all the analytics data will be placed in. Defaults to “tyk_analytics”.

pumps.elasticsearch.meta.elasticsearch_url

ENV: TYK_PMP_PUMPS_ELASTICSEARCH_META_ELASTICSEARCHURL
Type: string

If sniffing is disabled, the URL that all data will be sent to. Defaults to “http://localhost:9200”.

pumps.elasticsearch.meta.use_sniffing

ENV: TYK_PMP_PUMPS_ELASTICSEARCH_META_ENABLESNIFFING
Type: bool

If sniffing is enabled, the “elasticsearch_url” will be used to make a request to get a list of all the nodes in the cluster, the returned addresses will then be used. Defaults to false.

pumps.elasticsearch.meta.document_type

ENV: TYK_PMP_PUMPS_ELASTICSEARCH_META_DOCUMENTTYPE
Type: string

The type of the document that is created in ES. Defaults to “tyk_analytics”.

pumps.elasticsearch.meta.rolling_index

ENV: TYK_PMP_PUMPS_ELASTICSEARCH_META_ROLLINGINDEX
Type: bool

Appends the date to the end of the index name, so each days data is split into a different index name. E.g. tyk_analytics-2016.02.28. Defaults to false.

pumps.elasticsearch.meta.extended_stats

ENV: TYK_PMP_PUMPS_ELASTICSEARCH_META_EXTENDEDSTATISTICS
Type: bool

If set to true will include the following additional fields: Raw Request, Raw Response and User Agent.

pumps.elasticsearch.meta.generate_id

ENV: TYK_PMP_PUMPS_ELASTICSEARCH_META_GENERATEID
Type: bool

When enabled, generate _id for outgoing records. This prevents duplicate records when retrying ES.

pumps.elasticsearch.meta.decode_base64

ENV: TYK_PMP_PUMPS_ELASTICSEARCH_META_DECODEBASE64
Type: bool

Allows for the base64 bits to be decode before being passed to ES.

pumps.elasticsearch.meta.version

ENV: TYK_PMP_PUMPS_ELASTICSEARCH_META_VERSION
Type: string

Specifies the ES version. Use “3” for ES 3.X, “5” for ES 5.X, “6” for ES 6.X, “7” for ES 7.X . Defaults to “3”.

pumps.elasticsearch.meta.disable_bulk

ENV: TYK_PMP_PUMPS_ELASTICSEARCH_META_DISABLEBULK
Type: bool

Disable batch writing. Defaults to false.

pumps.elasticsearch.meta.bulk_config

Batch writing trigger configuration. Each option is an OR with eachother:

pumps.elasticsearch.meta.bulk_config.workers

ENV: TYK_PMP_PUMPS_ELASTICSEARCH_META_BULKCONFIG_WORKERS
Type: int

Number of workers. Defaults to 1.

pumps.elasticsearch.meta.bulk_config.flush_interval

ENV: TYK_PMP_PUMPS_ELASTICSEARCH_META_BULKCONFIG_FLUSHINTERVAL
Type: int

Specifies the time in seconds to flush the data and send it to ES. Default disabled.

pumps.elasticsearch.meta.bulk_config.bulk_actions

ENV: TYK_PMP_PUMPS_ELASTICSEARCH_META_BULKCONFIG_BULKACTIONS
Type: int

Specifies the number of requests needed to flush the data and send it to ES. Defaults to 1000 requests. If it is needed, can be disabled with -1.

pumps.elasticsearch.meta.bulk_config.bulk_size

ENV: TYK_PMP_PUMPS_ELASTICSEARCH_META_BULKCONFIG_BULKSIZE
Type: int

Specifies the size (in bytes) needed to flush the data and send it to ES. Defaults to 5MB. If it is needed, can be disabled with -1.

pumps.elasticsearch.meta.auth_api_key_id

ENV: TYK_PMP_PUMPS_ELASTICSEARCH_META_AUTHAPIKEYID
Type: string

API Key ID used for APIKey auth in ES. It’s send to ES in the Authorization header as ApiKey base64(auth_api_key_id:auth_api_key)

pumps.elasticsearch.meta.auth_api_key

ENV: TYK_PMP_PUMPS_ELASTICSEARCH_META_AUTHAPIKEY
Type: string

API Key used for APIKey auth in ES. It’s send to ES in the Authorization header as ApiKey base64(auth_api_key_id:auth_api_key)

pumps.elasticsearch.meta.auth_basic_username

ENV: TYK_PMP_PUMPS_ELASTICSEARCH_META_USERNAME
Type: string

Basic auth username. It’s send to ES in the Authorization header as username:password encoded in base64.

pumps.elasticsearch.meta.auth_basic_password

ENV: TYK_PMP_PUMPS_ELASTICSEARCH_META_PASSWORD
Type: string

Basic auth password. It’s send to ES in the Authorization header as username:password encoded in base64.

pumps.elasticsearch.meta.use_ssl

ENV: TYK_PMP_PUMPS_ELASTICSEARCH_META_USESSL
Type: bool

Enables SSL connection.

pumps.elasticsearch.meta.ssl_insecure_skip_verify

ENV: TYK_PMP_PUMPS_ELASTICSEARCH_META_SSLINSECURESKIPVERIFY
Type: bool

Controls whether the pump client verifies the Elastic Search server’s certificate chain and hostname.

pumps.elasticsearch.meta.ssl_cert_file

ENV: TYK_PMP_PUMPS_ELASTICSEARCH_META_SSLCERTFILE
Type: string

Can be used to set custom certificate file for authentication with Elastic Search.

pumps.elasticsearch.meta.ssl_key_file

ENV: TYK_PMP_PUMPS_ELASTICSEARCH_META_SSLKEYFILE
Type: string

Can be used to set custom key file for authentication with Elastic Search.

pumps.graylog.name

ENV: TYK_PMP_PUMPS_GRAYLOG_NAME
Type: string

The name of the pump. This is used to identify the pump in the logs. Deprecated, use type instead.

pumps.graylog.type

ENV: TYK_PMP_PUMPS_GRAYLOG_TYPE
Type: string

Sets the pump type. This is needed when the pump key does not equal to the pump name type. Current valid types are: mongo, mongo-pump-selective, mongo-pump-aggregate, csv, elasticsearch, influx, influx2, moesif, statsd, segment, graylog, splunk, hybrid, prometheus, logzio, dogstatsd, kafka, syslog, sql, sql_aggregate, stdout, timestream, mongo-graph, sql-graph, sql-graph-aggregate, resurfaceio.

pumps.graylog.filters

This feature adds a new configuration field in each pump called filters and its structure is the following:

"filters":{
  "api_ids":[],
  "org_ids":[],
  "response_codes":[],
  "skip_api_ids":[],
  "skip_org_ids":[],
  "skip_response_codes":[]
}

The fields api_ids, org_ids and response_codes works as allow list (APIs and orgs where we want to send the analytics records) and the fields skip_api_ids, skip_org_ids and skip_response_codes works as block list.

The priority is always block list configurations over allow list.

An example of configuration would be:

"csv": {
 "type": "csv",
 "filters": {
   "org_ids": ["org1","org2"]
 },
 "meta": {
   "csv_dir": "./bar"
 }
}

pumps.graylog.filters.org_ids

ENV: TYK_PMP_PUMPS_GRAYLOG_FILTERS_ORGSIDS
Type: []string

Filters pump data by an allow list of org_ids.

pumps.graylog.filters.api_ids

ENV: TYK_PMP_PUMPS_GRAYLOG_FILTERS_APIIDS
Type: []string

Filters pump data by an allow list of api_ids.

pumps.graylog.filters.response_codes

ENV: TYK_PMP_PUMPS_GRAYLOG_FILTERS_RESPONSECODES
Type: []int

Filters pump data by an allow list of response_codes.

pumps.graylog.filters.skip_org_ids

ENV: TYK_PMP_PUMPS_GRAYLOG_FILTERS_SKIPPEDORGSIDS
Type: []string

Filters pump data by a block list of org_ids.

pumps.graylog.filters.skip_api_ids

ENV: TYK_PMP_PUMPS_GRAYLOG_FILTERS_SKIPPEDAPIIDS
Type: []string

Filters pump data by a block list of api_ids.

pumps.graylog.filters.skip_response_codes

ENV: TYK_PMP_PUMPS_GRAYLOG_FILTERS_SKIPPEDRESPONSECODES
Type: []int

Filters pump data by a block list of response_codes.

pumps.graylog.timeout

ENV: TYK_PMP_PUMPS_GRAYLOG_TIMEOUT
Type: int

By default, a pump will wait forever for each write operation to complete; you can configure an optional timeout by setting the configuration option timeout. If you have deployed multiple pumps, then you can configure each timeout independently. The timeout is in seconds and defaults to 0.

The timeout is configured within the main pump config as shown here; note that this example would configure a 5 second timeout:

"pump_name": {
  ...
  "timeout":5,
  "meta": {...}
}

Tyk will inform you if the pump’s write operation is taking longer than the purging loop (configured via purge_delay) as this will mean that data is purged before being written to the target data sink.

If there is no timeout configured and pump’s write operation is taking longer than the purging loop, the following warning log will be generated: Pump {pump_name} is taking more time than the value configured of purge_delay. You should try to set a timeout for this pump.

If there is a timeout configured, but pump’s write operation is still taking longer than the purging loop, the following warning log will be generated: Pump {pump_name} is taking more time than the value configured of purge_delay. You should try lowering the timeout configured for this pump..

pumps.graylog.omit_detailed_recording

ENV: TYK_PMP_PUMPS_GRAYLOG_OMITDETAILEDRECORDING
Type: bool

Setting this to true will avoid writing raw_request and raw_response fields for each request in pumps. Defaults to false.

pumps.graylog.max_record_size

ENV: TYK_PMP_PUMPS_GRAYLOG_MAXRECORDSIZE
Type: int

Defines maximum size (in bytes) for Raw Request and Raw Response logs, this value defaults to 0. If it is not set then tyk-pump will not trim any data and will store the full information. This can also be set at a pump level. For example:

"csv": {
  "type": "csv",
  "max_record_size":1000,
  "meta": {
    "csv_dir": "./"
  }
}

pumps.graylog.ignore_fields

ENV: TYK_PMP_PUMPS_GRAYLOG_IGNOREFIELDS
Type: []string

IgnoreFields defines a list of analytics fields that will be ignored when writing to the pump. This can be used to avoid writing sensitive information to the Database, or data that you don’t really need to have. The field names must be the same as the JSON tags of the analytics record fields. For example: ["api_key", "api_version"].

pumps.graylog.meta.EnvPrefix

ENV: TYK_PMP_PUMPS_GRAYLOG_META_ENVPREFIX
Type: string

The prefix for the environment variables that will be used to override the configuration. Defaults to TYK_PMP_PUMPS_GRAYLOG_META

pumps.graylog.meta.host

ENV: TYK_PMP_PUMPS_GRAYLOG_META_GRAYLOGHOST
Type: string

Graylog host.

pumps.graylog.meta.port

ENV: TYK_PMP_PUMPS_GRAYLOG_META_GRAYLOGPORT
Type: int

Graylog port.

pumps.graylog.meta.tags

ENV: TYK_PMP_PUMPS_GRAYLOG_META_TAGS
Type: []string

List of tags to be added to the metric. The possible options are listed in the below example.

If no tag is specified the fallback behaviour is to don’t send anything. The possible values are:

• path
• method
• response_code
• api_version
• api_name
• api_id
• org_id
• tracked
• oauth_id
• raw_request
• raw_response
• request_time
• ip_address

pumps.hybrid.name

ENV: TYK_PMP_PUMPS_HYBRID_NAME
Type: string

The name of the pump. This is used to identify the pump in the logs. Deprecated, use type instead.

pumps.hybrid.type

ENV: TYK_PMP_PUMPS_HYBRID_TYPE
Type: string

Sets the pump type. This is needed when the pump key does not equal to the pump name type. Current valid types are: mongo, mongo-pump-selective, mongo-pump-aggregate, csv, elasticsearch, influx, influx2, moesif, statsd, segment, graylog, splunk, hybrid, prometheus, logzio, dogstatsd, kafka, syslog, sql, sql_aggregate, stdout, timestream, mongo-graph, sql-graph, sql-graph-aggregate, resurfaceio.

pumps.hybrid.filters

This feature adds a new configuration field in each pump called filters and its structure is the following:

"filters":{
  "api_ids":[],
  "org_ids":[],
  "response_codes":[],
  "skip_api_ids":[],
  "skip_org_ids":[],
  "skip_response_codes":[]
}

The fields api_ids, org_ids and response_codes works as allow list (APIs and orgs where we want to send the analytics records) and the fields skip_api_ids, skip_org_ids and skip_response_codes works as block list.

The priority is always block list configurations over allow list.

An example of configuration would be:

"csv": {
 "type": "csv",
 "filters": {
   "org_ids": ["org1","org2"]
 },
 "meta": {
   "csv_dir": "./bar"
 }
}

pumps.hybrid.filters.org_ids

ENV: TYK_PMP_PUMPS_HYBRID_FILTERS_ORGSIDS
Type: []string

Filters pump data by an allow list of org_ids.

pumps.hybrid.filters.api_ids

ENV: TYK_PMP_PUMPS_HYBRID_FILTERS_APIIDS
Type: []string

Filters pump data by an allow list of api_ids.

pumps.hybrid.filters.response_codes

ENV: TYK_PMP_PUMPS_HYBRID_FILTERS_RESPONSECODES
Type: []int

Filters pump data by an allow list of response_codes.

pumps.hybrid.filters.skip_org_ids

ENV: TYK_PMP_PUMPS_HYBRID_FILTERS_SKIPPEDORGSIDS
Type: []string

Filters pump data by a block list of org_ids.

pumps.hybrid.filters.skip_api_ids

ENV: TYK_PMP_PUMPS_HYBRID_FILTERS_SKIPPEDAPIIDS
Type: []string

Filters pump data by a block list of api_ids.

pumps.hybrid.filters.skip_response_codes

ENV: TYK_PMP_PUMPS_HYBRID_FILTERS_SKIPPEDRESPONSECODES
Type: []int

Filters pump data by a block list of response_codes.

pumps.hybrid.timeout

ENV: TYK_PMP_PUMPS_HYBRID_TIMEOUT
Type: int

By default, a pump will wait forever for each write operation to complete; you can configure an optional timeout by setting the configuration option timeout. If you have deployed multiple pumps, then you can configure each timeout independently. The timeout is in seconds and defaults to 0.

The timeout is configured within the main pump config as shown here; note that this example would configure a 5 second timeout:

"pump_name": {
  ...
  "timeout":5,
  "meta": {...}
}

Tyk will inform you if the pump’s write operation is taking longer than the purging loop (configured via purge_delay) as this will mean that data is purged before being written to the target data sink.

If there is no timeout configured and pump’s write operation is taking longer than the purging loop, the following warning log will be generated: Pump {pump_name} is taking more time than the value configured of purge_delay. You should try to set a timeout for this pump.

If there is a timeout configured, but pump’s write operation is still taking longer than the purging loop, the following warning log will be generated: Pump {pump_name} is taking more time than the value configured of purge_delay. You should try lowering the timeout configured for this pump..

pumps.hybrid.omit_detailed_recording

ENV: TYK_PMP_PUMPS_HYBRID_OMITDETAILEDRECORDING
Type: bool

Setting this to true will avoid writing raw_request and raw_response fields for each request in pumps. Defaults to false.

pumps.hybrid.max_record_size

ENV: TYK_PMP_PUMPS_HYBRID_MAXRECORDSIZE
Type: int

Defines maximum size (in bytes) for Raw Request and Raw Response logs, this value defaults to 0. If it is not set then tyk-pump will not trim any data and will store the full information. This can also be set at a pump level. For example:

"csv": {
  "type": "csv",
  "max_record_size":1000,
  "meta": {
    "csv_dir": "./"
  }
}

pumps.hybrid.ignore_fields

ENV: TYK_PMP_PUMPS_HYBRID_IGNOREFIELDS
Type: []string

IgnoreFields defines a list of analytics fields that will be ignored when writing to the pump. This can be used to avoid writing sensitive information to the Database, or data that you don’t really need to have. The field names must be the same as the JSON tags of the analytics record fields. For example: ["api_key", "api_version"].

pumps.hybrid.meta.EnvPrefix

ENV: TYK_PMP_PUMPS_HYBRID_META_ENVPREFIX
Type: string

The prefix for the environment variables that will be used to override the configuration. Defaults to TYK_PMP_PUMPS_HYBRID_META

pumps.hybrid.meta.ConnectionString

ENV: TYK_PMP_PUMPS_HYBRID_META_CONNECTIONSTRING
Type: string

MDCB URL connection string

pumps.hybrid.meta.RPCKey

ENV: TYK_PMP_PUMPS_HYBRID_META_RPCKEY
Type: string

Your organization ID to connect to the MDCB installation.

pumps.hybrid.meta.APIKey

ENV: TYK_PMP_PUMPS_HYBRID_META_APIKEY
Type: string

This the API key of a user used to authenticate and authorize the Hybrid Pump access through MDCB. The user should be a standard Dashboard user with minimal privileges so as to reduce any risk if the user is compromised.

pumps.hybrid.meta.ignore_tag_prefix_list

ENV: TYK_PMP_PUMPS_HYBRID_META_IGNORETAGPREFIXLIST
Type: []string

Specifies prefixes of tags that should be ignored if aggregated is set to true.

pumps.hybrid.meta.CallTimeout

ENV: TYK_PMP_PUMPS_HYBRID_META_CALLTIMEOUT
Type: int

Hybrid pump RPC calls timeout in seconds. Defaults to 10 seconds.

pumps.hybrid.meta.RPCPoolSize

ENV: TYK_PMP_PUMPS_HYBRID_META_RPCPOOLSIZE
Type: int

Hybrid pump connection pool size. Defaults to 5.

pumps.hybrid.meta.aggregationTime

ENV: TYK_PMP_PUMPS_HYBRID_META_AGGREGATIONTIME
Type: int

aggregationTime is to specify the frequency of the aggregation in minutes if aggregated is set to true.

pumps.hybrid.meta.Aggregated

ENV: TYK_PMP_PUMPS_HYBRID_META_AGGREGATED
Type: bool

Send aggregated analytics data to Tyk MDCB

pumps.hybrid.meta.TrackAllPaths

ENV: TYK_PMP_PUMPS_HYBRID_META_TRACKALLPATHS
Type: bool

Specifies if it should store aggregated data for all the endpoints if aggregated is set to true. By default, false which means that only store aggregated data for tracked endpoints.

pumps.hybrid.meta.store_analytics_per_minute

ENV: TYK_PMP_PUMPS_HYBRID_META_STOREANALYTICSPERMINUTE
Type: bool

Determines if the aggregations should be made per minute (true) or per hour (false) if aggregated is set to true.

pumps.hybrid.meta.UseSSL

ENV: TYK_PMP_PUMPS_HYBRID_META_USESSL
Type: bool

Use SSL to connect to Tyk MDCB

pumps.hybrid.meta.SSLInsecureSkipVerify

ENV: TYK_PMP_PUMPS_HYBRID_META_SSLINSECURESKIPVERIFY
Type: bool

Skip SSL verification

pumps.influx.name

ENV: TYK_PMP_PUMPS_INFLUX_NAME
Type: string

The name of the pump. This is used to identify the pump in the logs. Deprecated, use type instead.

pumps.influx.type

ENV: TYK_PMP_PUMPS_INFLUX_TYPE
Type: string

Sets the pump type. This is needed when the pump key does not equal to the pump name type. Current valid types are: mongo, mongo-pump-selective, mongo-pump-aggregate, csv, elasticsearch, influx, influx2, moesif, statsd, segment, graylog, splunk, hybrid, prometheus, logzio, dogstatsd, kafka, syslog, sql, sql_aggregate, stdout, timestream, mongo-graph, sql-graph, sql-graph-aggregate, resurfaceio.

pumps.influx.filters

This feature adds a new configuration field in each pump called filters and its structure is the following:

"filters":{
  "api_ids":[],
  "org_ids":[],
  "response_codes":[],
  "skip_api_ids":[],
  "skip_org_ids":[],
  "skip_response_codes":[]
}

The fields api_ids, org_ids and response_codes works as allow list (APIs and orgs where we want to send the analytics records) and the fields skip_api_ids, skip_org_ids and skip_response_codes works as block list.

The priority is always block list configurations over allow list.

An example of configuration would be:

"csv": {
 "type": "csv",
 "filters": {
   "org_ids": ["org1","org2"]
 },
 "meta": {
   "csv_dir": "./bar"
 }
}

pumps.influx.filters.org_ids

ENV: TYK_PMP_PUMPS_INFLUX_FILTERS_ORGSIDS
Type: []string

Filters pump data by an allow list of org_ids.

pumps.influx.filters.api_ids

ENV: TYK_PMP_PUMPS_INFLUX_FILTERS_APIIDS
Type: []string

Filters pump data by an allow list of api_ids.

pumps.influx.filters.response_codes

ENV: TYK_PMP_PUMPS_INFLUX_FILTERS_RESPONSECODES
Type: []int

Filters pump data by an allow list of response_codes.

pumps.influx.filters.skip_org_ids

ENV: TYK_PMP_PUMPS_INFLUX_FILTERS_SKIPPEDORGSIDS
Type: []string

Filters pump data by a block list of org_ids.

pumps.influx.filters.skip_api_ids

ENV: TYK_PMP_PUMPS_INFLUX_FILTERS_SKIPPEDAPIIDS
Type: []string

Filters pump data by a block list of api_ids.

pumps.influx.filters.skip_response_codes

ENV: TYK_PMP_PUMPS_INFLUX_FILTERS_SKIPPEDRESPONSECODES
Type: []int

Filters pump data by a block list of response_codes.

pumps.influx.timeout

ENV: TYK_PMP_PUMPS_INFLUX_TIMEOUT
Type: int

By default, a pump will wait forever for each write operation to complete; you can configure an optional timeout by setting the configuration option timeout. If you have deployed multiple pumps, then you can configure each timeout independently. The timeout is in seconds and defaults to 0.

The timeout is configured within the main pump config as shown here; note that this example would configure a 5 second timeout:

"pump_name": {
  ...
  "timeout":5,
  "meta": {...}
}

Tyk will inform you if the pump’s write operation is taking longer than the purging loop (configured via purge_delay) as this will mean that data is purged before being written to the target data sink.

If there is no timeout configured and pump’s write operation is taking longer than the purging loop, the following warning log will be generated: Pump {pump_name} is taking more time than the value configured of purge_delay. You should try to set a timeout for this pump.

If there is a timeout configured, but pump’s write operation is still taking longer than the purging loop, the following warning log will be generated: Pump {pump_name} is taking more time than the value configured of purge_delay. You should try lowering the timeout configured for this pump..

pumps.influx.omit_detailed_recording

ENV: TYK_PMP_PUMPS_INFLUX_OMITDETAILEDRECORDING
Type: bool

Setting this to true will avoid writing raw_request and raw_response fields for each request in pumps. Defaults to false.

pumps.influx.max_record_size

ENV: TYK_PMP_PUMPS_INFLUX_MAXRECORDSIZE
Type: int

Defines maximum size (in bytes) for Raw Request and Raw Response logs, this value defaults to 0. If it is not set then tyk-pump will not trim any data and will store the full information. This can also be set at a pump level. For example:

"csv": {
  "type": "csv",
  "max_record_size":1000,
  "meta": {
    "csv_dir": "./"
  }
}

pumps.influx.ignore_fields

ENV: TYK_PMP_PUMPS_INFLUX_IGNOREFIELDS
Type: []string

IgnoreFields defines a list of analytics fields that will be ignored when writing to the pump. This can be used to avoid writing sensitive information to the Database, or data that you don’t really need to have. The field names must be the same as the JSON tags of the analytics record fields. For example: ["api_key", "api_version"].

pumps.influx.meta.EnvPrefix

ENV: TYK_PMP_PUMPS_INFLUX_META_ENVPREFIX
Type: string

The prefix for the environment variables that will be used to override the configuration. Defaults to TYK_PMP_PUMPS_INFLUX_META

pumps.influx.meta.database_name

ENV: TYK_PMP_PUMPS_INFLUX_META_DATABASENAME
Type: string

InfluxDB pump database name.

pumps.influx.meta.address

ENV: TYK_PMP_PUMPS_INFLUX_META_ADDR
Type: string

InfluxDB pump host.

pumps.influx.meta.username

ENV: TYK_PMP_PUMPS_INFLUX_META_USERNAME
Type: string

InfluxDB pump database username.

pumps.influx.meta.password

ENV: TYK_PMP_PUMPS_INFLUX_META_PASSWORD
Type: string

InfluxDB pump database password.

pumps.influx.meta.fields

ENV: TYK_PMP_PUMPS_INFLUX_META_FIELDS
Type: []string

Define which Analytics fields should be sent to InfluxDB. Check the available fields in the example below. Default value is ["method", "path", "response_code", "api_key", "time_stamp", "api_version", "api_name", "api_id", "org_id", "oauth_id", "raw_request", "request_time", "raw_response", "ip_address"].

pumps.influx.meta.tags

ENV: TYK_PMP_PUMPS_INFLUX_META_TAGS
Type: []string

List of tags to be added to the metric.

pumps.kafka.name

ENV: TYK_PMP_PUMPS_KAFKA_NAME
Type: string

The name of the pump. This is used to identify the pump in the logs. Deprecated, use type instead.

pumps.kafka.type

ENV: TYK_PMP_PUMPS_KAFKA_TYPE
Type: string

Sets the pump type. This is needed when the pump key does not equal to the pump name type. Current valid types are: mongo, mongo-pump-selective, mongo-pump-aggregate, csv, elasticsearch, influx, influx2, moesif, statsd, segment, graylog, splunk, hybrid, prometheus, logzio, dogstatsd, kafka, syslog, sql, sql_aggregate, stdout, timestream, mongo-graph, sql-graph, sql-graph-aggregate, resurfaceio.

pumps.kafka.filters

This feature adds a new configuration field in each pump called filters and its structure is the following:

"filters":{
  "api_ids":[],
  "org_ids":[],
  "response_codes":[],
  "skip_api_ids":[],
  "skip_org_ids":[],
  "skip_response_codes":[]
}

The fields api_ids, org_ids and response_codes works as allow list (APIs and orgs where we want to send the analytics records) and the fields skip_api_ids, skip_org_ids and skip_response_codes works as block list.

The priority is always block list configurations over allow list.

An example of configuration would be:

"csv": {
 "type": "csv",
 "filters": {
   "org_ids": ["org1","org2"]
 },
 "meta": {
   "csv_dir": "./bar"
 }
}

pumps.kafka.filters.org_ids

ENV: TYK_PMP_PUMPS_KAFKA_FILTERS_ORGSIDS
Type: []string

Filters pump data by an allow list of org_ids.

pumps.kafka.filters.api_ids

ENV: TYK_PMP_PUMPS_KAFKA_FILTERS_APIIDS
Type: []string

Filters pump data by an allow list of api_ids.

pumps.kafka.filters.response_codes

ENV: TYK_PMP_PUMPS_KAFKA_FILTERS_RESPONSECODES
Type: []int

Filters pump data by an allow list of response_codes.

pumps.kafka.filters.skip_org_ids

ENV: TYK_PMP_PUMPS_KAFKA_FILTERS_SKIPPEDORGSIDS
Type: []string

Filters pump data by a block list of org_ids.

pumps.kafka.filters.skip_api_ids

ENV: TYK_PMP_PUMPS_KAFKA_FILTERS_SKIPPEDAPIIDS
Type: []string

Filters pump data by a block list of api_ids.

pumps.kafka.filters.skip_response_codes

ENV: TYK_PMP_PUMPS_KAFKA_FILTERS_SKIPPEDRESPONSECODES
Type: []int

Filters pump data by a block list of response_codes.

pumps.kafka.timeout

ENV: TYK_PMP_PUMPS_KAFKA_TIMEOUT
Type: int

By default, a pump will wait forever for each write operation to complete; you can configure an optional timeout by setting the configuration option timeout. If you have deployed multiple pumps, then you can configure each timeout independently. The timeout is in seconds and defaults to 0.

The timeout is configured within the main pump config as shown here; note that this example would configure a 5 second timeout:

"pump_name": {
  ...
  "timeout":5,
  "meta": {...}
}

Tyk will inform you if the pump’s write operation is taking longer than the purging loop (configured via purge_delay) as this will mean that data is purged before being written to the target data sink.

If there is no timeout configured and pump’s write operation is taking longer than the purging loop, the following warning log will be generated: Pump {pump_name} is taking more time than the value configured of purge_delay. You should try to set a timeout for this pump.

If there is a timeout configured, but pump’s write operation is still taking longer than the purging loop, the following warning log will be generated: Pump {pump_name} is taking more time than the value configured of purge_delay. You should try lowering the timeout configured for this pump..

pumps.kafka.omit_detailed_recording

ENV: TYK_PMP_PUMPS_KAFKA_OMITDETAILEDRECORDING
Type: bool

Setting this to true will avoid writing raw_request and raw_response fields for each request in pumps. Defaults to false.

pumps.kafka.max_record_size

ENV: TYK_PMP_PUMPS_KAFKA_MAXRECORDSIZE
Type: int

Defines maximum size (in bytes) for Raw Request and Raw Response logs, this value defaults to 0. If it is not set then tyk-pump will not trim any data and will store the full information. This can also be set at a pump level. For example:

"csv": {
  "type": "csv",
  "max_record_size":1000,
  "meta": {
    "csv_dir": "./"
  }
}

pumps.kafka.ignore_fields

ENV: TYK_PMP_PUMPS_KAFKA_IGNOREFIELDS
Type: []string

IgnoreFields defines a list of analytics fields that will be ignored when writing to the pump. This can be used to avoid writing sensitive information to the Database, or data that you don’t really need to have. The field names must be the same as the JSON tags of the analytics record fields. For example: ["api_key", "api_version"].

pumps.kafka.meta.EnvPrefix

ENV: TYK_PMP_PUMPS_KAFKA_META_ENVPREFIX
Type: string

The prefix for the environment variables that will be used to override the configuration. Defaults to TYK_PMP_PUMPS_KAFKA_META

pumps.kafka.meta.broker

ENV: TYK_PMP_PUMPS_KAFKA_META_BROKER
Type: []string

The list of brokers used to discover the partitions available on the kafka cluster. E.g. “localhost:9092”.

pumps.kafka.meta.client_id

ENV: TYK_PMP_PUMPS_KAFKA_META_CLIENTID
Type: string

Unique identifier for client connections established with Kafka.

pumps.kafka.meta.topic

ENV: TYK_PMP_PUMPS_KAFKA_META_TOPIC
Type: string

The topic that the writer will produce messages to.

pumps.kafka.meta.timeout

ENV: TYK_PMP_PUMPS_KAFKA_META_TIMEOUT
Type: interface{}

Timeout is the maximum amount of seconds to wait for a connect or write to complete.

pumps.kafka.meta.compressed

ENV: TYK_PMP_PUMPS_KAFKA_META_COMPRESSED
Type: bool

Enable “github.com/golang/snappy” codec to be used to compress Kafka messages. By default is false.

pumps.kafka.meta.meta_data

ENV: TYK_PMP_PUMPS_KAFKA_META_METADATA
Type: map[string]string

Can be used to set custom metadata inside the kafka message.

pumps.kafka.meta.use_ssl

ENV: TYK_PMP_PUMPS_KAFKA_META_USESSL
Type: bool

Enables SSL connection.

pumps.kafka.meta.ssl_insecure_skip_verify

ENV: TYK_PMP_PUMPS_KAFKA_META_SSLINSECURESKIPVERIFY
Type: bool

Controls whether the pump client verifies the kafka server’s certificate chain and host name.

pumps.kafka.meta.ssl_cert_file

ENV: TYK_PMP_PUMPS_KAFKA_META_SSLCERTFILE
Type: string

Can be used to set custom certificate file for authentication with kafka.

pumps.kafka.meta.ssl_key_file

ENV: TYK_PMP_PUMPS_KAFKA_META_SSLKEYFILE
Type: string

Can be used to set custom key file for authentication with kafka.

pumps.kafka.meta.sasl_mechanism

ENV: TYK_PMP_PUMPS_KAFKA_META_SASLMECHANISM
Type: string

SASL mechanism configuration. Only “plain” and “scram” are supported.

pumps.kafka.meta.sasl_username

ENV: TYK_PMP_PUMPS_KAFKA_META_USERNAME
Type: string

SASL username.

pumps.kafka.meta.sasl_password

ENV: TYK_PMP_PUMPS_KAFKA_META_PASSWORD
Type: string

SASL password.

pumps.kafka.meta.sasl_algorithm

ENV: TYK_PMP_PUMPS_KAFKA_META_ALGORITHM
Type: string

SASL algorithm. It’s the algorithm specified for scram mechanism. It could be sha-512 or sha-256. Defaults to “sha-256”.

pumps.kinesis.name

ENV: TYK_PMP_PUMPS_KINESIS_NAME
Type: string

The name of the pump. This is used to identify the pump in the logs. Deprecated, use type instead.

pumps.kinesis.type

ENV: TYK_PMP_PUMPS_KINESIS_TYPE
Type: string

Sets the pump type. This is needed when the pump key does not equal to the pump name type. Current valid types are: mongo, mongo-pump-selective, mongo-pump-aggregate, csv, elasticsearch, influx, influx2, moesif, statsd, segment, graylog, splunk, hybrid, prometheus, logzio, dogstatsd, kafka, syslog, sql, sql_aggregate, stdout, timestream, mongo-graph, sql-graph, sql-graph-aggregate, resurfaceio.

pumps.kinesis.filters

This feature adds a new configuration field in each pump called filters and its structure is the following:

"filters":{
  "api_ids":[],
  "org_ids":[],
  "response_codes":[],
  "skip_api_ids":[],
  "skip_org_ids":[],
  "skip_response_codes":[]
}

The fields api_ids, org_ids and response_codes works as allow list (APIs and orgs where we want to send the analytics records) and the fields skip_api_ids, skip_org_ids and skip_response_codes works as block list.

The priority is always block list configurations over allow list.

An example of configuration would be:

"csv": {
 "type": "csv",
 "filters": {
   "org_ids": ["org1","org2"]
 },
 "meta": {
   "csv_dir": "./bar"
 }
}

pumps.kinesis.filters.org_ids

ENV: TYK_PMP_PUMPS_KINESIS_FILTERS_ORGSIDS
Type: []string

Filters pump data by an allow list of org_ids.

pumps.kinesis.filters.api_ids

ENV: TYK_PMP_PUMPS_KINESIS_FILTERS_APIIDS
Type: []string

Filters pump data by an allow list of api_ids.

pumps.kinesis.filters.response_codes

ENV: TYK_PMP_PUMPS_KINESIS_FILTERS_RESPONSECODES
Type: []int

Filters pump data by an allow list of response_codes.

pumps.kinesis.filters.skip_org_ids

ENV: TYK_PMP_PUMPS_KINESIS_FILTERS_SKIPPEDORGSIDS
Type: []string

Filters pump data by a block list of org_ids.

pumps.kinesis.filters.skip_api_ids

ENV: TYK_PMP_PUMPS_KINESIS_FILTERS_SKIPPEDAPIIDS
Type: []string

Filters pump data by a block list of api_ids.

pumps.kinesis.filters.skip_response_codes

ENV: TYK_PMP_PUMPS_KINESIS_FILTERS_SKIPPEDRESPONSECODES
Type: []int

Filters pump data by a block list of response_codes.

pumps.kinesis.timeout

ENV: TYK_PMP_PUMPS_KINESIS_TIMEOUT
Type: int

By default, a pump will wait forever for each write operation to complete; you can configure an optional timeout by setting the configuration option timeout. If you have deployed multiple pumps, then you can configure each timeout independently. The timeout is in seconds and defaults to 0.

The timeout is configured within the main pump config as shown here; note that this example would configure a 5 second timeout:

"pump_name": {
  ...
  "timeout":5,
  "meta": {...}
}

Tyk will inform you if the pump’s write operation is taking longer than the purging loop (configured via purge_delay) as this will mean that data is purged before being written to the target data sink.

If there is no timeout configured and pump’s write operation is taking longer than the purging loop, the following warning log will be generated: Pump {pump_name} is taking more time than the value configured of purge_delay. You should try to set a timeout for this pump.

If there is a timeout configured, but pump’s write operation is still taking longer than the purging loop, the following warning log will be generated: Pump {pump_name} is taking more time than the value configured of purge_delay. You should try lowering the timeout configured for this pump..

pumps.kinesis.omit_detailed_recording

ENV: TYK_PMP_PUMPS_KINESIS_OMITDETAILEDRECORDING
Type: bool

Setting this to true will avoid writing raw_request and raw_response fields for each request in pumps. Defaults to false.

pumps.kinesis.max_record_size

ENV: TYK_PMP_PUMPS_KINESIS_MAXRECORDSIZE
Type: int

Defines maximum size (in bytes) for Raw Request and Raw Response logs, this value defaults to 0. If it is not set then tyk-pump will not trim any data and will store the full information. This can also be set at a pump level. For example:

"csv": {
  "type": "csv",
  "max_record_size":1000,
  "meta": {
    "csv_dir": "./"
  }
}

pumps.kinesis.ignore_fields

ENV: TYK_PMP_PUMPS_KINESIS_IGNOREFIELDS
Type: []string

IgnoreFields defines a list of analytics fields that will be ignored when writing to the pump. This can be used to avoid writing sensitive information to the Database, or data that you don’t really need to have. The field names must be the same as the JSON tags of the analytics record fields. For example: ["api_key", "api_version"].

pumps.kinesis.meta.EnvPrefix

ENV: TYK_PMP_PUMPS_KINESIS_META_ENVPREFIX
Type: string

The prefix for the environment variables that will be used to override the configuration. Defaults to TYK_PMP_PUMPS_KINESIS_META

pumps.kinesis.meta.StreamName

ENV: TYK_PMP_PUMPS_KINESIS_META_STREAMNAME
Type: string

A name to identify the stream. The stream name is scoped to the AWS account used by the application that creates the stream. It is also scoped by AWS Region. That is, two streams in two different AWS accounts can have the same name. Two streams in the same AWS account but in two different Regions can also have the same name.

pumps.kinesis.meta.Region

ENV: TYK_PMP_PUMPS_KINESIS_META_REGION
Type: string

AWS Region the Kinesis stream targets

pumps.kinesis.meta.BatchSize

ENV: TYK_PMP_PUMPS_KINESIS_META_BATCHSIZE
Type: int

Each PutRecords (the function used in this pump)request can support up to 500 records. Each record in the request can be as large as 1 MiB, up to a limit of 5 MiB for the entire request, including partition keys. Each shard can support writes up to 1,000 records per second, up to a maximum data write total of 1 MiB per second.

pumps.logzio.name

ENV: TYK_PMP_PUMPS_LOGZIO_NAME
Type: string

The name of the pump. This is used to identify the pump in the logs. Deprecated, use type instead.

pumps.logzio.type

ENV: TYK_PMP_PUMPS_LOGZIO_TYPE
Type: string

Sets the pump type. This is needed when the pump key does not equal to the pump name type. Current valid types are: mongo, mongo-pump-selective, mongo-pump-aggregate, csv, elasticsearch, influx, influx2, moesif, statsd, segment, graylog, splunk, hybrid, prometheus, logzio, dogstatsd, kafka, syslog, sql, sql_aggregate, stdout, timestream, mongo-graph, sql-graph, sql-graph-aggregate, resurfaceio.

pumps.logzio.filters

This feature adds a new configuration field in each pump called filters and its structure is the following:

"filters":{
  "api_ids":[],
  "org_ids":[],
  "response_codes":[],
  "skip_api_ids":[],
  "skip_org_ids":[],
  "skip_response_codes":[]
}

The fields api_ids, org_ids and response_codes works as allow list (APIs and orgs where we want to send the analytics records) and the fields skip_api_ids, skip_org_ids and skip_response_codes works as block list.

The priority is always block list configurations over allow list.

An example of configuration would be:

"csv": {
 "type": "csv",
 "filters": {
   "org_ids": ["org1","org2"]
 },
 "meta": {
   "csv_dir": "./bar"
 }
}

pumps.logzio.filters.org_ids

ENV: TYK_PMP_PUMPS_LOGZIO_FILTERS_ORGSIDS
Type: []string

Filters pump data by an allow list of org_ids.

pumps.logzio.filters.api_ids

ENV: TYK_PMP_PUMPS_LOGZIO_FILTERS_APIIDS
Type: []string

Filters pump data by an allow list of api_ids.

pumps.logzio.filters.response_codes

ENV: TYK_PMP_PUMPS_LOGZIO_FILTERS_RESPONSECODES
Type: []int

Filters pump data by an allow list of response_codes.

pumps.logzio.filters.skip_org_ids

ENV: TYK_PMP_PUMPS_LOGZIO_FILTERS_SKIPPEDORGSIDS
Type: []string

Filters pump data by a block list of org_ids.

pumps.logzio.filters.skip_api_ids

ENV: TYK_PMP_PUMPS_LOGZIO_FILTERS_SKIPPEDAPIIDS
Type: []string

Filters pump data by a block list of api_ids.

pumps.logzio.filters.skip_response_codes

ENV: TYK_PMP_PUMPS_LOGZIO_FILTERS_SKIPPEDRESPONSECODES
Type: []int

Filters pump data by a block list of response_codes.

pumps.logzio.timeout

ENV: TYK_PMP_PUMPS_LOGZIO_TIMEOUT
Type: int

By default, a pump will wait forever for each write operation to complete; you can configure an optional timeout by setting the configuration option timeout. If you have deployed multiple pumps, then you can configure each timeout independently. The timeout is in seconds and defaults to 0.

The timeout is configured within the main pump config as shown here; note that this example would configure a 5 second timeout:

"pump_name": {
  ...
  "timeout":5,
  "meta": {...}
}

Tyk will inform you if the pump’s write operation is taking longer than the purging loop (configured via purge_delay) as this will mean that data is purged before being written to the target data sink.

If there is no timeout configured and pump’s write operation is taking longer than the purging loop, the following warning log will be generated: Pump {pump_name} is taking more time than the value configured of purge_delay. You should try to set a timeout for this pump.

If there is a timeout configured, but pump’s write operation is still taking longer than the purging loop, the following warning log will be generated: Pump {pump_name} is taking more time than the value configured of purge_delay. You should try lowering the timeout configured for this pump..

pumps.logzio.omit_detailed_recording

ENV: TYK_PMP_PUMPS_LOGZIO_OMITDETAILEDRECORDING
Type: bool

Setting this to true will avoid writing raw_request and raw_response fields for each request in pumps. Defaults to false.

pumps.logzio.max_record_size

ENV: TYK_PMP_PUMPS_LOGZIO_MAXRECORDSIZE
Type: int

Defines maximum size (in bytes) for Raw Request and Raw Response logs, this value defaults to 0. If it is not set then tyk-pump will not trim any data and will store the full information. This can also be set at a pump level. For example:

"csv": {
  "type": "csv",
  "max_record_size":1000,
  "meta": {
    "csv_dir": "./"
  }
}

pumps.logzio.ignore_fields

ENV: TYK_PMP_PUMPS_LOGZIO_IGNOREFIELDS
Type: []string

IgnoreFields defines a list of analytics fields that will be ignored when writing to the pump. This can be used to avoid writing sensitive information to the Database, or data that you don’t really need to have. The field names must be the same as the JSON tags of the analytics record fields. For example: ["api_key", "api_version"].

pumps.logzio.meta.EnvPrefix

ENV: TYK_PMP_PUMPS_LOGZIO_META_ENVPREFIX
Type: string

The prefix for the environment variables that will be used to override the configuration. Defaults to TYK_PMP_PUMPS_LOGZIO_META

pumps.logzio.meta.check_disk_space

ENV: TYK_PMP_PUMPS_LOGZIO_META_CHECKDISKSPACE
Type: bool

Set the sender to check if it crosses the maximum allowed disk usage. Default value is true.

pumps.logzio.meta.disk_threshold

ENV: TYK_PMP_PUMPS_LOGZIO_META_DISKTHRESHOLD
Type: int

Set disk queue threshold, once the threshold is crossed the sender will not enqueue the received logs. Default value is 98 (percentage of disk).

pumps.logzio.meta.drain_duration

ENV: TYK_PMP_PUMPS_LOGZIO_META_DRAINDURATION
Type: string

Set drain duration (flush logs on disk). Default value is 3s.

pumps.logzio.meta.queue_dir

ENV: TYK_PMP_PUMPS_LOGZIO_META_QUEUEDIR
Type: string

The directory for the queue.

pumps.logzio.meta.token

ENV: TYK_PMP_PUMPS_LOGZIO_META_TOKEN
Type: string

Token for sending data to your logzio account.

pumps.logzio.meta.url

ENV: TYK_PMP_PUMPS_LOGZIO_META_URL
Type: string

If you do not want to use the default Logzio url i.e. when using a proxy. Default is https://listener.logz.io:8071.

pumps.moesif.name

ENV: TYK_PMP_PUMPS_MOESIF_NAME
Type: string

The name of the pump. This is used to identify the pump in the logs. Deprecated, use type instead.

pumps.moesif.type

ENV: TYK_PMP_PUMPS_MOESIF_TYPE
Type: string

Sets the pump type. This is needed when the pump key does not equal to the pump name type. Current valid types are: mongo, mongo-pump-selective, mongo-pump-aggregate, csv, elasticsearch, influx, influx2, moesif, statsd, segment, graylog, splunk, hybrid, prometheus, logzio, dogstatsd, kafka, syslog, sql, sql_aggregate, stdout, timestream, mongo-graph, sql-graph, sql-graph-aggregate, resurfaceio.

pumps.moesif.filters

This feature adds a new configuration field in each pump called filters and its structure is the following:

"filters":{
  "api_ids":[],
  "org_ids":[],
  "response_codes":[],
  "skip_api_ids":[],
  "skip_org_ids":[],
  "skip_response_codes":[]
}

The fields api_ids, org_ids and response_codes works as allow list (APIs and orgs where we want to send the analytics records) and the fields skip_api_ids, skip_org_ids and skip_response_codes works as block list.

The priority is always block list configurations over allow list.

An example of configuration would be:

"csv": {
 "type": "csv",
 "filters": {
   "org_ids": ["org1","org2"]
 },
 "meta": {
   "csv_dir": "./bar"
 }
}

pumps.moesif.filters.org_ids

ENV: TYK_PMP_PUMPS_MOESIF_FILTERS_ORGSIDS
Type: []string

Filters pump data by an allow list of org_ids.

pumps.moesif.filters.api_ids

ENV: TYK_PMP_PUMPS_MOESIF_FILTERS_APIIDS
Type: []string

Filters pump data by an allow list of api_ids.

pumps.moesif.filters.response_codes

ENV: TYK_PMP_PUMPS_MOESIF_FILTERS_RESPONSECODES
Type: []int

Filters pump data by an allow list of response_codes.

pumps.moesif.filters.skip_org_ids

ENV: TYK_PMP_PUMPS_MOESIF_FILTERS_SKIPPEDORGSIDS
Type: []string

Filters pump data by a block list of org_ids.

pumps.moesif.filters.skip_api_ids

ENV: TYK_PMP_PUMPS_MOESIF_FILTERS_SKIPPEDAPIIDS
Type: []string

Filters pump data by a block list of api_ids.

pumps.moesif.filters.skip_response_codes

ENV: TYK_PMP_PUMPS_MOESIF_FILTERS_SKIPPEDRESPONSECODES
Type: []int

Filters pump data by a block list of response_codes.

pumps.moesif.timeout

ENV: TYK_PMP_PUMPS_MOESIF_TIMEOUT
Type: int

By default, a pump will wait forever for each write operation to complete; you can configure an optional timeout by setting the configuration option timeout. If you have deployed multiple pumps, then you can configure each timeout independently. The timeout is in seconds and defaults to 0.

The timeout is configured within the main pump config as shown here; note that this example would configure a 5 second timeout:

"pump_name": {
  ...
  "timeout":5,
  "meta": {...}
}

Tyk will inform you if the pump’s write operation is taking longer than the purging loop (configured via purge_delay) as this will mean that data is purged before being written to the target data sink.

If there is no timeout configured and pump’s write operation is taking longer than the purging loop, the following warning log will be generated: Pump {pump_name} is taking more time than the value configured of purge_delay. You should try to set a timeout for this pump.

If there is a timeout configured, but pump’s write operation is still taking longer than the purging loop, the following warning log will be generated: Pump {pump_name} is taking more time than the value configured of purge_delay. You should try lowering the timeout configured for this pump..

pumps.moesif.omit_detailed_recording

ENV: TYK_PMP_PUMPS_MOESIF_OMITDETAILEDRECORDING
Type: bool

Setting this to true will avoid writing raw_request and raw_response fields for each request in pumps. Defaults to false.

pumps.moesif.max_record_size

ENV: TYK_PMP_PUMPS_MOESIF_MAXRECORDSIZE
Type: int

Defines maximum size (in bytes) for Raw Request and Raw Response logs, this value defaults to 0. If it is not set then tyk-pump will not trim any data and will store the full information. This can also be set at a pump level. For example:

"csv": {
  "type": "csv",
  "max_record_size":1000,
  "meta": {
    "csv_dir": "./"
  }
}

pumps.moesif.ignore_fields

ENV: TYK_PMP_PUMPS_MOESIF_IGNOREFIELDS
Type: []string

IgnoreFields defines a list of analytics fields that will be ignored when writing to the pump. This can be used to avoid writing sensitive information to the Database, or data that you don’t really need to have. The field names must be the same as the JSON tags of the analytics record fields. For example: ["api_key", "api_version"].

pumps.moesif.meta.EnvPrefix

ENV: TYK_PMP_PUMPS_MOESIF_META_ENVPREFIX
Type: string

The prefix for the environment variables that will be used to override the configuration. Defaults to TYK_PMP_PUMPS_MOESIF_META

pumps.moesif.meta.application_id

ENV: TYK_PMP_PUMPS_MOESIF_META_APPLICATIONID
Type: string

Moesif Application Id. You can find your Moesif Application Id from Moesif Dashboard -> Top Right Menu -> API Keys . Moesif recommends creating separate Application Ids for each environment such as Production, Staging, and Development to keep data isolated.

pumps.moesif.meta.request_header_masks

ENV: TYK_PMP_PUMPS_MOESIF_META_REQUESTHEADERMASKS
Type: []string

An option to mask a specific request header field.

pumps.moesif.meta.response_header_masks

ENV: TYK_PMP_PUMPS_MOESIF_META_RESPONSEHEADERMASKS
Type: []string

An option to mask a specific response header field.

pumps.moesif.meta.request_body_masks

ENV: TYK_PMP_PUMPS_MOESIF_META_REQUESTBODYMASKS
Type: []string

An option to mask a specific - request body field.

pumps.moesif.meta.response_body_masks

ENV: TYK_PMP_PUMPS_MOESIF_META_RESPONSEBODYMASKS
Type: []string

An option to mask a specific response body field.

pumps.moesif.meta.disable_capture_request_body

ENV: TYK_PMP_PUMPS_MOESIF_META_DISABLECAPTUREREQUESTBODY
Type: bool

An option to disable logging of request body. Default value is false.

pumps.moesif.meta.disable_capture_response_body

ENV: TYK_PMP_PUMPS_MOESIF_META_DISABLECAPTURERESPONSEBODY
Type: bool

An option to disable logging of response body. Default value is false.

pumps.moesif.meta.user_id_header

ENV: TYK_PMP_PUMPS_MOESIF_META_USERIDHEADER
Type: string

An optional field name to identify User from a request or response header.

pumps.moesif.meta.company_id_header

ENV: TYK_PMP_PUMPS_MOESIF_META_COMPANYIDHEADER
Type: string

An optional field name to identify Company (Account) from a request or response header.

pumps.moesif.meta.enable_bulk

ENV: TYK_PMP_PUMPS_MOESIF_META_ENABLEBULK
Type: bool

Set this to true to enable bulk_config.

pumps.moesif.meta.bulk_config

ENV: TYK_PMP_PUMPS_MOESIF_META_BULKCONFIG
Type: map[string]interface{}

Batch writing trigger configuration.

• "event_queue_size" - (optional) An optional field name which specify the maximum number of events to hold in queue before sending to Moesif. In case of network issues when not able to connect/send event to Moesif, skips adding new events to the queue to prevent memory overflow. Type: int. Default value is 10000.
• "batch_size" - (optional) An optional field name which specify the maximum batch size when sending to Moesif. Type: int. Default value is 200.
• "timer_wake_up_seconds" - (optional) An optional field which specifies a time (every n seconds) how often background thread runs to send events to moesif. Type: int. Default value is 2 seconds.

pumps.moesif.meta.authorization_header_name

ENV: TYK_PMP_PUMPS_MOESIF_META_AUTHORIZATIONHEADERNAME
Type: string

An optional request header field name to used to identify the User in Moesif. Default value is authorization.

pumps.moesif.meta.authorization_user_id_field

ENV: TYK_PMP_PUMPS_MOESIF_META_AUTHORIZATIONUSERIDFIELD
Type: string

An optional field name use to parse the User from authorization header in Moesif. Default value is sub.

pumps.mongo.name

ENV: TYK_PMP_PUMPS_MONGO_NAME
Type: string

The name of the pump. This is used to identify the pump in the logs. Deprecated, use type instead.

pumps.mongo.type

ENV: TYK_PMP_PUMPS_MONGO_TYPE
Type: string

Sets the pump type. This is needed when the pump key does not equal to the pump name type. Current valid types are: mongo, mongo-pump-selective, mongo-pump-aggregate, csv, elasticsearch, influx, influx2, moesif, statsd, segment, graylog, splunk, hybrid, prometheus, logzio, dogstatsd, kafka, syslog, sql, sql_aggregate, stdout, timestream, mongo-graph, sql-graph, sql-graph-aggregate, resurfaceio.

pumps.mongo.filters

This feature adds a new configuration field in each pump called filters and its structure is the following:

"filters":{
  "api_ids":[],
  "org_ids":[],
  "response_codes":[],
  "skip_api_ids":[],
  "skip_org_ids":[],
  "skip_response_codes":[]
}

The fields api_ids, org_ids and response_codes works as allow list (APIs and orgs where we want to send the analytics records) and the fields skip_api_ids, skip_org_ids and skip_response_codes works as block list.

The priority is always block list configurations over allow list.

An example of configuration would be:

"csv": {
 "type": "csv",
 "filters": {
   "org_ids": ["org1","org2"]
 },
 "meta": {
   "csv_dir": "./bar"
 }
}

pumps.mongo.filters.org_ids

ENV: TYK_PMP_PUMPS_MONGO_FILTERS_ORGSIDS
Type: []string

Filters pump data by an allow list of org_ids.

pumps.mongo.filters.api_ids

ENV: TYK_PMP_PUMPS_MONGO_FILTERS_APIIDS
Type: []string

Filters pump data by an allow list of api_ids.

pumps.mongo.filters.response_codes

ENV: TYK_PMP_PUMPS_MONGO_FILTERS_RESPONSECODES
Type: []int

Filters pump data by an allow list of response_codes.

pumps.mongo.filters.skip_org_ids

ENV: TYK_PMP_PUMPS_MONGO_FILTERS_SKIPPEDORGSIDS
Type: []string

Filters pump data by a block list of org_ids.

pumps.mongo.filters.skip_api_ids

ENV: TYK_PMP_PUMPS_MONGO_FILTERS_SKIPPEDAPIIDS
Type: []string

Filters pump data by a block list of api_ids.

pumps.mongo.filters.skip_response_codes

ENV: TYK_PMP_PUMPS_MONGO_FILTERS_SKIPPEDRESPONSECODES
Type: []int

Filters pump data by a block list of response_codes.

pumps.mongo.timeout

ENV: TYK_PMP_PUMPS_MONGO_TIMEOUT
Type: int

By default, a pump will wait forever for each write operation to complete; you can configure an optional timeout by setting the configuration option timeout. If you have deployed multiple pumps, then you can configure each timeout independently. The timeout is in seconds and defaults to 0.

The timeout is configured within the main pump config as shown here; note that this example would configure a 5 second timeout:

"pump_name": {
  ...
  "timeout":5,
  "meta": {...}
}

Tyk will inform you if the pump’s write operation is taking longer than the purging loop (configured via purge_delay) as this will mean that data is purged before being written to the target data sink.

If there is no timeout configured and pump’s write operation is taking longer than the purging loop, the following warning log will be generated: Pump {pump_name} is taking more time than the value configured of purge_delay. You should try to set a timeout for this pump.

If there is a timeout configured, but pump’s write operation is still taking longer than the purging loop, the following warning log will be generated: Pump {pump_name} is taking more time than the value configured of purge_delay. You should try lowering the timeout configured for this pump..

pumps.mongo.omit_detailed_recording

ENV: TYK_PMP_PUMPS_MONGO_OMITDETAILEDRECORDING
Type: bool

Setting this to true will avoid writing raw_request and raw_response fields for each request in pumps. Defaults to false.

pumps.mongo.max_record_size

ENV: TYK_PMP_PUMPS_MONGO_MAXRECORDSIZE
Type: int

Defines maximum size (in bytes) for Raw Request and Raw Response logs, this value defaults to 0. If it is not set then tyk-pump will not trim any data and will store the full information. This can also be set at a pump level. For example:

"csv": {
  "type": "csv",
  "max_record_size":1000,
  "meta": {
    "csv_dir": "./"
  }
}

pumps.mongo.ignore_fields

ENV: TYK_PMP_PUMPS_MONGO_IGNOREFIELDS
Type: []string

IgnoreFields defines a list of analytics fields that will be ignored when writing to the pump. This can be used to avoid writing sensitive information to the Database, or data that you don’t really need to have. The field names must be the same as the JSON tags of the analytics record fields. For example: ["api_key", "api_version"].

pumps.mongo.meta.EnvPrefix

ENV: TYK_PMP_PUMPS_MONGO_META_ENVPREFIX
Type: string

Prefix for the environment variables that will be used to override the configuration. Defaults to TYK_PMP_PUMPS_MONGO_META for Mongo Pump TYK_PMP_PUMPS_UPTIME_META for Uptime Pump TYK_PMP_PUMPS_MONGOAGGREGATE_META for Mongo Aggregate Pump TYK_PMP_PUMPS_MONGOSELECTIVE_META for Mongo Selective Pump TYK_PMP_PUMPS_MONGOGRAPH_META for Mongo Graph Pump.

pumps.mongo.meta.mongo_url

ENV: TYK_PMP_PUMPS_MONGO_META_MONGOURL
Type: string

The full URL to your MongoDB instance, this can be a clustered instance if necessary and should include the database and username / password data.

pumps.mongo.meta.mongo_use_ssl

ENV: TYK_PMP_PUMPS_MONGO_META_MONGOUSESSL
Type: bool

Set to true to enable Mongo SSL connection.

pumps.mongo.meta.mongo_ssl_insecure_skip_verify

ENV: TYK_PMP_PUMPS_MONGO_META_MONGOSSLINSECURESKIPVERIFY
Type: bool

Allows the use of self-signed certificates when connecting to an encrypted MongoDB database.

pumps.mongo.meta.mongo_ssl_allow_invalid_hostnames

ENV: TYK_PMP_PUMPS_MONGO_META_MONGOSSLALLOWINVALIDHOSTNAMES
Type: bool

Ignore hostname check when it differs from the original (for example with SSH tunneling). The rest of the TLS verification will still be performed.

pumps.mongo.meta.mongo_ssl_ca_file

ENV: TYK_PMP_PUMPS_MONGO_META_MONGOSSLCAFILE
Type: string

Path to the PEM file with trusted root certificates

pumps.mongo.meta.mongo_ssl_pem_keyfile

ENV: TYK_PMP_PUMPS_MONGO_META_MONGOSSLPEMKEYFILE
Type: string

Path to the PEM file which contains both client certificate and private key. This is required for Mutual TLS.

pumps.mongo.meta.mongo_db_type

ENV: TYK_PMP_PUMPS_MONGO_META_MONGODBTYPE
Type: int

Specifies the mongo DB Type. If it’s 0, it means that you are using standard mongo db. If it’s 1 it means you are using AWS Document DB. If it’s 2, it means you are using CosmosDB. Defaults to Standard mongo (0).

pumps.mongo.meta.omit_index_creation

ENV: TYK_PMP_PUMPS_MONGO_META_OMITINDEXCREATION
Type: bool

Set to true to disable the default tyk index creation.

pumps.mongo.meta.mongo_session_consistency

ENV: TYK_PMP_PUMPS_MONGO_META_MONGOSESSIONCONSISTENCY
Type: string

Set the consistency mode for the session, it defaults to Strong. The valid values are: strong, monotonic, eventual.

pumps.mongo.meta.driver

ENV: TYK_PMP_PUMPS_MONGO_META_MONGODRIVERTYPE
Type: string

MongoDriverType is the type of the driver (library) to use. The valid values are: “mongo-go” and “mgo”. Since v1.9, the default driver is “mongo-go”. Check out this guide to learn about MongoDB drivers supported by Tyk Pump.

pumps.mongo.meta.mongo_direct_connection

ENV: TYK_PMP_PUMPS_MONGO_META_MONGODIRECTCONNECTION
Type: bool

MongoDirectConnection informs whether to establish connections only with the specified seed servers, or to obtain information for the whole cluster and establish connections with further servers too. If true, the client will only connect to the host provided in the ConnectionString and won’t attempt to discover other hosts in the cluster. Useful when network restrictions prevent discovery, such as with SSH tunneling. Default is false.

pumps.mongo.meta.collection_name

ENV: TYK_PMP_PUMPS_MONGO_META_COLLECTIONNAME
Type: string

Specifies the mongo collection name.

pumps.mongo.meta.max_insert_batch_size_bytes

ENV: TYK_PMP_PUMPS_MONGO_META_MAXINSERTBATCHSIZEBYTES
Type: int

Maximum insert batch size for mongo selective pump. If the batch we are writing surpasses this value, it will be sent in multiple batches. Defaults to 10Mb.

pumps.mongo.meta.max_document_size_bytes

ENV: TYK_PMP_PUMPS_MONGO_META_MAXDOCUMENTSIZEBYTES
Type: int

Maximum document size. If the document exceed this value, it will be skipped. Defaults to 10Mb.

pumps.mongo.meta.collection_cap_max_size_bytes

ENV: TYK_PMP_PUMPS_MONGO_META_COLLECTIONCAPMAXSIZEBYTES
Type: int

Amount of bytes of the capped collection in 64bits architectures. Defaults to 5GB.

pumps.mongo.meta.collection_cap_enable

ENV: TYK_PMP_PUMPS_MONGO_META_COLLECTIONCAPENABLE
Type: bool

Enable collection capping. It’s used to set a maximum size of the collection.

pumps.mongoaggregate.name

ENV: TYK_PMP_PUMPS_MONGOAGGREGATE_NAME
Type: string

The name of the pump. This is used to identify the pump in the logs. Deprecated, use type instead.

pumps.mongoaggregate.type

ENV: TYK_PMP_PUMPS_MONGOAGGREGATE_TYPE
Type: string

Sets the pump type. This is needed when the pump key does not equal to the pump name type. Current valid types are: mongo, mongo-pump-selective, mongo-pump-aggregate, csv, elasticsearch, influx, influx2, moesif, statsd, segment, graylog, splunk, hybrid, prometheus, logzio, dogstatsd, kafka, syslog, sql, sql_aggregate, stdout, timestream, mongo-graph, sql-graph, sql-graph-aggregate, resurfaceio.

pumps.mongoaggregate.filters

This feature adds a new configuration field in each pump called filters and its structure is the following:

"filters":{
  "api_ids":[],
  "org_ids":[],
  "response_codes":[],
  "skip_api_ids":[],
  "skip_org_ids":[],
  "skip_response_codes":[]
}

The fields api_ids, org_ids and response_codes works as allow list (APIs and orgs where we want to send the analytics records) and the fields skip_api_ids, skip_org_ids and skip_response_codes works as block list.

The priority is always block list configurations over allow list.

An example of configuration would be:

"csv": {
 "type": "csv",
 "filters": {
   "org_ids": ["org1","org2"]
 },
 "meta": {
   "csv_dir": "./bar"
 }
}

pumps.mongoaggregate.filters.org_ids

ENV: TYK_PMP_PUMPS_MONGOAGGREGATE_FILTERS_ORGSIDS
Type: []string

Filters pump data by an allow list of org_ids.

pumps.mongoaggregate.filters.api_ids

ENV: TYK_PMP_PUMPS_MONGOAGGREGATE_FILTERS_APIIDS
Type: []string

Filters pump data by an allow list of api_ids.

pumps.mongoaggregate.filters.response_codes

ENV: TYK_PMP_PUMPS_MONGOAGGREGATE_FILTERS_RESPONSECODES
Type: []int

Filters pump data by an allow list of response_codes.

pumps.mongoaggregate.filters.skip_org_ids

ENV: TYK_PMP_PUMPS_MONGOAGGREGATE_FILTERS_SKIPPEDORGSIDS
Type: []string

Filters pump data by a block list of org_ids.

pumps.mongoaggregate.filters.skip_api_ids

ENV: TYK_PMP_PUMPS_MONGOAGGREGATE_FILTERS_SKIPPEDAPIIDS
Type: []string

Filters pump data by a block list of api_ids.

pumps.mongoaggregate.filters.skip_response_codes

ENV: TYK_PMP_PUMPS_MONGOAGGREGATE_FILTERS_SKIPPEDRESPONSECODES
Type: []int

Filters pump data by a block list of response_codes.

pumps.mongoaggregate.timeout

ENV: TYK_PMP_PUMPS_MONGOAGGREGATE_TIMEOUT
Type: int

By default, a pump will wait forever for each write operation to complete; you can configure an optional timeout by setting the configuration option timeout. If you have deployed multiple pumps, then you can configure each timeout independently. The timeout is in seconds and defaults to 0.

The timeout is configured within the main pump config as shown here; note that this example would configure a 5 second timeout:

"pump_name": {
  ...
  "timeout":5,
  "meta": {...}
}

Tyk will inform you if the pump’s write operation is taking longer than the purging loop (configured via purge_delay) as this will mean that data is purged before being written to the target data sink.

If there is no timeout configured and pump’s write operation is taking longer than the purging loop, the following warning log will be generated: Pump {pump_name} is taking more time than the value configured of purge_delay. You should try to set a timeout for this pump.

If there is a timeout configured, but pump’s write operation is still taking longer than the purging loop, the following warning log will be generated: Pump {pump_name} is taking more time than the value configured of purge_delay. You should try lowering the timeout configured for this pump..

pumps.mongoaggregate.omit_detailed_recording

ENV: TYK_PMP_PUMPS_MONGOAGGREGATE_OMITDETAILEDRECORDING
Type: bool

Setting this to true will avoid writing raw_request and raw_response fields for each request in pumps. Defaults to false.

pumps.mongoaggregate.max_record_size

ENV: TYK_PMP_PUMPS_MONGOAGGREGATE_MAXRECORDSIZE
Type: int

Defines maximum size (in bytes) for Raw Request and Raw Response logs, this value defaults to 0. If it is not set then tyk-pump will not trim any data and will store the full information. This can also be set at a pump level. For example:

"csv": {
  "type": "csv",
  "max_record_size":1000,
  "meta": {
    "csv_dir": "./"
  }
}

pumps.mongoaggregate.ignore_fields

ENV: TYK_PMP_PUMPS_MONGOAGGREGATE_IGNOREFIELDS
Type: []string

IgnoreFields defines a list of analytics fields that will be ignored when writing to the pump. This can be used to avoid writing sensitive information to the Database, or data that you don’t really need to have. The field names must be the same as the JSON tags of the analytics record fields. For example: ["api_key", "api_version"].

pumps.mongoaggregate.meta.EnvPrefix

ENV: TYK_PMP_PUMPS_MONGOAGGREGATE_META_ENVPREFIX
Type: string

Prefix for the environment variables that will be used to override the configuration. Defaults to TYK_PMP_PUMPS_MONGO_META for Mongo Pump TYK_PMP_PUMPS_UPTIME_META for Uptime Pump TYK_PMP_PUMPS_MONGOAGGREGATE_META for Mongo Aggregate Pump TYK_PMP_PUMPS_MONGOSELECTIVE_META for Mongo Selective Pump TYK_PMP_PUMPS_MONGOGRAPH_META for Mongo Graph Pump.

pumps.mongoaggregate.meta.mongo_url

ENV: TYK_PMP_PUMPS_MONGOAGGREGATE_META_MONGOURL
Type: string

The full URL to your MongoDB instance, this can be a clustered instance if necessary and should include the database and username / password data.

pumps.mongoaggregate.meta.mongo_use_ssl

ENV: TYK_PMP_PUMPS_MONGOAGGREGATE_META_MONGOUSESSL
Type: bool

Set to true to enable Mongo SSL connection.

pumps.mongoaggregate.meta.mongo_ssl_insecure_skip_verify

ENV: TYK_PMP_PUMPS_MONGOAGGREGATE_META_MONGOSSLINSECURESKIPVERIFY
Type: bool

Allows the use of self-signed certificates when connecting to an encrypted MongoDB database.

pumps.mongoaggregate.meta.mongo_ssl_allow_invalid_hostnames

ENV: TYK_PMP_PUMPS_MONGOAGGREGATE_META_MONGOSSLALLOWINVALIDHOSTNAMES
Type: bool

Ignore hostname check when it differs from the original (for example with SSH tunneling). The rest of the TLS verification will still be performed.

pumps.mongoaggregate.meta.mongo_ssl_ca_file

ENV: TYK_PMP_PUMPS_MONGOAGGREGATE_META_MONGOSSLCAFILE
Type: string

Path to the PEM file with trusted root certificates

pumps.mongoaggregate.meta.mongo_ssl_pem_keyfile

ENV: TYK_PMP_PUMPS_MONGOAGGREGATE_META_MONGOSSLPEMKEYFILE
Type: string

Path to the PEM file which contains both client certificate and private key. This is required for Mutual TLS.

pumps.mongoaggregate.meta.mongo_db_type

ENV: TYK_PMP_PUMPS_MONGOAGGREGATE_META_MONGODBTYPE
Type: int

Specifies the mongo DB Type. If it’s 0, it means that you are using standard mongo db. If it’s 1 it means you are using AWS Document DB. If it’s 2, it means you are using CosmosDB. Defaults to Standard mongo (0).

pumps.mongoaggregate.meta.omit_index_creation

ENV: TYK_PMP_PUMPS_MONGOAGGREGATE_META_OMITINDEXCREATION
Type: bool

Set to true to disable the default tyk index creation.

pumps.mongoaggregate.meta.mongo_session_consistency

ENV: TYK_PMP_PUMPS_MONGOAGGREGATE_META_MONGOSESSIONCONSISTENCY
Type: string

Set the consistency mode for the session, it defaults to Strong. The valid values are: strong, monotonic, eventual.

pumps.mongoaggregate.meta.driver

ENV: TYK_PMP_PUMPS_MONGOAGGREGATE_META_MONGODRIVERTYPE
Type: string

MongoDriverType is the type of the driver (library) to use. The valid values are: “mongo-go” and “mgo”. Since v1.9, the default driver is “mongo-go”. Check out this guide to learn about MongoDB drivers supported by Tyk Pump.

pumps.mongoaggregate.meta.mongo_direct_connection

ENV: TYK_PMP_PUMPS_MONGOAGGREGATE_META_MONGODIRECTCONNECTION
Type: bool

MongoDirectConnection informs whether to establish connections only with the specified seed servers, or to obtain information for the whole cluster and establish connections with further servers too. If true, the client will only connect to the host provided in the ConnectionString and won’t attempt to discover other hosts in the cluster. Useful when network restrictions prevent discovery, such as with SSH tunneling. Default is false.

pumps.mongoaggregate.meta.use_mixed_collection

ENV: TYK_PMP_PUMPS_MONGOAGGREGATE_META_USEMIXEDCOLLECTION
Type: bool

If set to true your pump will store analytics to both your organization defined collections z_tyk_analyticz_aggregate_{ORG ID} and your org-less tyk_analytics_aggregates collection. When set to ‘false’ your pump will only store analytics to your org defined collection.

pumps.mongoaggregate.meta.track_all_paths

ENV: TYK_PMP_PUMPS_MONGOAGGREGATE_META_TRACKALLPATHS
Type: bool

Specifies if it should store aggregated data for all the endpoints. By default, false which means that only store aggregated data for tracked endpoints.

pumps.mongoaggregate.meta.ignore_tag_prefix_list

ENV: TYK_PMP_PUMPS_MONGOAGGREGATE_META_IGNORETAGPREFIXLIST
Type: []string

Specifies prefixes of tags that should be ignored.

pumps.mongoaggregate.meta.threshold_len_tag_list

ENV: TYK_PMP_PUMPS_MONGOAGGREGATE_META_THRESHOLDLENTAGLIST
Type: int

Determines the threshold of amount of tags of an aggregation. If the amount of tags is superior to the threshold, it will print an alert. Defaults to 1000.

pumps.mongoaggregate.meta.store_analytics_per_minute

ENV: TYK_PMP_PUMPS_MONGOAGGREGATE_META_STOREANALYTICSPERMINUTE
Type: bool

Determines if the aggregations should be made per minute (true) or per hour (false).

pumps.mongoaggregate.meta.aggregation_time

ENV: TYK_PMP_PUMPS_MONGOAGGREGATE_META_AGGREGATIONTIME
Type: int

Determines the amount of time the aggregations should be made (in minutes). It defaults to the max value is 60 and the minimum is 1. If StoreAnalyticsPerMinute is set to true, this field will be skipped.

pumps.mongoaggregate.meta.enable_aggregate_self_healing

ENV: TYK_PMP_PUMPS_MONGOAGGREGATE_META_ENABLEAGGREGATESELFHEALING
Type: bool

Determines if the self healing will be activated or not. Self Healing allows pump to handle Mongo document’s max-size errors by creating a new document when the max-size is reached. It also divide by 2 the AggregationTime field to avoid the same error in the future.

pumps.mongoaggregate.meta.ignore_aggregations

ENV: TYK_PMP_PUMPS_MONGOAGGREGATE_META_IGNOREAGGREGATIONSLIST
Type: []string

This list determines which aggregations are going to be dropped and not stored in the collection. Posible values are: “APIID”,“errors”,“versions”,“apikeys”,“oauthids”,“geo”,“tags”,“endpoints”,“keyendpoints”, “oauthendpoints”, and “apiendpoints”.

pumps.mongoselective.name

ENV: TYK_PMP_PUMPS_MONGOSELECTIVE_NAME
Type: string

The name of the pump. This is used to identify the pump in the logs. Deprecated, use type instead.

pumps.mongoselective.type

ENV: TYK_PMP_PUMPS_MONGOSELECTIVE_TYPE
Type: string

Sets the pump type. This is needed when the pump key does not equal to the pump name type. Current valid types are: mongo, mongo-pump-selective, mongo-pump-aggregate, csv, elasticsearch, influx, influx2, moesif, statsd, segment, graylog, splunk, hybrid, prometheus, logzio, dogstatsd, kafka, syslog, sql, sql_aggregate, stdout, timestream, mongo-graph, sql-graph, sql-graph-aggregate, resurfaceio.

pumps.mongoselective.filters

This feature adds a new configuration field in each pump called filters and its structure is the following:

"filters":{
  "api_ids":[],
  "org_ids":[],
  "response_codes":[],
  "skip_api_ids":[],
  "skip_org_ids":[],
  "skip_response_codes":[]
}

The fields api_ids, org_ids and response_codes works as allow list (APIs and orgs where we want to send the analytics records) and the fields skip_api_ids, skip_org_ids and skip_response_codes works as block list.

The priority is always block list configurations over allow list.

An example of configuration would be:

"csv": {
 "type": "csv",
 "filters": {
   "org_ids": ["org1","org2"]
 },
 "meta": {
   "csv_dir": "./bar"
 }
}

pumps.mongoselective.filters.org_ids

ENV: TYK_PMP_PUMPS_MONGOSELECTIVE_FILTERS_ORGSIDS
Type: []string

Filters pump data by an allow list of org_ids.

pumps.mongoselective.filters.api_ids

ENV: TYK_PMP_PUMPS_MONGOSELECTIVE_FILTERS_APIIDS
Type: []string

Filters pump data by an allow list of api_ids.

pumps.mongoselective.filters.response_codes

ENV: TYK_PMP_PUMPS_MONGOSELECTIVE_FILTERS_RESPONSECODES
Type: []int

Filters pump data by an allow list of response_codes.

pumps.mongoselective.filters.skip_org_ids

ENV: TYK_PMP_PUMPS_MONGOSELECTIVE_FILTERS_SKIPPEDORGSIDS
Type: []string

Filters pump data by a block list of org_ids.

pumps.mongoselective.filters.skip_api_ids

ENV: TYK_PMP_PUMPS_MONGOSELECTIVE_FILTERS_SKIPPEDAPIIDS
Type: []string

Filters pump data by a block list of api_ids.

pumps.mongoselective.filters.skip_response_codes

ENV: TYK_PMP_PUMPS_MONGOSELECTIVE_FILTERS_SKIPPEDRESPONSECODES
Type: []int

Filters pump data by a block list of response_codes.

pumps.mongoselective.timeout

ENV: TYK_PMP_PUMPS_MONGOSELECTIVE_TIMEOUT
Type: int

By default, a pump will wait forever for each write operation to complete; you can configure an optional timeout by setting the configuration option timeout. If you have deployed multiple pumps, then you can configure each timeout independently. The timeout is in seconds and defaults to 0.

The timeout is configured within the main pump config as shown here; note that this example would configure a 5 second timeout:

"pump_name": {
  ...
  "timeout":5,
  "meta": {...}
}

Tyk will inform you if the pump’s write operation is taking longer than the purging loop (configured via purge_delay) as this will mean that data is purged before being written to the target data sink.

If there is no timeout configured and pump’s write operation is taking longer than the purging loop, the following warning log will be generated: Pump {pump_name} is taking more time than the value configured of purge_delay. You should try to set a timeout for this pump.

If there is a timeout configured, but pump’s write operation is still taking longer than the purging loop, the following warning log will be generated: Pump {pump_name} is taking more time than the value configured of purge_delay. You should try lowering the timeout configured for this pump..

pumps.mongoselective.omit_detailed_recording

ENV: TYK_PMP_PUMPS_MONGOSELECTIVE_OMITDETAILEDRECORDING
Type: bool

Setting this to true will avoid writing raw_request and raw_response fields for each request in pumps. Defaults to false.

pumps.mongoselective.max_record_size

ENV: TYK_PMP_PUMPS_MONGOSELECTIVE_MAXRECORDSIZE
Type: int

Defines maximum size (in bytes) for Raw Request and Raw Response logs, this value defaults to 0. If it is not set then tyk-pump will not trim any data and will store the full information. This can also be set at a pump level. For example:

"csv": {
  "type": "csv",
  "max_record_size":1000,
  "meta": {
    "csv_dir": "./"
  }
}

pumps.mongoselective.ignore_fields

ENV: TYK_PMP_PUMPS_MONGOSELECTIVE_IGNOREFIELDS
Type: []string

IgnoreFields defines a list of analytics fields that will be ignored when writing to the pump. This can be used to avoid writing sensitive information to the Database, or data that you don’t really need to have. The field names must be the same as the JSON tags of the analytics record fields. For example: ["api_key", "api_version"].

pumps.mongoselective.meta.EnvPrefix

ENV: TYK_PMP_PUMPS_MONGOSELECTIVE_META_ENVPREFIX
Type: string

Prefix for the environment variables that will be used to override the configuration. Defaults to TYK_PMP_PUMPS_MONGO_META for Mongo Pump TYK_PMP_PUMPS_UPTIME_META for Uptime Pump TYK_PMP_PUMPS_MONGOAGGREGATE_META for Mongo Aggregate Pump TYK_PMP_PUMPS_MONGOSELECTIVE_META for Mongo Selective Pump TYK_PMP_PUMPS_MONGOGRAPH_META for Mongo Graph Pump.

pumps.mongoselective.meta.mongo_url

ENV: TYK_PMP_PUMPS_MONGOSELECTIVE_META_MONGOURL
Type: string

The full URL to your MongoDB instance, this can be a clustered instance if necessary and should include the database and username / password data.

pumps.mongoselective.meta.mongo_use_ssl

ENV: TYK_PMP_PUMPS_MONGOSELECTIVE_META_MONGOUSESSL
Type: bool

Set to true to enable Mongo SSL connection.

pumps.mongoselective.meta.mongo_ssl_insecure_skip_verify

ENV: TYK_PMP_PUMPS_MONGOSELECTIVE_META_MONGOSSLINSECURESKIPVERIFY
Type: bool

Allows the use of self-signed certificates when connecting to an encrypted MongoDB database.

pumps.mongoselective.meta.mongo_ssl_allow_invalid_hostnames

ENV: TYK_PMP_PUMPS_MONGOSELECTIVE_META_MONGOSSLALLOWINVALIDHOSTNAMES
Type: bool

Ignore hostname check when it differs from the original (for example with SSH tunneling). The rest of the TLS verification will still be performed.

pumps.mongoselective.meta.mongo_ssl_ca_file

ENV: TYK_PMP_PUMPS_MONGOSELECTIVE_META_MONGOSSLCAFILE
Type: string

Path to the PEM file with trusted root certificates

pumps.mongoselective.meta.mongo_ssl_pem_keyfile

ENV: TYK_PMP_PUMPS_MONGOSELECTIVE_META_MONGOSSLPEMKEYFILE
Type: string

Path to the PEM file which contains both client certificate and private key. This is required for Mutual TLS.

pumps.mongoselective.meta.mongo_db_type

ENV: TYK_PMP_PUMPS_MONGOSELECTIVE_META_MONGODBTYPE
Type: int

Specifies the mongo DB Type. If it’s 0, it means that you are using standard mongo db. If it’s 1 it means you are using AWS Document DB. If it’s 2, it means you are using CosmosDB. Defaults to Standard mongo (0).

pumps.mongoselective.meta.omit_index_creation

ENV: TYK_PMP_PUMPS_MONGOSELECTIVE_META_OMITINDEXCREATION
Type: bool

Set to true to disable the default tyk index creation.

pumps.mongoselective.meta.mongo_session_consistency

ENV: TYK_PMP_PUMPS_MONGOSELECTIVE_META_MONGOSESSIONCONSISTENCY
Type: string

Set the consistency mode for the session, it defaults to Strong. The valid values are: strong, monotonic, eventual.

pumps.mongoselective.meta.driver

ENV: TYK_PMP_PUMPS_MONGOSELECTIVE_META_MONGODRIVERTYPE
Type: string

MongoDriverType is the type of the driver (library) to use. The valid values are: “mongo-go” and “mgo”. Since v1.9, the default driver is “mongo-go”. Check out this guide to learn about MongoDB drivers supported by Tyk Pump.

pumps.mongoselective.meta.mongo_direct_connection

ENV: TYK_PMP_PUMPS_MONGOSELECTIVE_META_MONGODIRECTCONNECTION
Type: bool

MongoDirectConnection informs whether to establish connections only with the specified seed servers, or to obtain information for the whole cluster and establish connections with further servers too. If true, the client will only connect to the host provided in the ConnectionString and won’t attempt to discover other hosts in the cluster. Useful when network restrictions prevent discovery, such as with SSH tunneling. Default is false.

pumps.mongoselective.meta.max_insert_batch_size_bytes

ENV: TYK_PMP_PUMPS_MONGOSELECTIVE_META_MAXINSERTBATCHSIZEBYTES
Type: int

Maximum insert batch size for mongo selective pump. If the batch we are writing surpass this value, it will be send in multiple batchs. Defaults to 10Mb.

pumps.mongoselective.meta.max_document_size_bytes

ENV: TYK_PMP_PUMPS_MONGOSELECTIVE_META_MAXDOCUMENTSIZEBYTES
Type: int

Maximum document size. If the document exceed this value, it will be skipped. Defaults to 10Mb.

pumps.prometheus.name

ENV: TYK_PMP_PUMPS_PROMETHEUS_NAME
Type: string

The name of the pump. This is used to identify the pump in the logs. Deprecated, use type instead.

pumps.prometheus.type

ENV: TYK_PMP_PUMPS_PROMETHEUS_TYPE
Type: string

Sets the pump type. This is needed when the pump key does not equal to the pump name type. Current valid types are: mongo, mongo-pump-selective, mongo-pump-aggregate, csv, elasticsearch, influx, influx2, moesif, statsd, segment, graylog, splunk, hybrid, prometheus, logzio, dogstatsd, kafka, syslog, sql, sql_aggregate, stdout, timestream, mongo-graph, sql-graph, sql-graph-aggregate, resurfaceio.

pumps.prometheus.filters

This feature adds a new configuration field in each pump called filters and its structure is the following:

"filters":{
  "api_ids":[],
  "org_ids":[],
  "response_codes":[],
  "skip_api_ids":[],
  "skip_org_ids":[],
  "skip_response_codes":[]
}

The fields api_ids, org_ids and response_codes works as allow list (APIs and orgs where we want to send the analytics records) and the fields skip_api_ids, skip_org_ids and skip_response_codes works as block list.

The priority is always block list configurations over allow list.

An example of configuration would be:

"csv": {
 "type": "csv",
 "filters": {
   "org_ids": ["org1","org2"]
 },
 "meta": {
   "csv_dir": "./bar"
 }
}

pumps.prometheus.filters.org_ids

ENV: TYK_PMP_PUMPS_PROMETHEUS_FILTERS_ORGSIDS
Type: []string

Filters pump data by an allow list of org_ids.

pumps.prometheus.filters.api_ids

ENV: TYK_PMP_PUMPS_PROMETHEUS_FILTERS_APIIDS
Type: []string

Filters pump data by an allow list of api_ids.

pumps.prometheus.filters.response_codes

ENV: TYK_PMP_PUMPS_PROMETHEUS_FILTERS_RESPONSECODES
Type: []int

Filters pump data by an allow list of response_codes.

pumps.prometheus.filters.skip_org_ids

ENV: TYK_PMP_PUMPS_PROMETHEUS_FILTERS_SKIPPEDORGSIDS
Type: []string

Filters pump data by a block list of org_ids.

pumps.prometheus.filters.skip_api_ids

ENV: TYK_PMP_PUMPS_PROMETHEUS_FILTERS_SKIPPEDAPIIDS
Type: []string

Filters pump data by a block list of api_ids.

pumps.prometheus.filters.skip_response_codes

ENV: TYK_PMP_PUMPS_PROMETHEUS_FILTERS_SKIPPEDRESPONSECODES
Type: []int

Filters pump data by a block list of response_codes.

pumps.prometheus.timeout

ENV: TYK_PMP_PUMPS_PROMETHEUS_TIMEOUT
Type: int

By default, a pump will wait forever for each write operation to complete; you can configure an optional timeout by setting the configuration option timeout. If you have deployed multiple pumps, then you can configure each timeout independently. The timeout is in seconds and defaults to 0.

The timeout is configured within the main pump config as shown here; note that this example would configure a 5 second timeout:

"pump_name": {
  ...
  "timeout":5,
  "meta": {...}
}

Tyk will inform you if the pump’s write operation is taking longer than the purging loop (configured via purge_delay) as this will mean that data is purged before being written to the target data sink.

If there is no timeout configured and pump’s write operation is taking longer than the purging loop, the following warning log will be generated: Pump {pump_name} is taking more time than the value configured of purge_delay. You should try to set a timeout for this pump.

If there is a timeout configured, but pump’s write operation is still taking longer than the purging loop, the following warning log will be generated: Pump {pump_name} is taking more time than the value configured of purge_delay. You should try lowering the timeout configured for this pump..

pumps.prometheus.omit_detailed_recording

ENV: TYK_PMP_PUMPS_PROMETHEUS_OMITDETAILEDRECORDING
Type: bool

Setting this to true will avoid writing raw_request and raw_response fields for each request in pumps. Defaults to false.

pumps.prometheus.max_record_size

ENV: TYK_PMP_PUMPS_PROMETHEUS_MAXRECORDSIZE
Type: int

Defines maximum size (in bytes) for Raw Request and Raw Response logs, this value defaults to 0. If it is not set then tyk-pump will not trim any data and will store the full information. This can also be set at a pump level. For example:

"csv": {
  "type": "csv",
  "max_record_size":1000,
  "meta": {
    "csv_dir": "./"
  }
}

pumps.prometheus.ignore_fields

ENV: TYK_PMP_PUMPS_PROMETHEUS_IGNOREFIELDS
Type: []string

IgnoreFields defines a list of analytics fields that will be ignored when writing to the pump. This can be used to avoid writing sensitive information to the Database, or data that you don’t really need to have. The field names must be the same as the JSON tags of the analytics record fields. For example: ["api_key", "api_version"].

pumps.prometheus.meta.EnvPrefix

ENV: TYK_PMP_PUMPS_PROMETHEUS_META_ENVPREFIX
Type: string

Prefix for the environment variables that will be used to override the configuration. Defaults to TYK_PMP_PUMPS_PROMETHEUS_META

pumps.prometheus.meta.listen_address

ENV: TYK_PMP_PUMPS_PROMETHEUS_META_ADDR
Type: string

The full URL to your Prometheus instance, :. For example localhost:9090.

pumps.prometheus.meta.path

ENV: TYK_PMP_PUMPS_PROMETHEUS_META_PATH
Type: string

The path to the Prometheus collection. For example /metrics.

pumps.prometheus.meta.aggregate_observations

ENV: TYK_PMP_PUMPS_PROMETHEUS_META_AGGREGATEOBSERVATIONS
Type: bool

This will enable an experimental feature that will aggregate the histogram metrics request time values before exposing them to prometheus. Enabling this will reduce the CPU usage of your prometheus pump but you will loose histogram precision. Experimental.

pumps.prometheus.meta.disabled_metrics

ENV: TYK_PMP_PUMPS_PROMETHEUS_META_DISABLEDMETRICS
Type: []string

Metrics to exclude from exposition. Currently, excludes only the base metrics.

pumps.prometheus.meta.track_all_paths

ENV: TYK_PMP_PUMPS_PROMETHEUS_META_TRACKALLPATHS
Type: bool

Specifies if it should expose aggregated metrics for all the endpoints. By default, false which means that all APIs endpoints will be counted as ‘unknown’ unless the API uses the track endpoint plugin.

pumps.prometheus.meta.custom_metrics

ENV: TYK_PMP_PUMPS_PROMETHEUS_META_CUSTOMMETRICS
Type: CustomMetrics

Custom Prometheus metrics.

pumps.splunk.name

ENV: TYK_PMP_PUMPS_SPLUNK_NAME
Type: string

The name of the pump. This is used to identify the pump in the logs. Deprecated, use type instead.

pumps.splunk.type

ENV: TYK_PMP_PUMPS_SPLUNK_TYPE
Type: string

Sets the pump type. This is needed when the pump key does not equal to the pump name type. Current valid types are: mongo, mongo-pump-selective, mongo-pump-aggregate, csv, elasticsearch, influx, influx2, moesif, statsd, segment, graylog, splunk, hybrid, prometheus, logzio, dogstatsd, kafka, syslog, sql, sql_aggregate, stdout, timestream, mongo-graph, sql-graph, sql-graph-aggregate, resurfaceio.

pumps.splunk.filters

This feature adds a new configuration field in each pump called filters and its structure is the following:

"filters":{
  "api_ids":[],
  "org_ids":[],
  "response_codes":[],
  "skip_api_ids":[],
  "skip_org_ids":[],
  "skip_response_codes":[]
}

The fields api_ids, org_ids and response_codes works as allow list (APIs and orgs where we want to send the analytics records) and the fields skip_api_ids, skip_org_ids and skip_response_codes works as block list.

The priority is always block list configurations over allow list.

An example of configuration would be:

"csv": {
 "type": "csv",
 "filters": {
   "org_ids": ["org1","org2"]
 },
 "meta": {
   "csv_dir": "./bar"
 }
}

pumps.splunk.filters.org_ids

ENV: TYK_PMP_PUMPS_SPLUNK_FILTERS_ORGSIDS
Type: []string

Filters pump data by an allow list of org_ids.

pumps.splunk.filters.api_ids

ENV: TYK_PMP_PUMPS_SPLUNK_FILTERS_APIIDS
Type: []string

Filters pump data by an allow list of api_ids.

pumps.splunk.filters.response_codes

ENV: TYK_PMP_PUMPS_SPLUNK_FILTERS_RESPONSECODES
Type: []int

Filters pump data by an allow list of response_codes.

pumps.splunk.filters.skip_org_ids

ENV: TYK_PMP_PUMPS_SPLUNK_FILTERS_SKIPPEDORGSIDS
Type: []string

Filters pump data by a block list of org_ids.

pumps.splunk.filters.skip_api_ids

ENV: TYK_PMP_PUMPS_SPLUNK_FILTERS_SKIPPEDAPIIDS
Type: []string

Filters pump data by a block list of api_ids.

pumps.splunk.filters.skip_response_codes

ENV: TYK_PMP_PUMPS_SPLUNK_FILTERS_SKIPPEDRESPONSECODES
Type: []int

Filters pump data by a block list of response_codes.

pumps.splunk.timeout

ENV: TYK_PMP_PUMPS_SPLUNK_TIMEOUT
Type: int

By default, a pump will wait forever for each write operation to complete; you can configure an optional timeout by setting the configuration option timeout. If you have deployed multiple pumps, then you can configure each timeout independently. The timeout is in seconds and defaults to 0.

The timeout is configured within the main pump config as shown here; note that this example would configure a 5 second timeout:

"pump_name": {
  ...
  "timeout":5,
  "meta": {...}
}

Tyk will inform you if the pump’s write operation is taking longer than the purging loop (configured via purge_delay) as this will mean that data is purged before being written to the target data sink.

If there is no timeout configured and pump’s write operation is taking longer than the purging loop, the following warning log will be generated: Pump {pump_name} is taking more time than the value configured of purge_delay. You should try to set a timeout for this pump.

If there is a timeout configured, but pump’s write operation is still taking longer than the purging loop, the following warning log will be generated: Pump {pump_name} is taking more time than the value configured of purge_delay. You should try lowering the timeout configured for this pump..

pumps.splunk.omit_detailed_recording

ENV: TYK_PMP_PUMPS_SPLUNK_OMITDETAILEDRECORDING
Type: bool

Setting this to true will avoid writing raw_request and raw_response fields for each request in pumps. Defaults to false.

pumps.splunk.max_record_size

ENV: TYK_PMP_PUMPS_SPLUNK_MAXRECORDSIZE
Type: int

Defines maximum size (in bytes) for Raw Request and Raw Response logs, this value defaults to 0. If it is not set then tyk-pump will not trim any data and will store the full information. This can also be set at a pump level. For example:

"csv": {
  "type": "csv",
  "max_record_size":1000,
  "meta": {
    "csv_dir": "./"
  }
}

pumps.splunk.ignore_fields

ENV: TYK_PMP_PUMPS_SPLUNK_IGNOREFIELDS
Type: []string

IgnoreFields defines a list of analytics fields that will be ignored when writing to the pump. This can be used to avoid writing sensitive information to the Database, or data that you don’t really need to have. The field names must be the same as the JSON tags of the analytics record fields. For example: ["api_key", "api_version"].

pumps.splunk.meta.EnvPrefix

ENV: TYK_PMP_PUMPS_SPLUNK_META_ENVPREFIX
Type: string

The prefix for the environment variables that will be used to override the configuration. Defaults to TYK_PMP_PUMPS_SPLUNK_META

pumps.splunk.meta.collector_token

ENV: TYK_PMP_PUMPS_SPLUNK_META_COLLECTORTOKEN
Type: string

Address of the datadog agent including host & port.

pumps.splunk.meta.collector_url

ENV: TYK_PMP_PUMPS_SPLUNK_META_COLLECTORURL
Type: string

Endpoint the Pump will send analytics too. Should look something like: https://splunk:8088/services/collector/event.

pumps.splunk.meta.ssl_insecure_skip_verify

ENV: TYK_PMP_PUMPS_SPLUNK_META_SSLINSECURESKIPVERIFY
Type: bool

Controls whether the pump client verifies the Splunk server’s certificate chain and host name.

pumps.splunk.meta.ssl_cert_file

ENV: TYK_PMP_PUMPS_SPLUNK_META_SSLCERTFILE
Type: string

SSL cert file location.

pumps.splunk.meta.ssl_key_file

ENV: TYK_PMP_PUMPS_SPLUNK_META_SSLKEYFILE
Type: string

SSL cert key location.

pumps.splunk.meta.ssl_server_name

ENV: TYK_PMP_PUMPS_SPLUNK_META_SSLSERVERNAME
Type: string

SSL Server name used in the TLS connection.

pumps.splunk.meta.obfuscate_api_keys

ENV: TYK_PMP_PUMPS_SPLUNK_META_OBFUSCATEAPIKEYS
Type: bool

Controls whether the pump client should hide the API key. In case you still need substring of the value, check the next option. Default value is false.

pumps.splunk.meta.obfuscate_api_keys_length

ENV: TYK_PMP_PUMPS_SPLUNK_META_OBFUSCATEAPIKEYSLENGTH
Type: int

Define the number of the characters from the end of the API key. The obfuscate_api_keys should be set to true. Default value is 0.

pumps.splunk.meta.fields

ENV: TYK_PMP_PUMPS_SPLUNK_META_FIELDS
Type: []string

Define which Analytics fields should participate in the Splunk event. Check the available fields in the example below. Default value is ["method", "path", "response_code", "api_key", "time_stamp", "api_version", "api_name", "api_id", "org_id", "oauth_id", "raw_request", "request_time", "raw_response", "ip_address"].

pumps.splunk.meta.ignore_tag_prefix_list

ENV: TYK_PMP_PUMPS_SPLUNK_META_IGNORETAGPREFIXLIST
Type: []string

Choose which tags to be ignored by the Splunk Pump. Keep in mind that the tag name and value are hyphenated. Default value is [].

pumps.splunk.meta.enable_batch

ENV: TYK_PMP_PUMPS_SPLUNK_META_ENABLEBATCH
Type: bool

If this is set to true, pump is going to send the analytics records in batch to Splunk. Default value is false.

pumps.splunk.meta.max_retries

ENV: TYK_PMP_PUMPS_SPLUNK_META_MAXRETRIES
Type: uint64

MaxRetries represents the maximum amount of retries to attempt if failed to send requests to splunk HEC. Default value is 0

pumps.sql.name

ENV: TYK_PMP_PUMPS_SQL_NAME
Type: string

The name of the pump. This is used to identify the pump in the logs. Deprecated, use type instead.

pumps.sql.type

ENV: TYK_PMP_PUMPS_SQL_TYPE
Type: string

Sets the pump type. This is needed when the pump key does not equal to the pump name type. Current valid types are: mongo, mongo-pump-selective, mongo-pump-aggregate, csv, elasticsearch, influx, influx2, moesif, statsd, segment, graylog, splunk, hybrid, prometheus, logzio, dogstatsd, kafka, syslog, sql, sql_aggregate, stdout, timestream, mongo-graph, sql-graph, sql-graph-aggregate, resurfaceio.

pumps.sql.filters

This feature adds a new configuration field in each pump called filters and its structure is the following:

"filters":{
  "api_ids":[],
  "org_ids":[],
  "response_codes":[],
  "skip_api_ids":[],
  "skip_org_ids":[],
  "skip_response_codes":[]
}

The fields api_ids, org_ids and response_codes works as allow list (APIs and orgs where we want to send the analytics records) and the fields skip_api_ids, skip_org_ids and skip_response_codes works as block list.

The priority is always block list configurations over allow list.

An example of configuration would be:

"csv": {
 "type": "csv",
 "filters": {
   "org_ids": ["org1","org2"]
 },
 "meta": {
   "csv_dir": "./bar"
 }
}

pumps.sql.filters.org_ids

ENV: TYK_PMP_PUMPS_SQL_FILTERS_ORGSIDS
Type: []string

Filters pump data by an allow list of org_ids.

pumps.sql.filters.api_ids

ENV: TYK_PMP_PUMPS_SQL_FILTERS_APIIDS
Type: []string

Filters pump data by an allow list of api_ids.

pumps.sql.filters.response_codes

ENV: TYK_PMP_PUMPS_SQL_FILTERS_RESPONSECODES
Type: []int

Filters pump data by an allow list of response_codes.

pumps.sql.filters.skip_org_ids

ENV: TYK_PMP_PUMPS_SQL_FILTERS_SKIPPEDORGSIDS
Type: []string

Filters pump data by a block list of org_ids.

pumps.sql.filters.skip_api_ids

ENV: TYK_PMP_PUMPS_SQL_FILTERS_SKIPPEDAPIIDS
Type: []string

Filters pump data by a block list of api_ids.

pumps.sql.filters.skip_response_codes

ENV: TYK_PMP_PUMPS_SQL_FILTERS_SKIPPEDRESPONSECODES
Type: []int

Filters pump data by a block list of response_codes.

pumps.sql.timeout

ENV: TYK_PMP_PUMPS_SQL_TIMEOUT
Type: int

By default, a pump will wait forever for each write operation to complete; you can configure an optional timeout by setting the configuration option timeout. If you have deployed multiple pumps, then you can configure each timeout independently. The timeout is in seconds and defaults to 0.

The timeout is configured within the main pump config as shown here; note that this example would configure a 5 second timeout:

"pump_name": {
  ...
  "timeout":5,
  "meta": {...}
}

Tyk will inform you if the pump’s write operation is taking longer than the purging loop (configured via purge_delay) as this will mean that data is purged before being written to the target data sink.

If there is no timeout configured and pump’s write operation is taking longer than the purging loop, the following warning log will be generated: Pump {pump_name} is taking more time than the value configured of purge_delay. You should try to set a timeout for this pump.

If there is a timeout configured, but pump’s write operation is still taking longer than the purging loop, the following warning log will be generated: Pump {pump_name} is taking more time than the value configured of purge_delay. You should try lowering the timeout configured for this pump..

pumps.sql.omit_detailed_recording

ENV: TYK_PMP_PUMPS_SQL_OMITDETAILEDRECORDING
Type: bool

Setting this to true will avoid writing raw_request and raw_response fields for each request in pumps. Defaults to false.

pumps.sql.max_record_size

ENV: TYK_PMP_PUMPS_SQL_MAXRECORDSIZE
Type: int

Defines maximum size (in bytes) for Raw Request and Raw Response logs, this value defaults to 0. If it is not set then tyk-pump will not trim any data and will store the full information. This can also be set at a pump level. For example:

"csv": {
  "type": "csv",
  "max_record_size":1000,
  "meta": {
    "csv_dir": "./"
  }
}

pumps.sql.ignore_fields

ENV: TYK_PMP_PUMPS_SQL_IGNOREFIELDS
Type: []string

IgnoreFields defines a list of analytics fields that will be ignored when writing to the pump. This can be used to avoid writing sensitive information to the Database, or data that you don’t really need to have. The field names must be the same as the JSON tags of the analytics record fields. For example: ["api_key", "api_version"].

pumps.sql.meta.EnvPrefix

ENV: TYK_PMP_PUMPS_SQL_META_ENVPREFIX
Type: string

The prefix for the environment variables that will be used to override the configuration. Defaults to TYK_PMP_PUMPS_SQL_META

pumps.sql.meta.type

ENV: TYK_PMP_PUMPS_SQL_META_TYPE
Type: string

The supported and tested types are sqlite and postgres.

pumps.sql.meta.connection_string

ENV: TYK_PMP_PUMPS_SQL_META_CONNECTIONSTRING
Type: string

Specifies the connection string to the database.

pumps.sql.meta.postgres

Postgres configurations.

pumps.sql.meta.postgres.prefer_simple_protocol

ENV: TYK_PMP_PUMPS_SQL_META_POSTGRES_PREFERSIMPLEPROTOCOL
Type: bool

Disables implicit prepared statement usage.

pumps.sql.meta.mysql

Mysql configurations.

pumps.sql.meta.mysql.default_string_size

ENV: TYK_PMP_PUMPS_SQL_META_MYSQL_DEFAULTSTRINGSIZE
Type: uint

Default size for string fields. Defaults to 256.

pumps.sql.meta.mysql.disable_datetime_precision

ENV: TYK_PMP_PUMPS_SQL_META_MYSQL_DISABLEDATETIMEPRECISION
Type: bool

Disable datetime precision, which not supported before MySQL 5.6.

pumps.sql.meta.mysql.dont_support_rename_index

ENV: TYK_PMP_PUMPS_SQL_META_MYSQL_DONTSUPPORTRENAMEINDEX
Type: bool

Drop & create when rename index, rename index not supported before MySQL 5.7, MariaDB.

pumps.sql.meta.mysql.dont_support_rename_column

ENV: TYK_PMP_PUMPS_SQL_META_MYSQL_DONTSUPPORTRENAMECOLUMN
Type: bool

change when rename column, rename column not supported before MySQL 8, MariaDB.

pumps.sql.meta.mysql.skip_initialize_with_version

ENV: TYK_PMP_PUMPS_SQL_META_MYSQL_SKIPINITIALIZEWITHVERSION
Type: bool

Auto configure based on currently MySQL version.

pumps.sql.meta.table_sharding

ENV: TYK_PMP_PUMPS_SQL_META_TABLESHARDING
Type: bool

Specifies if all the analytics records are going to be stored in one table or in multiple tables (one per day). By default, false. If false, all the records are going to be stored in tyk_aggregated table. Instead, if it’s true, all the records of the day are going to be stored in tyk_aggregated_YYYYMMDD table, where YYYYMMDD is going to change depending on the date.

pumps.sql.meta.log_level

ENV: TYK_PMP_PUMPS_SQL_META_LOGLEVEL
Type: string

Specifies the SQL log verbosity. The possible values are: info,error and warning. By default, the value is silent, which means that it won’t log any SQL query.

pumps.sql.meta.batch_size

ENV: TYK_PMP_PUMPS_SQL_META_BATCHSIZE
Type: int

Specifies the amount of records that are going to be written each batch. Type int. By default, it writes 1000 records max per batch.

pumps.sqlaggregate.name

ENV: TYK_PMP_PUMPS_SQLAGGREGATE_NAME
Type: string

The name of the pump. This is used to identify the pump in the logs. Deprecated, use type instead.

pumps.sqlaggregate.type

ENV: TYK_PMP_PUMPS_SQLAGGREGATE_TYPE
Type: string

Sets the pump type. This is needed when the pump key does not equal to the pump name type. Current valid types are: mongo, mongo-pump-selective, mongo-pump-aggregate, csv, elasticsearch, influx, influx2, moesif, statsd, segment, graylog, splunk, hybrid, prometheus, logzio, dogstatsd, kafka, syslog, sql, sql_aggregate, stdout, timestream, mongo-graph, sql-graph, sql-graph-aggregate, resurfaceio.

pumps.sqlaggregate.filters

This feature adds a new configuration field in each pump called filters and its structure is the following:

"filters":{
  "api_ids":[],
  "org_ids":[],
  "response_codes":[],
  "skip_api_ids":[],
  "skip_org_ids":[],
  "skip_response_codes":[]
}

The fields api_ids, org_ids and response_codes works as allow list (APIs and orgs where we want to send the analytics records) and the fields skip_api_ids, skip_org_ids and skip_response_codes works as block list.

The priority is always block list configurations over allow list.

An example of configuration would be:

"csv": {
 "type": "csv",
 "filters": {
   "org_ids": ["org1","org2"]
 },
 "meta": {
   "csv_dir": "./bar"
 }
}

pumps.sqlaggregate.filters.org_ids

ENV: TYK_PMP_PUMPS_SQLAGGREGATE_FILTERS_ORGSIDS
Type: []string

Filters pump data by an allow list of org_ids.

pumps.sqlaggregate.filters.api_ids

ENV: TYK_PMP_PUMPS_SQLAGGREGATE_FILTERS_APIIDS
Type: []string

Filters pump data by an allow list of api_ids.

pumps.sqlaggregate.filters.response_codes

ENV: TYK_PMP_PUMPS_SQLAGGREGATE_FILTERS_RESPONSECODES
Type: []int

Filters pump data by an allow list of response_codes.

pumps.sqlaggregate.filters.skip_org_ids

ENV: TYK_PMP_PUMPS_SQLAGGREGATE_FILTERS_SKIPPEDORGSIDS
Type: []string

Filters pump data by a block list of org_ids.

pumps.sqlaggregate.filters.skip_api_ids

ENV: TYK_PMP_PUMPS_SQLAGGREGATE_FILTERS_SKIPPEDAPIIDS
Type: []string

Filters pump data by a block list of api_ids.

pumps.sqlaggregate.filters.skip_response_codes

ENV: TYK_PMP_PUMPS_SQLAGGREGATE_FILTERS_SKIPPEDRESPONSECODES
Type: []int

Filters pump data by a block list of response_codes.

pumps.sqlaggregate.timeout

ENV: TYK_PMP_PUMPS_SQLAGGREGATE_TIMEOUT
Type: int

By default, a pump will wait forever for each write operation to complete; you can configure an optional timeout by setting the configuration option timeout. If you have deployed multiple pumps, then you can configure each timeout independently. The timeout is in seconds and defaults to 0.

The timeout is configured within the main pump config as shown here; note that this example would configure a 5 second timeout:

"pump_name": {
  ...
  "timeout":5,
  "meta": {...}
}

Tyk will inform you if the pump’s write operation is taking longer than the purging loop (configured via purge_delay) as this will mean that data is purged before being written to the target data sink.

If there is no timeout configured and pump’s write operation is taking longer than the purging loop, the following warning log will be generated: Pump {pump_name} is taking more time than the value configured of purge_delay. You should try to set a timeout for this pump.

If there is a timeout configured, but pump’s write operation is still taking longer than the purging loop, the following warning log will be generated: Pump {pump_name} is taking more time than the value configured of purge_delay. You should try lowering the timeout configured for this pump..

pumps.sqlaggregate.omit_detailed_recording

ENV: TYK_PMP_PUMPS_SQLAGGREGATE_OMITDETAILEDRECORDING
Type: bool

Setting this to true will avoid writing raw_request and raw_response fields for each request in pumps. Defaults to false.

pumps.sqlaggregate.max_record_size

ENV: TYK_PMP_PUMPS_SQLAGGREGATE_MAXRECORDSIZE
Type: int

Defines maximum size (in bytes) for Raw Request and Raw Response logs, this value defaults to 0. If it is not set then tyk-pump will not trim any data and will store the full information. This can also be set at a pump level. For example:

"csv": {
  "type": "csv",
  "max_record_size":1000,
  "meta": {
    "csv_dir": "./"
  }
}

pumps.sqlaggregate.ignore_fields

ENV: TYK_PMP_PUMPS_SQLAGGREGATE_IGNOREFIELDS
Type: []string

IgnoreFields defines a list of analytics fields that will be ignored when writing to the pump. This can be used to avoid writing sensitive information to the Database, or data that you don’t really need to have. The field names must be the same as the JSON tags of the analytics record fields. For example: ["api_key", "api_version"].

pumps.sqlaggregate.meta.EnvPrefix

ENV: TYK_PMP_PUMPS_SQLAGGREGATE_META_ENVPREFIX
Type: string

The prefix for the environment variables that will be used to override the configuration. Defaults to TYK_PMP_PUMPS_SQLAGGREGATE_META

pumps.sqlaggregate.meta.type

ENV: TYK_PMP_PUMPS_SQLAGGREGATE_META_TYPE
Type: string

The supported and tested types are sqlite and postgres.

pumps.sqlaggregate.meta.connection_string

ENV: TYK_PMP_PUMPS_SQLAGGREGATE_META_CONNECTIONSTRING
Type: string

Specifies the connection string to the database.

pumps.sqlaggregate.meta.postgres

ENV: TYK_PMP_PUMPS_SQLAGGREGATE_META_POSTGRES
Type: PostgresConfig

Postgres configurations.

pumps.sqlaggregate.meta.postgres.prefer_simple_protocol

ENV: TYK_PMP_PUMPS_SQLAGGREGATE_META_POSTGRES_PREFERSIMPLEPROTOCOL
Type: bool

Disables implicit prepared statement usage.

pumps.sqlaggregate.meta.mysql

ENV: TYK_PMP_PUMPS_SQLAGGREGATE_META_MYSQL
Type: MysqlConfig

Mysql configurations.

pumps.sqlaggregate.meta.mysql.default_string_size

ENV: TYK_PMP_PUMPS_SQLAGGREGATE_META_MYSQL_DEFAULTSTRINGSIZE
Type: uint

Default size for string fields. Defaults to 256.

pumps.sqlaggregate.meta.mysql.disable_datetime_precision

ENV: TYK_PMP_PUMPS_SQLAGGREGATE_META_MYSQL_DISABLEDATETIMEPRECISION
Type: bool

Disable datetime precision, which not supported before MySQL 5.6.

pumps.sqlaggregate.meta.mysql.dont_support_rename_index

ENV: TYK_PMP_PUMPS_SQLAGGREGATE_META_MYSQL_DONTSUPPORTRENAMEINDEX
Type: bool

Drop & create when rename index, rename index not supported before MySQL 5.7, MariaDB.

pumps.sqlaggregate.meta.mysql.dont_support_rename_column

ENV: TYK_PMP_PUMPS_SQLAGGREGATE_META_MYSQL_DONTSUPPORTRENAMECOLUMN
Type: bool

change when rename column, rename column not supported before MySQL 8, MariaDB.

pumps.sqlaggregate.meta.mysql.skip_initialize_with_version

ENV: TYK_PMP_PUMPS_SQLAGGREGATE_META_MYSQL_SKIPINITIALIZEWITHVERSION
Type: bool

Auto configure based on currently MySQL version.

pumps.sqlaggregate.meta.table_sharding

ENV: TYK_PMP_PUMPS_SQLAGGREGATE_META_TABLESHARDING
Type: bool

Specifies if all the analytics records are going to be stored in one table or in multiple tables (one per day). By default, false. If false, all the records are going to be stored in tyk_aggregated table. Instead, if it’s true, all the records of the day are going to be stored in tyk_aggregated_YYYYMMDD table, where YYYYMMDD is going to change depending on the date.

pumps.sqlaggregate.meta.log_level

ENV: TYK_PMP_PUMPS_SQLAGGREGATE_META_LOGLEVEL
Type: string

Specifies the SQL log verbosity. The possible values are: info,error and warning. By default, the value is silent, which means that it won’t log any SQL query.

pumps.sqlaggregate.meta.batch_size

ENV: TYK_PMP_PUMPS_SQLAGGREGATE_META_BATCHSIZE
Type: int

Specifies the amount of records that are going to be written each batch. Type int. By default, it writes 1000 records max per batch.

pumps.sqlaggregate.meta.track_all_paths

ENV: TYK_PMP_PUMPS_SQLAGGREGATE_META_TRACKALLPATHS
Type: bool

Specifies if it should store aggregated data for all the endpoints. By default, false which means that only store aggregated data for tracked endpoints.

pumps.sqlaggregate.meta.ignore_tag_prefix_list

ENV: TYK_PMP_PUMPS_SQLAGGREGATE_META_IGNORETAGPREFIXLIST
Type: []string

Specifies prefixes of tags that should be ignored.

pumps.sqlaggregate.meta.store_analytics_per_minute

ENV: TYK_PMP_PUMPS_SQLAGGREGATE_META_STOREANALYTICSPERMINUTE
Type: bool

Determines if the aggregations should be made per minute instead of per hour.

pumps.sqlaggregate.meta.omit_index_creation

ENV: TYK_PMP_PUMPS_SQLAGGREGATE_META_OMITINDEXCREATION
Type: bool

Set to true to disable the default tyk index creation.

pumps.statsd.name

ENV: TYK_PMP_PUMPS_STATSD_NAME
Type: string

The name of the pump. This is used to identify the pump in the logs. Deprecated, use type instead.

pumps.statsd.type

ENV: TYK_PMP_PUMPS_STATSD_TYPE
Type: string

Sets the pump type. This is needed when the pump key does not equal to the pump name type. Current valid types are: mongo, mongo-pump-selective, mongo-pump-aggregate, csv, elasticsearch, influx, influx2, moesif, statsd, segment, graylog, splunk, hybrid, prometheus, logzio, dogstatsd, kafka, syslog, sql, sql_aggregate, stdout, timestream, mongo-graph, sql-graph, sql-graph-aggregate, resurfaceio.

pumps.statsd.filters

This feature adds a new configuration field in each pump called filters and its structure is the following:

"filters":{
  "api_ids":[],
  "org_ids":[],
  "response_codes":[],
  "skip_api_ids":[],
  "skip_org_ids":[],
  "skip_response_codes":[]
}

The fields api_ids, org_ids and response_codes works as allow list (APIs and orgs where we want to send the analytics records) and the fields skip_api_ids, skip_org_ids and skip_response_codes works as block list.

The priority is always block list configurations over allow list.

An example of configuration would be:

"csv": {
 "type": "csv",
 "filters": {
   "org_ids": ["org1","org2"]
 },
 "meta": {
   "csv_dir": "./bar"
 }
}

pumps.statsd.filters.org_ids

ENV: TYK_PMP_PUMPS_STATSD_FILTERS_ORGSIDS
Type: []string

Filters pump data by an allow list of org_ids.

pumps.statsd.filters.api_ids

ENV: TYK_PMP_PUMPS_STATSD_FILTERS_APIIDS
Type: []string

Filters pump data by an allow list of api_ids.

pumps.statsd.filters.response_codes

ENV: TYK_PMP_PUMPS_STATSD_FILTERS_RESPONSECODES
Type: []int

Filters pump data by an allow list of response_codes.

pumps.statsd.filters.skip_org_ids

ENV: TYK_PMP_PUMPS_STATSD_FILTERS_SKIPPEDORGSIDS
Type: []string

Filters pump data by a block list of org_ids.

pumps.statsd.filters.skip_api_ids

ENV: TYK_PMP_PUMPS_STATSD_FILTERS_SKIPPEDAPIIDS
Type: []string

Filters pump data by a block list of api_ids.

pumps.statsd.filters.skip_response_codes

ENV: TYK_PMP_PUMPS_STATSD_FILTERS_SKIPPEDRESPONSECODES
Type: []int

Filters pump data by a block list of response_codes.

pumps.statsd.timeout

ENV: TYK_PMP_PUMPS_STATSD_TIMEOUT
Type: int

By default, a pump will wait forever for each write operation to complete; you can configure an optional timeout by setting the configuration option timeout. If you have deployed multiple pumps, then you can configure each timeout independently. The timeout is in seconds and defaults to 0.

The timeout is configured within the main pump config as shown here; note that this example would configure a 5 second timeout:

"pump_name": {
  ...
  "timeout":5,
  "meta": {...}
}

Tyk will inform you if the pump’s write operation is taking longer than the purging loop (configured via purge_delay) as this will mean that data is purged before being written to the target data sink.

If there is no timeout configured and pump’s write operation is taking longer than the purging loop, the following warning log will be generated: Pump {pump_name} is taking more time than the value configured of purge_delay. You should try to set a timeout for this pump.

If there is a timeout configured, but pump’s write operation is still taking longer than the purging loop, the following warning log will be generated: Pump {pump_name} is taking more time than the value configured of purge_delay. You should try lowering the timeout configured for this pump..

pumps.statsd.omit_detailed_recording

ENV: TYK_PMP_PUMPS_STATSD_OMITDETAILEDRECORDING
Type: bool

Setting this to true will avoid writing raw_request and raw_response fields for each request in pumps. Defaults to false.

pumps.statsd.max_record_size

ENV: TYK_PMP_PUMPS_STATSD_MAXRECORDSIZE
Type: int

Defines maximum size (in bytes) for Raw Request and Raw Response logs, this value defaults to 0. If it is not set then tyk-pump will not trim any data and will store the full information. This can also be set at a pump level. For example:

"csv": {
  "type": "csv",
  "max_record_size":1000,
  "meta": {
    "csv_dir": "./"
  }
}

pumps.statsd.ignore_fields

ENV: TYK_PMP_PUMPS_STATSD_IGNOREFIELDS
Type: []string

IgnoreFields defines a list of analytics fields that will be ignored when writing to the pump. This can be used to avoid writing sensitive information to the Database, or data that you don’t really need to have. The field names must be the same as the JSON tags of the analytics record fields. For example: ["api_key", "api_version"].

pumps.statsd.meta.EnvPrefix

ENV: TYK_PMP_PUMPS_STATSD_META_ENVPREFIX
Type: string

The prefix for the environment variables that will be used to override the configuration. Defaults to TYK_PMP_PUMPS_STATSD_META

pumps.statsd.meta.address

ENV: TYK_PMP_PUMPS_STATSD_META_ADDRESS
Type: string

Address of statsd including host & port.

pumps.statsd.meta.fields

ENV: TYK_PMP_PUMPS_STATSD_META_FIELDS
Type: []string

Define which Analytics fields should have its own metric calculation.

pumps.statsd.meta.tags

ENV: TYK_PMP_PUMPS_STATSD_META_TAGS
Type: []string

List of tags to be added to the metric.

pumps.statsd.meta.separated_method

ENV: TYK_PMP_PUMPS_STATSD_META_SEPARATEDMETHOD
Type: bool

Allows to have a separated method field instead of having it embedded in the path field.

pumps.stdout.name

ENV: TYK_PMP_PUMPS_STDOUT_NAME
Type: string

The name of the pump. This is used to identify the pump in the logs. Deprecated, use type instead.

pumps.stdout.type

ENV: TYK_PMP_PUMPS_STDOUT_TYPE
Type: string

Sets the pump type. This is needed when the pump key does not equal to the pump name type. Current valid types are: mongo, mongo-pump-selective, mongo-pump-aggregate, csv, elasticsearch, influx, influx2, moesif, statsd, segment, graylog, splunk, hybrid, prometheus, logzio, dogstatsd, kafka, syslog, sql, sql_aggregate, stdout, timestream, mongo-graph, sql-graph, sql-graph-aggregate, resurfaceio.

pumps.stdout.filters

This feature adds a new configuration field in each pump called filters and its structure is the following:

"filters":{
  "api_ids":[],
  "org_ids":[],
  "response_codes":[],
  "skip_api_ids":[],
  "skip_org_ids":[],
  "skip_response_codes":[]
}

The fields api_ids, org_ids and response_codes works as allow list (APIs and orgs where we want to send the analytics records) and the fields skip_api_ids, skip_org_ids and skip_response_codes works as block list.

The priority is always block list configurations over allow list.

An example of configuration would be:

"csv": {
 "type": "csv",
 "filters": {
   "org_ids": ["org1","org2"]
 },
 "meta": {
   "csv_dir": "./bar"
 }
}

pumps.stdout.filters.org_ids

ENV: TYK_PMP_PUMPS_STDOUT_FILTERS_ORGSIDS
Type: []string

Filters pump data by an allow list of org_ids.

pumps.stdout.filters.api_ids

ENV: TYK_PMP_PUMPS_STDOUT_FILTERS_APIIDS
Type: []string

Filters pump data by an allow list of api_ids.

pumps.stdout.filters.response_codes

ENV: TYK_PMP_PUMPS_STDOUT_FILTERS_RESPONSECODES
Type: []int

Filters pump data by an allow list of response_codes.

pumps.stdout.filters.skip_org_ids

ENV: TYK_PMP_PUMPS_STDOUT_FILTERS_SKIPPEDORGSIDS
Type: []string

Filters pump data by a block list of org_ids.

pumps.stdout.filters.skip_api_ids

ENV: TYK_PMP_PUMPS_STDOUT_FILTERS_SKIPPEDAPIIDS
Type: []string

Filters pump data by a block list of api_ids.

pumps.stdout.filters.skip_response_codes

ENV: TYK_PMP_PUMPS_STDOUT_FILTERS_SKIPPEDRESPONSECODES
Type: []int

Filters pump data by a block list of response_codes.

pumps.stdout.timeout

ENV: TYK_PMP_PUMPS_STDOUT_TIMEOUT
Type: int

By default, a pump will wait forever for each write operation to complete; you can configure an optional timeout by setting the configuration option timeout. If you have deployed multiple pumps, then you can configure each timeout independently. The timeout is in seconds and defaults to 0.

The timeout is configured within the main pump config as shown here; note that this example would configure a 5 second timeout:

"pump_name": {
  ...
  "timeout":5,
  "meta": {...}
}

Tyk will inform you if the pump’s write operation is taking longer than the purging loop (configured via purge_delay) as this will mean that data is purged before being written to the target data sink.

If there is no timeout configured and pump’s write operation is taking longer than the purging loop, the following warning log will be generated: Pump {pump_name} is taking more time than the value configured of purge_delay. You should try to set a timeout for this pump.

If there is a timeout configured, but pump’s write operation is still taking longer than the purging loop, the following warning log will be generated: Pump {pump_name} is taking more time than the value configured of purge_delay. You should try lowering the timeout configured for this pump..

pumps.stdout.omit_detailed_recording

ENV: TYK_PMP_PUMPS_STDOUT_OMITDETAILEDRECORDING
Type: bool

Setting this to true will avoid writing raw_request and raw_response fields for each request in pumps. Defaults to false.

pumps.stdout.max_record_size

ENV: TYK_PMP_PUMPS_STDOUT_MAXRECORDSIZE
Type: int

Defines maximum size (in bytes) for Raw Request and Raw Response logs, this value defaults to 0. If it is not set then tyk-pump will not trim any data and will store the full information. This can also be set at a pump level. For example:

"csv": {
  "type": "csv",
  "max_record_size":1000,
  "meta": {
    "csv_dir": "./"
  }
}

pumps.stdout.ignore_fields

ENV: TYK_PMP_PUMPS_STDOUT_IGNOREFIELDS
Type: []string

IgnoreFields defines a list of analytics fields that will be ignored when writing to the pump. This can be used to avoid writing sensitive information to the Database, or data that you don’t really need to have. The field names must be the same as the JSON tags of the analytics record fields. For example: ["api_key", "api_version"].

pumps.stdout.meta.EnvPrefix

ENV: TYK_PMP_PUMPS_STDOUT_META_ENVPREFIX
Type: string

The prefix for the environment variables that will be used to override the configuration. Defaults to TYK_PMP_PUMPS_STDOUT_META

pumps.stdout.meta.format

ENV: TYK_PMP_PUMPS_STDOUT_META_FORMAT
Type: string

Format of the analytics logs. Default is text if json is not explicitly specified. When JSON logging is used all pump logs to stdout will be JSON.

pumps.stdout.meta.log_field_name

ENV: TYK_PMP_PUMPS_STDOUT_META_LOGFIELDNAME
Type: string

Root name of the JSON object the analytics record is nested in.

pumps.syslog.name

ENV: TYK_PMP_PUMPS_SYSLOG_NAME
Type: string

The name of the pump. This is used to identify the pump in the logs. Deprecated, use type instead.

pumps.syslog.type

ENV: TYK_PMP_PUMPS_SYSLOG_TYPE
Type: string

Sets the pump type. This is needed when the pump key does not equal to the pump name type. Current valid types are: mongo, mongo-pump-selective, mongo-pump-aggregate, csv, elasticsearch, influx, influx2, moesif, statsd, segment, graylog, splunk, hybrid, prometheus, logzio, dogstatsd, kafka, syslog, sql, sql_aggregate, stdout, timestream, mongo-graph, sql-graph, sql-graph-aggregate, resurfaceio.

pumps.syslog.filters

This feature adds a new configuration field in each pump called filters and its structure is the following:

"filters":{
  "api_ids":[],
  "org_ids":[],
  "response_codes":[],
  "skip_api_ids":[],
  "skip_org_ids":[],
  "skip_response_codes":[]
}

The fields api_ids, org_ids and response_codes works as allow list (APIs and orgs where we want to send the analytics records) and the fields skip_api_ids, skip_org_ids and skip_response_codes works as block list.

The priority is always block list configurations over allow list.

An example of configuration would be:

"csv": {
 "type": "csv",
 "filters": {
   "org_ids": ["org1","org2"]
 },
 "meta": {
   "csv_dir": "./bar"
 }
}

pumps.syslog.filters.org_ids

ENV: TYK_PMP_PUMPS_SYSLOG_FILTERS_ORGSIDS
Type: []string

Filters pump data by an allow list of org_ids.

pumps.syslog.filters.api_ids

ENV: TYK_PMP_PUMPS_SYSLOG_FILTERS_APIIDS
Type: []string

Filters pump data by an allow list of api_ids.

pumps.syslog.filters.response_codes

ENV: TYK_PMP_PUMPS_SYSLOG_FILTERS_RESPONSECODES
Type: []int

Filters pump data by an allow list of response_codes.

pumps.syslog.filters.skip_org_ids

ENV: TYK_PMP_PUMPS_SYSLOG_FILTERS_SKIPPEDORGSIDS
Type: []string

Filters pump data by a block list of org_ids.

pumps.syslog.filters.skip_api_ids

ENV: TYK_PMP_PUMPS_SYSLOG_FILTERS_SKIPPEDAPIIDS
Type: []string

Filters pump data by a block list of api_ids.

pumps.syslog.filters.skip_response_codes

ENV: TYK_PMP_PUMPS_SYSLOG_FILTERS_SKIPPEDRESPONSECODES
Type: []int

Filters pump data by a block list of response_codes.

pumps.syslog.timeout

ENV: TYK_PMP_PUMPS_SYSLOG_TIMEOUT
Type: int

By default, a pump will wait forever for each write operation to complete; you can configure an optional timeout by setting the configuration option timeout. If you have deployed multiple pumps, then you can configure each timeout independently. The timeout is in seconds and defaults to 0.

The timeout is configured within the main pump config as shown here; note that this example would configure a 5 second timeout:

"pump_name": {
  ...
  "timeout":5,
  "meta": {...}
}

Tyk will inform you if the pump’s write operation is taking longer than the purging loop (configured via purge_delay) as this will mean that data is purged before being written to the target data sink.

If there is no timeout configured and pump’s write operation is taking longer than the purging loop, the following warning log will be generated: Pump {pump_name} is taking more time than the value configured of purge_delay. You should try to set a timeout for this pump.

If there is a timeout configured, but pump’s write operation is still taking longer than the purging loop, the following warning log will be generated: Pump {pump_name} is taking more time than the value configured of purge_delay. You should try lowering the timeout configured for this pump..

pumps.syslog.omit_detailed_recording

ENV: TYK_PMP_PUMPS_SYSLOG_OMITDETAILEDRECORDING
Type: bool

Setting this to true will avoid writing raw_request and raw_response fields for each request in pumps. Defaults to false.

pumps.syslog.max_record_size

ENV: TYK_PMP_PUMPS_SYSLOG_MAXRECORDSIZE
Type: int

Defines maximum size (in bytes) for Raw Request and Raw Response logs, this value defaults to 0. If it is not set then tyk-pump will not trim any data and will store the full information. This can also be set at a pump level. For example:

"csv": {
  "type": "csv",
  "max_record_size":1000,
  "meta": {
    "csv_dir": "./"
  }
}

pumps.syslog.ignore_fields

ENV: TYK_PMP_PUMPS_SYSLOG_IGNOREFIELDS
Type: []string

IgnoreFields defines a list of analytics fields that will be ignored when writing to the pump. This can be used to avoid writing sensitive information to the Database, or data that you don’t really need to have. The field names must be the same as the JSON tags of the analytics record fields. For example: ["api_key", "api_version"].

pumps.syslog.meta.meta_env_prefix

ENV: TYK_PMP_PUMPS_SYSLOG_META_ENVPREFIX
Type: string

The prefix for the environment variables that will be used to override the configuration. Defaults to TYK_PMP_PUMPS_SYSLOG_META

pumps.syslog.meta.transport

ENV: TYK_PMP_PUMPS_SYSLOG_META_TRANSPORT
Type: string

Possible values are udp, tcp, tls in string form.

pumps.syslog.meta.network_addr

ENV: TYK_PMP_PUMPS_SYSLOG_META_NETWORKADDR
Type: string

Host & Port combination of your syslog daemon ie: "localhost:5140".

pumps.syslog.meta.log_level

ENV: TYK_PMP_PUMPS_SYSLOG_META_LOGLEVEL
Type: int

The severity level, an integer from 0-7, based off the Standard: Syslog Severity Levels.

pumps.syslog.meta.tag

ENV: TYK_PMP_PUMPS_SYSLOG_META_TAG
Type: string

Prefix tag

When working with FluentD, you should provide a FluentD Parser based on the OS you are using so that FluentD can correctly read the logs.

"syslog": {
  "name": "syslog",
  "meta": {
    "transport": "udp",
    "network_addr": "localhost:5140",
    "log_level": 6,
    "tag": "syslog-pump"
  }

pumps.timestream.name

ENV: TYK_PMP_PUMPS_TIMESTREAM_NAME
Type: string

The name of the pump. This is used to identify the pump in the logs. Deprecated, use type instead.

pumps.timestream.type

ENV: TYK_PMP_PUMPS_TIMESTREAM_TYPE
Type: string

Sets the pump type. This is needed when the pump key does not equal to the pump name type. Current valid types are: mongo, mongo-pump-selective, mongo-pump-aggregate, csv, elasticsearch, influx, influx2, moesif, statsd, segment, graylog, splunk, hybrid, prometheus, logzio, dogstatsd, kafka, syslog, sql, sql_aggregate, stdout, timestream, mongo-graph, sql-graph, sql-graph-aggregate, resurfaceio.

pumps.timestream.filters

This feature adds a new configuration field in each pump called filters and its structure is the following:

"filters":{
  "api_ids":[],
  "org_ids":[],
  "response_codes":[],
  "skip_api_ids":[],
  "skip_org_ids":[],
  "skip_response_codes":[]
}

The fields api_ids, org_ids and response_codes works as allow list (APIs and orgs where we want to send the analytics records) and the fields skip_api_ids, skip_org_ids and skip_response_codes works as block list.

The priority is always block list configurations over allow list.

An example of configuration would be:

"csv": {
 "type": "csv",
 "filters": {
   "org_ids": ["org1","org2"]
 },
 "meta": {
   "csv_dir": "./bar"
 }
}

pumps.timestream.filters.org_ids

ENV: TYK_PMP_PUMPS_TIMESTREAM_FILTERS_ORGSIDS
Type: []string

Filters pump data by an allow list of org_ids.

pumps.timestream.filters.api_ids

ENV: TYK_PMP_PUMPS_TIMESTREAM_FILTERS_APIIDS
Type: []string

Filters pump data by an allow list of api_ids.

pumps.timestream.filters.response_codes

ENV: TYK_PMP_PUMPS_TIMESTREAM_FILTERS_RESPONSECODES
Type: []int

Filters pump data by an allow list of response_codes.

pumps.timestream.filters.skip_org_ids

ENV: TYK_PMP_PUMPS_TIMESTREAM_FILTERS_SKIPPEDORGSIDS
Type: []string

Filters pump data by a block list of org_ids.

pumps.timestream.filters.skip_api_ids

ENV: TYK_PMP_PUMPS_TIMESTREAM_FILTERS_SKIPPEDAPIIDS
Type: []string

Filters pump data by a block list of api_ids.

pumps.timestream.filters.skip_response_codes

ENV: TYK_PMP_PUMPS_TIMESTREAM_FILTERS_SKIPPEDRESPONSECODES
Type: []int

Filters pump data by a block list of response_codes.

pumps.timestream.timeout

ENV: TYK_PMP_PUMPS_TIMESTREAM_TIMEOUT
Type: int

By default, a pump will wait forever for each write operation to complete; you can configure an optional timeout by setting the configuration option timeout. If you have deployed multiple pumps, then you can configure each timeout independently. The timeout is in seconds and defaults to 0.

The timeout is configured within the main pump config as shown here; note that this example would configure a 5 second timeout:

"pump_name": {
  ...
  "timeout":5,
  "meta": {...}
}

Tyk will inform you if the pump’s write operation is taking longer than the purging loop (configured via purge_delay) as this will mean that data is purged before being written to the target data sink.

If there is no timeout configured and pump’s write operation is taking longer than the purging loop, the following warning log will be generated: Pump {pump_name} is taking more time than the value configured of purge_delay. You should try to set a timeout for this pump.

If there is a timeout configured, but pump’s write operation is still taking longer than the purging loop, the following warning log will be generated: Pump {pump_name} is taking more time than the value configured of purge_delay. You should try lowering the timeout configured for this pump..

pumps.timestream.omit_detailed_recording

ENV: TYK_PMP_PUMPS_TIMESTREAM_OMITDETAILEDRECORDING
Type: bool

Setting this to true will avoid writing raw_request and raw_response fields for each request in pumps. Defaults to false.

pumps.timestream.max_record_size

ENV: TYK_PMP_PUMPS_TIMESTREAM_MAXRECORDSIZE
Type: int

Defines maximum size (in bytes) for Raw Request and Raw Response logs, this value defaults to 0. If it is not set then tyk-pump will not trim any data and will store the full information. This can also be set at a pump level. For example:

"csv": {
  "type": "csv",
  "max_record_size":1000,
  "meta": {
    "csv_dir": "./"
  }
}

pumps.timestream.ignore_fields

ENV: TYK_PMP_PUMPS_TIMESTREAM_IGNOREFIELDS
Type: []string

IgnoreFields defines a list of analytics fields that will be ignored when writing to the pump. This can be used to avoid writing sensitive information to the Database, or data that you don’t really need to have. The field names must be the same as the JSON tags of the analytics record fields. For example: ["api_key", "api_version"].

pumps.timestream.meta.EnvPrefix

ENV: TYK_PMP_PUMPS_TIMESTREAM_META_ENVPREFIX
Type: string

The prefix for the environment variables that will be used to override the configuration. Defaults to TYK_PMP_PUMPS_TIMESTREAM_META

pumps.timestream.meta.AWSRegion

ENV: TYK_PMP_PUMPS_TIMESTREAM_META_AWSREGION
Type: string

The aws region that contains the timestream database

pumps.timestream.meta.TableName

ENV: TYK_PMP_PUMPS_TIMESTREAM_META_TABLENAME
Type: string

The table name where the data is going to be written

pumps.timestream.meta.DatabaseName

ENV: TYK_PMP_PUMPS_TIMESTREAM_META_DATABASENAME
Type: string

The timestream database name that contains the table being written to

pumps.timestream.meta.Dimensions

ENV: TYK_PMP_PUMPS_TIMESTREAM_META_DIMENSIONS
Type: []string

A filter of all the dimensions that will be written to the table. The possible options are [“Method”,“Host”,“Path”,“RawPath”,“APIKey”,“APIVersion”,“APIName”,“APIID”,“OrgID”,“OauthID”]

pumps.timestream.meta.Measures

ENV: TYK_PMP_PUMPS_TIMESTREAM_META_MEASURES
Type: []string

A filter of all the measures that will be written to the table. The possible options are [“ContentLength”,“ResponseCode”,“RequestTime”,“NetworkStats.OpenConnections”, “NetworkStats.ClosedConnection”,“NetworkStats.BytesIn”,“NetworkStats.BytesOut”, “Latency.Total”,“Latency.Upstream”,“GeoData.City.GeoNameID”,“IPAddress”, “GeoData.Location.Latitude”,“GeoData.Location.Longitude”,“UserAgent”,“RawRequest”,“RawResponse”, “RateLimit.Limit”,“Ratelimit.Remaining”,“Ratelimit.Reset”, “GeoData.Country.ISOCode”,“GeoData.City.Names”,“GeoData.Location.TimeZone”]

pumps.timestream.meta.WriteRateLimit

ENV: TYK_PMP_PUMPS_TIMESTREAM_META_WRITERATELIMIT
Type: bool

Set to true in order to save any of the RateLimit measures. Default value is false.

pumps.timestream.meta.ReadGeoFromRequest

ENV: TYK_PMP_PUMPS_TIMESTREAM_META_READGEOFROMREQUEST
Type: bool

If set true, we will try to read geo information from the headers if values aren’t found on the analytic record . Default value is false.

pumps.timestream.meta.WriteZeroValues

ENV: TYK_PMP_PUMPS_TIMESTREAM_META_WRITEZEROVALUES
Type: bool

Set to true, in order to save numerical values with value zero. Default value is false.

pumps.timestream.meta.NameMappings

ENV: TYK_PMP_PUMPS_TIMESTREAM_META_NAMEMAPPINGS
Type: map[string]string

A name mapping for both Dimensions and Measures names. It’s not required

analytics_storage_type

ENV: TYK_PMP_ANALYTICSSTORAGETYPE
Type: string

Sets the analytics storage type. Where the pump will be fetching data from. Currently, only the redis option is supported.

analytics_storage_config

Example Temporal storage configuration:

  "analytics_storage_config": {
    "type": "redis",
    "host": "localhost",
    "port": 6379,
    "hosts": null,
    "username": "",
    "password": "",
    "database": 0,
    "optimisation_max_idle": 100,
    "optimisation_max_active": 0,
    "enable_cluster": false,
    "use_ssl": false,
    "ssl_insecure_skip_verify": false
  },

analytics_storage_config.type

ENV: TYK_PMP_ANALYTICSSTORAGECONFIG_TYPE
Type: string

Deprecated.

analytics_storage_config.host

ENV: TYK_PMP_ANALYTICSSTORAGECONFIG_HOST
Type: string

Host value. For example: “localhost”.

analytics_storage_config.port

ENV: TYK_PMP_ANALYTICSSTORAGECONFIG_PORT
Type: int

Port value. For example: 6379.

analytics_storage_config.hosts

ENV: TYK_PMP_ANALYTICSSTORAGECONFIG_HOSTS
Type: map[string]string

Deprecated: use Addrs instead.

analytics_storage_config.addrs

ENV: TYK_PMP_ANALYTICSSTORAGECONFIG_ADDRS
Type: []string

Use instead of the host value if you’re running a Redis cluster with multiple instances.

analytics_storage_config.master_name

ENV: TYK_PMP_ANALYTICSSTORAGECONFIG_MASTERNAME
Type: string

Sentinel master name.

analytics_storage_config.sentinel_password

ENV: TYK_PMP_ANALYTICSSTORAGECONFIG_SENTINELPASSWORD
Type: string

Sentinel password.

analytics_storage_config.username

ENV: TYK_PMP_ANALYTICSSTORAGECONFIG_USERNAME
Type: string

Database username.

analytics_storage_config.password

ENV: TYK_PMP_ANALYTICSSTORAGECONFIG_PASSWORD
Type: string

Database password.

analytics_storage_config.database

ENV: TYK_PMP_ANALYTICSSTORAGECONFIG_DATABASE
Type: int

Database name.

analytics_storage_config.timeout

ENV: TYK_PMP_ANALYTICSSTORAGECONFIG_TIMEOUT
Type: int

How long to allow for new connections to be established (in milliseconds). Defaults to 5sec.

analytics_storage_config.optimisation_max_idle

ENV: TYK_PMP_ANALYTICSSTORAGECONFIG_MAXIDLE
Type: int

Maximum number of idle connections in the pool.

analytics_storage_config.optimisation_max_active

ENV: TYK_PMP_ANALYTICSSTORAGECONFIG_MAXACTIVE
Type: int

Maximum number of connections allocated by the pool at a given time. When zero, there is no limit on the number of connections in the pool. Defaults to 500.

analytics_storage_config.enable_cluster

ENV: TYK_PMP_ANALYTICSSTORAGECONFIG_ENABLECLUSTER
Type: bool

Enable this option if you are using a cluster instance. Default is false.

analytics_storage_config.redis_key_prefix

ENV: TYK_PMP_ANALYTICSSTORAGECONFIG_REDISKEYPREFIX
Type: string

Prefix the key names. Defaults to “analytics-”. Deprecated: use KeyPrefix instead.

analytics_storage_config.key_prefix

ENV: TYK_PMP_ANALYTICSSTORAGECONFIG_KEYPREFIX
Type: string

Prefix the key names. Defaults to “analytics-”.

analytics_storage_config.use_ssl

ENV: TYK_PMP_ANALYTICSSTORAGECONFIG_USESSL
Type: bool

Setting this to true to use SSL when connecting to the DB.

analytics_storage_config.ssl_insecure_skip_verify

ENV: TYK_PMP_ANALYTICSSTORAGECONFIG_SSLINSECURESKIPVERIFY
Type: bool

Set this to true to tell Pump to ignore database’s cert validation.

analytics_storage_config.ssl_ca_file

ENV: TYK_PMP_ANALYTICSSTORAGECONFIG_SSLCAFILE
Type: string

Path to the CA file.

analytics_storage_config.ssl_cert_file

ENV: TYK_PMP_ANALYTICSSTORAGECONFIG_SSLCERTFILE
Type: string

Path to the cert file.

analytics_storage_config.ssl_key_file

ENV: TYK_PMP_ANALYTICSSTORAGECONFIG_SSLKEYFILE
Type: string

Path to the key file.

analytics_storage_config.ssl_max_version

ENV: TYK_PMP_ANALYTICSSTORAGECONFIG_SSLMAXVERSION
Type: string

Maximum supported TLS version. Defaults to TLS 1.3, valid values are TLS 1.0, 1.1, 1.2, 1.3.

analytics_storage_config.ssl_min_version

ENV: TYK_PMP_ANALYTICSSTORAGECONFIG_SSLMINVERSION
Type: string

Minimum supported TLS version. Defaults to TLS 1.2, valid values are TLS 1.0, 1.1, 1.2, 1.3.

analytics_storage_config.redis_use_ssl

ENV: TYK_PMP_ANALYTICSSTORAGECONFIG_REDISUSESSL
Type: bool

Setting this to true to use SSL when connecting to the DB. Deprecated: use UseSSL instead.

analytics_storage_config.redis_ssl_insecure_skip_verify

ENV: TYK_PMP_ANALYTICSSTORAGECONFIG_REDISSSLINSECURESKIPVERIFY
Type: bool

Set this to true to tell Pump to ignore database’s cert validation. Deprecated: use SSLInsecureSkipVerify instead.

statsd_connection_string

ENV: TYK_PMP_STATSDCONNECTIONSTRING
Type: string

Connection string for StatsD monitoring for information please see the Instrumentation docs.

statsd_prefix

ENV: TYK_PMP_STATSDPREFIX
Type: string

Custom prefix value. For example separate settings for production and staging.

log_level

ENV: TYK_PMP_LOGLEVEL
Type: string

Set the logger details for tyk-pump. The posible values are: info,debug,error and warn. By default, the log level is info.

log_format

ENV: TYK_PMP_LOGFORMAT
Type: string

Set the logger format. The possible values are: text and json. By default, the log format is text.

Health Check

From v2.9.4, we have introduced a /health endpoint to confirm the Pump is running. You need to configure the following settings. This returns a HTTP 200 OK response if the Pump is running.

health_check_endpoint_name

ENV: TYK_PMP_HEALTHCHECKENDPOINTNAME
Type: string

The default is “health”.

health_check_endpoint_port

ENV: TYK_PMP_HEALTHCHECKENDPOINTPORT
Type: int

The default port is 8083.

omit_detailed_recording

ENV: TYK_PMP_OMITDETAILEDRECORDING
Type: bool

Setting this to true will avoid writing raw_request and raw_response fields for each request in pumps. Defaults to false.

max_record_size

ENV: TYK_PMP_MAXRECORDSIZE
Type: int

Defines maximum size (in bytes) for Raw Request and Raw Response logs, this value defaults to 0. If it is not set then tyk-pump will not trim any data and will store the full information. This can also be set at a pump level. For example:

"csv": {
  "type": "csv",
  "max_record_size":1000,
  "meta": {
    "csv_dir": "./"
  }
}

omit_config_file

ENV: TYK_PMP_OMITCONFIGFILE
Type: bool

Defines if tyk-pump should ignore all the values in configuration file. Specially useful when setting all configurations in environment variables.

enable_http_profiler

ENV: TYK_PMP_HTTPPROFILE
Type: bool

Enable debugging of Tyk Pump by exposing profiling information, the same as the gateway

raw_request_decoded

ENV: TYK_PMP_DECODERAWREQUEST
Type: bool

Setting this to true allows the Raw Request to be decoded from base 64 for all pumps. This is set to false by default.

raw_response_decoded

ENV: TYK_PMP_DECODERAWRESPONSE
Type: bool

Setting this to true allows the Raw Response to be decoded from base 64 for all pumps. This is set to false by default.