Configuration
editConfiguration
editAdapt the Elastic APM Go agent to your needs with one of the following methods—listed in descending order of precedence:
- APM Agent Configuration via Kibana (supported options are marked with )
- In code, using the Tracer Config API
- Environment variables
Configuration defined via Kibana will take precedence over the same configuration defined in code, which takes precedence over environment variables. If configuration is defined via Kibana, and then that is later removed, the agent will revert to configuration defined locally via either the Tracer Config API or environment variables.
To simplify development and testing,
the agent defaults to sending data to the Elastic APM Server at http://localhost:8200
.
To send data to an alternative location, you must configure
ELASTIC_APM_SERVER_URL. Depending on the configuration
of your server, you may also need to set ELASTIC_APM_API_KEY,
ELASTIC_APM_SECRET_TOKEN, and
ELASTIC_APM_VERIFY_SERVER_CERT. All other variables
have usable defaults.
Dynamic configuration
editConfiguration options marked with the badge can be changed at runtime when set from a supported source.
The Go Agent supports Central configuration,
which allows you to fine-tune certain configurations via the APM app.
This feature is enabled in the Agent by default, with ELASTIC_APM_CENTRAL_CONFIG
.
Configuration formats
editSome options require a unit, either duration or size. These need to be provided in a specific format.
Duration format
editThe duration format is used for options like timeouts. The unit is provided as a suffix directly after the number, without any whitespace.
Example: 5ms
Supported units:
-
ms
(milliseconds) -
s
(seconds) -
m
(minutes)
Size format
editThe size format is used for options such as maximum buffer sizes. The unit is provided as a suffix directly after the number, without any whitespace.
Example: 10KB
Supported units:
- B (bytes)
- KB (kilobytes)
- MB (megabytes)
- GB (gigabytes)
We use the power-of-two sizing convention, e.g. 1KB = 1024B.
ELASTIC_APM_SERVER_URL
editEnvironment | Default | Example |
---|---|---|
|
|
|
The URL for your Elastic APM Server. The Server supports both HTTP and HTTPS.
If you use HTTPS, then you may need to configure your client machines so
that the server certificate can be verified. You can disable certificate
verification with ELASTIC_APM_VERIFY_SERVER_CERT
.
ELASTIC_APM_SERVER_TIMEOUT
editEnvironment | Default | Example |
---|---|---|
|
|
|
The timeout for requests made to your Elastic APM server. When set to zero or a negative value, timeouts will be disabled.
ELASTIC_APM_SECRET_TOKEN
editEnvironment | Default | Example |
---|---|---|
|
"A random string" |
This string is used to ensure that only your agents can send data to your APM server. Both the agents and the APM server have to be configured with the same secret token.
The secret token is sent as plain-text in every request to the server, so you should also secure your communications using HTTPS. Unless you do so, your secret token could be observed by an attacker.
ELASTIC_APM_API_KEY
editEnvironment | Default | Example |
---|---|---|
|
"A base64-encoded string" |
This base64-encoded string is used to ensure that only your agents can send data to your APM server. The API key must be created using the APM Server command line tool.
The API Key is sent as plain-text in every request to the server, so you should also secure your communications using HTTPS. Unless you do so, your API Key could be observed by an attacker.
ELASTIC_APM_SERVICE_NAME
editEnvironment | Default | Example |
---|---|---|
|
Executable name |
|
The name of your service or application. This is used to keep all the errors and transactions of your service together and is the primary filter in the Elastic APM user interface.
If you do not specify ELASTIC_APM_SERVICE_NAME
, the Go agent will use the
executable name. e.g. if your executable is called "my-app.exe", then your
service will be identified as "my-app".
The service name must conform to this regular expression: ^[a-zA-Z0-9 _-]+$
.
In other words: your service name must only contain characters from the ASCII
alphabet, numbers, dashes, underscores, and spaces.
ELASTIC_APM_SERVICE_VERSION
editEnvironment | Default | Example |
---|---|---|
|
A string indicating the version of the deployed service |
A version string for the currently deployed version of the service.
If you don’t version your deployments, the recommended value for this field is the commit identifier
of the deployed revision, e.g. the output of git rev-parse HEAD
.
ELASTIC_APM_SERVICE_NODE_NAME
editEnvironment | Default | Example |
---|---|---|
|
|
Optional name used to differentiate between nodes in a service. Must be unique, otherwise data from multiple nodes will be aggregated together.
If you do not specify ELASTIC_APM_SERVICE_NODE_NAME
, service nodes will be identified using the container ID if available,
otherwise the host name.
This feature is fully supported in the APM Server versions >= 7.5.
ELASTIC_APM_ENVIRONMENT
editEnvironment | Default | Example |
---|---|---|
|
|
The name of the environment this service is deployed in, e.g. "production" or "staging".
Environments allow you to easily filter data on a global level in the APM app. It’s important to be consistent when naming environments across agents. See environment selector in the APM app for more information.
This feature is fully supported in the APM app in Kibana versions >= 7.2. You must use the query bar to filter for a specific environment in versions prior to 7.2.
ELASTIC_APM_ACTIVE
editEnvironment | Default | Example |
---|---|---|
|
true |
|
Enable or disable the agent. If set to false, then the Go agent does not send any data to the Elastic APM server, and instrumentation overhead is minimized.
ELASTIC_APM_RECORDING
editEnvironment | Default | Example |
---|---|---|
|
true |
|
Enable or disable recording of events. If set to false, then the Go agent does not send any events to the Elastic APM server, and instrumentation overhead is minimized, but the agent will continue to poll the server for configuration changes.
ELASTIC_APM_GLOBAL_LABELS
editEnvironment | Default | Example |
---|---|---|
|
|
Labels are added to all events. The format for labels is: key=value[,key=value[,...]]
.
Any labels set by application via the API will override global labels with the same keys.
This option requires APM Server 7.2 or greater, and will have no effect when using older server versions.
ELASTIC_APM_TRANSACTION_IGNORE_URLS
editEnvironment | Default | Example |
---|---|---|
|
|
A list of patterns to match HTTP requests to ignore. An incoming HTTP request whose request line matches any of the patterns will not be reported as a transaction.
This option supports the wildcard *
, which matches zero or more characters.
Examples: /foo/*/bar/*/baz*
, *foo*
. Matching is case insensitive by default.
Prefixing a pattern with (?-i)
makes the matching case sensitive.
This configuration was previously known as ELASTIC_APM_IGNORE_URLS
, which has been deprecated and will be removed in a future major
version of the agent.
ELASTIC_APM_SANITIZE_FIELD_NAMES
editEnvironment | Default | Example |
---|---|---|
|
|
|
A list of patterns to match the names of HTTP headers, cookies, and POST form fields to redact.
This option supports the wildcard *
, which matches zero or more characters.
Examples: /foo/*/bar/*/baz*
, *foo*
. Matching is case insensitive by default.
Prefixing a pattern with (?-i)
makes the matching case sensitive.
ELASTIC_APM_CAPTURE_HEADERS
editEnvironment | Default |
---|---|
|
|
For transactions that are HTTP requests, the Go agent can optionally capture request and response headers.
Possible values: true
, false
.
Captured headers are subject to sanitization, per ELASTIC_APM_SANITIZE_FIELD_NAMES
.
ELASTIC_APM_CAPTURE_BODY
editEnvironment | Default |
---|---|
|
|
For transactions that are HTTP requests, the Go agent can optionally capture the request body.
Possible values: errors
, transactions
, all
, off
.
Request bodies often contain sensitive values like passwords, credit card numbers, and so on. If your service handles data like this, enable this feature with care.
ELASTIC_APM_HOSTNAME
editEnvironment | Default | Example |
---|---|---|
|
|
|
The host name to use when sending error and transaction data to the APM server.
ELASTIC_APM_API_REQUEST_TIME
editEnvironment | Default |
---|---|
|
|
The amount of time to wait before ending a request to the Elastic APM server. When you report transactions, spans and errors, the agent will initiate a request and send them to the server when there is enough data to send; the request will remain open until this time has been exceeded, or until the maximum request size has been reached.
ELASTIC_APM_API_REQUEST_SIZE
editEnvironment | Default | Minimum | Maximum |
---|---|---|---|
|
|
|
|
The maximum size of request bodies to send to the Elastic APM server. The agent will maintain an in-memory buffer of compressed data for streaming to the APM server.
ELASTIC_APM_API_BUFFER_SIZE
editEnvironment | Default | Minimum | Maximum |
---|---|---|---|
|
|
|
|
The maximum number of bytes of uncompressed, encoded events to store in memory while the agent is busy. When the agent is able to, it will transfer buffered data to the request buffer, and start streaming it to the server. If the buffer fills up, new events will start replacing older ones.
ELASTIC_APM_TRANSACTION_MAX_SPANS
editEnvironment | Default |
---|---|
|
|
Limits the amount of spans that are recorded per transaction.
This is helpful in cases where a transaction creates a large number of spans (e.g. thousands of SQL queries). Setting an upper limit will prevent overloading the agent and the APM server with too much work for such edge cases.
ELASTIC_APM_EXIT_SPAN_MIN_DURATION
editEnvironment | Default |
---|---|
|
|
Sets the minimum duration for an exit span to be reported. Spans shorter or equal to this threshold will be dropped by the agent and reported as statistics in the span’s transaction, as long as the transaction didn’t end before the span was reported.
When span compression is enabled (ELASTIC_APM_SPAN_COMPRESSION_ENABLED
), the sum
of the compressed span composite is considered.
The minimum duration allowed for this setting is 1 microsecond (us
).
ELASTIC_APM_SPAN_FRAMES_MIN_DURATION
editEnvironment | Default |
---|---|
|
|
The APM agent will collect a stack trace for every recorded span whose duration exceeds this configured value. While this is very helpful to find the exact place in your code that causes the span, collecting this stack trace does have some processing and storage overhead.
This configuration has been deprecated and will be removed in a future major version of the agent.
ELASTIC_APM_SPAN_STACK_TRACE_MIN_DURATION
editEnvironment | Default |
---|---|
|
|
The APM agent will collect a stack trace for every recorded span whose duration exceeds this configured value. While this is very helpful to find the exact place in your code that causes the span, collecting this stack trace does have some processing and storage overhead.
This configuration was previously known as ELASTIC_APM_SPAN_FRAMES_MIN_DURATION
, which has been deprecated and will be removed in a future major
version of the agent.
ELASTIC_APM_STACK_TRACE_LIMIT
editEnvironment | Default |
---|---|
|
|
Limits the number of frames captured for each stack trace.
Setting the limit to 0 will disable stack trace collection, while any positive integer value will be used as the maximum number of frames to collect. Setting a negative value, such as -1, means that all frames will be collected.
ELASTIC_APM_TRANSACTION_SAMPLE_RATE
editEnvironment | Default |
---|---|
|
|
By default, the agent will sample every transaction (e.g. request to your service).
To reduce overhead and storage requirements, set the sample rate to a value
between 0.0
and 1.0
. We still record overall time and the result for unsampled
transactions, but no context information, tags, or spans.
ELASTIC_APM_METRICS_INTERVAL
editEnvironment | Default |
---|---|
|
30s |
The interval at which APM agent gathers and reports metrics. Set to 0s
to disable.
ELASTIC_APM_DISABLE_METRICS
editEnvironment | Default | Example |
---|---|---|
|
|
Disables the collection of certain metrics. If the name of a metric matches any of the wildcard expressions, it will not be collected.
This option supports the wildcard *
, which matches zero or more characters.
Examples: /foo/*/bar/*/baz*
, *foo*
. Matching is case insensitive by default.
Prefixing a pattern with (?-i)
makes the matching case sensitive.
ELASTIC_APM_BREAKDOWN_METRICS
editEnvironment | Default |
---|---|
|
|
Capture breakdown metrics. Set to false
to disable.
ELASTIC_APM_SERVER_CERT
editEnvironment | Default |
---|---|
|
If you have configured your APM Server with a self signed TLS certificate, or you
want to pin the server certificate, specify the path to the PEM-encoded
certificate via the ELASTIC_APM_SERVER_CERT
configuration.
ELASTIC_APM_SERVER_CA_CERT_FILE
editEnvironment | Default |
---|---|
|
The path to a PEM-encoded TLS Certificate Authority certificate that will be used for verifying the server’s TLS certificate chain.
ELASTIC_APM_VERIFY_SERVER_CERT
editEnvironment | Default |
---|---|
|
|
By default, the agent verifies the server’s certificate if you use an
HTTPS connection to the APM server. Verification can be disabled by
changing this setting to false
. This setting is ignored when
ELASTIC_APM_SERVER_CERT
is set.
ELASTIC_APM_LOG_FILE
editEnvironment | Default |
---|---|
|
ELASTIC_APM_LOG_FILE
specifies the output file for the agent’s default, internal
logger. The file will be created, or truncated if it exists, when the process starts.
By default, logging is disabled. You must specify ELASTIC_APM_LOG_FILE
to enable
it. This environment variable will be ignored if a logger is configured programatically.
There are two special file names that the agent recognizes: stdout
and stderr
.
These will configure the logger to write to standard output and standard error
respectively.
ELASTIC_APM_LOG_LEVEL
editEnvironment | Default |
---|---|
|
|
ELASTIC_APM_LOG_LEVEL
specifies the log level for the agent’s default, internal
logger. The only two levels used by the logger are "error" and "debug". By default,
logging is disabled. You must specify ELASTIC_APM_LOG_FILE
to enable it.
This environment variable will be ignored if a logger is configured programatically.
ELASTIC_APM_CENTRAL_CONFIG
editEnvironment | Default |
---|---|
|
|
Activate APM Agent central configuration via Kibana. By default the agent will poll the server
for agent configuration changes. This can be disabled by changing the setting to false
.
See APM Agent central configuration for more information.
This feature requires APM Server v7.3 or later.
ELASTIC_APM_USE_ELASTIC_TRACEPARENT_HEADER
editEnvironment |
Default |
|
|
To enable distributed tracing, the agent
adds trace context headers to outgoing HTTP requests made with module/apmhttp.
These headers (traceparent
and tracestate
) are defined in the
W3C Trace Context specification.
When this setting is true
, the agent will also add the header elastic-apm-traceparent
for backwards compatibility with older versions of Elastic APM agents.
ELASTIC_APM_CLOUD_PROVIDER
editEnvironment | Default | Example |
---|---|---|
|
|
|
This config value allows you to specify which cloud provider should be assumed for metadata collection. By default, the agent will use trial and error to automatically collect the cloud metadata.
Valid options are "none"
, "auto"
, "aws"
, "gcp"
, and "azure"
If this config value is set to "none"
, then no cloud metadata will be collected.
ELASTIC_APM_SPAN_COMPRESSION_ENABLED
editEnvironment | Default |
---|---|
|
|
When enabled, the agent will attempt to compress short exit spans that share the
same parent into a composite span. The exact duration for what is considered
short, depends on the compression strategy used (same_kind
or exact_match
).
In order for a span to be compressible, these conditions need to be met:
- Spans are exit spans.
- Spans are siblings (share the same parent).
- Spans have not propagated their context downstream.
- Each span duration is equal or lower to the compression strategy maximum duration.
-
Spans are compressed with
same_kind
strategy when these attributes are equal:-
span.type
. -
span.subtype
. -
span.context.destination.service.resource
-
-
Spans are compressed with
exact_match
strategy when all the previous conditions are met and thespan.name
is equal.
Compressing short exit spans should provide some storage savings for services that create a lot of consecutive short exit spans to for example databases or cache services which are generally uninteresting when viewing a trace.
This feature is experimental and requires APM Server v7.15 or later.
ELASTIC_APM_SPAN_COMPRESSION_EXACT_MATCH_MAX_DURATION
editEnvironment | Default |
---|---|
|
|
The maximum duration to consider for compressing sibling exit spans that are an exact match for compression.
ELASTIC_APM_SPAN_COMPRESSION_SAME_KIND_MAX_DURATION
editEnvironment | Default |
---|---|
|
|
The maximum duration to consider for compressing sibling exit spans that are of the same kind for compression.
ELASTIC_APM_TRACE_CONTINUATION_STRATEGY
editEnvironment | Default |
---|---|
|
|
This option allows some control over how the APM agent handles W3C trace-context headers on incoming requests. By default, the traceparent and tracestate headers are used per W3C spec for distributed tracing. However, in certain cases it can be helpful to not use the incoming traceparent header. Some example use cases:
- An Elastic-monitored service is receiving requests with traceparent headers from unmonitored services.
- An Elastic-monitored service is publicly exposed, and does not want tracing data (trace-ids, sampling decisions) to possibly be spoofed by user requests.
Valid options are continue
, restart
, and restart_external
:
- continue
-
The default behavior. An incoming
traceparent
value is used to continue the trace and determine the sampling decision. - restart
-
Always ignores the
traceparent
header of incoming requests. A new trace-id will be generated and the sampling decision will be made based ontransaction_sample_rate
. A span link will be made to the incomingtraceparent
. - restart_external
-
If an incoming request includes the
es
vendor flag intracestate
, then anytraceparent
will be considered internal and will be handled as described for continue above. Otherwise, anytraceparent
is considered external and will be handled as described for restart above.
Starting with Elastic Observability 8.2, span links are visible in trace views.