Profiler Auto instrumentation
editProfiler Auto instrumentation
editQuick start
editThe agent can automatically instrument .NET Framework, .NET Core, and .NET applications using the .NET CLR Profiling APIs. The Profiling APIs provide a way to instrument an application or dependency code without code changes.
This approach works with the following
Operating system |
||
Architecture |
windows |
Linux ** |
x64 |
.NET Framework 4.6.2+* .NET Core 2.1+ .NET 5+ |
.NET Core 2.1+ .NET 5+ |
* Due to binding issues introduced by Microsoft, we recommend at least .NET Framework 4.7.2 for best compatibility.
** Minimum GLIBC version 2.14.
The profiler-based agent only supports 64-bit applications. 32-bit applications aren’t supported.
The profiler-based agent does not currently support ARM.
It instruments the following assemblies:
Integration name | NuGet package version(s) | Assembly version(s) |
---|---|---|
AdoNet |
part of .NET |
System.Data 4.0.0 - 4.*.* |
part of .NET |
System.Data.Common 4.0.0 - 5.*.* |
|
AspNet |
part of .NET Framework |
System.Web 4.0.0 - 4.*.* |
Kafka |
Confluent.Kafka 1.4.0 - 1.*.* |
|
MySqlCommand |
MySql.Data 6.7.0 - 8.*.* |
|
NpgsqlCommand |
Npgsql 4.0.0 - 7.*.* |
|
OracleCommand |
Oracle.ManagedDataAccess 4.122.0 - 4.122.* |
|
Oracle.ManagedDataAccess 2.0.0 - 3.*.* |
||
RabbitMQ |
RabbitMQ.Client 3.6.9 - 6.*.* |
|
SqlCommand |
part of .NET |
System.Data 4.0.0 - 4.*.* |
System.Data.SqlClient 4.0.0 - 4.*.* |
||
Microsoft.Data.SqlClient 1.0.0 - 5.*.* |
||
SqliteCommand |
Microsoft.Data.Sqlite 2.0.0 - 7.*.* |
|
System.Data.SQLite 1.0.0 - 2.*.* |
The .NET CLR Profiling API allows only one profiler to be attached to a .NET process. In light of this limitation, only one solution that uses the .NET CLR profiling API should be used by an application.
Auto instrumentation using the .NET CLR Profiling API can be used in conjunction with
- the Public API to perform manual instrumentation.
-
NuGet packages that perform instrumentation using a
IDiagnosticsSubscriber
to subscribe to diagnostic events.
The version number of NuGet packages referenced by a project instrumented with a profiler must be the same as the version number of profiler zip file used.
General steps
editThe general steps in configuring profiler auto instrumentation are as follows; See Instrumenting containers and services for configuration for common deployment environments.
-
Download the
elastic_apm_profiler_<version>.zip
file from the Releases page of the .NET APM Agent GitHub repository, where<version>
is the version number to download. You can find the file under Assets. - Unzip the zip file into a folder on the host that is hosting the application to instrument.
-
Configure the following environment variables:
.NET Framework.
set COR_ENABLE_PROFILING = "1" set COR_PROFILER = "{FA65FE15-F085-4681-9B20-95E04F6C03CC}" set COR_PROFILER_PATH = "<unzipped directory>\elastic_apm_profiler.dll" set ELASTIC_APM_PROFILER_HOME = "<unzipped directory>" set ELASTIC_APM_PROFILER_INTEGRATIONS = "<unzipped directory>\integrations.yml" set ELASTIC_APM_SERVER_URL = "<apm server url>" set ELASTIC_APM_SECRET_TOKEN = "<secret token>"
<unzipped directory>
is the directory to which the zip file was unzipped in step 2.The URL of the APM server intake to which traces and metrics should be sent.
The secret token used by the APM Agent to authenticate with APM server.
.NET Core / .NET 5+ on windows.
set CORECLR_ENABLE_PROFILING = "1" set CORECLR_PROFILER = "{FA65FE15-F085-4681-9B20-95E04F6C03CC}" set CORECLR_PROFILER_PATH = "<unzipped directory>\elastic_apm_profiler.dll" set ELASTIC_APM_PROFILER_HOME = "<unzipped directory>" set ELASTIC_APM_PROFILER_INTEGRATIONS = "<unzipped directory>\integrations.yml" set ELASTIC_APM_SERVER_URL = "<apm server url>" set ELASTIC_APM_SECRET_TOKEN = "<secret token>"
<unzipped directory>
is the directory to which the zip file was unzipped in step 2.The URL of the APM server intake to which traces and metrics should be sent.
The secret token used by the APM Agent to authenticate with APM server.
.NET Core / .NET 5+ on Linux.
export CORECLR_ENABLE_PROFILING=1 export CORECLR_PROFILER={FA65FE15-F085-4681-9B20-95E04F6C03CC} export CORECLR_PROFILER_PATH="<unzipped directory>/libelastic_apm_profiler.so" export ELASTIC_APM_PROFILER_HOME="<unzipped directory>" export ELASTIC_APM_PROFILER_INTEGRATIONS="<unzipped directory>/integrations.yml" export ELASTIC_APM_SERVER_URL = "<apm server url>" export ELASTIC_APM_SECRET_TOKEN = "<secret token>"
<unzipped directory>
is the directory to which the zip file was unzipped in step 2.The URL of the APM server intake to which traces and metrics should be sent.
The secret token used by the APM Agent to authenticate with APM server.
Ensure you start your application in a context where the set environment variables are visible.
With this setup, the .NET runtime loads Elastic’s CLR profiler into the .NET process, which loads and instantiates the APM agent early in application startup. The profiler monitors methods of interest and injects code to instrument their execution.
Instrumenting containers and services
editUsing global environment variables causes profiler auto instrumentation to be loaded for any .NET process started on the host. Often, the environment variables should be set only for specific services or containers. The following sections demonstrate how to configure common containers and services.
Docker containers
editA build image containing the files for profiler auto instrumentation can be used as part of a multi-stage build
ARG AGENT_VERSION=<VERSION> FROM alpine:latest AS build ARG AGENT_VERSION WORKDIR /source # install unzip RUN apk update && apk add zip curl # pull down the zip file based on ${AGENT_VERSION} ARG and unzip RUN curl -L -o elastic_apm_profiler_${AGENT_VERSION}.zip https://github.com/elastic/apm-agent-dotnet/releases/download/v${AGENT_VERSION}/elastic_apm_profiler_${AGENT_VERSION}.zip && \ unzip elastic_apm_profiler_${AGENT_VERSION}.zip -d /elastic_apm_profiler_${AGENT_VERSION}
Replace |
The files can then be copied into a subsequent stage
COPY --from=build /elastic_apm_profiler_${AGENT_VERSION} /elastic_apm_profiler
Environment variables can be added to a Dockerfile to configure profiler auto instrumentation
ENV CORECLR_ENABLE_PROFILING=1 ENV CORECLR_PROFILER={FA65FE15-F085-4681-9B20-95E04F6C03CC} ENV CORECLR_PROFILER_PATH=/elastic_apm_profiler/libelastic_apm_profiler.so ENV ELASTIC_APM_PROFILER_HOME=/elastic_apm_profiler ENV ELASTIC_APM_PROFILER_INTEGRATIONS=/elastic_apm_profiler/integrations.yml ENTRYPOINT ["dotnet", "your-application.dll"]
You should also consider how to securely provide the the APM server URL and secret token as environment variables when you run your container. It is not recommended to include the secret token in the image as it may be accidently exposed.
windows services
editEnvironment variables can be added to specific windows services by adding an entry to the windows registry. Using PowerShell
.NET Framework service.
$environment = [string[]]@( "COR_ENABLE_PROFILING=1", "COR_PROFILER={FA65FE15-F085-4681-9B20-95E04F6C03CC}", "COR_PROFILER_PATH=<unzipped directory>\elastic_apm_profiler.dll", "ELASTIC_APM_PROFILER_HOME=<unzipped directory>", "ELASTIC_APM_PROFILER_INTEGRATIONS=<unzipped directory>\integrations.yml" "ELASTIC_APM_SERVER_URL=<apm server url>" "ELASTIC_APM_SECRET_TOKEN=<secret token>") Set-ItemProperty HKLM:SYSTEM\CurrentControlSet\Services\<service-name> -Name Environment -Value $environment
.NET Core service.
$environment = [string[]]@( "CORECLR_ENABLE_PROFILING=1", "CORECLR_PROFILER={FA65FE15-F085-4681-9B20-95E04F6C03CC}", "CORECLR_PROFILER_PATH=<unzipped directory>\elastic_apm_profiler.dll", "ELASTIC_APM_PROFILER_HOME=<unzipped directory>", "ELASTIC_APM_PROFILER_INTEGRATIONS=<unzipped directory>\integrations.yml" "ELASTIC_APM_SERVER_URL=<apm server url>" "ELASTIC_APM_SECRET_TOKEN=<secret token>") Set-ItemProperty HKLM:SYSTEM\CurrentControlSet\Services\<service-name> -Name Environment -Value $environment
|
|
The URL of the APM server intake to which traces and metrics should be sent. |
|
The secret token used by the APM Agent to authenticate with APM server. |
|
|
The service must then be restarted for the change to take effect. With PowerShell
Restart-Service <service-name>
Internet Information Services (IIS)
editFor IIS versions before IIS 10, it is not possible to set environment variables scoped to a specific application pool, so environment variables need to set globally.
For IIS 10 onwards, environment variables can be set for an application pool using AppCmd.exe. With PowerShell
.NET Framework.
$appcmd = "$($env:systemroot)\system32\inetsrv\AppCmd.exe" $appPool = "<application-pool>" $profilerHomeDir = "<unzipped directory>" $environment = @{ COR_ENABLE_PROFILING = "1" COR_PROFILER = "{FA65FE15-F085-4681-9B20-95E04F6C03CC}" COR_PROFILER_PATH = "$profilerHomeDir\elastic_apm_profiler.dll" ELASTIC_APM_PROFILER_HOME = "$profilerHomeDir" ELASTIC_APM_PROFILER_INTEGRATIONS = "$profilerHomeDir\integrations.yml" COMPlus_LoaderOptimization = "1" ELASTIC_APM_SERVER_URL = "<apm server url>" ELASTIC_APM_SECRET_TOKEN = "<secret token>" } $environment.Keys | ForEach-Object { & $appcmd set config -section:system.applicationHost/applicationPools /+"[name='$appPool'].environmentVariables.[name='$_',value='$($environment[$_])']" }
|
|
|
|
Forces assemblies not to be loaded domain-neutral. There is currently a limitation where Profiler auto-instrumentation cannot instrument assemblies when they are loaded domain-neutral. This limitation is expected to be removed in future, but for now, can be worked around by setting this environment variable. See the Microsoft documentation for further details. |
|
The URL of the APM server intake to which traces and metrics should be sent. |
|
The secret token used by the APM Agent to authenticate with APM server. |
.NET Core.
$appcmd = "$($env:systemroot)\system32\inetsrv\AppCmd.exe" $appPool = "<application-pool>" $profilerHomeDir = "<unzipped directory>" $environment = @{ CORECLR_ENABLE_PROFILING = "1" CORECLR_PROFILER = "{FA65FE15-F085-4681-9B20-95E04F6C03CC}" CORECLR_PROFILER_PATH = "$profilerHomeDir\elastic_apm_profiler.dll" ELASTIC_APM_PROFILER_HOME = "$profilerHomeDir" ELASTIC_APM_PROFILER_INTEGRATIONS = "$profilerHomeDir\integrations.yml" ELASTIC_APM_SERVER_URL = "<apm server url>" ELASTIC_APM_SECRET_TOKEN = "<secret token>" } $environment.Keys | ForEach-Object { & $appcmd set config -section:system.applicationHost/applicationPools /+"[name='$appPool'].environmentVariables.[name='$_',value='$($environment[$_])']" }
|
|
|
|
The URL of the APM server intake to which traces and metrics should be sent. |
|
The secret token used by the APM Agent to authenticate with APM server. |
Ensure that the location of the <unzipped directory>
is accessible and executable to the Identity
account under which the Application Pool runs.
Once environment variables have been set, stop and start IIS so that applications hosted in IIS will see the new environment variables.
net stop /y was net start w3svc
systemd / systemctl
editEnvironment variables can be added to specific services run with systemd by creating an environment.env file containing the following
CORECLR_ENABLE_PROFILING=1 CORECLR_PROFILER={FA65FE15-F085-4681-9B20-95E04F6C03CC} CORECLR_PROFILER_PATH=/<unzipped directory>/libelastic_apm_profiler.so ELASTIC_APM_PROFILER_HOME=/<unzipped directory> ELASTIC_APM_PROFILER_INTEGRATIONS=/<unzipped directory>/integrations.yml ELASTIC_APM_SERVER_URL=<apm server url> ELASTIC_APM_SECRET_TOKEN=<secret token>
|
|
The URL of the APM server intake to which traces and metrics should be sent. |
|
The secret token used by the APM Agent to authenticate with APM server. |
Then adding an EnvironmentFile
entry to the service’s configuration file
that references the path to the environment.env file
After adding the EnvironmentFile
entry, restart the service.
systemctl reload-or-restart <service>
Profiler environment variables
editThe profiler auto instrumentation has its own set of environment variables to manage the instrumentation. These are used in addition to agent configuration through environment variables.
-
ELASTIC_APM_PROFILER_HOME
-
The home directory of the profiler auto instrumentation. The home directory typically contains:
- platform specific profiler assemblies
- a directory for each compatible target framework, where each directory contains supporting managed assemblies for auto instrumentation.
- an integrations.yml file that determines which methods to target for auto instrumentation
-
ELASTIC_APM_PROFILER_INTEGRATIONS
(optional) -
The path to the integrations.yml file that determines which methods to target for
auto instrumentation. If not specified, the profiler will assume an
integrations.yml exists in the home directory specified by
ELASTIC_APM_PROFILER_HOME
environment variable. -
ELASTIC_APM_PROFILER_EXCLUDE_INTEGRATIONS
(optional) -
A semi-colon separated list of integrations to exclude from auto-instrumentation.
Valid values are those defined in the
Integration name
column in the integrations table above. -
ELASTIC_APM_PROFILER_EXCLUDE_PROCESSES
(optional) -
A semi-colon separated list of process names to exclude from auto-instrumentation.
For example,
dotnet.exe;powershell.exe
. Can be used in scenarios where profiler environment variables have a global scope that would end up auto-instrumenting applications that should not be.
The following processes are *always* excluded from profiling by default.
- powershell.exe
- ServerManager.exe
- ReportingServicesService.exe
- RSHostingService.exe
- RSMananagement.exe
- RSPortal.exe
-
RSConfigTool.exe
-
ELASTIC_APM_PROFILER_EXCLUDE_SERVICE_NAMES
(optional) -
A semi-colon separated list of APM service names to exclude from auto-instrumentation.
Values defined are checked against the value of
ELASTIC_APM_SERVICE_NAME
environment variable.
-
The following service names are *always* excluded from profiling by default.
-
SQLServerReportingServices
-
ELASTIC_OTEL_LOG_LEVEL
(optional) - The log level at which the profiler should log. Valid values are
-
- trace
- debug
- info
- warn
- error
- none
The default value is warn
. More verbose log levels like trace
and debug
can
affect the runtime performance of profiler auto instrumentation, so are recommended
only for diagnostics purposes.
This takes precedence over the now deprecated ELASTIC_APM_PROFILER_LOG
-
ELASTIC_OTEL_LOG_DIRECTORY
(optional) -
The directory in which to write profiler log files. If unset, defaults to
-
%PROGRAMDATA%\elastic\apm-agent-dotnet\logs
on windows -
/var/log/elastic/apm-agent-dotnet
on Linux
-
If the default directory cannot be written to for some reason, the profiler
will try to write log files to a logs
directory in the home directory specified
by ELASTIC_APM_PROFILER_HOME
environment variable.
This takes precedence over the now deprecated ELASTIC_APM_PROFILER_LOG_DIR
The user account under which the profiler process runs must have permission to write to the destination log directory. Specifically, ensure that when running on IIS, the AppPool identity has write permissions in the target directory.
-
ELASTIC_OTEL_LOG_TARGETS
(optional) -
A semi-colon separated list of targets for profiler logs. Valid values are
- file
- stdout
The default value is file
, which logs to the directory specified by
ELASTIC_APM_PROFILER_LOG_DIR
environment variable.
This takes precedence over the now deprecated ELASTIC_APM_PROFILER_LOG_TARGETS
Troubleshooting
editDLLs are blocked on windows
editwindows may automatically block downloaded DLL files if it considers them suspicious.
To unblock a DLL file on windows, you can do the following:
- Right-click the DLL file in File Explorer
- Select Properties
- In the General tab, look for the Security section at the bottom
- Select the Unblock check box and click OK