Semantic Conventions for network spans emitted by .NET

Status: Mixed

This article defines semantic conventions for HTTP client, DNS and TLS spans emitted by .NET.

.NET HttpClient reports HTTP client request spans according to HTTP Semantic Conventions.

In addition to stable HTTP client request spans, HTTP client handlers reports experimental spans describing HTTP connection establishment and its stages.

The connection lifetime is usually measured in minutes, so when application is under the load but connection pool is not overloaded, the rate of connection-related spans is expected to be much lower than the rate of HTTP client request spans.

Applications are encouraged to enable HTTP client request spans by default in production environments.

Connection-level spans are experimental - their semantics may be changed in the future in a breaking manner. Using connection-level instrumentation in production environments should be done after appropriate validation.

The connection-related spans are reported only by HttpClientHandler and SocketsHttpHandler which may not be supported on certain platforms or may not be used by a particular application.

HTTP client request

Status: Stable

.NET HttpClient reports client request spans according to HTTP Client Semantic Conventions with the following specific details:

  • network.protocol.name, network.peer.port, and http.request.resend_count attributes are not reported
  • url.full is redacted by default - query parameter values are replaced with *. Redaction can be disabled by setting AppContext switch System.Net.Http.DisableQueryRedaction to true.
  • When the error.type attribute is reported, it contains one of HTTP Request errors in snake_case, a full exception type name, or a string representation of received status code.
  • All attributes are reported after Activity is started, none are provided at creation time.
  • In case redirects occur, each redirected request is reported as a separate span.
  • SocketsHttpHandler may retry requests on connection failure. Such retries are not reported as separate spans.

Corresponding Activity.OperationName is System.Net.Http.HttpRequestOut, ActivitySource name - System.Net.Http. Span with HTTP semantics was added in .NET 9.

HTTP client request: wait for connection

Status: Development

The span describes the time it takes for the HTTP request to obtain a connection from the connection pool.

The span is reported only if there was no connection readily available when request has started. It’s reported as a child of HTTP client request span.

The span ends when the connection is obtained - it could happen when an existing connection becomes available or once a new connection is established, so the duration of Wait For Connection span is different from duration of the HTTP connection setup span.

The time it takes to get a connection from the pool is also reported by the http.client.request.time_in_queue metric.

Corresponding Activity.OperationName is Experimental.System.Net.Http.Connections.WaitForConnection, ActivitySource name - Experimental.System.Net.Http. Added in .NET 9.

Span name SHOULD be HTTP wait_for_connection {server.address}:{server.port}.

Span kind SHOULD be INTERNAL.

Span status SHOULD follow the Recording Errors document.

AttributeTypeDescriptionExamplesRequirement LevelStability
error.typestringOne of the HTTP Request errors in snake_case, or a full exception type.version_negotiation_error; System.OperationCanceledExceptionConditionally Required if and only if an error has occurred.Stable

error.type has the following list of well-known values. If one of them applies, then the respective value MUST be used; otherwise, a custom value MAY be used.

ValueDescriptionStability
_OTHERA fallback error value to be used when the instrumentation doesn’t define a custom value.Stable

HTTP connection setup

Status: Development

The span describes the establishment of the HTTP connection. It includes the time it takes to resolve the DNS, establish the socket connection, and perform the TLS handshake.

There is no parent-child relationship between the HTTP client request and the [HTTP connection setup]/docs/dotnet/dotnet-network-traces.md(/docs/dotnet/dotnet-network-traces.md#http-connection-setup) spans; the latter will always be a root span, defining a separate trace.

However, if the connection attempt represented by the HTTP connection setup span results in a successful HTTP connection, and that connection is picked up by a request to serve it, the instrumentation adds a link to the HTTP client request span pointing to the HTTP connection setup span. I.e., each request is linked to the connection that served this request.

Corresponding Activity.OperationName is Experimental.System.Net.Http.Connections.ConnectionSetup, ActivitySource name - Experimental.System.Net.Http.Connections. Added in .NET 9.

Span name SHOULD be HTTP connection_setup {server.address}:{server.port}.

Span kind SHOULD be INTERNAL.

Span status SHOULD follow the Recording Errors document.

AttributeTypeDescriptionExamplesRequirement LevelStability
error.typestringOne of the HTTP Request errors in snake_case, or a full exception type.name_resolution_error; System.OperationCanceledExceptionConditionally Required if and only if an error has occurred.Stable
network.peer.addressstringPeer IP address of the socket connection. [1]10.1.2.80; /tmp/my.sockRecommendedStable
server.addressstringServer domain name if available without reverse DNS lookup; otherwise, IP address or Unix domain socket name. [2]example.com; 10.1.2.80; /tmp/my.sockRecommendedStable
server.portintServer port number. [3]80; 8080; 443RecommendedStable
url.schemestringThe URI scheme component identifying the used protocol.https; ftp; telnetRecommendedStable

[1] network.peer.address: The network.peer.address attribute is available only if the connection was successfully established and only for IP sockets.

[2] server.address: When observed from the client side, and when communicating through an intermediary, server.address SHOULD represent the server address behind any intermediaries, for example proxies, if it’s available.

[3] server.port: When observed from the client side, and when communicating through an intermediary, server.port SHOULD represent the server port behind any intermediaries, for example proxies, if it’s available.


error.type has the following list of well-known values. If one of them applies, then the respective value MUST be used; otherwise, a custom value MAY be used.

ValueDescriptionStability
_OTHERA fallback error value to be used when the instrumentation doesn’t define a custom value.Stable

DNS lookup

Status: Development

The span describes DNS lookup or reverse lookup performed with one of the methods on System.Net.Dns class.

DNS spans track logical operations rather than physical DNS calls and the actual behavior depends on the resolver implementation which could be changed in the future versions of .NET. .NET 9 uses OS DNS resolver which may do zero or more physical lookups for one API call.

When the DNS lookup span is reported along with HTTP connection setup and socket connect span, the DNS lookup span span becomes a child of HTTP connection setup and a sibling of socket connect.

DNS lookup duration is also reported by dns.lookup.duration metric.

Corresponding Activity.OperationName is Experimental.System.Net.NameResolution.DnsLookup, ActivitySource name - Experimental.System.Net.NameResolution. Added in .NET 9.

Span name SHOULD be DNS lookup {dns.question.name} for DNS lookup (IP addresses from host name) and DNS reverse lookup {dns.question.name} for reverse lookup (host names from IP address).

Span kind SHOULD be INTERNAL.

Span status SHOULD follow the Recording Errors document.

AttributeTypeDescriptionExamplesRequirement LevelStability
error.typestringThe error code or exception name returned by System.Net.Dns. [1]host_not_found; try_againConditionally Required if and only if an error has occurred.Stable
dns.answersstring[]List of resolved IP addresses (for DNS lookup) or a single element containing domain name (for reverse lookup).["10.0.0.1", "2001:0db8:85a3:0000:0000:8a2e:0370:7334"]Recommended if DNS lookup was successful.Development
dns.question.namestringThe domain name or an IP address being queried.www.example.com; opentelemetry.ioRecommendedDevelopment

[1] error.type: The following errors are reported:

  • host_not_found
  • try_again
  • no_recovery
  • address_family_not_supported
  • the full exception type name

See SocketError for more details.


error.type has the following list of well-known values. If one of them applies, then the respective value MUST be used; otherwise, a custom value MAY be used.

ValueDescriptionStability
_OTHERA fallback error value to be used when the instrumentation doesn’t define a custom value.Stable

Socket connect

Status: Development

The span describes the establishment of the socket connection.

It’s different from HTTP connection setup span, which also covers the DNS lookup and TLS handshake.

When socket connect span is reported along with HTTP connection setup span, the socket span becomes a child of HTTP connection setup.

Corresponding Activity.OperationName is Experimental.System.Net.Sockets.Connect, ActivitySource name - Experimental.System.Net.Sockets. Added in .NET 9.

Span name SHOULD be socket connect {network.peer.address}:{network.peer.port} when socket address family has a notion of port and socket connect {network.peer.address} otherwise.

Span kind SHOULD be INTERNAL.

Span status SHOULD follow the Recording Errors document.

AttributeTypeDescriptionExamplesRequirement LevelStability
error.typestringSocket error code. [1]connection_refused; address_not_availableConditionally Required if and only if an error has occurred.Stable
network.peer.addressstringPeer address of the network connection - IP address or Unix domain socket name.10.1.2.80; /tmp/my.sockRecommendedStable
network.peer.portintPeer port number of the network connection.65123Recommended [2]Stable
network.transportstringOSI transport layer or inter-process communication method. [3]tcp; udp; unixRecommended [4]Stable
network.typestringOSI network layer or non-OSI equivalent. [5]ipv4; ipv6Recommended if network.peer.address is an IP address.Stable

[1] error.type: The following errors codes are reported:

  • network_down
  • address_already_in_use
  • interrupted
  • in_progress
  • already_in_progress
  • address_not_available
  • address_family_not_supported
  • connection_refused
  • fault
  • invalid_argument
  • is_connected
  • network_unreachable
  • host_unreachable
  • no_buffer_space_available
  • timed_out
  • access_denied
  • protocol_type

See socket errors on Windows and Linux for more details.

[2] network.peer.port: If port is supported for the socket address family.

[3] network.transport: The value SHOULD be normalized to lowercase.

Consider always setting the transport when setting a port number, since a port number is ambiguous without knowing the transport. For example different processes could be listening on TCP port 12345 and UDP port 12345.

[4] network.transport: If value is not tcp. When missing, the value is assumed to be tcp.

[5] network.type: The value SHOULD be normalized to lowercase.


error.type has the following list of well-known values. If one of them applies, then the respective value MUST be used; otherwise, a custom value MAY be used.

ValueDescriptionStability
_OTHERA fallback error value to be used when the instrumentation doesn’t define a custom value.Stable

network.transport has the following list of well-known values. If one of them applies, then the respective value MUST be used; otherwise, a custom value MAY be used.

ValueDescriptionStability
pipeNamed or anonymous pipe.Stable
quicQUICStable
tcpTCPStable
udpUDPStable
unixUnix domain socketStable

network.type has the following list of well-known values. If one of them applies, then the respective value MUST be used; otherwise, a custom value MAY be used.

ValueDescriptionStability
ipv4IPv4Stable
ipv6IPv6Stable

TLS handshake

Span name SHOULD be TLS client handshake {server.address} when authenticating on the client side and TLS server handshake when authenticating the server. Span kind SHOULD be INTERNAL in both cases.

Status: Development

The span describes TLS client or server handshake performed with System.Net.Security.SslStream.

When TLS span is reported for client-side authentication along with HTTP connection setup and socket connect span, the TLS span becomes a child of HTTP connection setup.

Corresponding Activity.OperationName is Experimental.System.Net.Security.TlsHandshake, ActivitySource name - Experimental.System.Net.Security. Added in .NET 9.

Span name SHOULD be TLS client handshake {server.address} when authenticating on the client side and TLS server handshake when authenticating the server.

Span kind SHOULD be INTERNAL in both cases.

Span status SHOULD follow the Recording Errors document.

AttributeTypeDescriptionExamplesRequirement LevelStability
error.typestringDescribes a class of error the operation ended with.System.Net.Security.Authentication.AuthenticationException; System.OperationCanceledExceptionConditionally Required if and only if an error has occurred.Stable
server.addressstringThe server name indication (SNI) used in the ‘Client Hello’ message during TLS handshake. [1]opentelemetry.io; example.comRecommended when authenticating the client.Stable
tls.protocol.namestringNormalized lowercase protocol name parsed from original string of the negotiated SSL/TLS protocol versionssl; tlsRecommended when availableDevelopment
tls.protocol.versionstringNumeric part of the version parsed from the original string of the negotiated SSL/TLS protocol version1.2; 3Recommended when availableDevelopment

[1] server.address: When observed from the client side, and when communicating through an intermediary, server.address SHOULD represent the server address behind any intermediaries, for example proxies, if it’s available.


error.type has the following list of well-known values. If one of them applies, then the respective value MUST be used; otherwise, a custom value MAY be used.

ValueDescriptionStability
_OTHERA fallback error value to be used when the instrumentation doesn’t define a custom value.Stable

tls.protocol.name has the following list of well-known values. If one of them applies, then the respective value MUST be used; otherwise, a custom value MAY be used.

ValueDescriptionStability
sslsslDevelopment
tlstlsDevelopment

Examples

HTTP request was performed on a connection that was immediately available

If connection is immediately available for the request, HttpClient creates one span for HTTP request and links it to the HTTP connection_setup span associated with this connection.

The HTTP connection_setup span has already ended at this point.

<- HTTP connection_setup - (trace=t1, span=s1) ->
<--- DNS --->
             <--- socket connect --->
                                     <--- TLS -->
                                                  <--- GET / (INTERNAL, trace=t2, span=s2, link_to=t1,s1) --->

HTTP request has to wait for connection setup

If connection was not immediately available for the request, HTTP client and handlers create HTTP request and Wait for connection spans. In this example, a new connection was created and the request was executed on it immediately after the connection was created. Instrumentation added a link to HTTP connection_setup span on the HTTP request GET span.

<--------- HTTP connection_setup (trace=t1, span=s1) -------->
<--- DNS --->
             <--------- socket connect -------->
                                                <--- TLS --->


<----------------------- GET / (trace=t2, span=s2, link_to=t1,s1) -------------------------------->
<--------- HTTP wait_for_connection (trace=t2, span=s3) ------>

HTTP request has to wait for connection setup and other requests on that connection to complete

If connection was not immediately available for the request, HTTP client and handlers create HTTP request and Wait for connection spans. In this example, request was performed on an existing connection, but this connection served other requests in the queue before it became available for this request.

<- HTTP connection_setup - (t1,s1) ->
                                        <--------------------- GET / (trace=t2, span=s2) ----------------------------------------->
                                        <---- HTTP wait_for_connection (trace=t2, span=s2, link_to=t1,s1) ---->

The HTTP connection_setup span has started before this request, the corresponding connection was serving other requests until it became available to the GET request above.

The long Wait for connection span here is indicating that there is a queue of requests and a high demand for connections in the pool.

HTTP request fails because connection cannot be established

If HTTP request fails before connection is established:

  • all attempts to establish connections are recorded as HTTP connection_setup spans
  • HTTP request GET span is recorded with the corresponding error type along with Wait for connection span.
  • HTTP request GET span is not linked to any of the HTTP connection_setup spans since these connections were never associated with corresponding request.
<- HTTP connection_setup - (trace=t1, span=s1) - ERROR ->
<------------------- DNS - timeout ---------------->

<---------- GET / (trace=t2, span=s2) - ERROR ---------->
<- HTTP wait_for_connection (trace=t2, span=s3) - ERROR ->