General RPC conventions

NOTICE Semantic Conventions are moving to a new location.

No changes to this document are allowed.

Status: Experimental

The conventions described in this section are RPC specific. When RPC operations occur, measurements about those operations are recorded to instruments. The measurements are aggregated and exported as metrics, which provide insight into those operations. By including RPC properties as attributes on measurements, the metrics can be filtered for finer grain analysis.

Warning Existing RPC instrumentations that are using v1.20.0 of this document (or prior):

  • SHOULD NOT change the version of the networking attributes that they emit until the HTTP semantic conventions are marked stable (HTTP stabilization will include stabilization of a core set of networking attributes which are also used in RPC instrumentations).
  • SHOULD introduce an environment variable OTEL_SEMCONV_STABILITY_OPT_IN in the existing major version which supports the following values:
    • none - continue emitting whatever version of the old experimental networking attributes the instrumentation was emitting previously. This is the default value.
    • http - emit the new, stable networking attributes, and stop emitting the old experimental networking attributes that the instrumentation emitted previously.
    • http/dup - emit both the old and the stable networking attributes, allowing for a seamless transition.
  • SHOULD maintain (security patching at a minimum) the existing major version for at least six months after it starts emitting both sets of attributes.
  • SHOULD drop the environment variable in the next major version (stable next major version SHOULD NOT be released prior to October 1, 2023).

Metric instruments

The following metric instruments MUST be used to describe RPC operations. They MUST be of the specified type and units.

Note: RPC server and client metrics are split to allow correlation across client/server boundaries, e.g. Lining up an RPC method latency to determine if the server is responsible for latency the client is seeing.

RPC Server

Below is a table of RPC server metric instruments.

NameInstrument Type (*)UnitUnit (UCUM)DescriptionStatusStreaming
rpc.server.durationHistogrammillisecondsmsmeasures duration of inbound RPCRecommendedN/A. While streaming RPCs may record this metric as start-of-batch to end-of-batch, it’s hard to interpret in practice.
rpc.server.request.sizeHistogramBytesBymeasures size of RPC request messages (uncompressed)OptionalRecorded per message in a streaming batch
rpc.server.response.sizeHistogramBytesBymeasures size of RPC response messages (uncompressed)OptionalRecorded per response in a streaming batch
rpc.server.requests_per_rpcHistogramcount{count}measures the number of messages received per RPC. Should be 1 for all non-streaming RPCsOptionalRequired
rpc.server.responses_per_rpcHistogramcount{count}measures the number of messages sent per RPC. Should be 1 for all non-streaming RPCsOptionalRequired

RPC Client

Below is a table of RPC client metric instruments. These apply to traditional RPC usage, not streaming RPCs.

NameInstrument Type (*)UnitUnit (UCUM)DescriptionStatusStreaming
rpc.client.durationHistogrammillisecondsmsmeasures duration of outbound RPCRecommendedN/A. While streaming RPCs may record this metric as start-of-batch to end-of-batch, it’s hard to interpret in practice.
rpc.client.request.sizeHistogramBytesBymeasures size of RPC request messages (uncompressed)OptionalRecorded per message in a streaming batch
rpc.client.response.sizeHistogramBytesBymeasures size of RPC response messages (uncompressed)OptionalRecorded per message in a streaming batch
rpc.client.requests_per_rpcHistogramcount{count}measures the number of messages received per RPC. Should be 1 for all non-streaming RPCsOptionalRequired
rpc.client.responses_per_rpcHistogramcount{count}measures the number of messages sent per RPC. Should be 1 for all non-streaming RPCsOptionalRequired

Attributes

Below is a table of attributes that SHOULD be included on client and server RPC measurements.

AttributeTypeDescriptionExamplesRequirement Level
rpc.systemstringA string identifying the remoting system. See below for a list of well-known identifiers.grpcRequired
rpc.servicestringThe full (logical) name of the service being called, including its package name, if applicable. [1]myservice.EchoServiceRecommended
rpc.methodstringThe name of the (logical) method being called, must be equal to the $method part in the span name. [2]exampleMethodRecommended
network.transportstringOSI Transport Layer or Inter-process Communication method. The value SHOULD be normalized to lowercase.tcp; udpRecommended
network.typestringOSI Network Layer or non-OSI equivalent. The value SHOULD be normalized to lowercase.ipv4; ipv6Recommended
server.addressstringRPC server host name. [3]example.comRequired
server.portintLogical server port number80; 8080; 443Conditionally Required: See below
server.socket.addressstringPhysical server IP address or Unix socket address.10.5.3.2See below
server.socket.portintPhysical server port.16456Recommended: [4]

[1]: This is the logical name of the service from the RPC interface perspective, which can be different from the name of any implementing class. The code.namespace attribute may be used to store the latter (despite the attribute name, it may include a class name; e.g., class with method actually executing the call on the server side, RPC client stub class on the client side).

[2]: This is the logical name of the method from the RPC interface perspective, which can be different from the name of any implementing method/function. The code.function attribute may be used to store the latter (e.g., method actually executing the call on the server side, RPC client stub method on the client side).

[3]: May contain server IP address, DNS name, or local socket name. When host component is an IP address, instrumentations SHOULD NOT do a reverse proxy lookup to obtain DNS name and SHOULD set server.address to the IP address provided in the host component.

[4]: If different than server.port and if server.socket.address is set.

Additional attribute requirements: At least one of the following sets of attributes is required:

rpc.system has the following list of well-known values. If one of them applies, then the respective value MUST be used, otherwise a custom value MAY be used.

ValueDescription
grpcgRPC
java_rmiJava RMI
dotnet_wcf.NET WCF
apache_dubboApache Dubbo
connect_rpcConnect RPC

To avoid high cardinality, implementations should prefer the most stable of server.address or server.socket.address, depending on expected deployment profile. For many cloud applications, this is likely server.address as names can be recycled even across re-instantiation of a server with a different ip.

For client-side metrics server.port is required if the connection is IP-based and the port is available (it describes the server port they are connecting to). For server-side spans server.port is optional (it describes the port the client is connecting from).

Service name

On the server process receiving and handling the remote procedure call, the service name provided in rpc.service does not necessarily have to match the service.name resource attribute. One process can expose multiple RPC endpoints and thus have multiple RPC service names. From a deployment perspective, as expressed by the service.* resource attributes, it will be treated as one deployed service with one service.name.

gRPC conventions

For remote procedure calls via gRPC, additional conventions are described in this section.

rpc.system MUST be set to "grpc".

gRPC Attributes

Below is a table of attributes that SHOULD be included on client and server RPC measurements when rpc.system is "grpc".

AttributeTypeDescriptionExamplesRequirement Level
rpc.grpc.status_codeintThe numeric status code of the gRPC request.0Required

rpc.grpc.status_code MUST be one of the following:

ValueDescription
0OK
1CANCELLED
2UNKNOWN
3INVALID_ARGUMENT
4DEADLINE_EXCEEDED
5NOT_FOUND
6ALREADY_EXISTS
7PERMISSION_DENIED
8RESOURCE_EXHAUSTED
9FAILED_PRECONDITION
10ABORTED
11OUT_OF_RANGE
12UNIMPLEMENTED
13INTERNAL
14UNAVAILABLE
15DATA_LOSS
16UNAUTHENTICATED

Connect RPC conventions

For remote procedure calls via connect, additional conventions are described in this section.

rpc.system MUST be set to "connect_rpc".

Connect RPC Attributes

Below is a table of attributes that SHOULD be included on client and server RPC measurements when rpc.system is "connect_rpc".

AttributeTypeDescriptionExamplesRequirement Level
rpc.connect_rpc.error_codestringThe error codes of the Connect request. Error codes are always string values.cancelledConditionally Required: [1]

[1]: If response is not successful and if error code available.

rpc.connect_rpc.error_code MUST be one of the following:

ValueDescription
cancelledcancelled
unknownunknown
invalid_argumentinvalid_argument
deadline_exceededdeadline_exceeded
not_foundnot_found
already_existsalready_exists
permission_deniedpermission_denied
resource_exhaustedresource_exhausted
failed_preconditionfailed_precondition
abortedaborted
out_of_rangeout_of_range
unimplementedunimplemented
internalinternal
unavailableunavailable
data_lossdata_loss
unauthenticatedunauthenticated