Status: Stable, Feature-freeze
The Tracing API consist of these main classes:
TracerProvideris the entry point of the API. It provides access to
Traceris the class responsible for creating
Spanis the API to trace an operation.
While languages and platforms have different ways of representing data, this section defines some generic requirements for this API.
OpenTelemetry can operate on time values up to nanosecond (ns) precision. The representation of those values is language specific.
A timestamp is the time elapsed since the Unix epoch.
- The minimal precision is milliseconds.
- The maximal precision is nanoseconds.
A duration is the elapsed time between two events.
- The minimal precision is milliseconds.
- The maximal precision is nanoseconds.
Tracers can be accessed with a
In implementations of the API, the
TracerProvider is expected to be the
stateful object that holds any configuration.
TracerProvider is expected to be accessed from a central place.
Thus, the API SHOULD provide a way to set/register and access
a global default
Notwithstanding any global
TracerProvider, some applications may want to or
have to use multiple
e.g. to have different configuration (like
SpanProcessors) for each
(and consequently for the
Tracers obtained from them),
or because its easier with dependency injection frameworks.
Thus, implementations of
TracerProvider SHOULD allow creating an arbitrary
TracerProvider MUST provide the following functions:
- Get a
Get a Tracer
This API MUST accept the following parameters:
name(required): This name SHOULD uniquely identify the instrumentation scope, such as the instrumentation library (e.g.
io.opentelemetry.contrib.mongodb), package, module or class name. If an application or library has built-in OpenTelemetry instrumentation, both Instrumented library and Instrumentation library may refer to the same library. In that scenario, the
namedenotes a module name or component name within that library or application. In case an invalid name (null or empty string) is specified, a working Tracer implementation MUST be returned as a fallback rather than returning null or throwing an exception, its
nameproperty SHOULD be set to an empty string, and a message reporting that the specified value is invalid SHOULD be logged. A library, implementing the OpenTelemetry API may also ignore this name and return a default instance for all calls, if it does not support “named” functionality (e.g. an implementation which is not even observability-related). A TracerProvider could also return a no-op Tracer here if application owners configure the SDK to suppress telemetry produced by this library.
version(optional): Specifies the version of the instrumentation scope if the scope has a version (e.g. a library version). Example value:
- [since 1.4.0]
schema_url(optional): Specifies the Schema URL that should be recorded in the emitted telemetry.
- [since 1.13.0]
attributes(optional): Specifies the instrumentation scope attributes to associate with emitted telemetry.
Tracers are identified by
schema_url fields. When more
Tracer of the same
schema_url is created, it
is unspecified whether or under which conditions the same or different
instances are returned. It is a user error to create Tracers with different
attributes but the same identity.
The term identical applied to Tracers describes instances where all identifying fields are equal. The term distinct applied to Tracers describes instances where at least one identifying field has a different value.
Implementations MUST NOT require users to repeatedly obtain a
with the same identity to pick up configuration changes. This can be
achieved either by allowing to work with an outdated configuration or by
ensuring that new configuration applies also to previously returned
Note: This could, for example, be implemented by storing any mutable
configuration in the
TracerProvider and having
Tracer implementation objects
have a reference to the
TracerProvider from which they were obtained. If
configuration must be stored per-tracer (such as disabling a certain tracer),
the tracer could, for example, do a look-up with its identity in a map
TracerProvider, or the
TracerProvider could maintain a registry of
Tracers and actively update their configuration if it changes.
The effect of associating a Schema URL with a
Tracer MUST be that the
telemetry emitted using the
Tracer will be associated with the Schema URL,
provided that the emitted data format is capable of representing such
This section defines all operations within the Tracing API that interact with the
The API MUST provide the following functionality to interact with a
- Extract the
- Combine the
Contextinstance, creating a new
The functionality listed above is necessary because API users SHOULD NOT have access to the Context Key used by the Tracing API implementation.
If the language has support for implicitly propagated
here), the API SHOULD also provide
the following functionality:
- Get the currently active span from the implicit context. This is equivalent to getting the implicit context, then extracting the
Spanfrom the context.
- Set the currently active span into a new context, and make that the implicit context. This is equivalent to combining the current implicit context’s values with the
Spanto create a new context, which is then made the current implicit context.
All the above functionalities operate solely on the context API, and they MAY be exposed as either static methods on the trace module, or as static methods on a class inside the trace module. This functionality SHOULD be fully implemented in the API when possible.
The tracer is responsible for creating
Tracers should usually not be responsible for configuration.
This should be the responsibility of the
Tracer MUST provide functions to:
- Create a new
Span(see the section on
SpanContext represents the portion of a
Span which must be serialized and
propagated along side of a distributed context.
SpanContexts are immutable.
SpanContext representation conforms to the W3C TraceContext
specification. It contains two
identifiers - a
TraceId and a
SpanId - along with a set of common
TraceFlags and system-specific
TraceId A valid trace identifier is a 16-byte array with at least one
SpanId A valid span identifier is an 8-byte array with at least one non-zero
TraceFlags contain details about the trace. Unlike TraceState values,
TraceFlags are present in all traces. The current version of the specification
only supports a single flag called sampled.
TraceState carries vendor-specific trace identification data, represented as a list
of key-value pairs. TraceState allows multiple tracing
systems to participate in the same trace. It is fully described in the W3C Trace Context
specific OTel values in
TraceState, see the TraceState Handling
IsRemote, a boolean indicating whether the SpanContext was received from somewhere
else or locally generated, see IsRemote.
The API MUST implement methods to create a
SpanContext. These methods SHOULD be the only way to
SpanContext. This functionality MUST be fully implemented in the API, and SHOULD NOT be
Retrieving the TraceId and SpanId
The API MUST allow retrieving the
SpanId in the following forms:
- Hex - returns the lowercase hex encoded
TraceId(result MUST be a 32-hex-character lowercase string) or
SpanId(result MUST be a 16-hex-character lowercase string).
- Binary - returns the binary representation of the
TraceId(result MUST be a 16-byte array) or
SpanId(result MUST be an 8-byte array).
The API SHOULD NOT expose details about how they are internally stored.
An API called
IsValid, that returns a boolean value, which is
true if the SpanContext has a
non-zero TraceID and a non-zero SpanID, MUST be provided.
An API called
IsRemote, that returns a boolean value, which is
true if the SpanContext was
propagated from a remote parent, MUST be provided.
When extracting a
SpanContext through the Propagators API,
IsRemote MUST return true, whereas for the SpanContext of any child spans it MUST return false.
TraceState is a part of
SpanContext, represented by an immutable list of string key/value pairs and
formally defined by the W3C Trace Context specification.
Tracing API MUST provide at least the following operations on
- Get value for a given key
- Add a new key/value pair
- Update an existing value for a given key
- Delete a key/value pair
These operations MUST follow the rules described in the W3C Trace Context specification.
All mutating operations MUST return a new
TraceState with the modifications applied.
TraceState MUST at all times be valid according to rules specified in W3C Trace Context specification.
Every mutating operations MUST validate input parameters.
If invalid value is passed the operation MUST NOT return
TraceState containing invalid data
and MUST follow the general error handling guidelines.
Please note, since
SpanContext is immutable, it is not possible to update
SpanContext with a new
Such changes then make sense only right before
or telemetry data exporting.
In both cases,
SpanExporters may create a modified
TraceState copy before serializing it to the wire.
Span represents a single operation within a trace. Spans can be nested to
form a trace tree. Each trace contains a root span, which typically describes
the entire operation and, optionally, one or more sub-spans for its sub-operations.
- The span name
- An immutable
SpanContextthat uniquely identifies the
- A parent span in the form of a
SpanContext, or null
- A start timestamp
- An end timestamp
- A list of
Links to other
- A list of timestamped
The span name concisely identifies the work represented by the Span, for example, an RPC method name, a function name, or the name of a subtask or stage within a larger computation. The span name SHOULD be the most general string that identifies a (statistically) interesting class of Spans, rather than individual Span instances while still being human-readable. That is, “get_user” is a reasonable name, while “get_user/314159”, where “314159” is a user ID, is not a good name due to its high cardinality. Generality SHOULD be prioritized over human-readability.
For example, here are potential span names for an endpoint that gets a hypothetical account information:
|Good, and account_id=42 would make a nice Span attribute|
|Also good (using the “HTTP route”)|
Span’s start and end timestamps reflect the elapsed real time of the
For example, if a span represents a request-response cycle (e.g. HTTP or an RPC), the span should have a start time that corresponds to the start time of the first sub-operation, and an end time of when the final sub-operation is complete. This includes:
- receiving the data from the request
- parsing of the data (e.g. from a binary or json format)
- any middleware or additional processing logic
- business logic
- construction of the response
- sending of the response
Child spans (or in some cases events) may be created to represent sub-operations which require more detailed observability. Child spans should measure the timing of the respective sub-operation, and may add additional attributes.
Span’s start time SHOULD be set to the current time on span
creation. After the
Span is created, it SHOULD be possible to
change its name, set its
Events, and set the
MUST NOT be changed after the
Span’s end time has been set.
Spans are not meant to be used to propagate information within a process. To
prevent misuse, implementations SHOULD NOT provide access to a
attributes besides its
Vendors may implement the
Span interface to effect vendor-specific logic.
However, alternative implementations MUST NOT allow callers to create
Spans MUST be created via a
There MUST NOT be any API for creating a
Span other than with a
In languages with implicit
Span creation MUST NOT
set the newly created
Span as the active
Span in the
Context by default, but this functionality
MAY be offered additionally as a separate operation.
The API MUST accept the following parameters:
The span name. This is a required parameter.
Contextor an indication that the new
Spanshould be a root
Span. The API MAY also have an option for implicitly using the current Context as parent as a default behavior. This API MUST NOT accept a
SpanContextas parent, only a full
The semantic parent of the Span MUST be determined according to the rules described in Determining the Parent Span from a Context.
SpanKind, default to
SpanKind.Internalif not specified.
The API documentation MUST state that adding attributes at span creation is preferred to calling
SetAttributelater, as samplers can only consider information already present during span creation.
Links - an ordered sequence of Links, see API definition here.
Start timestamp, default to current time. This argument SHOULD only be set when span creation time has already passed. If API is called at a moment of a Span logical start, API user MUST NOT explicitly set this argument.
Each span has zero or one parent span and zero or more child spans, which
represent causally related operations. A tree of related spans comprises a
trace. A span is said to be a root span if it does not have a parent. Each
trace includes a single root span, which is the shared ancestor of all other
spans in the trace. Implementations MUST provide an option to create a
a root span, and MUST generate a new
TraceId for each root span created.
For a Span with a parent, the
TraceId MUST be the same as the parent.
Also, the child span MUST inherit all
TraceState values of its parent by default.
Span is said to have a remote parent if it is the child of a
created in another process. Each propagators’ deserialization must set
IsRemote to true on a parent
Span creation knows if the
parent is remote.
Any span that is created MUST also be ended. This is the responsibility of the user. API implementations MAY leak memory or other resources (including, for example, CPU time for periodic work that iterates all spans) if the user forgot to end the span.
Determining the Parent Span from a Context
When a new
Span is created from a
Context may contain a
representing the currently active instance, and will be used as parent.
If there is no
Span in the
Context, the newly created
Span will be a root span.
SpanContext cannot be set as active in a
Context directly, but by
wrapping it into a Span.
For example, a
Propagator performing context extraction may need this.
Span creation, a user MUST have the ability to record links to other
Spans can be from the same or a different trace – see Links
Links cannot be added after
Link is structurally defined by the following properties:
Spanto link to.
- Zero or more
Attributesfurther describing the link.
The Span creation API MUST provide:
- An API to record a single
Linkproperties are passed as arguments. This MAY be called
AddLink. This API takes the
Spanto link to and optional
Attributes, either as individual parameters or as an immutable object encapsulating them, whichever is most appropriate for the language. Implementations MAY ignore links with an invalid
Links SHOULD preserve the order in which they’re set.
With the exception of the function to retrieve the
IsRecording, none of the below may be called after the
The Span interface MUST provide:
- An API that returns the
SpanContextfor the given
Span. The returned value may be used even after the
Spanis finished. The returned value MUST be the same for the entire Span lifetime. This MAY be called
Span is recording (
true) when the data provided to
it via functions like
SetStatus is captured in
some form (e.g. in memory). When a
Span is not recording (
false), all this data is discarded right away. Further attempts to set or add
data will not record, making the span effectively a no-op.
This flag may be
true despite the entire trace not being sampled. This
allows information about the individual Span to be recorded and processed without
sending it to the backend. An example of this scenario may be recording and
processing of all incoming requests for the processing and building of
SLA/SLO latency charts while sending only a subset - sampled spans - to the
backend. See also the sampling section of SDK design.
Span is ended, it SHOULD become non-recording and
SHOULD always return
false. The one known exception to this is
streaming implementations of the API that do not keep local state and cannot
change the value of
IsRecording after ending the span.
IsRecording SHOULD NOT take any parameters.
This flag SHOULD be used to avoid expensive computations of a Span attributes or
events in case when a Span is definitely not recorded. Note that any child
span’s recording is determined independently from the value of this flag
(typically based on the
sampled flag of a
Users of the API should only access the
IsRecording property when
instrumenting code and never access
SampledFlag unless used in context
Span MUST have the ability to set
Attributes associated with it.
The Span interface MUST provide:
- An API to set a single
Attributewhere the attribute properties are passed as arguments. This MAY be called
SetAttribute. To avoid extra allocations some implementations may offer a separate API for each of the possible value types.
The Span interface MAY provide:
- An API to set multiple
Attributesat once, where the
Attributesare passed in a single method call.
Setting an attribute with the same key as an existing attribute SHOULD overwrite the existing attribute’s value.
Note that the OpenTelemetry project documents certain “standard attributes” that have prescribed semantic meanings.
Note that Samplers can only consider information already present during span creation. Any changes done later, including new or changed attributes, cannot change their decisions.
Span MUST have the ability to add events. Events have a time associated
with the moment when they are added to the
Event is structurally defined by the following properties:
- Name of the event.
- A timestamp for the event. Either the time at which the event was added or a custom timestamp provided by the user.
- Zero or more
Attributesfurther describing the event.
The Span interface MUST provide:
- An API to record a single
Eventproperties are passed as arguments. This MAY be called
AddEvent. This API takes the name of the event, optional
Attributesand an optional
Timestampwhich can be used to specify the time at which the event occurred, either as individual parameters or as an immutable object encapsulating them, whichever is most appropriate for the language. If no custom timestamp is provided by the user, the implementation automatically sets the time at which this API is called on the event.
Events SHOULD preserve the order in which they are recorded. This will typically match the ordering of the events’ timestamps, but events may be recorded out-of-order using custom timestamps.
Consumers should be aware that an event’s timestamp might be before the start or after the end of the span if custom timestamps were provided by the user for the event or when starting or ending the span. The specification does not require any normalization if provided timestamps are out of range.
Note that the OpenTelemetry project documents certain “standard event names and keys” which have prescribed semantic meanings.
RecordException is a specialized variant of
AddEvent for recording exception events.
Status of the
Span. If used, this will override the default
status, which is
Status is structurally defined by the following properties:
StatusCode, one of the values listed below.
Descriptionthat provides a descriptive message of the
DescriptionMUST only be used with the
StatusCodevalue. An empty
Descriptionis equivalent with a not present one.
Note: The OTLP protocol definition
refers to the
Description property as
StatusCode is one of the following values:
- The default status.
- The operation has been validated by an Application developer or Operator to have completed successfully.
- The operation contains an error.
These values form a total order:
Ok > Error > Unset.
This means that setting
StatusCode=Ok will override any prior or future attempts to set
StatusCode=Unset. See below for more specific rules.
The Span interface MUST provide:
- An API to set the
Status. This SHOULD be called
SetStatus. This API takes the
StatusCode, and an optional
Description, either as individual parameters or as an immutable object encapsulating them, whichever is most appropriate for the language.
DescriptionMUST be IGNORED for
The status code SHOULD remain unset, except for the following circumstances:
An attempt to set value
Unset SHOULD be ignored.
When the status is set to
Error by Instrumentation Libraries, the
SHOULD be documented and predictable. The status code should only be set to
according to the rules defined within the semantic conventions. For operations
not covered by the semantic conventions, Instrumentation Libraries SHOULD
publish their own conventions, including possible values of
and what they mean.
Generally, Instrumentation Libraries SHOULD NOT set the status code to
unless explicitly configured to do so. Instrumentation Libraries SHOULD leave the
status code as
Unset unless there is an error, as described above.
Application developers and Operators may set the status code to
When span status is set to
Ok it SHOULD be considered final and any further
attempts to change it SHOULD be ignored.
Analysis tools SHOULD respond to an
Ok status by suppressing any errors they
would otherwise generate. For example, to suppress noisy errors such as 404s.
Only the value of the last call will be recorded, and implementations are free to ignore previous calls.
Span name. Upon this update, any sampling behavior based on
name will depend on the implementation.
Note that Samplers can only consider information already present during span creation. Any changes done later, including updated span name, cannot change their decisions.
Alternatives for the name update may be late
Span creation, when Span is
started with the explicit timestamp from the past at the moment where the final
Span name is known, or reporting a
Span with the desired name as a child
- The new span name, which supersedes whatever was passed in when the
Signals that the operation described by this span has now (or at the time optionally specified) ended.
Implementations SHOULD ignore all subsequent calls to
End and any other Span methods,
i.e. the Span becomes non-recording by being ended
(there might be exceptions when Tracer is streaming events
and has no mutable state associated with the
Language SIGs MAY provide methods other than
End in the API that also end the
span to support language-specific features like
with statements in Python.
However, all API implementations of such methods MUST internally call the
method and be documented to do so.
End MUST NOT have any effects on child spans.
Those may still be running and can be ended later.
End MUST NOT inactivate the
Span in any
Context it is active in.
It MUST still be possible to use an ended span as parent via a Context it is
contained in. Also, any mechanisms for putting the Span into a Context MUST
still work after the Span was ended.
- (Optional) Timestamp to explicitly set the end timestamp. If omitted, this MUST be treated equivalent to passing the current time.
Expect this operation to be called in the “hot path” of production applications. It needs to be designed to complete fast, if not immediately. This operation itself MUST NOT perform blocking I/O on the calling thread. Any locking used needs be minimized and SHOULD be removed entirely if possible. Some downstream SpanProcessors and subsequent SpanExporters called from this operation may be used for testing, proof-of-concept ideas, or debugging and may not be designed for production use themselves. They are not in the scope of this requirement and recommendation.
To facilitate recording an exception languages SHOULD provide a
RecordException method if the language uses exceptions.
This is a specialized variant of
so for anything not specified here, the same requirements as for
The signature of the method is to be determined by each language
and can be overloaded as appropriate.
The method MUST record an exception as an
Event with the conventions outlined in
the exceptions document.
The minimum required argument SHOULD be no more than only an exception object.
RecordException is provided, the method MUST accept an optional parameter
to provide any additional event attributes
(this SHOULD be done in the same way as for the
If attributes with the same name would be generated by the method already,
the additional attributes take precedence.
RecordException may be seen as a variant of
additional exception-specific parameters and all other parameters being optional
(because they have defaults from the exception semantic convention).
Span lifetime represents the process of recording the start and the end timestamps to the Span object:
- The start time is recorded when the Span is created.
- The end time needs to be recorded when the operation is ended.
Start and end time as well as Event’s timestamps MUST be recorded at a time of a calling of corresponding API.
Wrapping a SpanContext in a Span
The API MUST provide an operation for wrapping a
SpanContext with an object
Span interface. This is done in order to expose a
Span in operations such as in-process
If a new type is required for supporting this operation, it SHOULD NOT be exposed
publicly if possible (e.g. by only exposing a function that returns something
with the Span interface type). If a new type is required to be publicly exposed,
it SHOULD be named
The behavior is defined as follows:
GetContextMUST return the wrapped
falseto signal that events, attributes and other elements are not being recorded, i.e. they are being dropped.
The remaining functionality of
Span MUST be defined as no-op operations.
Note: This includes
End, so as an exception from the general rule,
it is not required (or even helpful) to end such a Span.
This functionality MUST be fully implemented in the API, and SHOULD NOT be overridable.
SpanKind describes the relationship between the Span, its parents,
and its children in a Trace.
SpanKind describes two independent
properties that benefit tracing systems during analysis.
The first property described by
SpanKind reflects whether the Span
is a “logical” remote child or parent. By “logical”, we mean that
the span is logically a remote child or parent, from the point of view
of the library that is being instrumented. Spans with a remote parent are
interesting because they are sources of external load. Spans with a
remote child are interesting because they reflect a non-local system
The second property described by
SpanKind reflects whether a child
Span represents a synchronous call. When a child span is synchronous,
the parent is expected to wait for it to complete under ordinary
circumstances. It can be useful for tracing systems to know this
property, since synchronous Spans may contribute to the overall trace
latency. Asynchronous scenarios can be remote or local.
In order for
SpanKind to be meaningful, callers SHOULD arrange that
a single Span does not serve more than one purpose. For example, a
server-side span SHOULD NOT be used directly as the parent of another
remote span. As a simple guideline, instrumentation should create a
new Span prior to extracting and serializing the SpanContext for a
Note: there are complex scenarios where a CLIENT span may have a child that is also logically a CLIENT span, or a PRODUCER span might have a local child that is a CLIENT span, depending on how the various libraries that are providing the functionality are built and instrumented. These scenarios, when they occur, should be detailed in the semantic conventions appropriate to the relevant libraries.
These are the possible SpanKinds:
SERVERIndicates that the span covers server-side handling of a synchronous RPC or other remote request. This span is often the child of a remote
CLIENTspan that was expected to wait for a response.
CLIENTIndicates that the span describes a request to some remote service. This span is usually the parent of a remote
SERVERspan and does not end until the response is received.
PRODUCERIndicates that the span describes the initiators of an asynchronous request. This parent span will often end before the corresponding child
CONSUMERspan, possibly even before the child span starts. In messaging scenarios with batching, tracing individual messages requires a new
PRODUCERspan per message to be created.
CONSUMERIndicates that the span describes a child of an asynchronous
INTERNALDefault value. Indicates that the span represents an internal operation within an application, as opposed to an operations with remote parents or children.
To summarize the interpretation of these kinds:
|Synchronous||Asynchronous||Remote Incoming||Remote Outgoing|
For languages which support concurrent execution the Tracing APIs provide specific guarantees and safeties. Not all of API functions are safe to be called concurrently.
TracerProvider - all methods are safe to be called concurrently.
Tracer - all methods are safe to be called concurrently.
Span - All methods of Span are safe to be called concurrently.
Event - Events are immutable and safe to be used concurrently.
Link - Links are immutable and safe to be used concurrently.
See Propagators Distribution for how propagators are to be distributed.
Behavior of the API in the absence of an installed SDK
In general, in the absence of an installed SDK, the Trace API is a “no-op” API.
This means that operations on a Tracer, or on Spans, should have no side effects
and do nothing. However, there is one important exception to this general rule,
and that is related to propagation of a
SpanContext: The API MUST return a
Span with the
SpanContext in the parent
Context (whether explicitly given or implicit current).
Span in the parent
Context is already non-recording, it SHOULD be returned directly
without instantiating a new
If the parent
Context contains no
Span, an empty non-recording Span MUST be
returned instead (i.e., having a
SpanContext with all-zero Span and Trace IDs,
empty Tracestate, and unsampled TraceFlags). This means that a
that has been provided by a configured
Propagator will be propagated through
to any child span and ultimately also
Inject, but that no new
will be created.