OpenTelemetry Tracing Specification now 1.0!
Our goal is to provide a generally available, production quality release for the tracing data source across most OpenTelemetry components in the first half of 2021. Several components have already reached this milestone! We expect metrics to reach the same status in the second half of 2021 and are targeting logs in 2022.
OpenTelemetry is developed on a signal by signal basis. Tracing, metrics, baggage, and logging are examples of signals. Signals are built on top of context propagation, a shared mechanism for correlating data across distributed systems.
Each signal consists of four core components: APIs, SDKs, the OTLP protocol, and the Collector. Signals also have contrib components, an ecosystem of plugins and instrumentation. All instrumentation shares the same semantic conventions, to ensure that they produce the same data when observing common operations, such as HTTP requests.
A detailed overview of signals and components can be found here.
Components follow a development lifecycle: Draft, Experimental, Stable, Deprecated, Removed.
Draft components are under design, and have not been added to the specification.
Experimental components are released and available for beta testing.
Stable components are backwards compatible and covered under long term support.
Deprecated components are stable, but may eventually be removed.
The complete definitions for lifecycles and long term support can be found here.
The following is a high level status report for currently available signals. Note that while the OpenTelemetry clients conform to a shared specification, they are developed independently.
- The tracing specification is now completely stable, and covered by long term support.
- The tracing specification is still extensible, but only in a backwards compatible manner.
- OpenTelemetry clients are versioned to v1.0 once their tracing implementation is complete.
- OpenTelemetry Metrics is currently under active development.
- The data model is stable and released as part of the OTLP protocol.
- Experimental support for metric pipelines are available in the Collector.
- Collector support for Prometheus is under developemnet, in collaboration with the Prometheus community.
- The metric API and SDK specification is currently being protoypted in Java, .NET, and Python.
- OpenTelemetry Baggage is now completely stable.
- Baggage is not an observability tool, it is a system for attaching arbitratry keys and values to a transaction, so that downstream services may access them. As such, there is no OTLP or Collector component to baggage.
- OpenTelemetry Logging is currently under active development.
- The data model is experimental and released as part of the OTLP protocol.
- Log processing for many data formats has been added to the Collector, thanks to the donation of Stanza to the the OpenTelemetry project.
- Log appenders are currently under develop in many languages. Log appenders allow OpenTelemetry tracing data, such as trace and span IDs, to be appended to existing logging systems.
- An OpenTelemetry logging SDK is currently under development. This allows OpenTelemetry clients to injest logging data from existing logging systems, outputting logs as part of OTLP along with tracing and metrics.
- An OpTelemetry logging API is not currently under development. We are focusing first on integration with existing logging systems. When metrics is complete, focus will shift to development of an OpenTelemetry logging API.
An effort to expand the availability and quality of OpenTelemetry instrumentation is scheduled for this summer.
- Stabilize and define long term support for instrumentation
- Provide instrumentation for a wider variety of important libraries
- Provide testing and CICD tools for writing and verifying instrumentation quality.
Updated instrumentations to pick the latest version of API/SDK (1.1.0-beta4).
## 1.0.0-RC1 / 0.21.0 - 2021-06-18 With this release we are introducing a split in module versions. The tracing API and SDK are entering the `v1.0.0` Release Candidate phase with `v1.0.0-RC1` while the experimental metrics API and SDK continue with `v0.x` releases at `v0.21.0`. Modules at major version 1 or greater will not depend on modules with major version 0. ### Added - Adds `otlpgrpc.WithRetry`option for configuring the retry policy for transient errors on the otlp/gRPC exporter. (#1832) - The following status codes are defined as transient errors: | gRPC Status Code | Description | | ---------------- | ----------- | | 1 | Cancelled | | 4 | Deadline Exceeded | | 8 | Resource Exhausted | | 10 | Aborted | | 10 | Out of Range | | 14 | Unavailable | | 15 | Data Loss | - Added `Status` type to the `go.opentelemetry.io/otel/sdk/trace` package to represent the status of a span. (#1874) - Added `SpanStub` type and its associated functions to the `go.opentelemetry.io/otel/sdk/trace/tracetest` package. This type can be used as a testing replacement for the `SpanSnapshot` that was removed from the `go.opentelemetry.io/otel/sdk/trace` package. (#1873) - Adds support for scheme in `OTEL_EXPORTER_OTLP_ENDPOINT` according to the spec. (#1886) - Adds `trace.WithSchemaURL` option for configuring the tracer with a Schema URL. (#1889) - Added an example of using OpenTelemetry Go as a trace context forwarder. (#1912) - `ParseTraceState` is added to the `go.opentelemetry.io/otel/trace` package. It can be used to decode a `TraceState` from a `tracestate` header string value. (#1937) - Added `Len` method to the `TraceState` type in the `go.opentelemetry.io/otel/trace` package. This method returns the number of list-members the `TraceState` holds. (#1937) - Creates package `go.opentelemetry.io/otel/exporters/otlp/otlptrace` that defines a trace exporter that uses a `otlptrace.Client` to send data. Creates package `go.opentelemetry.io/otel/exporters/otlp/otlptrace/otlptracegrpc` implementing a gRPC `otlptrace.Client` and offers convenience functions, `NewExportPipeline` and `InstallNewPipeline`, to setup and install a `otlptrace.Exporter` in tracing .(#1922) - Added `Baggage`, `Member`, and `Property` types to the `go.opentelemetry.io/otel/baggage` package along with their related functions. (#1967) - Added `ContextWithBaggage`, `ContextWithoutBaggage`, and `FromContext` functions to the `go.opentelemetry.io/otel/baggage` package. These functions replace the `Set`, `Value`, `ContextWithValue`, `ContextWithoutValue`, and `ContextWithEmpty` functions from that package and directly work with the new `Baggage` type. (#1967) - The `OTEL_SERVICE_NAME` environment variable is the preferred source for `service.name`, used by the environment resource detector if a service name is present both there and in `OTEL_RESOURCE_ATTRIBUTES`. (#1969) - Creates package `go.opentelemetry.io/otel/exporters/otlp/otlptrace/otlptracehttp` implementing an HTTP `otlptrace.Client` and offers convenience functions, `NewExportPipeline` and `InstallNewPipeline`, to setup and install a `otlptrace.Exporter` in tracing. (#1963) - Changes `go.opentelemetry.io/otel/sdk/resource.NewWithAttributes` to require a schema URL. The old function is still available as `resource.NewSchemaless`. This is a breaking change. (#1938) - Several builtin resource detectors now correctly populate the schema URL. (#1938) - Creates package `go.opentelemetry.io/otel/exporters/otlp/otlpmetric` that defines a metrics exporter that uses a `otlpmetric.Client` to send data. - Creates package `go.opentelemetry.io/otel/exporters/otlp/otlpmetric/otlpmetricgrpc` implementing a gRPC `otlpmetric.Client` and offers convenience functions, `New` and `NewUnstarted`, to create an `otlpmetric.Exporter`.(#1991) - Added `go.opentelemetry.io/otel/exporters/stdout/stdouttrace` exporter. (#2005) - Added `go.opentelemetry.io/otel/exporters/stdout/stdoutmetric` exporter. (#2005) - Added a `TracerProvider()` method to the `"go.opentelemetry.io/otel/trace".Span` interface. This can be used to obtain a `TracerProvider` from a given span that utilizes the same trace processing pipeline. (#2009) ### Changed - Make `NewSplitDriver` from `go.opentelemetry.io/otel/exporters/otlp` take variadic arguments instead of a `SplitConfig` item. `NewSplitDriver` now automatically implements an internal `noopDriver` for `SplitConfig` fields that are not initialized. (#1798) - `resource.New()` now creates a Resource without builtin detectors. Previous behavior is now achieved by using `WithBuiltinDetectors` Option. (#1810) - Move the `Event` type from the `go.opentelemetry.io/otel` package to the `go.opentelemetry.io/otel/sdk/trace` package. (#1846) - CI builds validate against last two versions of Go, dropping 1.14 and adding 1.16. (#1865) - BatchSpanProcessor now report export failures when calling `ForceFlush()` method. (#1860) - `Set.Encoded(Encoder)` no longer caches the result of an encoding. (#1855) - Renamed `CloudZoneKey` to `CloudAvailabilityZoneKey` in Resource semantic conventions according to spec. (#1871) - The `StatusCode` and `StatusMessage` methods of the `ReadOnlySpan` interface and the `Span` produced by the `go.opentelemetry.io/otel/sdk/trace` package have been replaced with a single `Status` method. This method returns the status of a span using the new `Status` type. (#1874) - Updated `ExportSpans` method of the`SpanExporter` interface type to accept `ReadOnlySpan`s instead of the removed `SpanSnapshot`. This brings the export interface into compliance with the specification in that it now accepts an explicitly immutable type instead of just an implied one. (#1873) - Unembed `SpanContext` in `Link`. (#1877) - Generate Semantic conventions from the specification YAML. (#1891) - Spans created by the global `Tracer` obtained from `go.opentelemetry.io/otel`, prior to a functioning `TracerProvider` being set, now propagate the span context from their parent if one exists. (#1901) - The `"go.opentelemetry.io/otel".Tracer` function now accepts tracer options. (#1902) - Move the `go.opentelemetry.io/otel/unit` package to `go.opentelemetry.io/otel/metric/unit`. (#1903) - Changed `go.opentelemetry.io/otel/trace.TracerConfig` to conform to the [Contributing guidelines](CONTRIBUTING.md#config.) (#1921) - Changed `go.opentelemetry.io/otel/trace.SpanConfig` to conform to the [Contributing guidelines](CONTRIBUTING.md#config). (#1921) - Changed `span.End()` now only accepts Options that are allowed at `End()`. (#1921) - Changed `go.opentelemetry.io/otel/metric.InstrumentConfig` to conform to the [Contributing guidelines](CONTRIBUTING.md#config). (#1921) - Changed `go.opentelemetry.io/otel/metric.MeterConfig` to conform to the [Contributing guidelines](CONTRIBUTING.md#config). (#1921) - Refactored option types according to the contribution style guide. (#1882) - Move the `go.opentelemetry.io/otel/trace.TraceStateFromKeyValues` function to the `go.opentelemetry.io/otel/oteltest` package. This function is preserved for testing purposes where it may be useful to create a `TraceState` from `attribute.KeyValue`s, but it is not intended for production use. The new `ParseTraceState` function should be used to create a `TraceState`. (#1931) - Updated `MarshalJSON` method of the `go.opentelemetry.io/otel/trace.TraceState` type to marshal the type into the string representation of the `TraceState`. (#1931) - The `TraceState.Delete` method from the `go.opentelemetry.io/otel/trace` package no longer returns an error in addition to a `TraceState`. (#1931) - Updated `Get` method of the `TraceState` type from the `go.opentelemetry.io/otel/trace` package to accept a `string` instead of an `attribute.Key` type. (#1931) - Updated `Insert` method of the `TraceState` type from the `go.opentelemetry.io/otel/trace` package to accept a pair of `string`s instead of an `attribute.KeyValue` type. (#1931) - Updated `Delete` method of the `TraceState` type from the `go.opentelemetry.io/otel/trace` package to accept a `string` instead of an `attribute.Key` type. (#1931) - Renamed `NewExporter` to `New` in the `go.opentelemetry.io/otel/exporters/stdout` package. (#1985) - Renamed `NewExporter` to `New` in the `go.opentelemetry.io/otel/exporters/metric/prometheus` package. (#1985) - Renamed `NewExporter` to `New` in the `go.opentelemetry.io/otel/exporters/trace/jaeger` package. (#1985) - Renamed `NewExporter` to `New` in the `go.opentelemetry.io/otel/exporters/trace/zipkin` package. (#1985) - Renamed `NewExporter` to `New` in the `go.opentelemetry.io/otel/exporters/otlp` package. (#1985) - Renamed `NewUnstartedExporter` to `NewUnstarted` in the `go.opentelemetry.io/otel/exporters/otlp` package. (#1985) - The `go.opentelemetry.io/otel/semconv` package has been moved to `go.opentelemetry.io/otel/semconv/v1.4.0` to allow for multiple [telemetry schema](https://github.com/open-telemetry/oteps/blob/main/text/0152-telemetry-schemas.md) versions to be used concurrently. (#1987) - Metrics test helpers in `go.opentelemetry.io/otel/oteltest` have been moved to `go.opentelemetry.io/otel/metric/metrictest`. (#1988) ### Deprecated - The `go.opentelemetry.io/otel/exporters/metric/prometheus` is deprecated, use `go.opentelemetry.io/otel/exporters/prometheus` instead. (#1993) - The `go.opentelemetry.io/otel/exporters/trace/jaeger` is deprecated, use `go.opentelemetry.io/otel/exporters/jaeger` instead. (#1993) - The `go.opentelemetry.io/otel/exporters/trace/zipkin` is deprecated, use `go.opentelemetry.io/otel/exporters/zipkin` instead. (#1993) ### Removed - Removed `resource.WithoutBuiltin()`. Use `resource.New()`. (#1810) - Unexported types `resource.FromEnv`, `resource.Host`, and `resource.TelemetrySDK`, Use the corresponding `With*()` to use individually. (#1810) - Removed the `Tracer` and `IsRecording` method from the `ReadOnlySpan` in the `go.opentelemetry.io/otel/sdk/trace`. The `Tracer` method is not a required to be included in this interface and given the mutable nature of the tracer that is associated with a span, this method is not appropriate. The `IsRecording` method returns if the span is recording or not. A read-only span value does not need to know if updates to it will be recorded or not. By definition, it cannot be updated so there is no point in communicating if an update is recorded. (#1873) - Removed the `SpanSnapshot` type from the `go.opentelemetry.io/otel/sdk/trace` package. The use of this type has been replaced with the use of the explicitly immutable `ReadOnlySpan` type. When a concrete representation of a read-only span is needed for testing, the newly added `SpanStub` in the `go.opentelemetry.io/otel/sdk/trace/tracetest` package should be used. (#1873) - Removed the `Tracer` method from the `Span` interface in the `go.opentelemetry.io/otel/trace` package. Using the same tracer that created a span introduces the error where an instrumentation library's `Tracer` is used by other code instead of their own. The `"go.opentelemetry.io/otel".Tracer` function or a `TracerProvider` should be used to acquire a library specific `Tracer` instead. (#1900) - The `TracerProvider()` method on the `Span` interface may also be used to obtain a `TracerProvider` using the same trace processing pipeline. (#2009) - The `http.url` attribute generated by `HTTPClientAttributesFromHTTPRequest` will no longer include username or password information. (#1919) - Removed `IsEmpty` method of the `TraceState` type in the `go.opentelemetry.io/otel/trace` package in favor of using the added `TraceState.Len` method. (#1931) - Removed `Set`, `Value`, `ContextWithValue`, `ContextWithoutValue`, and `ContextWithEmpty` functions in the `go.opentelemetry.io/otel/baggage` package. Handling of baggage is now done using the added `Baggage` type and related context functions (`ContextWithBaggage`, `ContextWithoutBaggage`, and `FromContext`) in that package. (#1967) - The `InstallNewPipeline` and `NewExportPipeline` creation functions in all the exporters (prometheus, otlp, stdout, jaeger, and zipkin) have been removed. These functions were deemed premature attempts to provide convenience that did not achieve this aim. (#1985) - The `go.opentelemetry.io/otel/exporters/otlp` exporter has been removed. Use `go.opentelemetry.io/otel/exporters/otlp/otlptrace` instead. (#1990) - The `go.opentelemetry.io/otel/exporters/stdout` exporter has been removed. Use `go.opentelemetry.io/otel/exporters/stdout/stdouttrace` or `go.opentelemetry.io/otel/exporters/stdout/stdoutmetric` instead. (#2005) ### Fixed - Only report errors from the `"go.opentelemetry.io/otel/sdk/resource".Environment` function when they are not `nil`. (#1850, #1851) - The `Shutdown` method of the simple `SpanProcessor` in the `go.opentelemetry.io/otel/sdk/trace` package now honors the context deadline or cancellation. (#1616, #1856) - BatchSpanProcessor now drops span batches that failed to be exported. (#1860) - Use `http://localhost:14268/api/traces` as default Jaeger collector endpoint instead of `http://localhost:14250`. (#1898) - Allow trailing and leading whitespace in the parsing of a `tracestate` header. (#1931) - Add logic to determine if the channel is closed to fix Jaeger exporter test panic with close closed channel. (#1870, #1973) - Avoid transport security when OTLP endpoint is a Unix socket. (#2001)
### :rocket: (Enhancement) * `opentelemetry-tracing` * [#2243](https://github.com/open-telemetry/opentelemetry-js/pull/2243) feat(tracing): auto flush BatchSpanProcessor on browser ([@kkruk-sumo](https://github.com/kkruk-sumo)) * `opentelemetry-resource-detector-aws`, `opentelemetry-semantic-conventions` * [#2268](https://github.com/open-telemetry/opentelemetry-js/pull/2268) feat(semantic-conventions): upgrade semantic conventions to version 1… ([@weyert](https://github.com/weyert)) * `opentelemetry-api-metrics`, `opentelemetry-context-async-hooks`, `opentelemetry-context-zone-peer-dep`, `opentelemetry-core`, `opentelemetry-exporter-collector-grpc`, `opentelemetry-exporter-collector-proto`, `opentelemetry-exporter-collector`, `opentelemetry-exporter-jaeger`, `opentelemetry-exporter-prometheus`, `opentelemetry-exporter-zipkin`, `opentelemetry-instrumentation-fetch`, `opentelemetry-instrumentation-grpc`, `opentelemetry-instrumentation-http`, `opentelemetry-instrumentation-xml-http-request`, `opentelemetry-instrumentation`, `opentelemetry-metrics`, `opentelemetry-node`, `opentelemetry-propagator-b3`, `opentelemetry-propagator-jaeger`, `opentelemetry-resource-detector-aws`, `opentelemetry-resource-detector-gcp`, `opentelemetry-resources`, `opentelemetry-sdk-node`, `opentelemetry-shim-opentracing`, `opentelemetry-tracing`, `opentelemetry-web` * [#2276](https://github.com/open-telemetry/opentelemetry-js/pull/2276) chore(deps): update dependency @opentelemetry/api to v1 ([@renovate-bot](https://github.com/renovate-bot)) ### :books: (Refine Doc) * [#2287](https://github.com/open-telemetry/opentelemetry-js/pull/2287) chore(doc): update compatibility matrix ([@vmarchaud](https://github.com/vmarchaud)) ### Committers: 3 * Krystian Kruk ([@kkruk-sumo](https://github.com/kkruk-sumo)) * Valentin Marchaud ([@vmarchaud](https://github.com/vmarchaud)) * Weyert de Boer ([@weyert](https://github.com/weyert))
## Changelog: ### API #### Enhancements - Parsing of the W3C Baggage header has been optimized. ### SDK #### Behavioral Changes - The implementation of SpanBuilder will no longer throw exceptions when null parameters are passed in. Instead, it will treat these calls as no-ops. #### Enhancements - Memory usage of the Tracing SDK has been greatly reduced when exporting via the OTLP or Jaeger exporters. - The OTLP protobuf version has been updated to v0.9.0 ### Extensions - A new experimental extension module has been added to provide a truly no-op implementation of the API. This is published under the `io.opentelemetry.extension.noopapi` name. - The `io.opentelemetry.sdk.autoconfigure` module now supports the `OTEL_SERVICE_NAME`/`otel.service.name` environment variable/system property for configuring the SDK's `Resource` implementation. ### Metrics (alpha) - The autoconfiguration code for metrics now supports durations to be provided with units attached to them (eg. "`100ms`"). This includes the following environment variables/system properties: - `OTEL_EXPORTER_OTLP_TIMEOUT`/`otel.exporter.otlp.timeout` - `OTEL_IMR_EXPORT_INTERVAL`/`otel.imr.export.interval` --- Many thanks to all the people who contributed to this release: @anuraaga @char16t @iNikem @jkwatson @kubawach @mxiamxia @piotr-sumo @wsargent @xyira
### Added - Allow span limits to be set programatically via TracerProvider. ([#1877](https://github.com/open-telemetry/opentelemetry-python/pull/1877)) ### Changed - Updated get_tracer to return an empty string when passed an invalid name ([#1854](https://github.com/open-telemetry/opentelemetry-python/pull/1854)) - Changed AttributeValue sequences to warn mypy users on adding None values to array ([#1855](https://github.com/open-telemetry/opentelemetry-python/pull/1855)) - Fixed exporter OTLP header parsing to match baggage header formatting. ([#1869](https://github.com/open-telemetry/opentelemetry-python/pull/1869)) - Added optional `schema_url` field to `Resource` class ([#1871](https://github.com/open-telemetry/opentelemetry-python/pull/1871)) - Update protos to latest version release 0.9.0 ([#1873](https://github.com/open-telemetry/opentelemetry-python/pull/1873))
This version of OpenTelemetry PHP will; Make HTTP exporters require Client, StreamFactory and RequestFactory parameters, as part of PSR flexibility improvements. Implement use of IdGenerator to generate trace ids instead of randomHex. Update the README.md to ease contributions. Update automated collector example to be cache independent and add children spans to example. Fix trace context key extraction . Change default port for GRPC exporter from 55680 to 55681. Include OTLP/GRPC Exporter Implementation. Fix the wrong variable in Otlp Exporter. Have guides for integrating OpenTelemetry with Laravel and Symfony applications. Include W3c trace context test service. Update components of automated examples. Add website documentation update action. Fix ParentBased spans for remote requests appearing as root spans. Update general sampling behaviour. Fix tracecontext header extraction. Include experimental new Relic exporter, span converter, example, and tests. Update SpanProcessor parameters onStart . Fix memory leak in Exporter. Add additional code owners to the project. Add ContextValueTrait and Scope to store in Context. Update namespace for CorrelationContext class . Add Basic example to send zipkin style data to a New Relic backend. Propagate SpanKind to Span constructor. Modified SpanStatus to conform with the feature-freeze specs. Add recordException from specs. Add an UpDownSumObserver Instrument for the Metrics API. Add IdGenerator interface . Add Trace Context Tracestate header propagation. Rename SpanContext methods isValidContext and isRemoteContext to be aligned with specification. Include TraceProvider lookups tracers only by name, instead of name and version. Change master to main in CI. Update PHP CS Fixer for PHP 8.0 support. Add semantic versioning rationale. Include PHP 8 Compatibility trials. Add proto folder to the project. Associate Spans with their TracerProvider. Add OTLP/HTTP JSON exporter . Add static analysis for tests and examples. Bump PHPStan to level 5, Psalm to level 3. Implement the sampler into the tracer and trace provider.
### v0.18.1 / 2021-06-08 * FIXED: Missing require to common in faraday
* `create_span` has been renamed `start_inactive_span` (#53) * Fix `add_event` Elixir function (#56) * Add accessors to deconstruct Span (#54) * Attributes and events that aren't a list are now silently dropped (#51) * Fix bug where there is no current span ctx and update_name is called (#52) * Readme, typespec and doc fixes #45 #46 #50 #59
## v0.29.0 Beta ## 🛑 Breaking changes 🛑 - Rename `service.Application` to `service.Collector` (#3268) - Provide case sensitivity in config yaml mappings by using Koanf instead of Viper (#3337) - Move zipkin constants to an internal package (#3431) - Disallow renaming metrics using metric relabel configs (#3410) - Move cgroup and iruntime utils from memory_limiter to internal folder (#3448) - Move model pdata interfaces to pdata, expose them publicly (#3455) ## 💡 Enhancements 💡 - Change obsreport helpers for scraper to use the same pattern as Processor/Exporter (#3327) - Convert `otlptext` to implement Marshaler interfaces (#3366) - Add encoder/decoder and marshaler/unmarshaler for OTLP protobuf (#3401) - Use the new marshaler/unmarshaler in `kafka` exporter (#3403) - Convert `zipkinv2` to to/from translator interfaces (#3409) - `zipkinv1`: Move to translator and encoders interfaces (#3419) - Use the new marshaler/unmarshaler in `kafka` receiver #3402 - Change `oltp` receiver to use the new unmarshaler, avoid grpc-gateway dependency (#3406) - Use the new Marshaler in the `otlphttp` exporter (#3433) - Add grpc response struct for all signals instead of returning interface in `otlp` receiver/exporter (#3437) - `zipkinv2`: Add encoders, decoders, marshalers (#3426) - `scrapererror` receiver: Return concrete error type (#3360) - `kafka` receiver: Add metrics support (#3452) - `prometheus` receiver: - Add store to track stale metrics (#3414) - Add `up` and `scrape_xxxx` internal metrics (#3116) ## 🧰 Bug fixes 🧰 - `prometheus` receiver: - Reject datapoints with duplicate label keys (#3408) - Scrapers are not stopped when receiver is shutdown (#3450) - `prometheusremotewrite` exporter: Adjust default retry settings (#3416) - `hostmetrics` receiver: Fix missing startTimestamp for `processes` scraper (#3461)