Instrumentation

Instrumentation is the act of adding observability code to an app yourself.

If you’re instrumenting an app, you need to use the OpenTelemetry SDK for your language. You’ll then use the SDK to initialize OpenTelemetry and the API to instrument your code. This will emit telemetry from your app, and any library you installed that also comes with instrumentation.

If you’re instrumenting a library, only install the OpenTelemetry API package for your language. Your library will not emit telemetry on its own. It will only emit telemetry when it is part of an app that uses the OpenTelemetry SDK. For more on instrumenting libraries, see Libraries.

For more information about the OpenTelemetry API and SDK, see the specification.

Setup

First, ensure you have the SDK package installed:

gem install opentelemetry-sdk

Then include configuration code that runs when your program initializes. Make sure that service.name is set by configuring a service name.

Traces

Acquiring a Tracer

To begin tracing, you will need to ensure you have an initialized Tracer that comes from a TracerProvider.

The easiest and most common way to do this is to use the globally-registered TracerProvider. If you are using instrumentation libraries, such as in a Rails app, then one will be registered for you.

# If in a Rails app, this lives in config/initializers/opentelemetry.rb
require "opentelemetry/sdk"

OpenTelemetry::SDK.configure do |c|
  c.service_name = '<YOUR_SERVICE_NAME>'
end

# 'Tracer' can be used throughout your code now
MyAppTracer = OpenTelemetry.tracer_provider.tracer('<YOUR_TRACER_NAME>')

With a Tracer acquired, you can manually trace code.

Get the current span

It’s very common to add information to the current span somewhere within your program. To do so, you can get the current span and add attributes to it.

require "opentelemetry/sdk"

def track_extended_warranty(extended_warranty)
  # Get the current span
  current_span = OpenTelemetry::Trace.current_span

  # And add useful stuff to it!
  current_span.add_attributes({
    "com.extended_warranty.id" => extended_warranty.id,
    "com.extended_warranty.timestamp" => extended_warranty.timestamp
  })
end

Creating New Spans

To create a span, you’ll need a configured Tracer.

Typically when you create a new span, you’ll want it to be the active/current span. To do that, use in_span:

require "opentelemetry/sdk"

def do_work
  MyAppTracer.in_span("do_work") do |span|
    # do some work that the 'do_work' span tracks!
  end
end

Creating nested spans

If you have a distinct sub-operation you’d like to track as a part of another one, you can create nested spans to represent the relationship:

require "opentelemetry/sdk"

def parent_work
  MyAppTracer.in_span("parent") do |span|
    # do some work that the 'parent' span tracks!

    child_work

    # do some more work afterwards
  end
end

def child_work
  MyAppTracer.in_span("child") do |span|
    # do some work that the 'child' span tracks!
  end
end

In the preceding example, two spans are created - named parent and child - with child nested under parent. If you view a trace with these spans in a trace visualization tool, child will be nested under parent.

Add attributes to a span

Attributes let you attach key/value pairs to a span so it carries more information about the current operation that it’s tracking.

You can use set_attribute to add a single attribute to a span:

require "opentelemetry/sdk"

current_span = OpenTelemetry::Trace.current_span

current_span.set_attribute("animals", ["elephant", "tiger"])

You can use add_attributes to add a map of attributes:

require "opentelemetry/sdk"

current_span = OpenTelemetry::Trace.current_span

current_span.add_attributes({
  "my.cool.attribute" => "a value",
  "my.first.name" => "Oscar"
})

You can also add attributes to a span as it’s being created:

require "opentelemetry/sdk"

MyAppTracer.in_span('foo', attributes: { "hello" => "world", "some.number" => 1024 }) do |span|
  #  do stuff with the span
end

⚠ Spans are thread safe data structures that require locks when they are mutated. You should therefore avoid calling set_attribute multiple times and instead assign attributes in bulk with a Hash, either during span creation or with add_attributes on an existing span.

⚠ Sampling decisions happen at the moment of span creation. If your sampler considers span attributes when deciding to sample a span, then you must pass those attributes as part of span creation. Any attributes added after creation will not be seen by the sampler, because the sampling decision has already been made.

Add semantic attributes

Semantic Attributes are pre-defined Attributes that are well-known naming conventions for common kinds of data. Using Semantic Attributes lets you normalize this kind of information across your systems.

To use Semantic Attributes in Ruby, add the appropriate gem:

gem install opentelemetry-semantic_conventions

Then you can use it in code:

require 'opentelemetry/sdk'
require 'opentelemetry/semantic_conventions'

current_span = OpenTelemetry::Trace.current_span

current_span.add_attributes({
  OpenTelemetry::SemanticConventions::Trace::HTTP_METHOD => "GET",
  OpenTelemetry::SemanticConventions::Trace::HTTP_URL => "https://opentelemetry.io/",
})

Add Span Events

A span event is a human-readable message on a span that represents “something happening” during it’s lifetime. For example, imagine a function that requires exclusive access to a resource that is under a mutex. An event could be created at two points - once, when we try to gain access to the resource, and another when we acquire the mutex.

require "opentelemetry/sdk"

span = OpenTelemetry::Trace.current_span

span.add_event("Acquiring lock")
if mutex.try_lock
  span.add_event("Got lock, doing work...")
  # some code here
  span.add_event("Releasing lock")
else
  span.add_event("Lock already in use")
end

A useful characteristic of events is that their timestamps are displayed as offsets from the beginning of the span, allowing you to easily see how much time elapsed between them.

Events can also have attributes of their own e.g.

require "opentelemetry/sdk"

span.add_event("Cancelled wait due to external signal", attributes: {
  "pid" => 4328,
  "signal" => "SIGHUP"
})

Add Span Links

A span can be created with zero or more span links that causally link it to another span. A link needs a span context to be created.

require "opentelemetry/sdk"

span_to_link_from = OpenTelemetry::Trace.current_span

link = OpenTelemetry::Trace::Link.new(span_to_link_from.context)

MyAppTracer.in_span("new-span", links: [link])
  # do something that 'new_span' tracks

  # The link in 'new_span' casually associated it with the span it's linked from,
  # but it is not necessarily a child span.
end

Span Links are often used to link together different traces that are related in some way, such as a long-running task that calls into sub-tasks asynchronously.

Links can also be created with additional attributes:

link = OpenTelemetry::Trace::Link.new(span_to_link_from.context, attributes: { "some.attribute" => 12 })

Set span status

A Status can be set on a Span, typically used to specify that a Span has not completed successfully - Error. By default, all spans are Unset, which means a span completed without error. The Ok status is reserved for when you need to explicitly mark a span as successful rather than stick with the default of Unset (i.e., “without error”).

The status can be set at any time before the span is finished.

require "opentelemetry/sdk"

current_span = OpenTelemetry::Trace.current_span

begin
  1/0 # something that obviously fails
rescue
  current_span.status = OpenTelemetry::Trace::Status.error("error message here!")
end

Record exceptions in spans

It can be a good idea to record exceptions when they happen. It’s recommended to do this in conjunction with setting span status.

require "opentelemetry/sdk"

current_span = OpenTelemetry::Trace.current_span

begin
  1/0 # something that obviously fails
rescue Exception => e
  current_span.status = OpenTelemetry::Trace::Status.error("error message here!")
  current_span.record_exception(e)
end

Recording an exception creates a Span Event on the current span with a stack trace as an attribute on the span event.

Exceptions can also be recorded with additional attributes:

current_span.record_exception(ex, attributes: { "some.attribute" => 12 })

Context Propagation

Distributed Tracing tracks the progression of a single Request, called a Trace, as it is handled by Services that make up an Application. A Distributed Trace transverses process, network and security boundaries. Glossary

This requires context propagation, a mechanism where identifiers for a trace are sent to remote processes.

ℹ The OpenTelemetry Ruby SDK will take care of context propagation as long as your service is leveraging auto-instrumented libraries. Please refer to the README for more details.

In order to propagate trace context over the wire, a propagator must be registered with the OpenTelemetry SDK. The W3 TraceContext and Baggage propagators are configured by default. Operators may override this value by setting OTEL_PROPAGATORS environment variable to a comma separated list of propagators. For example, to add B3 propagation, set OTEL_PROPAGATORS to the complete list of propagation formats you wish to support:

export OTEL_PROPAGATORS=tracecontext,baggage,b3

Propagators other than tracecontext and baggage must be added as gem dependencies to your Gemfile, e.g.:

gem 'opentelemetry-propagator-b3'

Metrics

The metrics API & SDK are currently under development.

Logs

The logs API & SDK are currently under development.

Next Steps

You’ll also want to configure an appropriate exporter to export your telemetry data to one or more telemetry backends.