Sampling

Sampling is a process that restricts the amount of spans that are generated by a system. The exact sampler you should use depends on your specific needs, but in general you should make a decision at the start of a trace, and allow the sampling decision to propagate to other services.

A Sampler can be set on the tracer provider using the WithSampler option, as follows:

provider := trace.NewTracerProvider(
    trace.WithSampler(trace.AlwaysSample()),
)

AlwaysSample and NeverSample are self-explanatory values. AlwaysSample means that every span is sampled, while NeverSample means that no span is sampled. When you’re getting started, or in a development environment, use AlwaysSample.

Other samplers include:

• TraceIDRatioBased, which samples a fraction of spans, based on the fraction given to the sampler. If you set .5, half of all the spans are sampled.
• ParentBased, is a sampler decorator which behaves differently, based on the parent of the span. If the span has no parent, the decorated sampler is used to make the sampling decision. By default, ParentBased samples spans that have parents that were sampled, and doesn’t sample spans whose parents were not sampled.

By default, the tracer provider uses a ParentBased sampler with the AlwaysSample sampler.

When in a production environment, consider using the ParentBased sampler with the TraceIDRatioBased sampler.

Custom samplers

If the built-in samplers don’t meet your needs, you can create a custom sampler by implementing the Sampler interface. A custom sampler must implement two methods:

• ShouldSample(parameters SamplingParameters) SamplingResult: Makes the sampling decision based on the provided parameters.
• Description() string: Returns a description of the sampler.

Preserving tracestate

psc := trace.SpanContextFromContext(parameters.ParentContext)
// Use psc.TraceState() in your SamplingResult

Example

The following example demonstrates a custom sampler that samples spans based on an attribute value while correctly preserving tracestate:

package main

import (
    "context"

    "go.opentelemetry.io/otel/attribute"
    sdktrace "go.opentelemetry.io/otel/sdk/trace"
    "go.opentelemetry.io/otel/trace"
)

// AttributeBasedSampler samples spans based on an attribute value.
// It always samples spans with the "high.priority" attribute set to true.
type AttributeBasedSampler struct {
    fallback sdktrace.Sampler
}

// NewAttributeBasedSampler creates a new AttributeBasedSampler.
func NewAttributeBasedSampler(fallback sdktrace.Sampler) *AttributeBasedSampler {
    return &AttributeBasedSampler{fallback: fallback}
}

func (s *AttributeBasedSampler) ShouldSample(p sdktrace.SamplingParameters) sdktrace.SamplingResult {
    // Always extract the parent span context to get the tracestate.
    psc := trace.SpanContextFromContext(p.ParentContext)

    // Check if any attribute indicates high priority.
    for _, attr := range p.Attributes {
        if attr.Key == "high.priority" && attr.Value.AsBool() {
            return sdktrace.SamplingResult{
                Decision:   sdktrace.RecordAndSample,
                Attributes: p.Attributes,
                // Critical: preserve the parent's tracestate
                Tracestate: psc.TraceState(),
            }
        }
    }

    // Fall back to the default sampler for other spans.
    result := s.fallback.ShouldSample(p)

    // Ensure tracestate is preserved even when using fallback.
    // Built-in samplers already handle this, but it's good practice to verify.
    return sdktrace.SamplingResult{
        Decision:   result.Decision,
        Attributes: result.Attributes,
        Tracestate: psc.TraceState(),
    }
}

func (s *AttributeBasedSampler) Description() string {
    return "AttributeBasedSampler"
}

Using your custom sampler

You can use your custom sampler with the tracer provider:

sampler := NewAttributeBasedSampler(sdktrace.TraceIDRatioBased(0.1))

provider := sdktrace.NewTracerProvider(
    sdktrace.WithSampler(sampler),
)

You can also compose it with the ParentBased sampler:

provider := sdktrace.NewTracerProvider(
    sdktrace.WithSampler(
        sdktrace.ParentBased(
            NewAttributeBasedSampler(sdktrace.TraceIDRatioBased(0.1)),
        ),
    ),
)

Additional considerations

When implementing custom samplers, keep these points in mind:

1. Ignoring the parent sampling decision: If you want to respect parent sampling decisions, wrap your sampler with ParentBased or check psc.IsSampled() manually.

2. Heavy computations in ShouldSample: The ShouldSample function is called synchronously for every span creation. Avoid expensive operations like network calls or complex computations that could impact performance.