# OpenTelemetry eBPF Instrumentation

> Learn how to use OpenTelemetry eBPF Instrumentation for automatic instrumentation.

---

LLMS index: [llms.txt](/llms.txt)

---

OpenTelemetry libraries provide telemetry collection for popular programming
languages and frameworks. However, getting started with distributed tracing can
be complex. In some compiled languages like Go or Rust, you must manually add
tracepoints to the code.

OpenTelemetry eBPF Instrumentation (OBI) is an auto-instrumentation tool to
easily get started with Application Observability. OBI uses eBPF to
automatically inspect application executables and the OS networking layer, and
capture trace spans related to web transactions and Rate Errors Duration (RED)
metrics for Linux HTTP/S and gRPC services. All data capture occurs without any
modifications to application code or configuration.

OBI offers the following features:

- **Wide language support**: Java (JDK 8+), .NET, Go, Python, Ruby, Node.js, C,
  C++, and Rust
- **Lightweight**: No code changes required, no libraries to install, no
  restarts needed
- **Efficient instrumentation**: Traces and metrics are captured by eBPF probes
  with minimal overhead
- **Distributed tracing**: Distributed trace spans are captured and reported to
  a collector
- **Log enrichment**: Enrich JSON logs with trace context for correlation
- **Kubernetes-native**: Provides configuration-free auto-instrumentation for
  Kubernetes applications
- **Visibility into encrypted communications**: Capture transactions over
  TLS/SSL without decryption
- **Context propagation**: Propagate trace context across services automatically
- **Protocol support**: HTTP/S, gRPC, gRPC-Web, MQTT, Memcached, and more
- **Database instrumentation**: PostgreSQL (including pgx driver), MySQL,
  MongoDB, Redis, Couchbase (N1QL/SQL++ and KV protocol)
- **GenAI instrumentation**: Trace and metrics for OpenAI and Anthropic Claude
  API calls with automatic payload extraction
- **Low cardinality metrics**: Prometheus-compatible metrics with low
  cardinality for cost reduction
- **Network observability**: Capture network flows between services with
  host-level TCP RTT statistics
- **Enhanced service discovery**: Improved service name lookup with DNS
  resolution
- **Collector integration**: Run OBI as an OpenTelemetry Collector receiver
  component

## Recent highlights (v0.7.0)

OBI v0.7.0 introduces several significant improvements:

- **StatsO11y**: New statistical metrics pipeline for host-level network
  statistics, starting with TCP RTT metrics
- **Expanded protocol coverage**: Added Memcached protocol tracing support
- **Enhanced GenAI instrumentation**: Added support for Anthropic Claude with
  automatic payload extraction
- **Python asyncio improvements**: Added context propagation support for Python
  asyncio workloads using `uvloop`
- **New example scenario**: Added an NGINX example covering direct routing,
  reverse proxying, and route-based telemetry across Docker, Kubernetes, and
  standalone deployments
- **Prometheus exemplars**: Support for exemplars in metrics export for better
  correlation with traces
- **New span types**: SQL instrumentation now emits server spans for database
  calls
- **Operational controls**: Added configurable `log_format` and Kubernetes API
  reconnect interval settings
- **Network diagnostics**: Added `obi_bpf_network_ignored_packets_total` for
  troubleshooting dropped network packets
- **Release artifacts**: CycloneDX SBOMs now included in release packages for
  supply chain security

For a complete list of changes and upgrade notes, see the
[release notes](https://github.com/open-telemetry/opentelemetry-ebpf-instrumentation/releases/tag/v0.7.0).

If you want to explore the new NGINX example, see the
[example walkthrough](https://github.com/open-telemetry/opentelemetry-ebpf-instrumentation/tree/v0.7.0/examples/nginx).

## Requirements

OBI requires the following to run:

- Linux kernel version 5.8 or later (or 4.18 for Redhat Enterprise Linux)
- An x86_64 or arm64 processor
- Runtime support for eBPF (most modern Linux distributions)
- Administrative privileges (root access) or the specific capabilities listed in
  the [configuration reference](security/)

![OBI eBPF architecture](./ebpf-arch.svg)

## Compatibility

OBI is tested with the following Linux distributions:

- Ubuntu 20.04 LTS, 21.04, 22.04 LTS and 23.04
- CentOS 7, 8, and 9
- AlmaLinux 8, 9
- Rocky Linux 8, 9
- Red Hat Enterprise Linux 8, 9
- Debian 11, 12
- openSUSE Leap 15.3, 15.4
- SUSE Linux Enterprise Server 15 SP4

- OBI also supports RedHat-based distributions: RHEL8, CentOS 8, Rocky8,
  AlmaLinux8, and others, which ship a Kernel 4.18 that backports eBPF-related
  patches.

- For instrumenting Go programs, compile with at least Go 1.17. OBI support Go
  applications built with a major **Go version no earlier than 3 versions**
  behind the current stable major release.
- Administrative access rights to execute OBI.

## Limitations

OBI has its limitations too. It only provides generic metrics and transaction
level trace span information. Language agents and manual instrumentation is
still recommended, so that you can specify the custom attributes and events you
want to capture.

While most eBPF programs require elevated privileges, OBI allows you to specify
finer grained permissions to run with minimum required permissions, such as:
`CAP_DAC_READ_SEARCH`, `CAP_SYS_PTRACE`, `CAP_PERFMON`, `CAP_BPF`,
`CAP_CHECKPOINT_RESTORE`, and others.

Some OBI functionality requires further permissions, for example using the
network observability probes with Linux Traffic Control requires
`CAP_NET_ADMIN`, but it's a feature you have to optionally enable.

For a comprehensive list of capabilities required by OBI, refer to
[Security, permissions and capabilities](security/).

## Get started with OBI

- Follow the [setup](setup/) documentation to get started with OBI either with
  Docker or Kubernetes.
- Learn about [trace-log correlation](./trace-log-correlation/) to connect
  traces with application logs and enrich JSON logs with trace context.
- Discover how to run
  [OBI as a Collector receiver](./configure/collector-receiver/) for centralized
  telemetry processing.

## Troubleshooting

- See the [troubleshooting](./troubleshooting) guide for help with common
  issues.

---

Section pages:

- [Configure OBI](/bn/docs/zero-code/obi/configure/): Learn how to configure OBI.
- [Network metrics](/bn/docs/zero-code/obi/network/): Configuring OBI to observe point-to-point network metrics.
- [Set up OBI](/bn/docs/zero-code/obi/setup/): Learn how to set up and run OBI.
- [OBI exported metrics](/bn/docs/zero-code/obi/metrics/): Learn about the HTTP/gRPC metrics OBI can export.
- [Distributed traces with OBI](/bn/docs/zero-code/obi/distributed-traces/): Learn about OBI's distributed traces support.
- [Measuring total request times, instead of service times](/bn/docs/zero-code/obi/requesttime/): How to measure total request times from the point of view of the client
- [OBI security, permissions, and capabilities](/bn/docs/zero-code/obi/security/): Privileges and capabilities required by OBI
- [Troubleshooting](/bn/docs/zero-code/obi/troubleshooting/): Troubleshooting OBI common issues and errors
- [OBI and Cilium compatibility](/bn/docs/zero-code/obi/cilium-compatibility/): Compatibility notes when running OBI alongside Cilium
- [OBI metrics cardinality](/bn/docs/zero-code/obi/cardinality/): Overview of how to calculate the cardinality of metrics produced by a default OBI installation, considering the size and complexity of the instrumented environment.
- [Trace-log correlation](/bn/docs/zero-code/obi/trace-log-correlation/): Learn how OBI correlates application logs with distributed traces for faster debugging and troubleshooting.
