Performance Benchmark of OpenTelemetry API
This document describes common performance benchmark guidelines on how to measure and report the performance of OpenTelemetry SDKs.
The goal of this benchmark is to provide a tool to get the basic performance overhead of the OpenTelemetry SDK for given events throughput on the target platform.
- No parent
- Default Span Kind and Status.
- Associated to a resource with attributes
service.versionand 10 characters string value for each attribute, and attribute
service.instance.idwith a unique UUID. See Service for details.
- 1 attribute with a signed 64-bit integer value.
- 1 event without any attributes.
AlwaysOnsampler should be enabled.
Spanis created and immediately ended.
For the languages with bootstrap cost like JIT compilation, a warm-up phase is
recommended to take place before the measurement, which runs under the same
Number of spans which could be created and exported via OTLP exporter in 1 second per logical core and average number over all logical cores, with each span containing 10 attributes, and each attribute containing two 20 characters strings, one as attribute name the other as value.
CPU Usage Measurement
With given number of span throughput specified by user, or 10,000 spans per second as default if user does not input the number, measure and report the CPU usage for SDK with both default configured simple and batching span processors together with OTLP exporter. The benchmark should create an out-of-process OTLP receiver which listens on the exporting target or adopts existing OTLP exporter which runs out-of-process, responds with success status immediately and drops the data. The collector should not add significant CPU overhead to the measurement. Because the benchmark does not include user processing logic, the total CPU consumption of benchmark program could be considered as approximation of SDK’s CPU consumption.
The total running time for one test iteration is suggested to be at least 15 seconds. The average and peak CPU usage should be reported.
Memory Usage Measurement
Measure dynamic memory consumption, e.g. heap, for the same scenario as above CPU Usage section with 15 seconds duration.
All the numbers above should be measured multiple times (suggest 10 times at least) and reported.