LLM call examples

Simple chat completion

This is an example of telemetry generated for a chat completion call with system and user messages.

%%{init:
{
  "sequence": { "messageAlign": "left", "htmlLabels":true },
  "themeVariables": { "noteBkgColor" : "green", "noteTextColor": "black", "activationBkgColor": "green", "htmlLabels":true }
}
}%%
sequenceDiagram
    participant A as Application
    participant I as Instrumented Client
    participant M as Model
    A->>+I: #U+200D
    I->>M: input = [system: You are a helpful bot, user: Tell me a joke about OpenTelemetry]
    Note left of I: GenAI Client span
    I-->M: assistant: Why did the developer bring OpenTelemetry to the party? Because it always knows how to trace the fun!
    I-->>-A: #U+200D

GenAI client span when content capturing is disabled

GenAI client span when content capturing is enabled on span attributes

gen_ai.input.messages value

[
  {
    "role": "system",
    "parts": [
      {
        "type": "text",
        "content": "You are a helpful bot"
      }
    ]
  },
  {
    "role": "user",
    "parts": [
      {
        "type": "text",
        "content": "Tell me a joke about OpenTelemetry"
      }
    ]
  }
]

gen_ai.output.messages value

[
  {
    "role": "assistant",
    "parts": [
      {
        "type": "text",
        "content": " Why did the developer bring OpenTelemetry to the party? Because it always knows how to trace the fun!"
      }
    ],
    "finish_reason": "stop"
  }
]

GenAI telemetry when content capturing is enabled on event attributes

Span:

Event:

gen_ai.input.messages value

[
  {
    "role": "system",
    "parts": [
      {
        "type": "text",
        "content": "You are a helpful bot"
      }
    ]
  },
  {
    "role": "user",
    "parts": [
      {
        "type": "text",
        "content": "Tell me a joke about OpenTelemetry"
      }
    ]
  }
]

gen_ai.output.messages value

[
  {
    "role": "assistant",
    "parts": [
      {
        "type": "text",
        "content": " Why did the developer bring OpenTelemetry to the party? Because it always knows how to trace the fun!"
      }
    ],
    "finish_reason": "stop"
  }
]

Tool calls (functions)

This is an example of telemetry generated for a chat completion call with user message and function definition that results in a model requesting application to call provided function. Application executes a function and requests another completion now with the tool response.

%%{init:
{
  "sequence": { "messageAlign": "left", "htmlLabels":true },
  "themeVariables": { "noteBkgColor" : "green", "noteTextColor": "black", "activationBkgColor": "green", "htmlLabels":true }
}
}%%
sequenceDiagram
    participant A as Application
    participant I as Instrumented Client
    participant M as Model
    A->>+I: #U+200D
    I->>M: input = [user: What's the weather in Paris?]
    Note left of I: GenAI Client span 1
    I-->M: assistant: Call to the get_weather tool with Paris as the location argument.
    I-->>-A: #U+200D
    A -->> A: parse tool parameters<br/>execute tool<br/>update chat history
    A->>+I: #U+200D
    I->>M: input = [user: What's the weather in Paris?, assistant: get_weather tool call, tool: rainy, 57°F]
    Note left of I: GenAI Client span 2
    I-->M: assistant: The weather in Paris is rainy and overcast, with temperatures around 57°F
    I-->>-A: #U+200D

GenAI client spans when content capturing is disabled

The relationship between below spans depends on how user application code is written. They are likely to be siblings if there is an encompassing span.

GenAI client span 1:

Tool call:

If tool call is instrumented according to execute-tool span definition, it may look like

GenAI client span 2:

GenAI client spans when content capturing is enabled on span attributes

The relationship between below spans depends on how user application code is written. They are likely to be siblings if there is an encompassing span.

GenAI client span 1:

gen_ai.input.messages value

[
  {
    "role": "user",
    "parts": [
      {
        "type": "text",
        "content": "Weather in Paris?"
      }
    ]
  }
]

gen_ai.output.messages value

[
  {
    "role": "assistant",
    "parts": [
      {
        "type": "tool_call",
        "id": "call_VSPygqKTWdrhaFErNvMV18Yl",
        "name": "get_weather",
        "arguments": {
          "location": "Paris"
        }
      }
    ],
    "finish_reason": "tool_call"
  }
]

Tool call:

If tool call is instrumented according to execute-tool span definition, it may look like this:

GenAI client span 2:

gen_ai.input.messages value

[
  {
    "role": "user",
    "parts": [
      {
        "type": "text",
        "content": "Weather in Paris?"
      }
    ]
  },
  {
    "role": "assistant",
    "parts": [
      {
        "type": "tool_call",
        "id": "call_VSPygqKTWdrhaFErNvMV18Yl",
        "name": "get_weather",
        "arguments": {
          "location": "Paris"
        }
      }
    ]
  },
  {
    "role": "tool",
    "parts": [
      {
        "type": "tool_call_response",
        "id": " call_VSPygqKTWdrhaFErNvMV18Yl",
        "response": "rainy, 57°F"
      }
    ]
  }
]

gen_ai.output.messages value

[
  {
    "role": "assistant",
    "parts": [
      {
        "type": "text",
        "content": "The weather in Paris is currently rainy with a temperature of 57°F."
      }
    ],
    "finish_reason": "stop"
  }
]

System instructions along with chat history (content enabled)

Some providers allow to provide instructions separately from the chat history provided in the inputs or in addition to system (developer, etc) message provided in the input.

This example demonstrates en edge case when conflicting instructions are provided to the OpenAI responses API. In this case instructions are recorded in the gen_ai.system_instructions attribute.

response = openai.responses.create(
  model="gpt-4",
  instructions="You must never tell jokes",
  input=[{"role": "system", "content": "You are a helpful assistant"}, {"role": "user", "content": "Tell me a joke"}])

Span:

gen_ai.system_instructions value

[
  {
    "type": "text",
    "content": "You must never tell jokes"
  }
]

gen_ai.input.messages value

[
  {
    "role": "system",
    "parts": [
      {
        "type": "text",
        "content": "You are a helpful bot"
      }
    ]
  },
  {
    "role": "user",
    "parts": [
      {
        "type": "text",
        "content": "Tell me a joke about OpenTelemetry"
      }
    ]
  }
]

gen_ai.output.messages value

[
  {
    "role": "assistant",
    "parts": [
      {
        "type": "text",
        "content": "I'm sorry, but I can't assist with that"
      }
    ],
    "finish_reason": "stop"
  }
]

Tool calls (built-in)

[!NOTE]

The format of gen_ai.output.messages is not yet specified for built-in tool calls (check #2585 for the details).

This is an example of telemetry generated for a chat completion call with code_interpreter tool that results in a model provider executing a tool and returning response along with tool call details.

%%{init:
{
  "sequence": { "messageAlign": "left", "htmlLabels":true },
  "themeVariables": { "noteBkgColor" : "green", "noteTextColor": "black", "activationBkgColor": "green", "htmlLabels":true }
}
}%%
sequenceDiagram
  participant A as Application
  participant I as Instrumented Client
  participant M as Model

  A ->>+ I:
  I ->> M: input = [system: You are a helpful bot, user: Write Python code that generates a random number, executes it, and returns the result.]
  Note left of I: GenAI Client span
  I --> M: tool:code='import random ....'<br>assistant: The generated random number is 95.
  I -->>- A:

GenAI client span:

gen_ai.input.messages value

TODO

gen_ai.output.messages value

TODO

Chat completion with multiple choices

This example covers the scenario when user requests model to generate two completions for the same prompt:

%%{init:
{
  "sequence": { "messageAlign": "left", "htmlLabels":true },
  "themeVariables": { "noteBkgColor" : "green", "noteTextColor": "black", "activationBkgColor": "green", "htmlLabels":true }
}
}%%
sequenceDiagram
    participant A as Application
    participant I as Instrumented Client
    participant M as Model
    A->>+I: #U+200D
    I->>M: input = [system: You are a helpful bot, user: Tell me a joke about OpenTelemetry]
    Note left of I: GenAI Client span
    I-->M: assistant: Why did the developer bring OpenTelemetry to the party? Because it always knows how to trace the fun!<br/> assistant: Why did OpenTelemetry get promoted? It had great span of control!
    I-->>-A: #U+200D

GenAI client span when content capturing is enabled on span attributes

gen_ai.input.messages value

[
  {
    "role": "system",
    "parts": [
      {
        "type": "text",
        "content": "You are a helpful bot"
      }
    ]
  },
  {
    "role": "user",
    "parts": [
      {
        "type": "text",
        "content": "Tell me a joke about OpenTelemetry"
      }
    ]
  }
]

gen_ai.output.messages value

[
  {
    "role": "assistant",
    "parts": [
      {
        "type": "text",
        "content": " Why did the developer bring OpenTelemetry to the party? Because it always knows how to trace the fun!"
      }
    ],
    "finish_reason": "stop"
  },
  {
    "role": "assistant",
    "parts": [
      {
        "type": "text",
        "content": " Why did OpenTelemetry get promoted? It had great span of control!"
      }
    ],
    "finish_reason": "stop"
  }
]