Semantic conventions for feature flags in events

Status: Development

Motivation

Feature flags are commonly used in modern applications to decouple feature releases from deployments. Many feature flagging tools support the ability to update flag configurations in near real-time from a remote feature flag management service. They also commonly allow rulesets to be defined that return values based on contextual information. For example, a feature could be enabled only for a specific subset of users based on context (e.g. users email domain, membership tier, country).

Since feature flags are dynamic and affect runtime behavior, it’s important to collect relevant feature flag telemetry signals. This can be used to determine the impact a feature has on a request, enabling enhanced observability use cases, such as A/B testing or progressive feature releases.

Recording an evaluation

Feature flag evaluations SHOULD be recorded as attributes on the Event passed to the Logger emit operations.

Note: Events are in-development and not yet available in some languages. Check the spec-compliance matrix to see the implementation status in the corresponding language.

Evaluation event

The table below indicates which attributes should be added to the Event and their types.

Status:

The event name MUST be feature_flag.evaluation.

Defines feature flag evaluation as an event.

A feature_flag.evaluation event SHOULD be emitted whenever a feature flag value is evaluated, which may happen many times over the course of an application lifecycle. For example, a website A/B testing different animations may evaluate a flag each time a button is clicked. A feature_flag.evaluation event is emitted on each evaluation even if the result is the same.

Attributes:

Key	Stability	Requirement Level	Value Type	Description	Example Values
feature_flag.key		Required	string	The lookup key of the feature flag.	logo-color
error.type		Conditionally Required [1]	string	Describes a class of error the operation ended with. [2]	provider_not_ready; targeting_key_missing; provider_fatal; general
feature_flag.result.value		Conditionally Required [3]	any	The evaluated value of the feature flag. [4]	#ff0000; true; 3
feature_flag.result.variant		Conditionally Required [5]	string	A semantic identifier for an evaluated flag value. [6]	red; true; on
error.message		Recommended [7]	string	A message providing more detail about an error in human-readable form. [8]	Unexpected input type: string; The user has exceeded their storage quota
feature_flag.context.id		Recommended	string	The unique identifier for the flag evaluation context. For example, the targeting key.	5157782b-2203-4c80-a857-dbbd5e7761db
feature_flag.provider.name		Recommended	string	Identifies the feature flag provider.	Flag Manager
feature_flag.result.reason		Recommended	string	The reason code which shows how a feature flag value was determined.	static; targeting_match; error; default
feature_flag.set.id		Recommended	string	The identifier of the flag set to which the feature flag belongs.	proj-1; ab98sgs; service1/dev
feature_flag.version		Recommended	string	The version of the ruleset used during the evaluation. This may be any stable value which uniquely identifies the ruleset.	1; 01ABCDEF

[1] error.type: If and only if an error occurred during flag evaluation.

[2] error.type: If one of these values applies, then it MUST be used; otherwise, a custom value MAY be used.

Value	Description	Stability
flag_not_found	The flag could not be found.
invalid_context	The evaluation context does not meet provider requirements.
parse_error	An error was encountered parsing data, such as a flag configuration.
provider_fatal	The provider has entered an irrecoverable error state.
provider_not_ready	The value was resolved before the provider was initialized.
targeting_key_missing	The provider requires a targeting key and one was not provided in the evaluation context.
type_mismatch	The type of the flag value does not match the expected type.
general	The error was for a reason not enumerated above.

[3] feature_flag.result.value: If and only if feature flag provider does not supply variant or equivalent concept. Otherwise, feature_flag.result.value should be treated as opt-in.

[4] feature_flag.result.value: With some feature flag providers, feature flag results can be quite large or contain private or sensitive details. Because of this, feature_flag.result.variant is often the preferred attribute if it is available.

It may be desirable to redact or otherwise limit the size and scope of feature_flag.result.value if possible. Because the evaluated flag value is unstructured and may be any type, it is left to the instrumentation author to determine how best to achieve this.

[5] feature_flag.result.variant: If feature flag provider supplies a variant or equivalent concept.

[6] feature_flag.result.variant: A semantic identifier, commonly referred to as a variant, provides a means for referring to a value without including the value itself. This can provide additional context for understanding the meaning behind a value. For example, the variant red maybe be used for the value #c05543.

[7] error.message: If and only if an error occurred during flag evaluation and error.type does not sufficiently describe the error.

[8] error.message: Should not simply duplicate the value of error.type, but should provide more context. For example, if error.type is invalid_context the error.message may enumerate which context keys are missing or invalid.

error.type has the following list of well-known values. If one of them applies, then the respective value MUST be used; otherwise, a custom value MAY be used.

Value	Description	Stability
_OTHER	A fallback error value to be used when the instrumentation doesn’t define a custom value.

feature_flag.result.reason has the following list of well-known values. If one of them applies, then the respective value MUST be used; otherwise, a custom value MAY be used.

Value	Description	Stability
cached	The resolved value was retrieved from cache.
default	The resolved value fell back to a pre-configured value (no dynamic evaluation occurred or dynamic evaluation yielded no result).
disabled	The resolved value was the result of the flag being disabled in the management system.
error	The resolved value was the result of an error.
split	The resolved value was the result of pseudorandom assignment.
stale	The resolved value is non-authoritative or possibly out of date
static	The resolved value is static (no dynamic evaluation).
targeting_match	The resolved value was the result of a dynamic evaluation, such as a rule or specific user-targeting.
unknown	The reason for the resolved value could not be determined.