Recommend JSON object encoding of an attribute for non-OTLP

[pellared]

What

Adds in-development guidance for representing a single OpenTelemetry attribute as a string in non-OTLP protocols.

The recommended form is a single-entry JSON object, for example:

{"http.request.method":"GET"}

Implementation:

• [DO NOT MERGE] attribute: add String method for KeyValue opentelemetry-go#8205 in OTel Go we are using non-OTLP string represnetation for debugging

Why

• Support String() method on attribute.Value and attribute.KeyValue opentelemetry-go#7810

The narrower purpose is to fill a gap in the existing non-OTLP string representation guidance. The specification already has stable guidance for representing AnyValue values as strings when a non-OTLP protocol cannot represent some AnyValue types natively. What is missing is the corresponding representation for a single Attribute, which is a key plus an AnyValue. This PR proposes that the single attribute is represented as a single-member JSON object, with the member value following the existing non-OTLP AnyValue representation rules.

I originally thought about key:value (and even key=value beforehand), but it is ambiguous when : appears in keys or string values and is not a robust debugging representation.

Using a JSON object improves clarity for debugging output by:

• aligning with existing non-OTLP, JSON-based guidance for complex AnyValue types
• making the representation easier to read and copy when needed

This also is consistent with how OTel Java, .NET, Python, Go are representing attributes in standard output exporters. Therefore, I think it would be most familiar representation for the users.

Why not OTLP/JSON?
For the OTel Go use case, the compact (lossy) non-OTLP representation is practical for human-readable output such as test failures. String() is intended for debugging and logging, and its output is not expected to be lossless nor suitable for deserialization. A canonical OTLP JSON KeyValue would need wrapper fields such as {"key":"http.request.method","value":{"stringValue":"GET"}}, which is much more verbose than {"http.request.method":"GET"} and harder to scan in failure messages.
Moreover, OTel Go attribute.Value.String() already follows the non-OTLP AnyValue rules: open-telemetry/opentelemetry-go#8142 (as OTel Go maintainers, we want to use an OTel-wide representation even for debugging output). We want attribute.KeyValue.String() to follow the same rules, but the string representation of a single attribute is not yet specified; this PR fills that gap and is currently blocking open-telemetry/opentelemetry-go#8205.

[jack-berg]

non-OTLP or debugging contexts.

Can you clarify what this applies to? I can think of a couple of potential candidates:

• Zipkin and other non-OTLP exporters which need to encode AnyValue types
• Console exporter which writes to console/stout in an unspecified format
• Generic "toString" representation called when you invoke printf or equivalent

Is this what you had in mind? Any other contexts you know of at the moment?

[pellared]

@jack-berg, mostly

Generic "toString" representation called when you invoke printf or equivalent

I thought that maybe some exporters may need a representation of a single attribute (we already have a defintion how to encode a map of AnyValue which is equivalent to a collection of attributes). This is why I decided to also include "For non-OTLP protocols".

[tigrannajaryan]

I am not sure I understand the purpose of this PR. Where is this representation is supposed to be used?

For Zipkip for example we have specific guideline on how to represent attributes: https://github.com/open-telemetry/opentelemetry-specification/blob/main/specification/trace/sdk_exporters/zipkin.md#attribute

If this is just for debugging purposes I why don't we use canonical OTLP JSON representation?

[pellared]

@tigrannajaryan, thanks, that is a fair question. I do not think this PR should be understood as redefining Zipkin, Prometheus, or other protocol-specific mappings, and I also do not think it should require every debugging exporter to use this exact output.

I am not sure I understand the purpose of this PR. Where is this representation is supposed to be used?

The narrower purpose is to fill a gap in the existing non-OTLP string representation guidance. The specification already has stable guidance for representing AnyValue values as strings when a non-OTLP protocol cannot represent some AnyValue types natively. What is missing is the corresponding representation for a single Attribute, which is a key plus an AnyValue. This PR proposes that the single attribute is represented as a single-member JSON object, with the member value following the existing non-OTLP AnyValue representation rules.

For Zipkip for example we have specific guideline on how to represent attributes: https://github.com/open-telemetry/opentelemetry-specification/blob/main/specification/trace/sdk_exporters/zipkin.md#attribute

For protocols with their own mapping, that mapping remains authoritative. Zipkin is a good example: the Zipkin exporter spec already says that OpenTelemetry span attributes are reported as Zipkin tags and defines its own conversion rules. This PR should not supersede that. I also had a shorter reply here: #5028 (comment)

If this is just for debugging purposes I why don't we use canonical OTLP JSON representation?

On "why not canonical OTLP JSON": OTLP JSON is the right answer when the goal is to emit OTLP. For example, the File Exporter spec explicitly serializes telemetry data using OTLP JSON. However, the existing and stable non-OTLP AnyValue representation is not the OTLP JSON representation. For example, top-level strings are represented as-is, byte arrays are base64-encoded, arrays and maps use JSON arrays/objects, and the representation is documented as lossy.
For the OTel Go use case, the compact non-OTLP representation is practical for human-readable output such as test failures. String() is intended for debugging and logging, and its output is not expected to be lossless nor suitable for deserialization. A canonical OTLP JSON KeyValue would need wrapper fields such as {"key":"http.request.method","value":{"stringValue":"GET"}}, which is much more verbose than {"http.request.method":"GET"} and harder to scan in failure messages.
Moreover, OTel Go attribute.Value.String() already follows the non-OTLP AnyValue rules: open-telemetry/opentelemetry-go#8142. We want attribute.KeyValue.String() to follow the same rules, but the string representation of a single attribute is not yet specified; this PR fills that gap and is currently blocking open-telemetry/opentelemetry-go#8205.

I think it is worth noting that, PR #4848 added that guidance, and the stable Prometheus/OpenMetrics Metric Attributes section requires non-string attribute values to follow it. That Prometheus mapping was stabilized in #4963 on March 18. If the concern is that this representation is fundamentally problematic, I would expect the consequences to be much more significant for the Prometheus exporter, where this is part of a stable exporter contract, than for human-readable debugging messages. Yet, I am not aware of reported issues with the non-OTLP AnyValue representation so far. CC @open-telemetry/prometheus-interoperability

I updated the PR description.

[tigrannajaryan]

The specification already has stable guidance for representing AnyValue values as strings when a non-OTLP protocol cannot represent some AnyValue types natively.

We have this in spec precisely because conversion to formats like Prometheus need well-defined conversion rules that are followed by all implementations. This is important for interoperability and definitely belongs in the spec.

Debugging output format on the other hand has no such interoperability requirements, since it is for humans to read and it doesn't have the same need for strict normative specification as the data formats or network protocols have.

If we want to have a centralized Otel-wide format for debugging output I would put it in a separate non-normative document, as a guidance to implementations, but not as strict requirement.

If there are other use cases besides debugging output please list them.

[pellared]

Debugging output format on the other hand has no such interoperability requirements, since it is for humans to read and it doesn't have the same need for strict normative specification as the data formats or network protocols have.

Yes, I agree with that, and that was the distinction I was trying to make earlier.

Also, based on the earlier comments about lossy representation and type information, should we be concerned about the existing Prometheus/OpenMetrics interoperability guidance that already depends on the stable non-OTLP AnyValue representation? If so then we should track this in a separate issue.

If we want to have a centralized Otel-wide format for debugging output I would put it in a separate non-normative document, as a guidance to implementations, but not as strict requirement.

As OTel Go maintainers, we want to use an OTel-wide representation for debugging output. I agree that debugging guidance itself can live in a separate non-normative document and this is not the concern of this PR (thus #5028 (comment)). However, the document should refer to a stable representation defined by the spec, rather than define the representation itself. In other words, I would prefer this layering:

1. the spec defines a stable non-OTLP representation for a single Attribute, extending the existing stable AnyValue representation (this PR);
2. a non-normative debugging document can then recommend using that representation where it is useful (next PR). EDIT: I think this may use normative language e.g. "implementations MAY use this representation for debugging purposes e.g. as the format used by toString methods"

So the immediate use case is OTel Go debugging/logging output, but the reason for this PR is to define the representation that such guidance can point to.

[jack-berg]

@trask do you see anything blocking in here?

[jack-berg]

To me this seems duplicative of the "#### Maps" section on line 168. It already describes how to encode using what I believe is the same output.

An attribute set is analogous to a map. A single attribute is analogous to a single entry map.

Rather than repeating that section, could we make the (obvious) analogy between attributes and maps, and link over to the existing maps section for how it should be represented?

[pellared]

A single attribute is analogous to a single entry map.

Indeed. And this is what this section is specifying.

Rather than repeating that section, could we make the (obvious) analogy between attributes and maps, and link over to the existing maps section for how it should be represented?

I am not repeating the whole section. Note that this PR is already linking to the maps section.

Also note that e.g. https://github.com/open-telemetry/opentelemetry-specification/blob/main/specification/common/README.md#maps does not tell anything about the attribute name/value. I did my best to refer to the existing specification while also adding the missing parts. This is why this PR has e.g.

The attribute key SHOULD be represented as a JSON object member
name.

Other reviewers also asked to have more examples and have this section to be more clear (even it if is more verbose).

An attribute set is analogous to a map.

It is not specified anywhere. It a subject for a separate PR as we would need it for open-telemetry/opentelemetry-go#8209. I do not want to open large PRs and I plan to create it once this PR lands.

[jack-berg]

Fair enough - I approve.