diff --git a/content/docs/observability/features/masking.mdx b/content/docs/observability/features/masking.mdx index 63466c3585..831a389292 100644 --- a/content/docs/observability/features/masking.mdx +++ b/content/docs/observability/features/masking.mdx @@ -16,16 +16,31 @@ Learn more about Langfuse's data security and privacy measures concerning the st ## How it works -1. You define a custom masking function and pass it to the Langfuse client constructor. -2. All event inputs, outputs, and metadata are processed through this function. -3. The masked data is then sent to the Langfuse server. - -This approach ensures that you have complete control over the event input, output, and metadata traced by your application. +Langfuse supports two client-side masking hooks. Choose the hook based on where +the data is created. + +| Hook | SDK | Use for | Important behavior | +| --- | --- | --- | --- | +| `mask` | Python, JS/TS | Data written through Langfuse SDK APIs, such as observation `input`, `output`, and `metadata` | Runs when Langfuse SDK data is recorded. It is the simplest option for data you pass directly to Langfuse. | +| `mask_otel_spans` | Python | Final OpenTelemetry span attributes before this Langfuse client exports them to Langfuse | Runs after span filtering and media handling. It is the right option for third-party OTEL instrumentation and final exported span attributes. | +| `should_export_span` / `shouldExportSpan` | Python, JS/TS | Dropping or keeping whole spans | Use this for span-level filtering. Do not use masking callbacks to drop spans. | + +`mask_otel_spans` only changes the copy of the OpenTelemetry spans exported by +the Langfuse Python SDK. It does not mutate the original OpenTelemetry span. If +the same span is also exported to a third-party observability backend, such as +Datadog, Honeycomb, Grafana Tempo, or an OpenTelemetry Collector, that exporter +receives its own unmodified span copy. + +Use `mask` when you control the data written through Langfuse SDK methods. Use +`mask_otel_spans` when sensitive data is emitted by third-party OpenTelemetry +instrumentation or when you need to inspect the final OTEL attributes that will +be sent to Langfuse. -Define a masking function. The masking function will apply to all event inputs, outputs, and metadata regardless of the Langfuse-maintained integration you are using. +Define a masking function. The `mask` function applies to event inputs, outputs, +and metadata written through Langfuse SDK APIs. ```python def masking_function(data: any, **kwargs) -> any: @@ -148,6 +163,98 @@ const handler = new CallbackHandler({ +## Mask OpenTelemetry span attributes in Python [#mask-otel-spans] + +Use `mask_otel_spans` when you need to redact OpenTelemetry spans before the +Langfuse Python SDK sends them to Langfuse. This is especially useful for spans +created by third-party instrumentations such as OpenInference, OpenLLMetry, +OpenLIT, LiteLLM, or provider-specific OTEL libraries. + +The callback receives one OpenTelemetry export batch. A batch is not guaranteed +to contain a complete trace or request. Return `None` to leave the batch +unchanged, or return sparse patches for the spans you want to change. + +```python +from typing import Optional + +from langfuse import Langfuse +from langfuse.types import ( + MaskOtelSpansParams, + MaskOtelSpansResult, + OtelSpanPatch, +) + +SENSITIVE_ATTRIBUTE_PREFIXES = ( + "gen_ai.prompt.", + "gen_ai.completion.", + "llm.input_messages.", + "llm.output_messages.", +) +SENSITIVE_ATTRIBUTE_KEYS = { + "gen_ai.prompt", + "gen_ai.completion", +} + + +def mask_otel_spans( + *, params: MaskOtelSpansParams +) -> Optional[MaskOtelSpansResult]: + patches = {} + + for identifier, span in params.spans.items(): + sensitive_keys = tuple( + key + for key in span.attributes + if key in SENSITIVE_ATTRIBUTE_KEYS + or key.startswith(SENSITIVE_ATTRIBUTE_PREFIXES) + ) + + if not sensitive_keys: + continue + + patches[identifier] = OtelSpanPatch( + delete_attributes=sensitive_keys, + set_attributes={"masking.applied": True}, + ) + + return MaskOtelSpansResult(span_patches=patches) + + +langfuse = Langfuse(mask_otel_spans=mask_otel_spans) +``` + +`mask_otel_spans` runs after `should_export_span` accepts a span and after +export-stage media handling converts supported media payloads into Langfuse +media references. The callback can: + +- Read span IDs, parent span ID, name, instrumentation scope, attributes, and resource attributes. +- Delete exact attribute keys. +- Set or replace OpenTelemetry-compatible attribute values. + +The callback cannot change span IDs, span names, parent relationships, resource +attributes, events, links, or instrumentation scope. + + + If `mask_otel_spans` raises an exception or returns an invalid batch result, + Langfuse drops the whole export batch. If one returned span patch is invalid, + Langfuse drops only that span from the Langfuse export. Keep the function + deterministic and add explicit fallback behavior. + + +### Using external PII services + +If you use an IO-bound PII detection or redaction service, `mask_otel_spans` is +usually the right place to call it for third-party OTEL span data. Normal batch +exports run outside the main application path, so this avoids blocking the code +that creates or ends spans. + +Keep the callback synchronous, bounded, and batch-oriented: + +- Batch candidate attributes from `params.spans` and call the PII service once per export batch where possible. +- Use strict network timeouts. +- Decide whether failures should drop the batch, delete sensitive attributes, or export the original values. +- Avoid request-local state, the current active span, and async-only APIs. During `flush()` or shutdown, the callback may run on the caller thread. + ## Examples Now, we'll show you examples how to use the masking feature. We'll use the Langfuse decorator for this, but you can also use the low-level SDK or the JS/TS SDK analogously. diff --git a/content/docs/observability/features/multi-modality.mdx b/content/docs/observability/features/multi-modality.mdx index 46b96834d6..4bbdb29340 100644 --- a/content/docs/observability/features/multi-modality.mdx +++ b/content/docs/observability/features/multi-modality.mdx @@ -10,6 +10,10 @@ Langfuse supports multi-modal traces including **text, images, audio, and other By default, **[base64 encoded data URIs](https://developer.mozilla.org/en-US/docs/Web/URI/Schemes/data#syntax) are handled automatically by the Langfuse SDKs**. They are extracted from the payloads commonly used in multi-modal LLMs, uploaded to Langfuse's object storage, and linked to the trace. +In the Python SDK, media handling also runs at export time for supported media +shapes found in third-party OpenTelemetry span attributes that are exported +through the Langfuse client. + This also works if you: 1. Reference media files via external URLs. @@ -54,6 +58,22 @@ This works with standard Data URI ([MDN](https://developer.mozilla.org/en-US/doc This [notebook](/guides/cookbook/example_multi_modal_traces) includes a couple of examples using the OpenAI SDK and LangChain. +For Python SDK exports, Langfuse can also detect media in supported +OpenTelemetry span attributes emitted by third-party instrumentation. Supported +export-stage shapes include: + +- Direct base64 data URI strings, such as `data:image/png;base64,...`. +- JSON string attributes that contain supported media hints. +- String sequence attributes. +- Anthropic-style objects with `type`, `media_type`, and `data`. +- Vertex-style objects with `type`, `mime_type`, and `data`. +- Google Gemini / Vertex `inline_data` and `inlineData` payloads. + +Export-stage media handling runs before +[`mask_otel_spans`](/docs/observability/features/masking#mask-otel-spans). If a +media payload is detected successfully, the masking callback sees the Langfuse +media reference token instead of the original base64 content. + ### External media (URLs) Langfuse supports in-line rendering of media files via URLs if they follow common formats. In this case, the media file is not uploaded to Langfuse's object storage but simply rendered in the UI directly from the source. @@ -112,7 +132,7 @@ from langfuse.media import LangfuseMedia # Create a LangfuseMedia object from a file with open("static/bitcoin.pdf", "rb") as pdf_file: -pdf_bytes = pdf_file.read() + pdf_bytes = pdf_file.read() # Wrap media in LangfuseMedia class @@ -153,7 +173,7 @@ with langfuse.start_as_current_observation(as_type="span", name="analyze-documen "original": pdf_media }) -```` +``` diff --git a/content/docs/observability/sdk/advanced-features.mdx b/content/docs/observability/sdk/advanced-features.mdx index aae4ff5deb..506a4cc4f0 100644 --- a/content/docs/observability/sdk/advanced-features.mdx +++ b/content/docs/observability/sdk/advanced-features.mdx @@ -163,12 +163,19 @@ You can read more about using Langfuse with an existing OpenTelemetry setup [her ## Mask sensitive data -If your trace data (inputs, outputs, metadata) might contain sensitive information (PII, secrets), you can provide a mask function during client initialization. This function will be applied to all relevant data before it’s sent to Langfuse. +If your trace data might contain sensitive information (PII, secrets), you can +provide a masking function before data is sent to Langfuse. Use the dedicated +[masking guide](/docs/observability/features/masking) for the full decision +tree across SDK-level masking, OpenTelemetry export-stage masking, and span +filtering. -The `mask` function should accept data as a keyword argument and return the masked data. The returned data must be JSON-serializable. +The `mask` function should accept data as a keyword argument and return the +masked data. The returned data must be JSON-serializable. Use this for data +written through Langfuse SDK APIs such as observation `input`, `output`, and +`metadata`. ```python @@ -186,6 +193,12 @@ def pii_masker(data: any, **kwargs) -> any: langfuse = Langfuse(mask=pii_masker) ``` + +For spans emitted by third-party OpenTelemetry instrumentation, use +`mask_otel_spans` instead. It runs on the final span attributes exported by the +Langfuse Python SDK, after span filtering and media handling, and only affects +the spans sent to Langfuse. See +[Mask OpenTelemetry span attributes in Python](/docs/observability/features/masking#mask-otel-spans). diff --git a/content/integrations/native/opentelemetry.mdx b/content/integrations/native/opentelemetry.mdx index 55654558ed..f709f08b83 100644 --- a/content/integrations/native/opentelemetry.mdx +++ b/content/integrations/native/opentelemetry.mdx @@ -116,6 +116,15 @@ The quickest path to start tracing with Langfuse is the new **OTEL-native Langfu Because it lives in the shared OpenTelemetry context, spans from other OTEL-instrumented libraries can be exported to Langfuse too. By default, Langfuse focuses on LLM-relevant spans (Langfuse SDK spans, spans with `gen_ai.*` attributes, and known LLM instrumentors). To export everything, use a permissive custom filter as described in the [advanced SDK docs](/docs/observability/sdk/advanced-features#filtering-by-instrumentation-scope). + + Python SDK export hooks such as + [`mask_otel_spans`](/docs/observability/features/masking#mask-otel-spans) + and export-stage media handling run only when spans are exported through the + Langfuse Python SDK. Spans sent directly to `/api/public/otel` through a + collector or raw OTLP exporter do not run Langfuse SDK masking or media upload + logic. + + Get started by following the dedicated guide for the Python implementation here: [/docs/observability/sdk/overview](/docs/observability/sdk/overview). ### OpenTelemetry endpoint