Skip to content

docs(otel): document remappings #2989

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
nimarb wants to merge 1 commit into
base: main
Choose a base branch
from
Open

docs(otel): document remappings #2989

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 1 addition & 0 deletions content/docs/observability/features/meta.json
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -14,6 +14,7 @@
"user-feedback",
"log-levels",
"agent-graphs",
"tool-calls",
"masking",
"mcp-tracing",
"multi-modality",
Expand Down
61 changes: 61 additions & 0 deletions content/docs/observability/features/tool-calls.mdx
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,61 @@
---
description: Langfuse automatically renders tool definitions and tool calls from your agent traces, making it easy to debug function calling.
sidebarTitle: Tool Calls
---

Check failure on line 4 in content/docs/observability/features/tool-calls.mdx

View check run for this annotation

Claude / Claude Code Review

Missing required `title:` in frontmatter 

Missing required `title:` field in this new file's frontmatter — it's the only file in all of `content/docs/` that omits it. The Fumadocs frontmatter schema (`baseFrontmatterSchema` → `frontmatterSchema` in `source.config.ts`) declares `title` as a required `z.string()`, so the MDX collection build will fail with a Zod `ValidationError`; even if it didn't, `lib/mdx-page.ts` `buildSectionMetadata` would crash because `enrichOgTitle` calls `title.toLowerCase()` on `undefined`. Fix: add e.g. `title
Comment on lines +1 to +4
Copy link
Copy Markdown

@chatgpt-codex-connector chatgpt-codex-connector Bot May 22, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P0 Badge Add required title frontmatter to tool-calls docs page

This new docs file omits the required title frontmatter key, which causes the docs loader to fail during build ([MDX] invalid frontmatter ... title: expected string, received undefined when running pnpm next build). Because this page is part of the content/docs collection, the missing field breaks site compilation rather than just degrading this one page.

Useful? React with 👍 / 👎.

Comment on lines +1 to +4
Copy link
Copy Markdown

@claude claude Bot May 22, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🔴 Missing required title: field in this new file's frontmatter — it's the only file in all of content/docs/ that omits it. The Fumadocs frontmatter schema (baseFrontmatterSchema → frontmatterSchema in source.config.ts) declares title as a required z.string(), so the MDX collection build will fail with a Zod ValidationError; even if it didn't, lib/mdx-page.ts buildSectionMetadata would crash because enrichOgTitle calls title.toLowerCase() on undefined. Fix: add e.g. title: Tool Calls (or title: Tool Call Visualization to match the H1) on a new line in the frontmatter.

Extended reasoning...

What the bug is

The new content/docs/observability/features/tool-calls.mdx file has only description: and sidebarTitle: in its frontmatter — no title:. A scan of every .mdx under content/docs confirms it is the only file out of ~99 docs without title:, and the only one out of 20 sibling features/*.mdx pages without it. So it is at minimum a glaring convention violation, but more importantly it breaks two real code paths.

Code path 1: MDX collection build fails on frontmatter validation

source.config.ts:26 defines:

const baseFrontmatterSchema = frontmatterSchema.extend({ ... });

frontmatterSchema is imported from fumadocs-mdx/config. In fumadocs-mdx 14.2.7 (per the package.json pin ^14.2.7), this is pageSchema = z.object({ title: z.string(), description: z.string().optional(), ... }) — title is required, not optional. The docs collection uses sidebarFrontmatterSchema = baseFrontmatterSchema.extend({ shortTitle, sidebarTitle }) (source.config.ts:80), and z.extend preserves the required-ness of title. fumadocs-mdx's compile path then calls validate(collection.schema, data, …) and throws a ValidationError("invalid frontmatter in …") on missing fields, so pnpm build (or the MDX collection step) will fail with something like invalid frontmatter in .../tool-calls.mdx: title: Required.

Code path 2: Metadata generation crashes

Even if validation were lenient, lib/mdx-page.ts:119 reads:

const seoTitle = pageData.seoTitle || page.data.title;
const ogTitle = pageData.seoTitle ? seoTitle : enrichOgTitle(seoTitle, slug, sectionTitle);

With no title and no seoTitle, seoTitle is undefined and we enter enrichOgTitle at lib/mdx-page.ts:84:

const lower = title.toLowerCase().trim();

That throws TypeError: Cannot read properties of undefined (reading 'toLowerCase') while rendering /docs/observability/features/tool-calls. Note sidebarTitle: Tool Calls doesn't help — lib/source.ts's shortTitleTransformer consumes sidebarTitle only for the sidebar label, never as a title fallback for metadata.

Step-by-step proof (what happens if you build right now)

  1. fumadocs-mdx scans content/docs and loads each .mdx frontmatter.
  2. It calls sidebarFrontmatterSchema.parse({ description: "…", sidebarTitle: "Tool Calls" }) for this file (no title key).
  3. title: z.string() from pageSchema fails with { code: "invalid_type", expected: "string", received: "undefined", path: ["title"] }.
  4. fumadocs-mdx throws ValidationError("invalid frontmatter in content/docs/observability/features/tool-calls.mdx: title: Required"). Build aborts.
  5. Hypothetically bypassing Signup for product updates via email #4: rendering /docs/observability/features/tool-calls calls buildSectionMetadata, seoTitle = undefined || undefined = undefined, enrichOgTitle(undefined, …) → undefined.toLowerCase() → TypeError. Page 500s.

How to fix

One-line change in the frontmatter:

---
title: Tool Calls
description: Langfuse automatically renders tool definitions and tool calls from your agent traces, making it easy to debug function calling.
sidebarTitle: Tool Calls
---

Or title: Tool Call Visualization to mirror the H1. The same fix should be applied to the duplicate pages/docs/observability/features/tool-calls.mdx file (or that file should be deleted — both copies currently lack title:).


import { GhDiscussionsPreview } from "@/components/gh-discussions/GhDiscussionsPreview";

# Tool Call Visualization

Tool calls are the heartbeat of agents. Langfuse automatically renders all tools available to an LLM at the top of each generation, allowing you to instantly see if the LLM selected the right one.

<Frame fullWidth>
![Tool call visualization](/images/changelog/2025-11-05-langfuse-for-agents/lw4-d3-tools.png)
</Frame>

## What's Displayed

**Tool Definitions**: All tools available to the LLM during that interaction:

- Tool name and call status
- Call count badge ("called", "called 2x", "not called")
- Expandable description and parameter schema
- Toggle between formatted and JSON views

**Tool Calls in Chat**: Called tools appear with their arguments and call IDs. Numbering matches the available tools list for easy validation.

**Automatic Sorting**: Called tools appear first, followed by unused tools.

## Supported Frameworks

Tool visualization works automatically with:

- OpenAI (Chat Completions & Responses API)
- Google Gemini / Vertex AI
- Vercel AI SDK
- LangGraph / LangChain
- Pydantic AI
- Microsoft Agent Framework

No changes to your instrumentation code required. If you're using tools with a supported framework, visualization appears automatically.

## How Tools Are Detected

Langfuse detects **available tools** from supported framework and OpenTelemetry metadata, normalizes them into the generation input, and renders them at the top of each generation. Langfuse detects **called tools** from the model output and renders them inline in the chat view with their arguments and call IDs.

For the exact OpenTelemetry and framework metadata shapes that are remapped into `input.tools`, `tool_definitions`, `tool_calls`, and `tool_call_names`, see [Tool definitions and tool calls](/integrations/native/opentelemetry#tool-definitions-and-tool-calls) in the OpenTelemetry mapping docs.

## Why This Matters

1. **Faster debugging**: Instantly see if the right tools were available and called
2. **Better prompt engineering**: Understand which tool descriptions lead to actual usage
3. **Cost optimization**: Identify unused tools to reduce token usage

## Related

- [Agent Graphs](/docs/observability/features/agent-graphs)
- [Observation Types](/docs/observability/features/observation-types)

## GitHub Discussions

<GhDiscussionsPreview labels={["feat-tool-calls"]} />
15 changes: 15 additions & 0 deletions content/integrations/native/opentelemetry.mdx
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -442,6 +442,21 @@ These attributes are applied to individual observations (spans) within a trace (
| <a id="obs-version"></a>`version` | The [version](/docs/observability/features/releases-and-versioning) of the observation. | • `langfuse.version`: `string` |
| <a id="obs-environment"></a>`environment` | The deployment [environment](/docs/observability/features/environments) where the observation was generated. | • `langfuse.environment`<br/>• `deployment.environment`<br/>• `deployment.environment.name` |

### Tool definitions and tool calls [#tool-definitions-and-tool-calls]

Langfuse normalizes available tool definitions and called tool invocations into dedicated observation fields. Available tools are attached to the generation input as `input.tools`; named tools are also stored in `tool_definitions` for filtering, exports, and dashboards. Called tools are parsed from the generation output and stored in `tool_calls` and `tool_call_names`.

| Langfuse concept | Source attribute or shape | Mapping behavior |
| ------------------------ | --------------------------------------------------------------------------------------------------------------------------------------------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| Available tools | `gen_ai.tool.definitions` | Moves valid named tool definitions to `input.tools` and removes the duplicate metadata attribute. Each item must have a tool name, matching the OpenTelemetry GenAI tool definition schema. |
| Available tools | `ai.prompt.tools` | Moves Vercel AI SDK tool definitions to `input.tools` and removes the duplicate metadata attribute. Named tools are extracted into `tool_definitions`; provider tools without names, such as `{ type: "web_search_preview" }`, remain in `input.tools` for display. |
| Available tools | `model_request_parameters.function_tools` | Moves pydantic-ai function tools to `input.tools` and removes the nested `function_tools` key when valid. |
| Available tools | `llm.tools.<N>.tool.json_schema` | Moves valid indexed OpenInference-style tool schema attributes to `input.tools` in index order and removes the moved attributes. |
| Available tools fallback | Observation `metadata.tools` | Compatibility fallback for already-shaped tool metadata. Only arrays where every item is a named tool definition are moved; generic user metadata such as `{ id: "hammer" }` or `{ type: "manual" }` is preserved. |
| Called tools | `output.tool_calls`, `output.toolCalls`, `choices[].message.tool_calls`, message `tool_calls`, AI SDK `tool-call` parts, Anthropic `tool_use` parts | Extracts tool invocation payloads into `tool_calls` and tool names into `tool_call_names`. |

If a tool definition source is moved into `input.tools`, Langfuse removes the duplicate source metadata to keep observations clean. If the value does not look like a supported tool definition, Langfuse leaves it untouched in metadata.

<Callout type="info">
**Filtering by metadata key in Langfuse**

Expand Down
61 changes: 61 additions & 0 deletions pages/docs/observability/features/tool-calls.mdx
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,61 @@
---
description: Langfuse automatically renders tool definitions and tool calls from your agent traces, making it easy to debug function calling.
sidebarTitle: Tool Calls
---

# Tool Call Visualization

Tool calls are the heartbeat of agents. Langfuse automatically renders all tools available to an LLM at the top of each generation, allowing you to instantly see if the LLM selected the right one.

<Frame fullWidth>

Check failure on line 10 in pages/docs/observability/features/tool-calls.mdx

View check run for this annotation

Claude / Claude Code Review

Stray duplicate tool-calls.mdx under pages/ should be deleted 

This PR adds a stray duplicate `pages/docs/observability/features/tool-calls.mdx` alongside the canonical `content/docs/observability/features/tool-calls.mdx`. This site uses Fumadocs App Router (routes resolved from `content/docs` via `app/docs/[[...slug]]`) and `next.config.mjs` does not extend `pageExtensions` to `.mdx`, so the `pages/` file is never served — it is dead content that has already drifted from the canonical copy within this same PR. Please delete `pages/docs/observability/featur
Comment on lines +1 to +10
Copy link
Copy Markdown

@claude claude Bot May 22, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🔴 This PR adds a stray duplicate pages/docs/observability/features/tool-calls.mdx alongside the canonical content/docs/observability/features/tool-calls.mdx. This site uses Fumadocs App Router (routes resolved from content/docs via app/docs/[[...slug]]) and next.config.mjs does not extend pageExtensions to .mdx, so the pages/ file is never served — it is dead content that has already drifted from the canonical copy within this same PR. Please delete pages/docs/observability/features/tool-calls.mdx.

Extended reasoning...

What the bug is

This PR adds two copies of the same documentation page:

  • content/docs/observability/features/tool-calls.mdx — the canonical page, wired into the sidebar via content/docs/observability/features/meta.json
  • pages/docs/observability/features/tool-calls.mdx — a stray duplicate

The pages/ copy is the only file in the entire pages/ directory (git log --all -- pages/ shows only this PR's commit ad01125), and it is never rendered by the site.

Why it is dead

The repo migrated from Nextra to Fumadocs and now routes /docs/* exclusively through the Next.js App Router:

  1. source.config.ts declares defineDocs({ dir: 'content/docs', ... }).
  2. app/docs/[[...slug]]/page.tsx is a catch-all that resolves slugs via source.getPage() from the Fumadocs collection rooted at content/docs.
  3. next.config.mjs wires only createMDX() from fumadocs-mdx/next; it does not override pageExtensions, so Next.js's default ['tsx','ts','jsx','js'] applies. .mdx files in pages/ are not picked up as routes.
  4. scripts/authors/config.js even contains an explicit comment that the pages/ directory is retained only so the pre-migration git history can be walked for author attribution — not for routing.

So the pages/ file is invisible to users; the /docs/observability/features/tool-calls URL is served from the content/ copy.

Concrete proof of the drift hazard

The two files differ already, in the same PR that introduced them:

  • Line 38 of pages/.../tool-calls.mdx: No changes to your instrumentation code required—if you're using tools with a supported framework, visualization appears automatically. (em-dash, one sentence)
  • Line 40 of content/.../tool-calls.mdx: No changes to your instrumentation code required. If you're using tools with a supported framework, visualization appears automatically. (period, two sentences)
  • The GhDiscussionsPreview import statement is at the top of the content/ copy and at the bottom of the pages/ copy.

This is exactly the maintenance hazard the deletion avoids: future copy edits will land on only one of the two files, and the dead copy will silently bit-rot.

Step-by-step proof of impact

  1. A reader visits https://langfuse.com/docs/observability/features/tool-calls.
  2. Next.js App Router matches app/docs/[[...slug]]/page.tsx with slug = ['observability','features','tool-calls'].
  3. That page calls source.getPage(slug), which reads the Fumadocs content collection rooted at content/docs.
  4. The reader sees content/docs/observability/features/tool-calls.mdx — never the pages/ copy.
  5. A future contributor edits one copy (say, fixing a typo in the em-dash line). The other copy is now stale, but no build error fires because the dead file is never rendered or type-checked.

Fix

Delete pages/docs/observability/features/tool-calls.mdx. The canonical file in content/docs/observability/features/tool-calls.mdx already has the correct sidebar entry in meta.json and the correct import placement at the top.

![Tool call visualization](/images/changelog/2025-11-05-langfuse-for-agents/lw4-d3-tools.png)
</Frame>

## What's Displayed

**Tool Definitions**: All tools available to the LLM during that interaction:

- Tool name and call status
- Call count badge ("called", "called 2x", "not called")
- Expandable description and parameter schema
- Toggle between formatted and JSON views

**Tool Calls in Chat**: Called tools appear with their arguments and call IDs. Numbering matches the available tools list for easy validation.

**Automatic Sorting**: Called tools appear first, followed by unused tools.

## Supported Frameworks

Tool visualization works automatically with:

- OpenAI (Chat Completions & Responses API)
- Google Gemini / Vertex AI
- Vercel AI SDK
- LangGraph / LangChain
- Pydantic AI
- Microsoft Agent Framework

No changes to your instrumentation code required—if you're using tools with a supported framework, visualization appears automatically.

## How Tools Are Detected

Langfuse detects **available tools** from supported framework and OpenTelemetry metadata, normalizes them into the generation input, and renders them at the top of each generation. Langfuse detects **called tools** from the model output and renders them inline in the chat view with their arguments and call IDs.

For the exact OpenTelemetry and framework metadata shapes that are remapped into `input.tools`, `tool_definitions`, `tool_calls`, and `tool_call_names`, see [Tool definitions and tool calls](/integrations/native/opentelemetry#tool-definitions-and-tool-calls) in the OpenTelemetry mapping docs.

## Why This Matters

1. **Faster debugging**: Instantly see if the right tools were available and called
2. **Better prompt engineering**: Understand which tool descriptions lead to actual usage
3. **Cost optimization**: Identify unused tools to reduce token usage

## Related

- [Agent Graphs](/docs/observability/features/agent-graphs)
- [Observation Types](/docs/observability/features/observation-types)

## GitHub Discussions

import { GhDiscussionsPreview } from "@/components/gh-discussions/GhDiscussionsPreview";

<GhDiscussionsPreview labels={["feat-tool-calls"]} />
Loading