Skip to content

fix(groq): make _combine_llm_outputs None-safe and non-mutating#38759

Open
Andrei Boldyrev (abcgco) wants to merge 4 commits into
langchain-ai:masterfrom
abcgco:fix-groq-combine-llm-outputs
Open

fix(groq): make _combine_llm_outputs None-safe and non-mutating#38759
Andrei Boldyrev (abcgco) wants to merge 4 commits into
langchain-ai:masterfrom
abcgco:fix-groq-combine-llm-outputs

Conversation

@abcgco

Copy link
Copy Markdown
Contributor

Description:

ChatGroq._combine_llm_outputs merged nested token_usage dicts by storing the first generation's nested dict by reference and +=-ing into it — mutating the caller's llm_output, so callbacks/tracing for generation 0 report inflated nested counts (e.g. cached_tokens). A later None clobbered accumulated numbers, and None followed by a number raised TypeError.

This replaces the merge with a recursive _update_token_usage aligned with the pattern merged for Fireworks in #38751: rebuilds dicts instead of mutating, skips None values, preserves accumulator-only nested keys, seeds first-seen nested dict nodes so they merge instead of colliding with a numeric default, hardens token_usage access with .get(), logs-and-passes-through unexpected leaf types, and types the helper via a recursive TokenUsageTree alias.

Groq deltas from the Fireworks version: numeric leaves are int | float — Groq ships float timings such as queue_time/prompt_time inside token_usage (see the response samples in this module's docstrings) — and type mismatches raise TypeError per this package's ruff TRY004 (Fireworks' config doesn't enforce it).

Issue: Fixes #38659

Dependencies: none

Tests: parametrized None/float handling, an immutability guard (source llm_output unchanged, combined dict not aliased), accumulator-key preservation, first-seen nested-dict seeding, parametrized mismatch raises, unexpected-leaf warn-and-pass-through. pytest tests/unit_tests: 78 passed; ruff check/format, mypy, and lint_imports clean.

The merge loop stored the first occurrence of a nested usage dict by
reference and then mutated it in place, corrupting the first
generation's llm_output before on_llm_end callbacks observe it. Its
None-guard was also inverted: a later None silently clobbered an
accumulated value, and None followed by a number raised TypeError.
Replace with a recursive _update_token_usage (langchain-openai
pattern, extended to floats for Groq's timing fields) and skip None
values.
Mirror the shape merged in langchain-ai#38751: preserve accumulator-only nested keys,
seed first-seen nested dict nodes so they merge instead of colliding with
a numeric default, harden token_usage access with .get(), log and pass
through unexpected leaf types instead of raising, and type the helper via
a recursive TokenUsageTree alias.

Groq-specific deltas from the fireworks version: numeric leaves stay
int | float (Groq ships float timings such as queue_time / prompt_time),
and type mismatches raise TypeError per this package's ruff TRY004.
…llm-outputs

# Conflicts:
#	libs/partners/groq/langchain_groq/chat_models.py
#	libs/partners/groq/tests/unit_tests/test_chat_models.py
@github-actions github-actions Bot added fix For PRs that implement a fix groq `langchain-groq` package issues & PRs integration PR made that is related to a provider partner package integration size: M 200-499 LOC labels Jul 9, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

external fix For PRs that implement a fix groq `langchain-groq` package issues & PRs integration PR made that is related to a provider partner package integration new-contributor size: M 200-499 LOC

Projects

None yet

Development

Successfully merging this pull request may close these issues.

ChatGroq._combine_llm_outputs mutates per-generation token_usage in place and mishandles None values (silent clobber / TypeError)

1 participant