Skip to content

feat(groq): native content-block streaming events#118

Open
LennyMalcolm0 wants to merge 3 commits into
masterfrom
devasign-clone/pr-38029
Open

feat(groq): native content-block streaming events#118
LennyMalcolm0 wants to merge 3 commits into
masterfrom
devasign-clone/pr-38029

Conversation

@LennyMalcolm0

@LennyMalcolm0 LennyMalcolm0 commented Jul 14, 2026

Copy link
Copy Markdown
Owner

Stacked on #124 (core BlockStreamTracker + Anthropic), which it branches from. Depends only on the core primitive — sibling of the OpenAI chain. Retargets to master once #124 lands.

Gives ChatGroq native _stream_chat_model_events / _astream_chat_model_events hooks so stream_events(version="v3") maps groq's stream directly to content-block events.

How it works

A bespoke converter (convert_groq_stream / async twin) builds text, reasoning, and tool-call blocks directly from groq's raw OpenAI-shaped delta, feeding the shared BlockStreamTracker. groq streams tool-call arguments incrementally (id+name on the first delta, fragments on later deltas at the same index), so the tracker accumulates them and finalizes to a parsed dict. Usage is read from x_groq.usage; the arguments == "null" quirk is normalized to "{}".

It deliberately does not depend on langchain-openai — it reuses only groq's own _create_usage_metadata.

The win over the bridge

groq's existing chunk parser puts tool_calls and reasoning in additional_kwargs, so the compat bridge surfaces neither as content blocks. The native converter surfaces both — tool calls as tool_call blocks and reasoning as reasoning blocks.

Worth careful review

  • Tool-arg accumulation across deltas (keyed by tool:{index}); a test covers parallel tool calls at distinct indices.
  • message_id is keyword-only on the hooks, kept out of the request kwargs; message-start carries core's LangChain run id.
  • Scope cut: groq server-tool results (executed_tools) are not yet surfaced as blocks (noted in-code); the bridge still carried them in additional_kwargs.
  • Sync/async are faithful twins.

Cloned from langchain-ai#38029 for DevAsign stress testing.

@devasign-test-app

devasign-test-app Bot commented Jul 16, 2026

Copy link
Copy Markdown

AI review: changes requested

The pull request successfully implements native content-block streaming events for Groq by adding sync and async hooks to ChatGroq. It introduces convert_groq_stream and its async twin to parse raw OpenAI-shaped deltas directly into text, reasoning, and tool-call blocks using the shared BlockStreamTracker. All acceptance criteria are fully met with comprehensive unit tests covering parallel tool calls and argument accumulation.

Acceptance criteria

Met

  • ChatGroq implements both _stream_chat_model_events and _astream_chat_model_events hooks to natively support stream_events(version="v3").
  • Implement convert_groq_stream and its asynchronous counterpart to build text, reasoning, and tool-call blocks directly from Groq's raw OpenAI-shaped delta, feeding the shared BlockStreamTracker.
  • Accumulate tool-call arguments incrementally across deltas using index-based keying (e.g., tool:{index}) and finalize them to a parsed dictionary.
  • Normalize the Groq quirk where tool arguments are returned as the string "null" to an empty JSON object string "{}".
  • Extract usage information from x_groq.usage and reuse Groq's own _create_usage_metadata helper without introducing a dependency on langchain-openai.
  • Ensure message_id is a keyword-only argument on both _stream_chat_model_events and _astream_chat_model_events and is excluded from the request kwargs sent to the Groq API.
  • Ensure the message-start event carries the core LangChain run ID.
  • Include unit tests verifying parallel tool calls at distinct indices and tool-argument accumulation across deltas.

Additional findings

Blockers

  • Critical error (libs/partners/groq/langchain_groq/chat_models.py)
    In libs/partners/groq/langchain_groq/chat_models.py, the native streaming event hooks call self.client.create(...) (line 788) and self.async_client.create(...) (line 812). However, the Groq SDK client (groq.Groq / groq.AsyncGroq) does not expose a create method directly on the client instance; chat completions must be created via client.chat.completions.create. This will cause an immediate AttributeError and crash at runtime when native streaming events are requested.

Warnings

  • Deferred / incomplete work (libs/partners/groq/langchain_groq/_stream_events.py)
    Incidental: In libs/partners/groq/langchain_groq/_stream_events.py, the code notes that "groq server-tool results (delta["executed_tools"]) are not surfaced as blocks yet; tracked for a follow-up."

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants