Feature: Add Z.AI (Zhipu AI) provider support

[Societus]

Feature: Add Z.AI (Zhipu AI) provider support

Summary

Add first-class support for Z.AI (Zhipu AI) as an LLM provider. Z.AI offers OpenAI-compatible APIs with two plan types and several capable models including the GLM-5 family.

Motivation

Z.AI provides competitive models (GLM-5-turbo, GLM-5.1, GLM-4.7) at accessible pricing through both subscription and pay-as-you-go plans. Their API is OpenAI-compatible, making integration straightforward. However, the GLM-5 family models are reasoning models with thinking enabled by default, which causes a significant problem for repowise: reasoning tokens consume the entire output budget, leaving no tokens for the actual response content. This means Z.AI support requires handling the thinking toggle as a first-class concern, not just wiring up another OpenAI-compatible endpoint.

API Details

Endpoints

Z.AI has two API plans with different base URLs:

Plan	Base URL	Billing
Coding	https://api.z.ai/api/coding/paas/v4	Subscription (resource package)
General	https://api.z.ai/api/paas/v4	Pay-as-you-go

Both endpoints use identical request/response formats. The difference is purely billing -- which models are available and how you're charged.

Note: Based on how other tools handle this (e.g., separate provider entries for different OpenRouter plans or Anthropic AWS vs direct), the maintainers may prefer to split these into two separate provider entries (e.g., zai-coding and zai) rather than a single configurable provider. I'll leave that decision to you.

Available Models

Tested and confirmed working (coding plan):

• glm-5-turbo
• glm-5.1
• glm-5
• glm-4.7

Authentication

Bearer token via Authorization header. The API is OpenAI-compatible and works with the openai Python SDK's AsyncOpenAI client.

Reasoning Model Handling (Critical)

The GLM-5 family has reasoning/thinking enabled by default. This means every token in the output budget goes to chain-of-thought reasoning first, and content tokens only appear after reasoning completes. For repowise's structured generation prompts, this wastes 85-95% of the token budget on invisible thinking.

The fix: Z.AI supports disabling thinking via the thinking parameter in the request body:

{
  "model": "glm-5.1",
  "messages": [...],
  "thinking": { "type": "disabled" }
}

This produces clean, direct output with zero reasoning token overhead:

$ curl https://api.z.ai/api/coding/paas/v4/chat/completions \
  -H "Authorization: Bearer $KEY" \
  -d '{"model":"glm-5.1","messages":[{"role":"user","content":"Explain DHT in one sentence."}],"max_tokens":200,"thinking":{"type":"disabled"}}'

# Response: 35 completion tokens, 0 reasoning tokens
# "A Distributed Hash Table (DHT) is a decentralized storage system..."

Without thinking.type=disabled, the same request produces 469 reasoning tokens and only 37 content tokens -- a 92% overhead.

Note: Other Z.AI model families (GLM-4, self-hosted via vLLM/SGLang) use a different parameter: chat_template_kwargs: { enable_thinking: false }. The provider should default to the thinking parameter style since that's what the Z.AI hosted API expects, but could potentially support both.

Proposed Implementation

Architecture

class ZAIProvider(BaseProvider):
    """Z.AI (Zhipu AI) chat provider.

    Uses the OpenAI-compatible API with thinking disabled by default
    for efficient structured output generation.
    """

    _CODING_BASE_URL = "https://api.z.ai/api/coding/paas/v4"
    _GENERAL_BASE_URL = "https://api.z.ai/api/paas/v4"
    _DEFAULT_MODEL = "glm-5.1"

Environment Variables

Variable	Purpose	Default
ZAI_API_KEY	API key for authentication	(required)
ZAI_BASE_URL	Override API base URL	https://api.z.ai/api/coding/paas/v4
ZAI_THINKING	Enable/disable thinking	disabled

Files to Create/Modify

• repowise/core/providers/llm/zai.py -- New provider class
• repowise/core/providers/llm/registry.py -- Register "zai" provider
• repowise/core/providers/llm/__init__.py -- Update docstring
• repowise/core/rate_limiter.py -- Add PROVIDER_DEFAULTS["zai"]
• repowise/cli/helpers.py -- Add ZAI_API_KEY auto-detection
• repowise/server/provider_config.py -- Add to server provider catalog (if applicable)

Key Design Decisions

1. Thinking disabled by default. The primary use case for repowise is structured generation, where reasoning tokens are pure waste. The provider should default to thinking: { type: "disabled" } on every request.

2. Configurable thinking mode. Allow users to enable thinking via ZAI_THINKING=enabled if they want to use a reasoning model's full capability (e.g., for complex architectural decision extraction).

3. Uses openai package dependency. Same as the ollama provider -- no new dependencies needed since Z.AI's API is OpenAI-compatible.

4. extra_body for non-standard params. The thinking parameter is passed via the openai SDK's extra_body kwarg, which injects it as a top-level field in the JSON request body.

Test Results

Verified working with a local patch using the openai provider + extra_body:

Provider: openai / glm-5.1 (via Z.AI Coding API)
Phase 1: Ingestion -- 278 files ingested (works, no LLM needed)
Phase 2: Decision extraction -- tested inline marker structuring
Phase 3: Generation -- symbol_spotlight pages generated successfully

Compared to a reasoning model (qwopus3.5-27b-v3 via vLLM) on the same codebase:

• Reasoning model: 0/24 file_pages succeeded (all timed out, 100% failure)
• GLM-5.1 with thinking disabled: direct output, zero reasoning overhead

API Documentation

• Z.AI API docs: https://open.bigmodel.cn/dev/api
• OpenAI-compatible endpoint reference: https://open.bigmodel.cn/dev/api/normal-model/glm-5

[RaghavChamadiya]

Really thorough writeup, appreciate the detail on the reasoning token handling. We're open to a PR for this. The thinking toggle is something we'll probably need for other reasoning models too, so if you implement it generically that would be ideal.

One question - would you prefer a single zai provider with a configurable plan, or two entries (zai-coding / zai-general)? I'm leaning single provider with a plan option.

[Societus]

I might have a chance to put my tests so far into a PR in the next couple days, I've still got some changes I am testing, ran into some rate limiting and concurrency issues once I gave it real tasks, probably should get dealt with before I put it out though. I've got a discussion started with Z.AI about the rate limiting per-plan, since I don't know whether they scale them by load or if they are assigned to the plan level. From test runs I've done so far it could result in arbitrarily failed generations which just waste time.

As for thinking toggles, something I am learning is that they vary by cloud provider and model family, so there may not be too much freedom in creating a generic toggle that works everywhere. Out of curiosity I also found out that the toggles are effectively useless if trying to disable thinking on ollama/lmstudio local models (vLLM somewhat works with modification to repowise prompts), which was where I started that lead to this issue: wanting to 100% local gen repowise wikis on a private codebase. Basically, it seems a bit more involved than ideal, so new providers will likely need their own version of Z.AI's thinking: { type: "disabled" }

As for provider options, I think a single provider with a selection for endpoint (coding plan/subscription vs PAYG) would be the cleanest option.

[Societus]

Update: Tier-aware rate limiting PR stack

Following up on the rate limiting discussion from the original Z.AI provider work. I have opened a two-PR stack that addresses plan-aware concurrency:

PR Stack

1. feat: add generic tier-aware rate limiting framework #82 — Generic tier-aware rate limiting framework
   Adds RATE_LIMIT_TIERS and resolve_rate_limiter() to BaseProvider. Provider-agnostic — any future provider with subscription tiers can use it without writing tier resolution logic. 8 tests.

2. feat(zai): adopt tier framework for plan-aware rate limiting #83 — Z.AI adopts the tier framework
   Wires Z.AI into the framework with Lite/Pro/Max rate limits, ZAI_TIER env var, bumped retries (5 retries / 30s max wait for Z.AI load-shedding tolerance). 13 new tests.

Rate Limit Basis

Z.AI support confirmed (April 2026):

• Limits are aggregate across all models, dynamic and load-dependent
• GLM-5 family models consume 2-3x quota per prompt (reasoning token multiplier)
• Conservative defaults: Lite 10 RPM, Pro 30 RPM, Max 60 RPM
• Recommended: exponential backoff on 429 errors (implemented in provider retry logic)
• Ref: https://docs.z.ai/devpack/usage-policy

The old monolithic PR (#80) has been closed in favor of this layered approach.

Future Work

Once the generic framework lands, adding MiniMax or any other tiered provider becomes a thin layer — just define RATE_LIMIT_TIERS and call resolve_rate_limiter().