Skip to content

feat: Add reasoning token-aware PTU sizing for reasoning models (o1, o3, o3-mini, o4-mini) #2

Description

@ricmmartins

Problem

The calculator supports reasoning models (o1, o3, o3-mini, o4-mini) as selectable SKUs with throughput and pricing data. However, PTU sizing is based only on total tokens per minute and input/output token pricing — it does not account for reasoning tokens or reserved capacity.

Customers using reasoning_effort=high can be significantly undersized because reasoning tokens consume PTU capacity but are not surfaced by the AOAI service yet. This can result in:

  • 429 rate limit errors
  • Output truncation
  • Request failures even on single requests

Proposed Solution

Add a reasoning-aware sizing mode that accounts for the hidden reasoning token overhead.

Changes needed

  1. Model configs (src/*.json) — Add isReasoningModel flag and reasoningMultipliers per effort level (low/medium/high)
  2. UI (src/App.jsx) — Add a reasoning effort dropdown (low/medium/high) that appears only when a reasoning model is selected
  3. PTU sizing logic (src/App.jsx, src/corrected_pricing_service.js) — Apply a reasoning multiplier to TPM before calculating required PTUs
  4. PAYG calculation (src/official_token_pricing.js) — Add reasoning tokens as a third token bucket in cost comparison
  5. Research — Define accurate reasoning token multipliers per model and effort level (empirical testing or AOAI team collaboration)

Key consideration

The hardest part is defining accurate reasoning token multipliers since the AOAI service doesn't expose reasoning token counts yet. Initial implementation could use estimated multipliers based on empirical benchmarks, with a plan to refine once the service exposes this data.

References

  • Reasoning tokens can scale exponentially with prompt length and reasoning steps
  • max_completion_tokens vs max_tokens behavior differs for reasoning models

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions