OpenAI-Compatible Cost Guardrail Kit

Catch LLM budget drift before it becomes an invoice surprise.

This independent, non-official kit gives teams a small offline-first CLI for testing OpenAI-compatible usage logs against budget policies. Drop in usage/cost fixtures and a JSON budget policy, then get per-feature budget usage, hard and soft limit warnings, model-route cost rows, fallback/escalation suggestions, and a what-to-monitor checklist.

Endpoint reference for compatible gateway testing: https://www.tken.shop/v1

Try TKEN-compatible developer tooling

Disclosure: I work on TKEN-related developer tooling.

This repository is independent and non-official. It is not affiliated with, endorsed by, or sponsored by OpenAI or any provider named in examples.

Why This Exists

LLM cost tracking usually gets attention after a feature ships. This kit is designed for earlier checks:

• Review fixture usage by feature, environment, route, and model.
• Compare spend against soft and hard budget thresholds.
• See which model routes are driving cost.
• Draft fallback and escalation policy before production traffic.
• Keep the price table local so reviews can happen without live provider calls.

Example prices in this repo are placeholders for demo math only. Replace them with current provider documentation before using the output for planning, billing, alerts, or customer-facing decisions.

Quickstart

npm run demo

Or run the CLI directly:

node bin/cost-guardrail.js \
  --usage fixtures/usage.example.json \
  --policy fixtures/budget-policy.example.json \
  --prices fixtures/provider-prices.placeholder.json

Example Output

Feature budget usage
- support-triage: $5.71 / $8.00 soft, $10.00 hard (57.1% of hard) [ok]
- code-review: $7.15 / $6.00 soft, $9.00 hard (79.4% of hard) [soft_limit]

Model-route cost rows
- support-triage via primary: gpt-placeholder-large input=$1.25 output=$3.75 total=$5.00
- support-triage via fallback: gpt-placeholder-small input=$0.18 output=$0.53 total=$0.71

Suggested fallback/escalation policy
- code-review: switch new low-priority jobs to fallback route; notify owner if projected daily cost remains above soft limit.

Input Files

The CLI reads three JSON files:

• usage: feature usage rows with input/output token counts, route, model, and optional priority.
• policy: feature budget limits and default fallback/escalation behavior.
• prices: local model price rows expressed per 1M input/output tokens.

See the included fixtures:

• fixtures/usage.example.json
• fixtures/budget-policy.example.json
• fixtures/provider-prices.placeholder.json

Output Sections

• Feature budget usage: actual spend compared with soft and hard limits.
• Warnings: hard/soft limit notices with feature owner hints.
• Model-route cost rows: route-level model costs.
• Suggested fallback/escalation policy: conservative next actions based on current status.
• What to monitor: checklist for production observability.

Safety Notes

• Do not commit API keys, gateway keys, customer data, or raw production logs.
• Keep provider price metadata under review; example numbers here are placeholder fixtures.
• Treat CLI output as planning support, not billing truth.
• Redact prompts and user identifiers before sharing fixtures.

Validate

npm run check

Documentation

• Setup
• UTM links
• Publish checklist

License

MIT