Studio: add option to disable the in-memory API monitor#7156
Studio: add option to disable the in-memory API monitor#7156gaurav0107 wants to merge 2 commits into
Conversation
Studio records every OpenAI-compatible API request into an in-memory
ApiMonitor (the "API Monitor" activity view). Users who run Studio purely
as an inference API and handle logging/telemetry with an external gateway
have no use for it and want to turn it off.
Add an opt-in UNSLOTH_STUDIO_DISABLE_API_MONITOR env var (house
{"1","true","yes","on"} truthy parsing). When set, ApiMonitor.start()
returns an empty id so every downstream mutator short-circuits on its
existing `if not entry_id` guard and no entry is ever recorded; snapshot(),
active_count() and get() return empty/zero/None naturally. Off by default,
so existing behaviour is unchanged and no call sites need to change.
Fixes unslothai#7036
for more information, see https://pre-commit.ci
There was a problem hiding this comment.
Code Review
This pull request introduces an opt-in kill switch for the in-memory API monitor via the UNSLOTH_STUDIO_DISABLE_API_MONITOR environment variable. When disabled, the ApiMonitor class acts as a no-op, returning a falsy ID on start and preventing any telemetry or logging from being recorded. Comprehensive unit tests have been added to verify the default behavior, the disabled state, and the environment variable parsing. There are no review comments, and I have no additional feedback to provide.
Important
The consumer version of Gemini Code Assist on GitHub is being sunset. Starting June 18, 2026, new organization installations will be blocked, and all code review activity will officially cease on July 17, 2026.
For more details on the timeline and next steps, please review the Help Documentation.
Summary
Studio records every OpenAI-compatible API request into an in-memory
ApiMonitor(the "API Monitor" activity view). A user running Studio purely asan inference API, with logging/telemetry handled by an external gateway (e.g.
LiteLLM on a separate host), asked for a way to turn this off (#7036).
This adds an opt-in
UNSLOTH_STUDIO_DISABLE_API_MONITORenvironment variable,parsed with the same truthy set (
{"1","true","yes","on"}) the backend alreadyuses elsewhere (
transformers_latest.py,llama_admission.py, ...). When set:ApiMonitor.start()returns an empty (falsy) id, so every downstream mutator(
append_reply/set_reply/set_usage/finish/fail) short-circuitson its existing
if not entry_idguard and no entry is ever recorded.snapshot(),active_count()andget()return empty / zero /Nonenaturally, so the Monitor view stays empty.
It is off by default, so existing behaviour is unchanged and no call sites
needed to change — the singleton is simply constructed as
ApiMonitor(enabled = not _api_monitor_disabled()).Scope note: "API monitor" here means the in-memory OpenAI-compatible API
activity monitor in
core/inference/api_monitor.py, not the HTTP access log(
LoggingMiddlewareinloggers/handlers.py). Happy to extend to the accesslog too if that was the intent.
Validation
python -m pytest tests/test_api_monitor.py-> 17 passed (5 new tests:disabled-is-noop, enabled-by-default-records, and env-var truthy/falsy/unset
parsing).
ruff check studio/backend/core/inference/api_monitor.py studio/backend/tests/test_api_monitor.py-> clean.python -m compileallon both touched files -> clean.UNSLOTH_STUDIO_DISABLE_API_MONITOR=1->start()returns
""andsnapshot()is[]; unset -> records normally.api_monitor.start()call sites inroutes/inference.pyassign to an
Optional[str]monitor_idthat is consumed only through theguarded mutators, so returning
""when disabled is safe.Fixes #7036