Skip to content

Studio: add option to disable the in-memory API monitor#7156

Open
gaurav0107 wants to merge 2 commits into
unslothai:mainfrom
gaurav0107:fix/7036-add-option-to-disable-api-monitor-logs
Open

Studio: add option to disable the in-memory API monitor#7156
gaurav0107 wants to merge 2 commits into
unslothai:mainfrom
gaurav0107:fix/7036-add-option-to-disable-api-monitor-logs

Conversation

@gaurav0107

Copy link
Copy Markdown
Contributor

Summary

Studio records every OpenAI-compatible API request into an in-memory
ApiMonitor (the "API Monitor" activity view). A user running Studio purely as
an inference API, with logging/telemetry handled by an external gateway (e.g.
LiteLLM on a separate host), asked for a way to turn this off (#7036).

This adds an opt-in UNSLOTH_STUDIO_DISABLE_API_MONITOR environment variable,
parsed with the same truthy set ({"1","true","yes","on"}) the backend already
uses elsewhere (transformers_latest.py, llama_admission.py, ...). When set:

  • ApiMonitor.start() returns an empty (falsy) id, so every downstream mutator
    (append_reply / set_reply / set_usage / finish / fail) short-circuits
    on its existing if not entry_id guard and no entry is ever recorded.
  • snapshot(), active_count() and get() return empty / zero / None
    naturally, so the Monitor view stays empty.

It is off by default, so existing behaviour is unchanged and no call sites
needed to change — the singleton is simply constructed as
ApiMonitor(enabled = not _api_monitor_disabled()).

Scope note: "API monitor" here means the in-memory OpenAI-compatible API
activity monitor in core/inference/api_monitor.py, not the HTTP access log
(LoggingMiddleware in loggers/handlers.py). Happy to extend to the access
log too if that was the intent.

Validation

  • python -m pytest tests/test_api_monitor.py -> 17 passed (5 new tests:
    disabled-is-noop, enabled-by-default-records, and env-var truthy/falsy/unset
    parsing).
  • ruff check studio/backend/core/inference/api_monitor.py studio/backend/tests/test_api_monitor.py -> clean.
  • python -m compileall on both touched files -> clean.
  • Manual singleton smoke: UNSLOTH_STUDIO_DISABLE_API_MONITOR=1 -> start()
    returns "" and snapshot() is []; unset -> records normally.
  • Impact audit: all 9 api_monitor.start() call sites in routes/inference.py
    assign to an Optional[str] monitor_id that is consumed only through the
    guarded mutators, so returning "" when disabled is safe.

Fixes #7036

gaurav0107 and others added 2 commits July 16, 2026 00:31
Studio records every OpenAI-compatible API request into an in-memory
ApiMonitor (the "API Monitor" activity view). Users who run Studio purely
as an inference API and handle logging/telemetry with an external gateway
have no use for it and want to turn it off.

Add an opt-in UNSLOTH_STUDIO_DISABLE_API_MONITOR env var (house
{"1","true","yes","on"} truthy parsing). When set, ApiMonitor.start()
returns an empty id so every downstream mutator short-circuits on its
existing `if not entry_id` guard and no entry is ever recorded; snapshot(),
active_count() and get() return empty/zero/None naturally. Off by default,
so existing behaviour is unchanged and no call sites need to change.

Fixes unslothai#7036

@gemini-code-assist gemini-code-assist Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Code Review

This pull request introduces an opt-in kill switch for the in-memory API monitor via the UNSLOTH_STUDIO_DISABLE_API_MONITOR environment variable. When disabled, the ApiMonitor class acts as a no-op, returning a falsy ID on start and preventing any telemetry or logging from being recorded. Comprehensive unit tests have been added to verify the default behavior, the disabled state, and the environment variable parsing. There are no review comments, and I have no additional feedback to provide.

Important

The consumer version of Gemini Code Assist on GitHub is being sunset. Starting June 18, 2026, new organization installations will be blocked, and all code review activity will officially cease on July 17, 2026.
For more details on the timeline and next steps, please review the Help Documentation.

@gaurav0107 gaurav0107 marked this pull request as ready for review July 15, 2026 19:09
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[Feature Request] Add option to disable API monitor logs

1 participant