Studio: enforce 60s minimum on idle auto-unload TTL (0 stays off)#7185
Studio: enforce 60s minimum on idle auto-unload TTL (0 stays off)#7185NilayYadav wants to merge 2 commits into
Conversation
for more information, see https://pre-commit.ci
There was a problem hiding this comment.
Code Review
This pull request introduces a minimum floor of 60 seconds for the model idle auto-unload setting (with 0 remaining as 'off') to prevent frequent model reloading during active chats. The changes include backend validation, clamping of legacy persisted values and environment variables (with a warning), frontend validation mirroring, updated localization strings across multiple languages, and comprehensive unit tests. I have no feedback to provide as there are no review comments.
Important
The consumer version of Gemini Code Assist on GitHub is being sunset. Starting June 18, 2026, new organization installations will be blocked, and all code review activity will officially cease on July 17, 2026.
For more details on the timeline and next steps, please review the Help Documentation.
Problem
The opt-in idle auto-unload (#6392) accepts any TTL >= 0. A tiny value like 5s silently tears the model down between turns of an active chat, so every next turn pays a full weight reload plus re-prefill of the whole conversation (seconds of re-work per turn, growing with chat length).
Change
MIN_AUTO_UNLOAD_IDLE_SECONDS = 60; 0 keeps meaning "off".set_openai_auto_switchrejects 1-59 with a clear ValueError (PUT route already maps it to 400).UNSLOTH_MODEL_IDLE_TTL(one-time warning for the env var, since headless deploys have no UI to surface an error), so pre-floor persisted values are covered too.Tests
5-> 60,0-> off,600unchanged.python -m pytest studio/backend/tests/test_openai_auto_switch.py: 154 passed. Frontendtsc -bclean.