Skip to content

don't kill live llama-servers when a new Studio instance starts#7182

Open
NilayYadav wants to merge 2 commits into
unslothai:mainfrom
NilayYadav:fix-orphan-reaper
Open

don't kill live llama-servers when a new Studio instance starts#7182
NilayYadav wants to merge 2 commits into
unslothai:mainfrom
NilayYadav:fix-orphan-reaper

Conversation

@NilayYadav

Copy link
Copy Markdown
Collaborator

Starting a second Studio instance killed the first instance's llama-server mid-generation chats died with "Lost connection to the model server".

The startup orphan reaper (_kill_orphaned_servers) killed every llama-server under a Studio install root without checking whether it was actually orphaned. It also killed user-launched llama-servers and its own instance's embed server (surfacing as a bogus "GPU start failed" CPU fallback).

Fix: skip processes whose parent is still alive, reusing _pid_parent_is_alive(the same guard the pidfile reaper already uses), in both the psutil path and the Linux pgrep fallback. True orphans (parent gone) are still reaped.

Repro: load a GGUF model, start a long generation, run unsloth studio -p <port> in another terminal — the stream died within ~3s. With the fix it completes, and a genuinely orphaned server is still cleaned up.

@gemini-code-assist gemini-code-assist Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Code Review

This pull request updates the _kill_orphaned_servers method in llama_cpp.py to prevent killing llama-servers whose parent processes are still alive, ensuring they are not incorrectly classified as orphans. Corresponding unit tests in test_llama_cpp_wait_for_vram_settle.py have been updated, and a new test has been added to verify that servers with active parents are spared. I have no feedback to provide as the changes are clean and well-tested.

Important

The consumer version of Gemini Code Assist on GitHub is being sunset. Starting June 18, 2026, new organization installations will be blocked, and all code review activity will officially cease on July 17, 2026.
For more details on the timeline and next steps, please review the Help Documentation.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant