Feature: Add local LLM provider support (LM Studio / Ollama)

[luca55466]

Feature Summary

The app is currently tightly coupled to OpenAI. While it is technically possible
to point the app at a local LLM (e.g. LM Studio) by overriding OPENAI_BASE_URL,
this workaround has several rough edges that make it error-prone and confusing for
contributors and users.

Motivation / Use Case

Local LLM support is valuable for:

• Development without an OpenAI API key
• Privacy-sensitive deployments where data must not leave the machine
• Cost-free testing and experimentation

Proposed Solution

Current problems

• OPENAI_API_KEY has to be set to a dummy value like lm-studio — misleading
  for anyone reading the config
• No LLM_PROVIDER concept exists — there is no clear way to declare intent
  (use OpenAI vs. use a local model)
• No connection validation on startup — if the local LLM server is not running,
  the app silently starts and only fails on the first request
• host.docker.internal (required to reach the host machine from inside Docker)
  only works on Docker Desktop (Windows/Mac) — Linux users need a different approach
• No documentation on how to set up a local LLM with this project

Proposed solution

1. Add LLM_PROVIDER setting to settings.py:
   LLM_PROVIDER=openai # default
   LLM_PROVIDER=lm-studio # local via LM Studio
   LLM_PROVIDER=ollama # local via Ollama

2. Validate LLM connectivity on startup — if LLM_PROVIDER is not openai,
   ping the base URL and log a clear error if unreachable

3. Handle Linux Docker networking — on Linux, host.docker.internal is not
   automatically available; the compose file should handle this with
   extra_hosts: ["host.docker.internal:host-gateway"]

4. Document local LLM setup in README or docs/SETUP.md with step-by-step
   instructions for LM Studio and Ollama

Example .env after this change

# Use local LLM via LM Studio
LLM_PROVIDER=lm-studio
OPENAI_MODEL=deepseek-r1-distill-llama-8b
OPENAI_BASE_URL=http://host.docker.internal:1234/v1

# Use OpenAI (default)
LLM_PROVIDER=openai
OPENAI_API_KEY=sk-...
OPENAI_MODEL=gpt-4o

### Alternatives Considered

_No response_

### Feature Area

LLM / AI Models

### Estimated Effort

Medium (1–3 days)

### Suggested Labels

- [x] enhancement
- [ ] good first issue
- [ ] help wanted
- [x] documentation

### Checklist

- [x] I have searched existing issues and this is not a duplicate.
- [x] This feature aligns with the project's medical safety guidelines.

[luca55466]

I'd like to work on this — could you assign it to me?

[arvindsis11]

Thanks for the detailed proposal — this is a solid improvement and addresses real usability gaps around local LLM support.

I agree with the direction, especially introducing an explicit LLM_PROVIDER and adding startup validation. This should make configuration much clearer and reduce confusion for contributors.

A couple of suggestions to consider while implementing:

• Keep the provider interface extensible so adding future providers remains simple
• Ensure fallback/error messages are actionable (especially for Docker/Linux networking cases)
• Try to avoid breaking existing OpenAI-based setups

Happy to have you work on this 👍