Skip to content

Why custom model is restricted to Ollama only? #4126

Description

@privacyadmin

Self hosted version supports only Ollama.

vLLM is also OpenAPI compatible but it seems hardcoded to accept only ollama/llama2 as the only option when loading available models.

Suggestion: Please consider removing the hardcoded Ollama only option for custom provider.

Thank you

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions