feat: add LiteLLM generator

[RheagalFire]

Summary

• Add LiteLLMGenerator in goldenverba/components/generation/LiteLLMGenerator.py that routes RAG answers through litellm.acompletion() to 100+ providers (OpenAI, Anthropic, Bedrock, Azure, Vertex, Gemini, Ollama, OpenRouter, Groq, DeepSeek, etc.) using provider-native API keys.

• Registered in goldenverba/components/managers.py in both the default and hosted generators lists so it shows up in the UI generator dropdown.

• New litellm optional extra in setup.py so the base install stays lean; users install with pip install 'goldenverba[litellm]'.

  Prior art

Searched the repo for existing LiteLLM-related work before drafting this PR:

• PR Add support for Palm, Claude-2, Cohere Llama2, CodeLlama (100+LLMs)  #8 by @ishaan-jaff (LiteLLM maintainer, Sept 2023) proposed initial LiteLLM support. Closed Nov 2023 without merge. This PR supersedes Add support for Palm, Claude-2, Cohere Llama2, CodeLlama (100+LLMs)  #8 with a pattern that matches the repo's current generator abstraction (previously the generate_stream interface did not exist in its current shape).
• Issue Add support for LiteLLM #56 asked for first-class LiteLLM support. Closed Dec 2023 with a partial workaround (set OPENAI_BASE_URL so the OpenAI generator hits a LiteLLM proxy). That workaround requires users to run a separate LiteLLM proxy server and still shows up as the OpenAI generator in the UI. A later comment on the same issue (July 2024, @tan-yong-sheng) asked
  whether LiteLLM embedding models are supported; no follow-up was posted. This PR ships the proper first-class generator that @priamai originally requested (UI model dropdown, no external proxy required).
• No other open or closed PRs/issues reference litellm.
• No references to litellm in goldenverba/ source before this PR.

Motivation

Verba currently ships 8 provider-specific generators (OpenAI, Anthropic, Cohere, Gemini, Groq, Novita, Ollama, Upstage). Each new provider requires a new file plus ongoing maintenance. LiteLLM unifies 100+ providers behind a single interface, so users pick any of them with a model-name prefix and LiteLLM handles routing and auth.

Changes

• goldenverba/components/generation/LiteLLMGenerator.py: new Generator subclass with config fields (Model, API Key, URL, System Message), async generate_stream using litellm.acompletion(stream=True), requires_library=["litellm"] so the UI availability check works. Lazy-imports litellm with a clear install-hint ImportError if the extra is not installed.
• goldenverba/components/managers.py: import and append LiteLLMGenerator() to both generators lists.
• setup.py: "litellm": ["litellm"] optional extra.
• goldenverba/tests/test_litellm_generator.py: 6 mocked unit tests covering metadata, config surface, prepare_messages, stream chunk/finish_reason forwarding, optional-kwarg omission, and the ImportError path when the extra is missing.

Testing and Usage

1. Unit tests for the new generator: pytest goldenverba/tests/test_litellm_generator.py -v

goldenverba/tests/test_litellm_generator.py::test_litellm_generator_metadata PASSED [ 16%]                                                                                                                                                                                                                                                                              
goldenverba/tests/test_litellm_generator.py::test_litellm_generator_config_surface PASSED [ 33%]
goldenverba/tests/test_litellm_generator.py::test_prepare_messages_builds_system_user_pair PASSED [ 50%]                                                                                                                                                                                                                                                                
goldenverba/tests/test_litellm_generator.py::test_generate_stream_yields_chunks_and_finish PASSED [ 66%]                                                                                                                                                                                                                                                                
goldenverba/tests/test_litellm_generator.py::test_generate_stream_omits_api_key_when_blank PASSED [ 83%]                                                                                                                                                                                                                                                                
goldenverba/tests/test_litellm_generator.py::test_generate_stream_raises_import_error_without_litellm PASSED [100%]                                                                                                                                                                                                                                                     
============================== 6 passed in 14.46s ===============================                                                                                                                                                                                                                                                                                       

What they cover: metadata/config surface the UI renders, prepare_messages structure (system message + user query with context), litellm.acompletion receives the model/stream/api_key/api_base the user configured, blank credentials are omitted (so LiteLLM falls back to provider-specific env vars like ANTHROPIC_API_KEY, AWS_*), missing extra raises a
clean ImportError pointing at goldenverba[litellm].

2. Format check: black --check on the three new/modified files -> 3 files would be left unchanged.

3. Live E2E against Azure OpenAI (exercises full RAG answer path: UI -> GeneratorManager.generate_stream -> LiteLLMGenerator.generate_stream -> litellm.acompletion -> Azure OpenAI Chat Completions -> streamed token parsing):

-> LiteLLM model: azure/gpt-4o
-> api_base: https://.openai.azure.com
[stream] Hello
[stream] !
[stream] Verba
[stream] LiteLLM
[stream] OK
[stream] (finish_reason=stop)
[result] 4/7 chunks captured, finish_reason=stop
[pass] live Azure gpt-4o streamed via LiteLLM OK

This proves the integration chain end to end: config forwarded correctly, litellm.acompletion dispatches with the Azure-prefixed model + api_key + api_base, streamed deltas are unwrapped into the {"message", "finish_reason"} dicts Verba's UI expects.

Risk / Compatibility

• Additive only. Existing OpenAIGenerator, AnthropicGenerator, and all other generators are untouched.
• litellm is an optional extra. Base installs unaffected. When the extra is not installed, the UI availability check (requires_library=["litellm"] -> importlib.import_module in verba_manager.verify_installed_libraries) marks the generator as unavailable, so it is simply hidden from the dropdown.
• Both the default (dev) and Production generators lists in managers.py include LiteLLMGenerator(), matching the pattern of the other lightweight generators (Anthropic, Cohere, Upstage).

Example usage

Install the optional extra:

pip install 'goldenverba[litellm]'

Export a provider-specific key (LiteLLM auto-resolves per model prefix):

export ANTHROPIC_API_KEY=sk-ant-...

Or for Azure OpenAI:

export AZURE_API_KEY=...
export AZURE_API_BASE=https://.openai.azure.com
export AZURE_API_VERSION=2025-01-01-preview

Launch Verba, pick LiteLLM from the Generator dropdown in the UI, and set the Model field to any LiteLLM-supported prefix:

• anthropic/claude-3-5-sonnet-20241022
• azure/gpt-4o
• bedrock/anthropic.claude-3-sonnet-20240229-v1:0
• gemini/gemini-1.5-pro
• ollama/llama3

[orca-security-eu]

Orca Security Scan Summary

Status	Check	Issues by priority
Passed	Infrastructure as Code	0   0   0   0	View in Orca
Passed	SAST	0   0   0   0	View in Orca
Passed	Secrets	0   0   0   0	View in Orca
Passed	Vulnerabilities	0   0   0   0	View in Orca

[weaviate-git-bot]

To avoid any confusion in the future about your contribution to Weaviate, we work with a Contributor License Agreement. If you agree, you can simply add a comment to this PR that you agree with the CLA so that we can merge.

beep boop - the Weaviate bot 👋🤖

PS:
Are you already a member of the Weaviate Forum?

[RheagalFire]

@thomashacker Requesting a review.

[RheagalFire]

I agree with the CLA.