LiteLLM

Overview

LiteLLM is a lightweight proxy that allows us to expose various LLM providers (Mistral, OpenCode Zen, etc.) via a single OpenAI-compatible API. It includes a management UI, a Postgres database for state persistence (keys, models), and native support for MCP (Model Context Protocol) servers.

Ports

Service	Port	Protocol	Description
API / UI	`14000`	HTTP	OpenAI API & Management Dashboard
Database	`5432`	TCP	Internal Postgres 16 (Localhost only)

External Access

The service is available internally at http://<homelab-ip>:14000 and externally via Nginx proxy at https://litellm-home.javiersc.com. The UI is protected by credentials stored in 1Password.

Models

Currently configured models in config.yaml:

mistral-small: Proxies to Mistral AI (mistral-small-latest).
big-pickle: Proxies to OpenCode Zen (big-pickle model).

Database

LiteLLM uses a Postgres 16-alpine container for persistence. This allows it to store:

API Keys and usage tracking.
Model configurations.
MCP server registry.

The database is only accessible from localhost and uses the MASTER_KEY as the password for the litellm user.

MCP Integration

LiteLLM connects to Home Assistant as an MCP server.

CURL Usage (Critical)

To use tools via the API, you must use the specialized LiteLLM MCP format. Using standard OpenAI tool definitions will cause a 400 Bad Request error from the upstream provider.

curl http://localhost:14000/v1/chat/completions \
  -H "Content-Type: application/json" \
  -H "Authorization: Bearer scarves8blanche.LOAD" \
  -d '{
    "model": "big-pickle",
    "messages": [{"role": "user", "content": "Dime qué hay en la lista de la compra"}],
    "tools": [
      {
        "type": "mcp",
        "server_url": "litellm_proxy/mcp/home_assistant",
        "server_label": "home_assistant",
        "require_approval": "never"
      }
    ]
  }'

type: "mcp": Forces the proxy to intercept the tool call.
server_url: Must be litellm_proxy/mcp/<server_id>.
require_approval: "never": Required for automatic execution.

Secrets

Secrets are fetched by litellm-secrets.service (using homelab.mkSecretService) into /run/litellm/env at boot, then litellm-config.service renders config.yaml and container environment files. No secrets are stored in the Nix store.

Secret	1Password Path	Description
`MISTRAL_API_KEY`	`op://Homelab/PicoClaw - Mistral AI Token/credential`	Mistral AI API Key
`OPENCODE_ZEN_TOKEN`	`op://Homelab/OpenCode - Zen Token/credential`	OpenCode Zen API Key
`MASTER_KEY`	`op://Homelab/LiteLLM - Master Key/credential`	Used for DB auth and encryption
`UI_USER`	`op://Homelab/Homelab/username`	Admin UI Username
`UI_PASS`	`op://Homelab/Homelab/password`	Admin UI Password
`HA_TOKEN`	`op://Homelab/Home Assistant - OpenClaw Token/credential`	HA MCP Access Token

Backup

State Directory: /var/lib/litellm (persistent data).
Database Directory: /var/lib/litellm/postgres (Postgres data).
Runtime Configs: /run/litellm/ (generated config.yaml, litellm.env, db.env — RAM-based, regenerated on boot).
Backup Status: /var/lib/litellm is included in daily Restic backups. /run/litellm/ is excluded (tmpfs).

Troubleshooting

Restarting the service

LiteLLM follows a three-stage startup chain: litellm-secrets → litellm-config → podman-litellm + DB.

sudo systemctl restart litellm-secrets litellm-config podman-litellm-db podman-litellm

Checking Logs

# Secret fetching & config rendering
sudo journalctl -u litellm-secrets -f
sudo journalctl -u litellm-config -f

# Main LiteLLM logs
sudo journalctl -u podman-litellm -f

# Database logs
sudo journalctl -u podman-litellm-db -f