llama.cpp
Connect to a local llama.cpp llama-server to show model, throughput and request status on your dashboard.
The llama.cpp integration lets Homarr connect to a llama-server instance so you can see the health of your local LLM, which model is loaded, and how fast it is generating — all from your dashboard.
Widgets & Capabilities
Adding the integration
Create the connection under Management → Integrations. See Managing integrations.
URL configuration
Use the root URL of your llama-server, for example http://192.168.1.50:8080. Homarr calls the /health, /v1/models, /metrics and /slots endpoints under that URL automatically.
Metrics
Generation speed and token counters are read from the Prometheus endpoint at /metrics. Start llama-server with the --metrics flag so these values are available:
llama-server --model /path/to/model.gguf --metricsWithout --metrics the widget still shows health, model information, and active-request speed from /slots; metrics-backed counters and aggregate speed stay empty.
Context and per-request speed
Context (KV cache) usage and the in-flight request data are read from the /slots endpoint, which is available on every recent llama-server build and requires no extra flags. The per-request generation speed shown while a request is running is derived by tracking the request's decoded token count across polls, so it reflects the average speed of the request that is currently being generated.
Secrets
llama-server does not require authentication by default. The integration only needs the URL of your server.
No credentials required