llama.cpp
System Monitoring
Shows the health, loaded model and generation speed of a local llama.cpp llama-server.
This widget shows the live status of your local llama.cpp llama-server: whether it is online, which model is loaded (with quantization, context size, file size and parameter count), the generation speed in tokens per second, and how many requests are being processed. When the server is started with --metrics, it also reports total token counts and the prompt cache hit rate.
Generation speed
The widget picks the most meaningful speed for the current state (hover the speed for an explanation of which one is shown):
- While a request is being generated — the average speed of that request since it started, derived from the token count reported by
/slots. - While the server is busy but the per-request data is unavailable — the live speed from the
/metricsrate gauge. - While the server is idle — the average generation speed since the server started (cumulative tokens ÷ seconds), so the value stays meaningful instead of dropping to zero.
Supported Integrations
llama.cppConnect to a local llama.cpp llama-server to show model, throughput and request status on your dashboard.Displays health, model details, generation speed and request status
Adding the widget
Add the widget from the board's Add board content menu in edit mode.
Configuration
| Name | Description | Values | Default value |
|---|---|---|---|
| Show title | Displays the llama.cpp label and status badge at the top of the widget | yes / no | yes |
| Show model info | Displays the loaded model name, quantization, context size and file size | yes / no | yes |
| Show cache hit rate | Displays the prompt cache hit rate as a percentage bar | yes / no | yes |
| Show context usage | Displays the context (KV cache) token usage as a progress bar | yes / no | yes |