Grafana dashboards

Dashboard sets

The repository contains two dashboard sets:

• examples/grafana/dashboards/: the original 11 dashboards.
• examples/grafana/dashboards-v2/: an advanced bundle with expanded AI navigation and dedicated input/cache + cost/savings visibility.

For v2 screenshots and feature highlights, see Grafana dashboards v2.

Legacy set (11 dashboards)

The repository ships eleven ready-to-provision dashboards in examples/grafana/dashboards/, generated by examples/grafana/generate.py and validated in CI against a real Prometheus and Grafana with synthetic data (see Validation).

File	Dashboard	Answers
01-overview.json	Overview	How many tokens, requests and dollars in the selected range; savings; exporter status.
02-tokens.json	Tokens	Tokens by direction and model, share over time, top models, cache hit and reasoning share.
03-time-patterns.json	Time patterns	Tokens, requests and cost per bucket (hour, day, week, month); hour of day; weekday; period vs previous period.
04-reported-cost.json	Reported cost	Reported spend, month to date, monthly projection, budget, cost per request and per 1M tokens.
05-equivalent-cost.json	Equivalent cost & savings	What local and free models would cost on a pay-per-token API, savings, price coverage, what-if prices.
06-reference-models.json	Reference model comparison	What all your usage would cost if it had run on a reference model (for example Claude Sonnet or GPT-5).
07-efficiency.json	Efficiency	Tokens per request, output per request, output/prompt ratio, $ per 1M tokens.
08-reliability.json	Reliability	Errors, aborts, truncations, stop reasons, usage and cost missing.
09-model-drilldown.json	Model drilldown	Everything about one provider/model pair.
10-local-vs-llamacpp.json	Local models vs llama.cpp	OMP local usage next to the llama.cpp server metrics (speed, slots, cache), side by side.
11-exporter-health.json	Exporter health	Import cycle, invalid records, files, memory, CPU, price catalog and unresolved prices.

All dashboards share the tag omp-usage, uids prefixed with omp-usage- and links to each other.

Provisioning

1. Copy or mount either examples/grafana/dashboards/ (legacy) or examples/grafana/dashboards-v2/ (advanced) into the Grafana container, for example at /var/lib/grafana/dashboards/omp-usage.
2. Add the dashboard provider examples/grafana/provisioning/dashboards/omp-usage.yml to /etc/grafana/provisioning/dashboards/. It loads the files into the folder "OMP usage".
3. If you do not provision the datasource already, use examples/grafana/provisioning/datasources/prometheus.yml. The dashboards default to the datasource uid prometheus; any other Prometheus datasource can be selected with the Data source variable.

Docker Compose (Dokploy works the same way):

services:
  grafana:
    image: grafana/grafana:13.2.3
    volumes:
      - ./omp-usage/examples/grafana/dashboards:/var/lib/grafana/dashboards/omp-usage:ro
      - ./omp-usage/examples/grafana/provisioning/dashboards/omp-usage.yml:/etc/grafana/provisioning/dashboards/omp-usage.yml:ro

To load v2 instead of legacy, change the first volume source to ./omp-usage/examples/grafana/dashboards-v2.

Manual import also works: Dashboards → New → Import → upload a JSON file, then select your Prometheus datasource.

Datasource scrape interval

Set the datasource "Scrape interval" (jsonData.timeInterval) to the scrape_interval of the omp_usage job (30 s in the example). Grafana uses it for $__rate_interval and as the minimum bucket.

Variables

Variable	Used in	Meaning
Data source	all	Prometheus datasource (default uid prometheus).
Job	all	Prometheus job of the exporter (default omp_usage). Multi-select for several machines.
Provider, Model	most	Filter; Model depends on Provider. Single value in Model drilldown.
Bucket	most	Bar width for "per bucket" panels: 1h, 6h, 1d, 7d, 30d. Totals in legends are exact.
Compare	Time patterns	Offset of the "previous period" series (1d, 7d, 30d).
Budget	Reported cost	Monthly budget in USD, used by the gauge and the projection.
What-if models, In / Out / Cache R / Cache W (USD/1M)	Equivalent	Try prices without changing the pricing file.
Reference	References	Reference models from references in the pricing file; one repeated panel per model.
Local provider, llama.cpp job	Local	OMP provider of the local server (default llama.cpp) and its Prometheus job (default llm).

How the numbers are computed

Every "selected range" number is increase(counter[$__range]), so it follows the Grafana time picker. Every "per bucket" panel uses increase(counter[$__interval]) with the panel interval set to $bucket, so each bar is the exact amount in that bucket.

# Tokens per model in the selected range
sum by (provider, model) (increase(omp_llm_tokens_total{job=~"$job"}[$__range]))

# Tokens per model per bucket (panel interval = $bucket)
sum by (provider, model) (increase(omp_llm_tokens_total{job=~"$job"}[$__interval]))

# Cache hit ratio: cache_read over all prompt tokens
sum(increase(omp_llm_tokens_total{direction="cache_read"}[$__range]))
/
sum(increase(omp_llm_tokens_total{direction=~"input|cache_read|cache_write"}[$__range]))

increase() attributes tokens to the scrape that saw them, not to the original event time, and history imported from old JSONL files lands at import time. See metrics.

Equivalent cost

With --pricing-file the exporter publishes prices per provider/model/direction. The dashboards multiply tokens by price at query time:

sum by (provider, model) (
  increase(omp_llm_tokens_total{job=~"$job"}[$__range])
  * on (provider, model, direction) group_left
  max by (provider, model, direction) (omp_llm_price_usd_per_million_tokens{job=~"$job"})
) / 1e6

• Reported cost is what OMP reported (omp_llm_reported_cost_usd_total). It is never replaced or mixed with prices.
• Equivalent cost is tokens × price: "what this would cost on a pay-per-token API". Directions without a price (for example cache_write when the model has no cache write price) are not counted. They are not treated as zero.
• Effective cost is the reported cost when it is above zero, otherwise the equivalent cost. Local and free models stop showing $0.
• Saved is, per priced model, equivalent cost minus reported cost, summed. Unpriced models are left out, so the number is never inflated by unknown usage. The panel "Priced share of tokens" shows how much of the usage has a price.
• Prices are the current gauge values applied to the whole range. If a price changes, past usage is shown at the new price.

To price a local model like an OpenRouter model, map it in the pricing file (see pricing):

{
  "models": [
    { "provider": "llama.cpp", "model": "qwen3.6-35b-a3b", "openrouter": "qwen/qwen3.6-35b-a3b" }
  ]
}

:free OpenRouter models are priced at their paid variant automatically; openrouter/free stays at 0 because its underlying model is unknown.

What-if prices

The Equivalent cost dashboard has text variables for input, output, cache read and cache write prices (USD per 1M tokens) and a model selector. The what-if panels apply those prices to the selected models only. Nothing is stored; use them to test a price before adding it to the pricing file.

Reference models

references in the pricing file (for example anthropic/claude-sonnet-4.5, openai/gpt-5) are exported as omp_llm_reference_price_usd_per_million_tokens{reference_model,direction}. The References dashboard prices all selected usage at each reference model:

sum by (reference_model) (
  sum by (direction) (increase(omp_llm_tokens_total{job=~"$job"}[$__range]))
  * on (direction) group_right
  max by (reference_model, direction) (omp_llm_reference_price_usd_per_million_tokens{job=~"$job"})
) / 1e6

Time patterns

"Tokens by hour of day" and "by weekday" query hourly buckets and group them with Grafana transformations (Format time → Group by), so they use the dashboard time zone. Use a range of at least a week for a meaningful profile.

Local models and llama.cpp

The llama.cpp server metrics (llamacpp:*, job llm) have no model label and include requests that did not come from OMP. The same local generation can be counted in both omp_llm_tokens_total{provider="llama.cpp"} and llamacpp:tokens_predicted_total. The Local dashboard puts them next to each other and computes ratios (for example "OMP share of llama.cpp generation"), but never adds them together. Change the Local provider variable if your local OMP provider has another name.

Recording rules and alerts

examples/prometheus/ has a scrape config, optional recording rules, alerts and promtool unit tests. See Prometheus. The dashboards do not need the recording rules.

Changing the dashboards

Edit examples/grafana/generate.py, not the JSON:

python3 examples/grafana/generate.py           # regenerate
python3 examples/grafana/generate.py --check   # CI: fails if JSON is out of date

Dashboards edited in the Grafana UI can be exported and kept in your own repository; provisioning sets allowUiUpdates: true but a reprovision overwrites UI changes.

Validation

examples/grafana/validate/ checks every query of every panel against a real Prometheus and Grafana:

python3 examples/grafana/validate/synth.py --days 21 > data.om   # 21 days of synthetic usage
promtool tsdb create-blocks-from openmetrics data.om tsdb
prometheus --storage.tsdb.path=tsdb ...                            # plus Grafana with the provisioning files
python3 examples/grafana/validate/check_panels.py --to <last sample epoch>

The check fails on any query error and on any panel without data. CI runs the same steps (.github/workflows/dashboards.yml).