Skip to main content

Prometheus metrics

Talos exposes Prometheus metrics on a dedicated port (default: 4422).

Endpoint

GET http://localhost:4422/metrics

HTTP metrics

MetricTypeLabelsDescription
http_requests_totalCountermethod, code, endpointTotal HTTP requests
http_request_duration_secondsHistogrammethod, code, endpointRequest latency
http_request_size_bytesHistogrammethod, codeRequest payload size
http_response_size_bytesHistogrammethod, codeResponse payload size
http_requests_in_flightGauge--Concurrent requests

Labels

LabelDescriptionUsed by
methodHTTP method (lowercase)All except http_requests_in_flight
codeHTTP status codeAll except http_requests_in_flight
endpointRoute template (e.g., /v2alpha1/admin/issuedApiKeys/{key_id})http_requests_total, http_request_duration_seconds

Configuration

serve:
metrics:
host: "0.0.0.0"
port: 4422

Proxy metrics (Commercial)

The edge proxy exposes additional metrics under the talos_proxy_ namespace.

MetricTypeLabelsDescription
talos_proxy_cache_hits_totalCounter--Total number of cache hits
talos_proxy_cache_misses_totalCounter--Total number of cache misses
talos_proxy_upstream_requests_totalCounterstatusRequests forwarded to upstream
talos_proxy_upstream_latency_secondsHistogram--Upstream request latency
talos_proxy_request_duration_secondsHistogramcachedTotal request duration

Labels

LabelDescriptionUsed by
statusHTTP status code of upstream responsetalos_proxy_upstream_requests_total
cachedWhether the response was served from cache (true/false)talos_proxy_request_duration_seconds

Grafana

Scrape the metrics endpoint with Prometheus and visualize with Grafana. Key panels:

  • Request rate: rate(http_requests_total[5m])
  • P99 latency: histogram_quantile(0.99, rate(http_request_duration_seconds_bucket[5m]))
  • Error rate: rate(http_requests_total{code=~"5.."}[5m])