Skip to main content

v26.3.4

v26.3.4​

Metrics endpoints now support OpenMetrics and trace exemplars​

The Prometheus metrics endpoints of Kratos, Keto, Hydra, Oathkeeper, and Talos now additionally serve the OpenMetrics exposition format. The format is selected automatically through standard HTTP content negotiation: scrapers that ask for OpenMetrics receive it, and all other scrapers keep receiving the classic text format. No configuration or scraper change is necessary.

OpenMetrics enables exemplars, which attach a trace reference to a metric sample. Two sets of metrics use this:

  • The HTTP request duration histograms and request counters attach an exemplar when a request runs under a sampled trace, so latency dashboards can link slow-request outliers and error spikes to the traces that caused them.
  • The ory_x_popx_cockroach_transaction_retries_total counter attaches an exemplar whose value is the transaction's retry count when a transaction is retried during a traced request, and the trace records a db.transaction.retry span event. Dashboards can link retry spikes directly to the traces that caused them.

That counter's caller label now also more reliably attributes automatic CockroachDB transaction retries to the business operation that opened the transaction, for example the OAuth2 refresh token flow, instead of a generic persistence Transaction wrapper method. This makes it possible to see which operation is causing database contention. Dashboards or alerts that match specific caller label values may need updating, because existing label values change with this release.