How to remove no longer valid Gauges

I use the Prometheus Java Client to export session information of my application. We want to show how long sessions have been idle.

The problem is that we have a maximum of 1000 sessions and sessions are removed after a certain period. Unfortunately they do not disappear from Prometheus:

My code looks like this:

static final Gauge sessionInactivity = Gauge.build()
    .name("sessions_inactivity_duration")
    .labelNames("internal_key", "external_key", "browser")
    .help("Number of milliseconds a certain session has been inactive")
    .register();

sessionInactivity.labels(internalKey, externalKey, browser).set(inactivityTime);

I tried to do sessionInactivity.clear() during scrapes but obviously this does not empty the content of the Gauge.

Answer

This sort of per-request handling is not suited to a metric system like Prometheus. This would be considered profiling, for which something more custom would be in order.

It’s also recommend to export the timestamp for this sort of thing, not how long ago it was. This is resilient to the thing updating the time no longer updating, and you can do the subtraction from time() on the Prometheus end.