feat(observability): attribute dashboard panels per cluster and server - #151
Open
gabriele-wolfox wants to merge 3 commits into
Open
feat(observability): attribute dashboard panels per cluster and server#151gabriele-wolfox wants to merge 3 commits into
gabriele-wolfox wants to merge 3 commits into
Conversation
gabriele-wolfox
force-pushed
the
dev/49
branch
3 times, most recently
from
August 24, 2026 10:21
0ebf69a to
a983b47
Compare
The Grafana dashboard was only validated against a single server and cluster, where sums and maxima are unambiguous. With several servers and clusters it folded independent entities into single misleading values and could not tell them apart. Add a cluster_name attribute to the plugin backup metrics (klio.plugin.backup.*) so backup panels can be attributed per cluster even when several clusters share a namespace. Rework the dashboard so every aggregation groups by its identifying label and nothing is folded silently: - Identify servers by the OpenTelemetry service.name instead of the pod host name, which collides when two servers share a name in different namespaces. - Scope server panels by service.name and cluster_name rather than the namespace, so a cluster backed up cross-namespace is attributed to the right server. - Group every panel by cluster, server and tier, and identify each series in its legend, so per-tier backup and WAL state is never folded (tier-1 and tier-2 have independent relay progress and retention). - Render the PostgreSQL timelines as stepped time series rather than a single current value, so a promotion or failover is visible as the step where the line jumps. - Derive a cluster label from the Kopia snapshot source so the base snapshot panels are per cluster too. Update the metric catalog descriptions and the OpenTelemetry and Grafana documentation, and regenerate the committed dashboard JSON. Assisted-by: Claude Signed-off-by: Gabriele Quaresima <gabriele.quaresima@enterprisedb.com>
Regenerate the three dashboard section screenshots against a multi-server, multi-cluster, two-tier environment so they reflect the reworked panels: per-cluster and per-tier series, servers identified by service name, and the PostgreSQL timelines rendered as stepped time series. Assisted-by: Claude Signed-off-by: Gabriele Quaresima <gabriele.quaresima@enterprisedb.com>
Backup failures did not show on the dashboard: the runs counter series for an outcome and failure category only appears on the first backup of that kind, so it is born at 1 and rate()/increase() (and the success-ratio panel built on them) cannot see that first event. Seed the runs counter at 0 for every outcome and failure category, and the in-progress gauge, when the plugin starts, so a first failure is a visible 0->1, per the Prometheus "avoid missing metrics" guidance: https://prometheus.io/docs/practices/instrumentation/#avoid-missing-metrics Assisted-by: Claude Signed-off-by: Gabriele Quaresima <gabriele.quaresima@enterprisedb.com>
gabriele-wolfox
force-pushed
the
dev/49
branch
from
August 27, 2026 08:14
deb2469 to
c01ba17
Compare
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
The Grafana dashboard was only validated against a single server and cluster, where sums and maxima are unambiguous. With several servers and clusters it folded independent entities into single misleading values and could not tell them apart.
Add a
cluster_nameattribute to the plugin backup metrics (klio.plugin.backup.*) so backup panels can be attributed per cluster even when several clusters share a namespace.Rework the dashboard so every aggregation groups by its identifying label and nothing is folded silently:
service.nameandcluster_namerather than the namespace, so a cluster backed up cross-namespace is attributed to the right server.Update the metric catalog descriptions and the OpenTelemetry and Grafana documentation, and regenerate the committed dashboard JSON.
Assisted-by: Claude
Closes #149