Metrics are useful when they help answer whether a service is healthy, whether users are affected, and what action should happen next.
Start with service questions
Can users complete important operations? Are requests failing or slowing down? Is the system approaching a resource limit? Are background jobs falling behind? Map each question to a small set of measurements.
Use the four golden signals
Latency, traffic, errors, and saturation provide a useful starting point. Add domain-specific indicators such as payment completion, publishing delay, or conversion processing failures where they represent critical service outcomes.
Avoid high cardinality
Do not use user IDs, request IDs, or arbitrary URLs as metric labels. These create excessive time series and can expose sensitive identifiers. Use bounded dimensions such as route template, status class, or service name.
Document interpretation
Each metric should have a definition, unit, owner, and expected response. A graph without context is not an operational practice.
Checklist
- Map metrics to service and user questions.
- Track latency, traffic, errors, and saturation.
- Keep labels low-cardinality.
- Define owners and expected responses.
- Remove metrics that do not inform decisions.