Observability costs can grow quickly as traffic, trace volume, metric labels, and log retention increase. Cost controls should preserve useful diagnostic coverage rather than simply discard data indiscriminately.
Control metric cardinality
Avoid unbounded labels such as user IDs, request IDs, raw URLs, and arbitrary exception text. Use aggregation and exemplars to connect metrics to traces when supported.
Apply tiered retention
Keep high-resolution data for a shorter period and aggregated trends for longer. Define different retention for logs, traces, metrics, and audit records according to operational and compliance needs.
Sample intelligently
Sample routine successful traces more heavily than repetitive low-value events, while retaining error and slow traces at a higher rate where feasible. Keep sampling policy documented so missing telemetry is understood.
Review cost by signal
Track ingestion volume, storage growth, query cost, and high-cardinality sources. Assign ownership to teams generating unusually large telemetry volumes.
Checklist
- Prevent unbounded metric labels.
- Set retention by telemetry type and purpose.
- Use documented sampling policies.
- Monitor ingestion and query costs.
- Preserve critical incident and audit signals.