Alerts

It seems that over the last couple of years, I’ve been building a filtering, grouping, and routing system for alerts for the fourth time. Each time I try to do it using standard tools, but it always ends up as a separate service. All because of the lack of templates for the full webhook body in Prometheus and Grafana Unified Alerting, and the difficulty of debugging those templates. It’s almost disappointing: an extra 700 lines of code in a critical process.