Incidents
0 open
admin
demo data
Open
0
Last 90 days
3
Mean time to detect
2min
Mean time to resolve
34min
Alert rules
alerts go to email and Telegram, always to a human
| Rule | Threshold | Window | Route | State |
|---|---|---|---|---|
| Delivery p95 latency | > 1,000 ms | 5 min | on-call | armed |
| Worker queue depth | > 5,000 | 2 min | on-call | armed |
| Rejection rate spike | > 4x baseline | 10 min | on-call | armed |
| Repeated auth failures | > 20 per IP | 5 min | security | armed |
| License polling from many IPs | > 5 distinct | 1 h | security | armed |
| Payment webhook failures | > 3 consecutive | 15 min | billing | armed |
| Backup restore test | fails | weekly | on-call | armed |
History
12 days ago · resolved in 22 minutes
Elevated delivery latency, EU region
A noisy neighbour on the shared network path pushed p95 to 1.4 s for part of the London session. No signals were lost; 41 expired on their time-to-live and were logged as expired.
- Detected by the p95 alert, 2 minutes in
- Traffic moved to the secondary path
- Affected customers emailed with their own numbers
38 days ago · resolved in 51 minutes
Razorpay webhook backlog
Provider-side delay meant activations lagged after payment. No customer was charged twice; idempotency held. Manual activation applied to 14 accounts while the backlog cleared.
76 days ago · resolved in 29 minutes
Planned key rotation, ran long
Quarterly signing-key rotation. Connectors briefly rejected signed responses because the overlap window was too short. Fixed by widening the window to two keys for 24 hours, which is now the standing procedure.