Vespa Cloud - Metrics pipeline broken — Grafana dashboards reporting no incoming data – Incident details

Metrics pipeline broken — Grafana dashboards reporting no incoming data

Resolved
Major outage
Started 9 days agoLasted about 2 hours

Affected

Infrastructure

Major outage from 12:50 AM to 2:56 AM

Enclave

Major outage from 12:50 AM to 2:56 AM

AWS

Major outage from 12:50 AM to 2:56 AM

GCP

Major outage from 12:50 AM to 2:56 AM

Zones

Major outage from 12:50 AM to 2:56 AM

dev

Major outage from 12:50 AM to 2:56 AM

Updates
  • Resolved
    Resolved
    This incident has been resolved.
  • Monitoring
    Monitoring
    We implemented a fix and are currently monitoring the result.
  • Investigating
    Investigating
    We are currently investigating this incident.