Vespa Cloud - Issues with new host provisioning – Incident details

All systems operational

Issues with new host provisioning

Resolved
Operational
Started 16 days agoLasted about 4 hours

Affected

Console

Operational from 11:28 AM to 3:15 PM

Infrastructure

Degraded performance from 11:28 AM to 3:15 PM

Zones

Major outage from 11:28 AM to 12:39 PM, Degraded performance from 12:39 PM to 3:15 PM

dev.aws-us-east-1c

Major outage from 11:28 AM to 12:39 PM, Degraded performance from 12:39 PM to 3:15 PM

dev.gcp-us-central1-f

Major outage from 11:28 AM to 12:39 PM, Degraded performance from 12:39 PM to 3:15 PM

perf.aws-us-east-1c

Major outage from 11:28 AM to 12:39 PM, Degraded performance from 12:39 PM to 3:15 PM

Updates
  • Resolved
    Resolved
    This incident has been resolved.
  • Monitoring
    Monitoring
    We implemented a fix and are currently monitoring the result.
  • Identified
    Identified

    We have identified the root cause and are taking the necessary steps to resolve the issue.

  • Investigating
    Investigating

    We are experiencing a technical issue that is impacting provisioning of new Vespa hosts. Provisioning requests may fail or be delayed while we work to resolve the problem.

    Existing deployments remain unaffected.

    We are currently investigating this incident.