Vespa Cloud - Node failures in dev environment – Incident details

Node failures in dev environment

ResolvedPartial outage1 hour 32 minutes

Affected

Zones

Partial outage from 1:35 PM to 3:07 PM

dev

Partial outage from 1:35 PM to 3:07 PM

dev.aws-us-east-1c

Partial outage from 1:35 PM to 3:07 PM

  • ResolvedUTC

    This incident has been resolved.

  • UpdateUTC

    We implemented a fix and are currently monitoring the result. Due to the fact that there is no redundant storage in the dev environment, some documents may have been lost. We have restored as much as possible.

  • MonitoringUTC

    We implemented a fix and are currently monitoring the result.

  • InvestigatingUTC

    Several nodes have been failed out in the dev environment. We are currently investigating this incident.