Vespa Cloud - Performance regression on release 8.528.12 – Incident details

All systems operational

Performance regression on release 8.528.12

Resolved
Operational
Started about 1 month agoLasted about 3 hours

Affected

Zones

Degraded performance from 11:36 AM to 12:17 PM, Operational from 12:17 PM to 2:59 PM

dev.aws-us-east-1c

Degraded performance from 11:36 AM to 12:17 PM, Operational from 12:17 PM to 2:59 PM

dev.gcp-us-central1-f

Degraded performance from 11:36 AM to 12:17 PM, Operational from 12:17 PM to 2:59 PM

perf.aws-us-east-1c

Degraded performance from 11:36 AM to 12:17 PM, Operational from 12:17 PM to 2:59 PM

test.aws-us-east-1c

Degraded performance from 11:36 AM to 12:17 PM, Operational from 12:17 PM to 2:59 PM

test.gcp-us-central1-f

Degraded performance from 11:36 AM to 12:17 PM, Operational from 12:17 PM to 2:59 PM

Updates
  • Resolved
    Resolved

    We don't see any further impact.
    Please don't hesitate to contact our support through the Vespa Cloud panel if you still experience any disruption.

  • Monitoring
    Monitoring

    All production applications have been rolled back the the previous version, there should not be any impact at the current moment.

    We will be monitoring the situation for a while.

  • Identified
    Identified

    All the upgrades in Vespa Cloud are being stopped and any application already upgraded will be automatically downgraded.

  • Investigating
    Investigating

    We are have identified a regression on the vespa release 8.528.12.