Vespa Cloud - Performance regression on release 8.528.12 – Incident details

All systems operational

Performance regression on release 8.528.12

Resolved
Operational
Started over 1 year agoLasted about 3 hours

Affected

Zones

Degraded performance from 11:36 AM to 12:17 PM, Operational from 12:17 PM to 2:59 PM

dev

Degraded performance from 11:36 AM to 12:17 PM, Operational from 12:17 PM to 2:59 PM

dev.aws-us-east-1c

Degraded performance from 11:36 AM to 12:17 PM, Operational from 12:17 PM to 2:59 PM

dev.gcp-us-central1-f

Degraded performance from 11:36 AM to 12:17 PM, Operational from 12:17 PM to 2:59 PM

perf

Degraded performance from 11:36 AM to 12:17 PM, Operational from 12:17 PM to 2:59 PM

perf.aws-us-east-1c

Degraded performance from 11:36 AM to 12:17 PM, Operational from 12:17 PM to 2:59 PM

Updates
  • Resolved
    UTC
    Resolved

    We don't see any further impact.
    Please don't hesitate to contact our support through the Vespa Cloud panel if you still experience any disruption.

  • Monitoring
    UTC
    Monitoring

    All production applications have been rolled back the the previous version, there should not be any impact at the current moment.

    We will be monitoring the situation for a while.

  • Identified
    UTC
    Identified

    All the upgrades in Vespa Cloud are being stopped and any application already upgraded will be automatically downgraded.

  • Investigating
    UTC
    Investigating

    We are have identified a regression on the vespa release 8.528.12.