Vespa Cloud - The vespa release rolled out today, vespa-8.534.59, introduced an issue affecting applications with containers running embedders – Incident details

All systems operational

The vespa release rolled out today, vespa-8.534.59, introduced an issue affecting applications with containers running embedders

Resolved
Degraded performance
Started over 1 year agoLasted 14 hours 10 minutes

Affected

Infrastructure

Operational from 8:30 AM to 2:54 PM, Degraded performance from 2:54 PM to 10:40 PM

Updates
  • Resolved
    UTC
    Resolved
    This incident has been resolved.
  • Monitoring
    UTC
    Monitoring

    We have implemented a fix and created a new release, which is now being rolled out across our infrastructure. Once the deployment is complete, the issue reported in this incident will be considered resolved.

  • Update
    UTC
    Update
    We are continuing to work on a fix for this incident.
  • Identified
    UTC
    Identified
    Impact: * Any application changes will fail to be successfully deployed * Autoscaling: New nodes added to such container clusters will fail to start We have implemented a fix that we currently are testing [Embedding Reference](https://docs.vespa.ai/en/reference/embedding-reference.html)