Vespa Cloud - Notice history

All systems operational

Console - Operational

100% - uptime
Apr 2024 · 99.70%May · 100.0%Jun · 100.0%

Previous page

Next page

Apr 2024
May 2024
Jun 2024

Infrastructure - Operational

99% - uptime
Apr 2024 · 99.15%May · 97.61%Jun · 100.0%

Previous page

Next page

Apr 2024
May 2024
Jun 2024
100% - uptime

AWS - Operational

100% - uptime
Apr 2024 · 100.0%May · 100.0%Jun · 100.0%

Previous page

Next page

Apr 2024
May 2024
Jun 2024

GCP - Operational

100% - uptime
Apr 2024 · 100.0%May · 100.0%Jun · 100.0%

Previous page

Next page

Apr 2024
May 2024
Jun 2024
100% - uptime

Notice history

Jun 2024

Node provisioning in AWS is not working
  • Resolved
    UTC
    Resolved
    This incident has been resolved.
  • Identified
    UTC
    Identified

    During the release of vespa-8.362.16 today, we identified failing deployments to test and staging. This failed due to a change that broke provisioning of nodes from AWS.

    We are deploying a fix for this now

Console might not load
  • Resolved
    UTC
    Resolved

    We have deployed a fix that should prevent this from happening again

  • Update
    UTC
    Update

    This is caused by a changed to an un-versioned embedded file. We will address this as soon as possible

  • Identified
    UTC
    Identified

    Some might get "You encountered a bug", with "TypeError: Cannot read properties of undefined (reading 'apiUrl') in the error trace. We will address this as soon as possible.

    To work around this issue do a forced refresh.

May 2024

System Test and Staging Test runs is not finishing
  • Resolved
    UTC
    Resolved

    The patch applied to address the issue from Thursday, May 23 was not active on one of our controllers, this would cause System Test and Staging Test runs to fail some times. We have reapplied the patch now.

  • Monitoring
    UTC
    Monitoring

    We have applied a workaround for this issue. All current application have been pinned to vespa-8.345.20

  • Identified
    UTC
    Identified

    The fix for the previous incident, https://status.vespa.ai/clwjjxvt07700bvoom119zon5, was not fully successful, as it failed the System Test and Staging Test runs for applications deployed before the incident

System Test and Staging Test runs in new production deployments is not working
  • Resolved
    UTC
    Resolved
    This incident has been resolved.
  • Identified
    UTC
    Identified

    Due to a failure during the upgrade of our infrastructure today, the upgrade was incomplete and was causing issues for the System Test and Staging Test runs for applications. We have pinned all applications to their current vespa version to work around this.

    If you deploy a new application to production, you will be blocked by this. Please reach out to us if you see that your new application is failing deployment in the System Test and Staging Test runs.

Apr 2024

Vespa Cloud Console Access
  • Resolved
    UTC
    Resolved
    This incident has been resolved.
  • Monitoring
    UTC
    Monitoring

    A problem related to Google login was identified and has been remedied. Users will be asked to approve Google login when logging in the first time. Currently monitoring issue resolution.

Metrics missing in /prometheus/v1 API on vespa-8.334.22
  • Resolved
    UTC
    Resolved

    The metrics issue is fixed in vespa-8.336.22 that released today.

  • Identified
    UTC
    Identified

    We have detected a bug in the vespa-8.334.22 release that is preventing metrics to be retrieved using the /prometheus/v1 API.
    Please reach out to us if you have been upgraded and are using this API.

Vespa application and Hugging Face model hub
  • Resolved
    UTC
    Resolved

    We tested new deployments and it is working correctly.

  • Investigating
    UTC
    Investigating

    We are investigating deployment related issues for applications using model url where the url  points to Huggingface model hub.

The billing view in the console does not show bills
  • Resolved
    UTC
    Resolved
    This incident has been resolved.
  • Update
    UTC
    Update

    This affects all tenants which have an invoice created at August 2023 or earlier.

  • Identified
    UTC
    Identified

    We are continuing to work on a fix for this incident.

    Users will see "Unexpected error occurred"

Apr 2024 to Jun 2024

Next