Vespa Cloud - Notice history

All systems operational

Console - Operational

100% - uptime
Jul 2024 · 100.0%Aug · 100.0%Sep · 100.0%

Previous page

Next page

Jul 2024
Aug 2024
Sep 2024

Infrastructure - Operational

100% - uptime
Jul 2024 · 100.0%Aug · 100.0%Sep · 100.0%

Previous page

Next page

Jul 2024
Aug 2024
Sep 2024
100% - uptime

dev.aws-us-east-1c - Operational

100% - uptime
Jul 2024 · 100.0%Aug · 99.94%Sep · 100.0%

Previous page

Next page

Jul 2024
Aug 2024
Sep 2024

dev.gcp-us-central1-f - Operational

100% - uptime
Jul 2024 · 100.0%Aug · 100.0%Sep · 100.0%

Previous page

Next page

Jul 2024
Aug 2024
Sep 2024

perf.aws-us-east-1c - Operational

100% - uptime
Jul 2024 · 100.0%Aug · 100.0%Sep · 100.0%

Previous page

Next page

Jul 2024
Aug 2024
Sep 2024

test.aws-us-east-1c - Operational

100% - uptime
Jul 2024 · 100.0%Aug · 100.0%Sep · 100.0%

Previous page

Next page

Jul 2024
Aug 2024
Sep 2024

test.gcp-us-central1-f - Operational

100% - uptime
Jul 2024 · 100.0%Aug · 100.0%Sep · 100.0%

Previous page

Next page

Jul 2024
Aug 2024
Sep 2024

staging.aws-us-east-1c - Operational

100% - uptime
Jul 2024 · 100.0%Aug · 100.0%Sep · 100.0%

Previous page

Next page

Jul 2024
Aug 2024
Sep 2024

staging.gcp-us-central1-f - Operational

100% - uptime
Jul 2024 · 100.0%Aug · 100.0%Sep · 100.0%

Previous page

Next page

Jul 2024
Aug 2024
Sep 2024

prod.aws-us-east-1c - Operational

100% - uptime
Jul 2024 · 100.0%Aug · 100.0%Sep · 100.0%

Previous page

Next page

Jul 2024
Aug 2024
Sep 2024

prod.aws-us-west-2a - Operational

100% - uptime
Jul 2024 · 100.0%Aug · 100.0%Sep · 100.0%

Previous page

Next page

Jul 2024
Aug 2024
Sep 2024

prod.aws-eu-west-1a - Operational

100% - uptime
Jul 2024 · 100.0%Aug · 100.0%Sep · 100.0%

Previous page

Next page

Jul 2024
Aug 2024
Sep 2024

prod.ap-northeast-1a - Operational

100% - uptime
Jul 2024 · 100.0%Aug · 100.0%Sep · 100.0%

Previous page

Next page

Jul 2024
Aug 2024
Sep 2024

prod.gcp-europe-west3-b - Operational

100% - uptime
Jul 2024 · 100.0%Aug · 100.0%Sep · 100.0%

Previous page

Next page

Jul 2024
Aug 2024
Sep 2024

prod.gcp-us-central1-f - Operational

100% - uptime
Jul 2024 · 100.0%Aug · 100.0%Sep · 100.0%

Previous page

Next page

Jul 2024
Aug 2024
Sep 2024
100% - uptime

AWS - Operational

99% - uptime
Jul 2024 · 97.58%Aug · 100.0%Sep · 100.0%

Previous page

Next page

Jul 2024
Aug 2024
Sep 2024

GCP - Operational

100% - uptime
Jul 2024 · 100.0%Aug · 100.0%Sep · 100.0%

Previous page

Next page

Jul 2024
Aug 2024
Sep 2024

Notice history

Sep 2024

Vespa Cloud console maintenance
  • Completed
    September 04, 2024 at 07:20
    Completed
    September 04, 2024 at 07:20
    Maintenance has completed successfully
  • In progress
    September 04, 2024 at 07:15
    In progress
    September 04, 2024 at 07:15
    Maintenance is now in progress
  • Planned
    September 04, 2024 at 07:15
    Planned
    September 04, 2024 at 07:15

    We are planning a Vespa Cloud console maintenance. The console will be unavailable for a few minutes

New deployments, and node increase/replacement might fail in dev.gcp-us-central1-f
  • Resolved
    Resolved
    This incident has been resolved.
  • Investigating
    Investigating

    New hosts are not being provisioned in dev.gcp-us-central1-f. This prevents new dev deployments, and node replacements. We are currently investigating this incident.

Aug 2024

Node failures in dev environment
  • Resolved
    Resolved

    This incident has been resolved.

  • Update
    Update

    We implemented a fix and are currently monitoring the result. Due to the fact that there is no redundant storage in the dev environment, some documents may have been lost. We have restored as much as possible.

  • Monitoring
    Monitoring

    We implemented a fix and are currently monitoring the result.

  • Investigating
    Investigating

    Several nodes have been failed out in the dev environment. We are currently investigating this incident.

Jul 2024

New deployments, and node increase/replacement might fail for enclave
  • Resolved
    Resolved
    This incident has been resolved.
  • Monitoring
    Monitoring

    We implemented a fix and are currently monitoring the result.

  • Update
    Update

    We are still investigating. Existing deployments are currently stable

  • Investigating
    Investigating
    We are currently investigating this incident.
  • Resolved
    Resolved
    This incident has been resolved.
  • Monitoring
    Monitoring
    We implemented a fix and are currently monitoring the result.
  • Investigating
    Investigating
    We are currently investigating this incident.
  • Resolved
    Resolved
    This incident has been resolved.
  • Monitoring
    Monitoring

    The immediate issue has been resolved, we are monitoring the system

Application deployment may fail due to certificate provisioning
  • Resolved
    Resolved

    We have not seen any certificate issues the last day, resolving.

  • Monitoring
    Monitoring

    We implemented a fix and are currently monitoring the result.

    Please reach out to support if you are experiencing issues with deployments

  • Update
    Update
    We are continuing to work on a fix for this incident.
  • Identified
    Identified
    We are continuing to work on a fix for this incident.
  • Resolved
    Resolved

    We have now resolved this issue, thanks for hanging in there

  • Monitoring
    Monitoring

    Some certificate provisioning requests may fail at this time. This will appear as various errors deploying applications in Vespa Cloud.

    We have deployed a fix and we are working to stabilise the system and work through its backlog of certificate requests.

  • Update
    Update

    We are continuing to work on a fix for this incident. A side effect has appeared now.

    Deployments are failing with 'UNEXPECTED ERROR', due to 'Endpoints failed to show up within 15 minutes!'

  • Identified
    Identified

    We have identified the problem and will deploy a patch as soon as possible

  • Investigating
    Investigating

    We are currently investigating this incident.

    In the deployment view this will be shown as
    "Unexpected exception running run 474 of <zone> for <tenant>.<application>: Timed out after waiting PT5M to acquire lock '/controller/v1/locks/certificatePool'"


Outdated cost per hour values in console
  • Resolved
    Resolved
    This incident has been resolved.
  • Identified
    Identified

    The cost per hour values seen in the application lists will display old values and are currently not being updated. The underlying issue has been identified and a fix is under development.

Jul 2024 to Sep 2024

Next