On Monday, June 10, 2024, there was a service-impacting outage affecting a limited set of customers utilizing our NetApp service, which includes VMware, CloudSites, Cloud Load Balancers, Cloud VPN, and SAN-based services. Upon discovering the NetApp storage array(s) were inaccessible, our team launched an investigation and began working on a swift resolution. We immediately engaged the NetApp North America leadership and their engineering teams to assist us. By Monday evening, we restored service to an initial set of customers, with the majority up and running by the next morning.
This week, our focus has been to ensure all customer issues are resolved and services fully restored. While we’ve made significant progress, we are still addressing some remaining issues and intermittent disruptions, and we remain vigilant to keep systems online and stable.
Our investigation into the root cause of the outage is ongoing. While this comprehensive analysis will take time, it is crucial for developing strategies to prevent incidents in the future. We expect to conclude the investigation in the coming weeks.
This product is known for its performance and stability, which continues to be one of the main reasons customers choose this platform with us. In fact, we have experienced very high NetApp performance since the inception, with very limited and sparse outages. Our confidence continues to be high in our ability to provide continued high performance and stability for this product. Our teams are working diligently to deliver exactly that for our customers.
We sincerely apologize for the inconvenience this disruption has caused and thank you for your patience and understanding as we work to regain your trust. Your business is important to us, and we are committed to ensuring you have the stable and reliable environment you deserve.
If you are still experiencing any further issues with your services, please do not hesitate to contact a member of our support team via live-chat, ticket, email (support@liquidweb.com), or by phone at (800)-580-4985, (517)-322-0434 (international).
Posted Jun 12, 2024 - 19:51 CDT
Monitoring
Cloud Sites services have been restored and we continue to monitor for any further problems. If you are still experiencing issues, please contact our support team via live chat, ticket, or by phone at (800)-580-4985 or (517)-322-0434 (international).
We value your business, and apologize for the problems that have occurred. We recognize the serious nature of this incident and are working diligently to mitigate future issues.
Posted Jun 11, 2024 - 09:50 CDT
Identified
Cloudsites Windows customers are still experiencing issues and we are working to restore these services.
Posted Jun 11, 2024 - 08:27 CDT
Monitoring
Services have been restored for our impacted customers; however, there still may be some intermittent issues as we continue to address individual restoration activities.
We continue to monitor for any further problems. If you are still experiencing issues, please contact our support team via live chat, ticket, or by phone at (800)-580-4985 or (517)-322-0434 (international).
We value your business, and apologize for the problems that have occurred. We recognize the serious nature of this incident and are working diligently to mitigate future issues.
Posted Jun 11, 2024 - 07:42 CDT
Update
Our team is actively collaborating with the NetApp North America leadership and senior engineers to address the current issues with our NetApp-based products and services. We expect to be nearing a resolution path by mid-morning. We will keep you informed with updates on our progress and aim to provide a more detailed update at that time.
Posted Jun 11, 2024 - 05:56 CDT
Update
Our team is continuing to work with the NetApp North America leadership team and senior engineers to address the current situation involving our NetApp-based products and services. We will continue to provide updates on any progress we have made and aim to provide a more detailed update in the morning.
Posted Jun 11, 2024 - 02:19 CDT
Update
We continue to work closely with the NetApp North America leadership team and senior engineers to address the current situation involving our NetApp-based products and services. We'll continue to provide updates throughout the evening and aim to provide a more detailed update tomorrow morning.
Posted Jun 10, 2024 - 23:00 CDT
Update
Earlier today, we began investigating degraded performance and were able to isolate the issue to some areas of our NetApp-based products and services, which include VMware, CloudSites, and SAN-based services. We have engaged the NetApp North America leadership team and have multiple active bridges with senior NetApp engineering teams working towards a resolution.
We will continue to post updates throughout the evening. At this point, we aim to provide a more detailed update tomorrow morning. Please know that our team is dedicated to resolving this issue and will be working nonstop until it is resolved.
Posted Jun 10, 2024 - 21:08 CDT
Update
We are diligently working to resolve the issue affecting CloudSites services. Rest assured, our team is fully committed to finding a resolution as soon as possible.
At this time, we do not have an estimated restoration time for the affected services, but we will provide updates as soon as more information becomes available.
If you are experiencing a separate issue, please note that there may be delays, and we will address requests as soon as possible.
Posted Jun 10, 2024 - 18:32 CDT
Update
We are continuing to actively work on resolving the issue affecting Cloud Sites Services. Please note this issue also affects access to the Cloud Sites Control Panel.
Our team is making progress in identifying and pursuing a resolution path, investigating all potential solutions to ensure our customers are back online and running smoothly.
We will keep you updated on our progress as soon as information becomes available. Please be assured that bringing our customers back online is our number one priority.
If you are experiencing a separate issue, please note that we are experiencing significant delays and will address requests as soon as possible.
Posted Jun 10, 2024 - 16:14 CDT
Update
We are actively working on resolving the issue affecting Cloud Sites Services. Please note this issue also affects access to the Cloud Sites Control Panel.
Our team is making progress in identifying and pursuing a resolution path, investigating all potential solutions to ensure our customers are back online and running smoothly.
We will keep you updated on our progress as soon as information becomes available. Please be assured that bringing our customers back online is our number one priority.
If you are experiencing a separate issue, please note that we are experiencing significant delays and will address requests as soon as possible.
Posted Jun 10, 2024 - 14:41 CDT
Update
Systems and Network Engineering Teams are continuing to work through this issue. Your understanding and cooperation during this period are greatly appreciated as the teams work diligently to restore normal service. Should you have any inquiries, please do not hesitate to reach out to a member of our support team via live-chat, ticket, or by phone at (800)-580-4985, (517)-322-0434 (international)
Posted Jun 10, 2024 - 14:02 CDT
Update
Systems and Network Engineering Teams are still working to resolve this issue. Thank you for your continued patience. If you have any questions, please contact a member of our support team via live-chat, ticket, or by phone at (800)-580-4985, (517)-322-0434 (international).
Posted Jun 10, 2024 - 12:28 CDT
Update
Systems and Network Engineering Teams continue to work on implementing a fix for this issue. Impacted services will remain interrupted until the fix is fully implemented. The scope of this incident has been redefined to include all NetApp based products and services, which includes all Cloudsites services.
Thank you for your continued patience while our teams work to resolve this issue. If you have any questions, please contact a member of our support team via live-chat, ticket, or by phone at (800)-580-4985, (517)-322-0434 (international).
Posted Jun 10, 2024 - 11:12 CDT
Identified
Systems and Network Engineering Teams have identified the source of the service impact and are working to implement a fix. Impacted services will remain interrupted until the fix is fully implemented. We will provide further updates as soon as they become available.
Thank you for your continued patience while our teams work to resolve this issue. If you have any questions, please contact a member of our support team via live-chat, ticket, or by phone at (800)-580-4985, (517)-322-0434 (international).
Posted Jun 10, 2024 - 08:32 CDT
Update
We are continuing to investigate this.
Posted Jun 10, 2024 - 08:15 CDT
Investigating
We are currently investigating a problem affecting VMware, CloudSites, and the load balancers in our environment. Our team is examining the problem, and we will provide updates as soon as we make progress.
Thank you for your patience while we work to resolve this problem. If you have any questions, please contact a member of our support team via live-chat, ticket, or by phone at (800)-580-4985, (517)-322-0434 (international).