Intermedia Cloud Communications’ Post

💡 Important Reminder for All Tech Professionals 💡 In light of the recent global outage caused by a system update at CrowdStrike, I wanted to remind everyone of the critical importance of thorough testing before rolling out any updates. This incident underscores the need to meticulously evaluate all components, including sub-channels, to prevent such disruptions. 🔍 Key Takeaways: 1. Thorough Testing: Always ensure that every part of the system is rigorously tested to identify and fix potential issues before an update goes live. 2. Learning from Mistakes: Outages can be a tough learning experience, but they also offer valuable insights into improving our processes. 3. Stay Resilient: To everyone affected, hang in there. These challenges make us stronger and drive us to enhance our systems. Let’s continue striving for excellence and reliability in our tech solutions. #TechCommunity #SystemUpdate #QualityAssurance #Resilience #LearningExperience https://lnkd.in/gpGX_vYX

In 2020 my team had a foresight to introduce a custom CrowdStrike release process to prevent this exact outcome. We met with CrowdStrike support team and asked to have our release process as a standard out of the box option, but clearly that did not happen. Our release process looks like this: DEV=N INT=DEV > INT and stable/unchanged for 7 days PROD=INT > PROD and stable/unchanged for 7 days This introduces a 2 week test buffer for stability checks and prevents this very silliness.

Good morning everyone, Hope you all had a good weekend. However, it'd be understandable if many of you had a rough time considering the recent CrowdStrike issue that crippled numerous organizations across Banks, Airlines and various other industries. If you're still experiencing the issue, please consult your IT Team on the following fix before implementing it. 1. Boot into Safe Mode (https://lnkd.in/gtrK8bG3) 2. Locate to C:\Windows\System32\drivers\CrowdStrike 3. Delete “C-00000291*.sys” file 4. Restart your PC normally Hope this helps and have a smashing week ahead everyone!

The CrowdStrike outage illustrates the vast reliance both businesses and consumers place upon technology and our expectation that it will always “continue to work.” However, one small bug and a massive rollout plan can lead to catastrophic and expensive results that occurred with airlines canceling all global flights, hospitals operating on “paper mode,” and even financial institutions having their systems impacted. This defect was a costly reminder that software systems and consumers alike need to have plans in place to mitigate risk and avoid the reliance that can lead to these sorts of outages. Discussing the idea that human oversight in a technological world can lead to these sorts of struggles, Dave Lee of the Harvard Business Review wrote, “Technology alone cannot safeguard against risk; it requires human judgment and strategic oversight to fully address the complexities of modern operation environments.” The full fallout of this outage is yet to be seen, but it does create the idea of having plans in place to mitigate risk. To learn more about the CrowdStrike outage and how you can support your business, please visit our blog: https://lnkd.in/gX-e3URU

Want to learn how your business can minimize its potential future damages in the event of a cyber-attack? Check out this Legal Alert prepared by my Maslon colleagues.

It wasn't a relaxing weekend for anyone in the world of IT. If you're still reeling from Friday's fall out, read Info-Tech Research Group's key response steps for the global CrowdStrike outage: https://lnkd.in/gc-CgVzx #businesscontinuity #incidentresponse #crisismanagement #vendormanagement #changemanagement

The CrowdStrike incident shows the high cost of not properly testing IT systems. When companies skip thorough testing, they risk serious problems that can lead to big financial losses and damage their reputation. For example, the CrowdStrike outage drained an estimated $5.4 billion from Fortune 500 companies, and Delta's CEO reported an IT outage costing the airline $500 million. This incident is a clear reminder of how vital it is to invest in proper testing to catch issues early and ensure reliability. Neglecting testing can be very costly in the long run. Jokes and memes aside, thorough testing isn't just a "nice-to-have"; it's essential for maintaining IT system integrity and trustworthiness. Investing in comprehensive testing helps ensure system reliability, prevents financial losses, and protects a company's reputation in an increasingly digital world. Learn from others, and protect your assets.

One of the lessons we learned from Crowdstrike incident about Redundancy is super important in technology because it helps prevent downtime and data loss. Think of it like having a backup plan. If one system fails, another one can take over, keeping things running smoothly.

For those impacted by today's Crowdsrike agent issue, this is really useful

Following on from our article earlier this month, and now that the dust has started to settle with the CrowdStrike outage, we have taken some time to consider what options customers have when looking for contractual remedies in IT contracts and the challenges this can bring!