Mike Hicks’ Post

Networks and the Internet have become fundamental to human needs. Anything that disrupts them has a bitter impact on our daily lives. One of the most extensive cloud outages happened in July 2024. The outage disrupted operations across multiple industries, halting flights and upending everything from banking and IT services to healthcare systems. Why did this happen? Let’s explore some of the reasons. Accelerated by the COVID-19 pandemic, governments and businesses have become increasingly dependent on a handful of interconnected technology companies that run on the cloud. In a routine job, a misconfigured network device in the Central US region triggered the failure in many networks. Unfortunately, the problem was worsened by an automated failover system that didn’t function as intended, spreading the issue to other regions and making this a historic event. With such massive interconnected solutions, Robust business continuity planning is crucial. A single-cloud strategy has to be adapted to multi-cloud strategies and multi-provider dependency. The outage has opened new debates. While a car manufacturer faces heavy penalties for faulty brakes, a software provider can often issue another update and move on. Do we get an era where software companies must pay a price for defective products? #techAiData #biggestCloudOutage2024 #Aptastories

Let's analyze the outage by examining the data at the heart of the internet: BGP I'll show you the data lying at the heart of the internet in real-time and explain the connection between global outages and monopolization. I recommend closely following the analogies and terms below. This way, you will gain many inside and understand how to detect global internet issues. You see the plots of the fluid BGP and ping data from the last three days. The first thing that stands out is an anomaly, due to an outage. Details are important: - High dependency occurs when most BGP paths to a particular AS pass through one or a few upstream ASes, indicating that the network heavily relies on these upstream providers for its internet connectivity. - Values close to 100% in the AS Dependency metric indicate a high level of reliance on specific ASes. Yes, think of the first thing that comes to your mind. High dependency on a single or a few ASes means that if these ASes experience an outage, the dependent network (and potentially its customer cone) will likely face significant connectivity issues. Outages in highly dependent ASes can cause cascading failures, where multiple networks experience disruptions due to their reliance on the affected AS. A high dependency on an upstream AS means that an outage in that AS can lead to widespread service disruptions across all dependent networks. An AS with a high dependency on a few upstream providers (values close to 100% in AS Dependency metrics) is more vulnerable to disruptions if those upstream providers experience an outage. In peer logic, we know who dominates these globally. A few companies... So what? For example, if a financial institution’s network heavily depends on a specific AS for connectivity, an outage in that AS can disrupt online banking services, trading platforms, and internal communications. Coinsidence? High dependency can exacerbate BGP route flapping during an outage, where routes rapidly change back and forth, causing further instability in the network. So a "software" incident can turn into a global BGP disaster. There are examples in history. Let's explain it with an analogy: - Imagine a city that gets its electricity from a single power plant. If that plant experiences a failure, the entire city will be without power because there is no alternative source. The city with only one power plant would experience a complete blackout, akin to a network with high AS dependency facing widespread connectivity issues if its main upstream provider goes down. Lots of details but I think you get the gist. The presentation consists of real data. And I have been following this with my live stream code that has been following BGP raw. When you examine this data well, you can know where to focus the incident response. Sometimes, we use this inside to detect widespread internet censorship. Note that. ^-^/ #freedomofinternet #cybersecurity #crowdstrike #microsoft #outage #informationsecurity #bgp

This recent #DataCenterFailure out of Singapore emphasizes the important aspects of data center management that we’re always advising our customers to keep in mind. And, the best way to sleep soundly at night, is to adopt a regular #PreventativeMaintenance program, completed only by #CertifiedTechnicians that have been properly trained by the manufacturer of the equipment. Read this article to find out the real ramifications of NOT executing these types of programs... #Vertiv Vertiv #BeyondTheProduct #ThermalManagementServices https://lnkd.in/d2NiCExY

Today's digital consumers have high expectations. How fast can performance issues arise? Can you resolve them before these demanding users notice? Latency-sensitive services and multi-domain applications demand precision, and service providers need real-time, carrier-grade service assurance tools to stay ahead. Uncover the 8️⃣ core elements that underpin carrier-grade service assurance here: https://okt.to/c31tK4 to enhance your understanding of this critical topic.

Insightful article on securing your multi-cloud network ecosystem. #multicloudsecurity #networkperformance

In this interview with Amber Jackson from Mobile Magazine, I explain why telecoms businesses should use packet-level data when undertaking digital transformation projects.

On Friday, 20th July 2024, our global dependency on technology was laid bare. The widespread outage disrupted everyday activities: accessing cash, traveling, and shopping became unexpectedly difficult. With over 5000 flight cancellations, the NHS and pharmacies unable to provide essential services, we faced disruption on a scale we've never seen before. As technology evolves, our reliance on it grows. Consumers will never accept a return to lesser services. Therefore, organizations must strategically manage their infrastructure and systems. Knowing where data and applications are hosted—be it in the cloud or secure data centres—is crucial. This awareness allows us to anticipate and mitigate the impacts of unforeseen changes. Infrastructure is everything; the data centre’s where things are supported, whether or not we go to cloud, whether or not we put it into our own secure data centres that are located on premises within our own organisations or in colocations where you have a lot more control on the IT elements. By exercising more control over the IT environments that you are running will allow you to understand the impacts to unexpected and uncontrolled changes to this environment. The strategic approach in terms of your support organisations and where their applications are being hosted The outage, triggered by a minor update error, reminds us of the importance of balance. While we must trust in security updates to protect against threats, vigilance is essential. Thorough governance around software releases, including rigorous testing, is necessary. Business continuity planning is another critical aspect. Implementing sandbox environments, stringent governance, and digital twins can prevent disruptions before they reach production. The costs of these measures are far less than the financial and operational impacts of major outages. The recent events underscore the necessity of vigilant, strategic IT management. Let's take these lessons to heart to build a more resilient digital future 💪 Colin Woods Indy Kambo #Tech #BusinessContinuity #Datacentre #infrastructure #freedomtechsolutions #security

Today's digital consumers have high expectations. How fast can performance issues arise? Can you resolve them before these demanding users notice? Latency-sensitive services and multi-domain applications demand precision, and service providers need real-time, carrier-grade service assurance tools to stay ahead. Uncover the 8️⃣ core elements that underpin carrier-grade service assurance here: https://okt.to/0Ggv8f to enhance your understanding of this critical topic.

🚨 Global IT Glitch: What Happened and How We're Moving Forward 🚨 Recently, a major global IT outage disrupted multiple sectors, from airlines to banking and media. This incident, primarily linked to Microsoft's Azure platform and a problematic update from CrowdStrike, highlighted vulnerabilities in our interconnected world. Here's a quick rundown of what went wrong and the proactive measures being taken to prevent future occurrences. 𝐀𝐢𝐫𝐥𝐢𝐧𝐞𝐬 𝐆𝐫𝐨𝐮𝐧𝐝𝐞𝐝: Major airlines like Delta, United, and IndiGo faced system outages, leading to manual check-ins and significant delays. 𝐁𝐚𝐧𝐤𝐢𝐧𝐠 𝐚𝐧𝐝 𝐌𝐞𝐝𝐢𝐚 𝐇𝐢𝐭: Banks experienced disruptions in online services, while media outlets, including Sky TV in the UK, went offline. 𝐌𝐢𝐜𝐫𝐨𝐬𝐨𝐟𝐭 𝐀𝐳𝐮𝐫𝐞 𝐚𝐧𝐝 𝐂𝐫𝐨𝐰𝐝𝐒𝐭𝐫𝐢𝐤𝐞: A configuration change in Azure and a faulty update from CrowdStrike caused widespread connectivity issues and the infamous Blue Screen of Death (BSOD) for Windows users. 𝐆𝐥𝐨𝐛𝐚𝐥 𝐈𝐧𝐟𝐫𝐚𝐬𝐭𝐫𝐮𝐜𝐭𝐮𝐫𝐞 𝐈𝐦𝐩𝐚𝐜𝐭𝐞𝐝: Flights were grounded, stock exchanges like the London Stock Exchange faced interruptions, and emergency services, including 911 operators, experienced outages. 𝐓𝐞𝐥𝐞𝐜𝐨𝐦𝐦𝐮𝐧𝐢𝐜𝐚𝐭𝐢𝐨𝐧𝐬 𝐀𝐟𝐟𝐞𝐜𝐭𝐞𝐝: Cellular networks, including major carriers like Verizon, saw significant disruptions impacting millions of users. To prevent such disruptions in the future, Microsoft has rolled out several critical measures: 𝐑𝐞𝐠𝐢𝐨𝐧𝐚𝐥 𝐂𝐨𝐧𝐟𝐢𝐠𝐮𝐫𝐚𝐭𝐢𝐨𝐧 𝐏𝐨𝐥𝐢𝐜𝐢𝐞𝐬: Limiting changes to a single region to prevent widespread impact. 𝐌𝐞𝐭𝐚𝐝𝐚𝐭𝐚 𝐌𝐚𝐧𝐚𝐠𝐞𝐦𝐞𝐧𝐭: Correcting metadata tags to prevent accidental deletion of critical resources. 𝐖𝐨𝐫𝐤𝐥𝐨𝐚𝐝 𝐑𝐞𝐠𝐢𝐨𝐧𝐚𝐥𝐢𝐳𝐚𝐭𝐢𝐨𝐧: Reducing the number of regions for workloads to minimize widespread effects. 𝐄𝐧𝐡𝐚𝐧𝐜𝐞𝐝 𝐈𝐧𝐜𝐢𝐝𝐞𝐧𝐭 𝐑𝐞𝐬𝐩𝐨𝐧𝐬𝐞: Improving automation for faster and more efficient response during incidents. 𝐅𝐚𝐬𝐭𝐞𝐫 𝐂𝐨𝐦𝐦𝐮𝐧𝐢𝐜𝐚𝐭𝐢𝐨𝐧: Updating tools for quicker notification and clearer communication during outages. 𝐈𝐦𝐩𝐫𝐨𝐯𝐞𝐝 𝐓𝐞𝐬𝐭𝐢𝐧𝐠: Increasing test coverage and verification procedures for configuration updates. 𝐏𝐚𝐫𝐚𝐥𝐥𝐞𝐥𝐢𝐳𝐚𝐭𝐢𝐨𝐧 𝐨𝐟 𝐑𝐞𝐜𝐨𝐯𝐞𝐫𝐲: Enhancing the speed of recovery by parallelizing model pool buildouts. 𝐀𝐜𝐭𝐢𝐯𝐞 𝐌𝐨𝐧𝐢𝐭𝐨𝐫𝐢𝐧𝐠: Developing additional active monitoring systems to detect issues early without relying solely on customer reports. This event underscores the importance of robust, resilient IT infrastructure. Let's stay informed and prepared. If you have insights or experiences to share about navigating IT disruptions, I'd love to hear from you. Together, we can build stronger systems and ensure smoother operations! #ITsecurity #CloudComputing #TechNews #BusinessContinuity #Microsoft #Azure #LinkedIn

Tired of slow systems lagging behind you? 🤔 Ever waited for screens to load, or games to respond? That's latency at work! 🐌 Understanding Latency: Factors, Measurement, and Impact on Applications and Systems ✪ Latency meaning: 1️⃣ Latency refers to the time it takes for a request to travel from its point of origin to its destination and receive a response. 🚀 2️⃣ It involves various factors such as processing time, network travel time, and queueing time, and can be measured in seconds, milliseconds, or nanoseconds. 📏 #definition ✪ How does Latency work?: 🛠️ 1️⃣ Latency occurs due to the time taken for transmitting an action, server processing, transmitting the response, and updating the screen. ⏳ 2️⃣ High latency can cause delays in real-time applications, such as online gaming, resulting in shot registration delays and inconsistencies. 🎮 #functionality ✪ What is Network Latency?: 🌐 1️⃣ Network Latency is the time it takes for data to travel from one point in a network to another and is measured in milliseconds or microseconds. 🕰️ 2️⃣ It involves steps like data initiation, encapsulation, transmission, processing, propagation, and reaching the destination. 📡 #network ✪ What is System Latency?: 💻 1️⃣ System latency refers to the overall time it takes for a request to go from its origin in the system to its destination and receive a response. 🔄 2️⃣ It encompasses processing time on both client and server, network transfers, and rendering delays. 📊 #system ✪ How does High Latency occur?: ❗ 1️⃣ High latency can be caused by physical distance, network congestion, inefficient network infrastructure, wireless interference, slow hardware, software inefficiency, database access, and resource competition. 🚧 #causes ✪ How to measure Latency?: 📏 1️⃣ Methods for measuring latency include tools like Ping, Traceroute, MTR, network monitoring tools, time stamps, performance profiling tools, and APM tools. 🛠️ 2️⃣ Tips for accurate measurement include repeating measurements, controlling variables, and using appropriate tools. 🔍 #measurement ✪ Latency in Transactions: 💳 1️⃣ Refers to the time for processing a request and receiving a response in transactions. ⌛ 2️⃣ Steps involved: Initiating Payment, Network Transmission, Payment Processing, Response Transmission, Confirmation Screen. 🔄 #transactions ✪ Latency in Gaming: 🎮 1️⃣ Refers to the delay between a player's action and response in gaming. ⏳ 2️⃣ Steps involved: Player Action, Network Transmission, Server Processing, Response Transmission, Your Screen Update. 🔄 #gaming P.S. High latency doesn't have to be your reality! 😉 Rahul Maheshwari #latency #performance #systems #technology #optimization