The Chain and the Link
The recent outage at Comcast due to a Level3 configuration mistake, is a great example of 'the chain only being as strong as its weakest link'.
Last week's misconfiguration at Level 3 triggered a wave of outages across North America's larger SPs.
Due to an attempt to optimize routing policies, Level3 actually created a route leak. A route leak means that traffic on route to a destination was inefficiently routed, causing delays and poor response time. Or traffic fails to reach the destination all the together, causing much more agony. The latter comment is an understatement, since it caused serious issues. Not just issues at the backbone and provider level, also at peering providers across the nation's borders. This misconfiguration truly caused a wave that ripples across the network, across borders, unobstructed.
This outage demonstrates that human errors still happen. They can have a major impact. Some says that they only happened 'occasionally' over the last decade. Fair enough, except that 10 years ago we didn't rely on the Internet as much as we do today. So, there is, in my opinion, no excuse possible, more focus on change management, more control over who can access what and assess the impact in a 'what-if'. Reliable and predictable Internet connectivity is key.
My reference to the Chain and the Link expression stands... the Web can apparently only be as strong as weakest Route. That's not a comfortable feeling.
[Picture shown by downdetector.com]