Please ensure Javascript is enabled for purposes of website accessibility

Amazon Web Services Failed and We Still Streamed Netflix Perfectly -- Here's Why

By Tim Beyers - Sep 28, 2015 at 8:00PM

You’re reading a free article with opinions that may differ from The Motley Fool’s Premium Investing Services. Become a Motley Fool member today to get instant access to our top analyst recommendations, in-depth research, investing resources, and more. Learn More

Netflix learned from the Christmas Eve outage of 2012.

Despite an outage, I was still able to stream Person of Interest flawlessly. Image source: CBS.

When Amazon.com's (AMZN 3.15%) Web Services infrastructure suffered cascading problems on Sunday, Sept. 20, one notable media outlet, techtimes.com, reported it as a "monstrous" outage.

Um, no. The Internet did not fall to pieces.

Shoppers still browsed Amazon, Tinder users still found dates, Redditors... did something else, and my wife and I still managed to binge-stream episodes of Person of Interest via Netflix (NFLX 2.90%), the highest-profile site affected by the outage.

How it happened
According to Amazon's account of the outage, AWS's DynamoDB experienced what the company called a "service event" due to problems with how the database handles metadata -- i.e., data that describes data -- and the storage servers used to capture information in tables.

In its postmortem, Amazon said:

On Sunday morning, a portion of the metadata service responses exceeded the retrieval and transmission time allowed by storage servers. As a result, some of the storage servers were unable to obtain their membership data, and removed themselves from taking requests.

Why you should care
Still confused? In the simplest terms, DynamoDB "timed out" over and over and over again, leading to a cascading flow of problems. Think of it as the digital equivalent of clogged pipes. Eventually, nothing gets through.

Frankly, I think Amazon did about as well it could have under the circumstances. Outages happen. Only the (small-f) foolish don't plan for them.

In Amazon's case, it took about six hours to recover to full service -- not quite the scale of the outage that plagued Netflix on Christmas Eve 2012. That one cut off viewers for 7.5 hours, making grinches out of millions who hoped to celebrate the holiday spirit watching Netflix.

Why you should love Netflix even more today
Netflix had far fewer problems this go-round. We didn't even notice. To be fair, that's at least partially because of where we live -- Colorado -- as most of the problem was centered in Amazon's East Coast data center operations. Yet that's not the only reason; Netflix has created a whole system to account for problems in delivering streams to its 60 million-plus customers around the world.

According to an insightful account at TechRepublic, the company has built automated troublemakers dubbed its "Simian Army." Their only job: poke, prod, and otherwise try to break the network and expose weaknesses. You know how the government hires hackers? The Simian Army is to Netflix what hackers are to the feds; they help surface and fix issues before they become problems.

It's because of this level of "chaos engineering," as Netflix calls it, that the company was able to easily weather the fallout of the DynamoDB outage. And it's why I still won't sell my shares, despite the clear premium at which they trade.

According to TechRepublic, Netflix was able to quickly redirect traffic to datacenters in an unaffected area. "Netflix was able to do this because it practices what it refers to as multi-region, active-active replication - where all of the data needed for its services is replicated between different AWS regions in a way that allows rapid recovery from failures," TechRepublic reported.

What do you think of Amazon stock at current prices? What about Netflix stock? Tell me on Twitter, or reach out on Google Plus. I may use your comment in a follow-up article.

Invest Smarter with The Motley Fool

Join Over 1 Million Premium Members Receiving…

  • New Stock Picks Each Month
  • Detailed Analysis of Companies
  • Model Portfolios
  • Live Streaming During Market Hours
  • And Much More
Get Started Now

Stocks Mentioned

Netflix, Inc. Stock Quote
Netflix, Inc.
NFLX
$179.95 (2.90%) $5.08
Amazon.com, Inc. Stock Quote
Amazon.com, Inc.
AMZN
$109.56 (3.15%) $3.35

*Average returns of all recommendations since inception. Cost basis and return based on previous market day close.

Related Articles

Motley Fool Returns

Motley Fool Stock Advisor

Market-beating stocks from our award-winning analyst team.

Stock Advisor Returns
316%
 
S&P 500 Returns
112%

Calculated by average return of all stock recommendations since inception of the Stock Advisor service in February of 2002. Returns as of 07/03/2022.

Discounted offers are only available to new members. Stock Advisor list price is $199 per year.

Premium Investing Services

Invest better with The Motley Fool. Get stock recommendations, portfolio guidance, and more from The Motley Fool's premium services.