In an NFL game, every big play is a content earthquake. The moment Atlanta Falcons wide receiver Julio Jones catches a touchdown by ripping the ball out of a hapless defender's hands, the effects ripple far and wide. Somebody in the stands with a good view uploads their video to Instagram. Fantasy scores and online gamecasts update. Twitter goes nuts. Friends text "DID YOU SEE THAT" with a bunch of wide-eye emoji. And you get a push notification on your phone from ESPN or Bleacher Report, alerting you to the highlight.
If you're watching the game on a livestream rather than through your cable box, all that stuff usually happens before you even see the play. Whether you're on Hulu, Sling, YouTube TV, or some shady Russian website, every sports fan knows the pain of their stream lagging 30 seconds behind, maybe even more. Add in the occasional re-buffering and all the times when the picture goes so low-resolution it looks like you're watching the National Football Legos, and every fan starts dreaming of paying Comcast too much money again.
That's why, when Amazon won the right to stream Thursday Night Football during the 2017 season, the company decided not to pad the experience with bells and whistles. (Other than a "shoppable" pre-game show, anyway.) 1 Instead, the team sought to achieve the previously impossible: make the stream perfect.
Even delivering an excellent stream to someone with fast internet and a new set-top box poses challenges. But Amazon's Thursday Night Football plan included viewers in more than 200 countries, using more than 600 devices. Many of those people would have slow Wi-Fi and old devices. Others would be watching on their phones, slurping down who-knows-how-fast LTE and 3G. They all just wanted to watch the game, and they'd all get really annoyed if it didn't work.
Amazon has plenty of experience in streaming video, of course. Its Prime Video service streams to tens of millions of subscribers. Amazon's been investing for years in all things streaming media: buying content, building the back-end infrastructure, even creating and selling the devices people use to watch stuff. Few companies can claim as integrated a streaming system.
But live video is a bit different. Live sports even more so. If your feed of The Good Place is 30 seconds delayed, you likely won't notice, but you'll definitely notice the lag in a football game. "Low latency was a critical project for us when we started this project, which is not a big criteria when you look at an HBO stream," says BA Winston, the global head of digital video playback and delivery for Amazon Video. "Bringing it as close as possible to a cable or broadcast channel was critical."
The system the teams devised is quite complicated—even the "really simplified" flowchart Winston made for me involved lots of intertwining arrows, color-coded boxes, and words like "Heuristics Optimization." It seems to boil down to this: Amazon's job begins once NBC or CBS or whoever's covering the game for TV creates its master feed of the game. (Amazon adds its own commentary in some regions and languages, and keeps the standard broadcast in others.) That feed comes into Amazon's data centers, where it is immediately dispersed to servers around the world through a system called AWS Direct Connect. Next, Amazon's software encodes that video in all the resolutions, formats, and compressions it needs for the hundreds of devices it supports. It works within DRM boundaries, optimizes for bandwidth restrictions, and spits out the best version of the feed for every device.
Just before it heads out to your device, Amazon sends the feed through its in-house ad program. Rather than loading ads in separately on people's devices like most services, Amazon adds them into the stream directly. That, according to Keith Wymbs, the chief marketing officer at AWS Elemental, which provides a lot of the infrastructure for the service, solves a key problem in live streaming: combating ad-blockers. "Transcoding the ads so they match to the core content," he says, both helps advertisers and improves the stream quality. And with AWS Elemental, those ads can still be as personalized and targeted as the advertiser wants. To viewers, it just looks like TV.
After all that takes place, Amazon sends the final feed out to its many CDN partners, which are the data centers and servers all over the world that deliver the content to customers. Amazon's own CloudFront service is a massive CDN, but the company works with lots of partners as well. And then, finally, you get the video. It sounds like a lot, but ideally it all happens in just a few seconds.
A couple of weeks ago, during a criminally boring Thursday Night Football game between the bad Denver Broncos and the terrible Indianapolis Colts, I put the whole thing to the test. I flipped on three feeds of the game at once: one on TV, one on Amazon's Prime Video app for iOS, and one in Chrome on my laptop. Neither Amazon stream ever caught up to the TV, but they were solidly close: My laptop ran about 20 seconds behind, and on the iPhone it was only about seven seconds back. The stream never died, even over a bum LTE connection, and only dropped to 8-bit sorts of resolutions a couple of times. It still wasn't as clean as watching it on TV, but the setup legitimately worked. Which is no small feat.
After 10 games, and more than 17 million viewers in more than 200 countries, Amazon's run of football broadcasting ends on Christmas—at least for now. But the infrastructure the company created has a bright and large future. Earlier this fall Amazon announced a service called AWS Media Services, which lets anyone spin up a video channel, live or on-demand, about as easily as hosting a website on AWS. "We're now to the point where the infrastructure, at least the core part of it, can be spun up in minutes," Wymbs says. All the ad-targeting, all the bandwidth, all the transcoding and DRM, all available to anyone with some video and a credit card.
Meanwhile, other providers are about to learn how hard it is to stream football. NBC signed a deal to stream Sunday Night Football to anyone with a cable subscription, starting next season. Verizon, which has been streaming NFL games to its subscribers for years, will simultaneously start streaming everything to everyone through Yahoo Sports and AOL. They're all dedicated to football because it's the rare event popular enough to make people go wherever they must in order to see it. They're also starting with football because they know, as Amazon learned, that live sports may be the hardest problem in streaming. If they can churn out a super-reliable, low-latency, ad-targeting stream of the most popular sport in America, they can stream anything.
1UPDATE: This story has been updated to reflect the existence of Amazon's "shoppable" pre-game show, which does in fact exist.
