We are listening, are you?

Did you know Audible produced the first portable audio player in 1998? It held 4mb of data, or about two hours of audio and fit in your pocket.

Now imagine all of the devices created since then, from the first generation iPod to the iPhone, mp3 players and tablets, and iMacs to Chromebooks. At Audible, we want to bring the power of a well told story to everyone, across whatever devices they have. This is what led us to create our streaming architecture as well as a web based player we call the Cloud Player.

You can access the Cloud Player from http://audible.com/lib Click “Play” under any book in your library to launch in a separate window.

Streaming Platform

We began by looking within Amazon to see if other teams were doing anything similar. The Amazon MP3 team was working on streaming for Prime Music and the Instant Video team was using SmoothStreaming for their movie and TV show content. The Instant Video team was processing terabytes of video content daily so it made sense to use their workflow to also process Audible’s content. Audible team members worked alongside the Instant Video team to extend the systems and build in features specific to Audible.

As new content is ingested, the audio files are processed by this system, transcoded to different bit rates and then encrypted with PlayReady DRM. The encrypted files are then processed into streaming assets which contain a manifest, essentially a listing of all of the audio fragments, as well as the fragments themselves. We can request the audio in non-sequential order and play from any point with only minimal lag time. Finally, we upload to Akamai which provides optimizations for caching and content distribution.

Cloud Player allows our users to listen to their books instantly, without having to install any software.

Cloud Player

In order to enable playback for our users, we decided to build a web based streaming player. This allows our users to listen to their books without having to install any software in most cases. Initially we choose to utilize Silverlight for the player due to the fairly wide install base and because Smooth Streaming with DRM is supported by the player without additional libraries. The player is an HTML and javascript component for the UI which controls the Silverlight player. We built it in a modular way in case we ever decided to move away from Silverlight. The player manages the DRM, downloads and buffers the audio and allows for scrubbing around the audio stream. We also store and retrieve the user’s position, and keep it synched across all devices, allowing for seamless moving from device to device. We call this system “Whispersync”.

HLS/HDS

Silverlight worked well for us and our users, but we knew we would want to expand to devices that didn’t support Silverlight like mobile devices, and the bevy of living room players. In order to support this, we decided to use a combination of HLS and HDS. HLS, or HTTP Live Streaming, is Apple’s streaming protocol and is supported natively in Safari and on iOS. Many devices like Chromecast, Roku, and Amazon Fire TV also support HLS. HDS, or HTTP Dynamic Streaming, is Adobe’s HTTP Adaptive streaming protocol. We decided to HDS this as it fills the gaps for browsers that do not support HLS. Flash, which has a much higher install base than Silverlight can be used when HLS is not supported. Between the two, we support nearly every browser, platform, and device. As the streaming assets are quite similar to Smooth Streaming, with fragments and a manifest file, playback works similarly as well. We process the files in the same flow, and in fact still encode to Smooth Streaming as it supports offline playback better than HLS and HDS.

Because of the modularization of the player and playback component, we were able to move from Silverlight to HLS/HDS behind the scenes and not impact the customer experience. We actually noticed an improvement in buffering times and reduction of playback errors after the migration as well. It’s a good thing we moved quickly because Chrome began blocking Silverlight for Mac late last year and our users were able to continue use without any interruption.

We’ve received a ton of positive feedback from customers who don’t have smart phones and actually prefer the Cloud Player.

Clips

Another cool thing we can do with HTTP Adaptive Streaming is generating dynamic clips from full books. Because we use a manifest file to point to the relevant fragments, we can dynamically create a manifest that contains only the fragments we want for other uses, e.g. shorter samples of books for the site. We can even create clips based on things like popular highlights in the Kindle book or even allow users to cut their own clips for sharing.

My favorite part of working at a customer obsessed company like Audible is that the things we take the time to build are actually used by real people. We’ve received thank you notes from customers who don’t have smart phones and depend on Cloud Player. We’ve also received touching notes from users who have difficulty using some devices, and now enjoy listening to their books via the web based player. But the most meaningful praise for me was when my friend, who knew I worked for Audible, but not on what specifically said, “tell the people who made the cloud player that they have done a fantastic job”. I made sure to let them know!