Phil and Stuart talk iPlayer Transcoding
/
Phil Cluff and Stuart Hicks getting ready for the off
We love it when ex-students come back to the department to tell us what they've been up to. Phil Cluff graduated a few years ago and Stuart Hicks a couple of years after that. Since graduation they've both been working for the BBC on the iPlayer team.
If your'e not from the UK you might not appreciate just how wonderful BBC iPlayer is. It lets you consume BBC TV output on any device, in any place. All the output of the BBC channels, plus the content from BBC local TV, is made available shortly after broadcast so you really don't have to worry about missing things. The programmes are available for a number of days after transmission and they are also made available for download. It's awesome. Phil and Stuart told us that 3% of the consumption of BBC output is now via iPlayer and it is growing rapidly. There have been around 28 Million downloads of the iPlayer apps across the various platforms, they get around 10 million requests a day and release around 700 hours of new content a week. Amazing stuff.
Of course, getting all that content onto the internet is a tricky job. The original content has got to be converted into the different video formats and sizes that are consumed by the users, and this must happen as soon after broadcast as possible. Nobody wants to have to wait until tomorrow to see today's news. There are huge peaks and troughs in demand and this makes it tricky to manage the computing resource that you need for the task.
A little while back the BBC decided to move their transcoding (as this is called) activities into the cloud. Phil and Stuart described the cloud as "A collection of services that someone else runs for you, which lets you not care about hardware". Which is just what the BBC needed. So they put together a project to build a cloud based "Video Factory" from scratch in 18 months. And they did it.
Large computing projects are notable for failing or being expensively late. The Video Factory was neither of these things. Phil put this down to the way the design was based on a number of small components, each of which performs one step in the process of getting the video into your iPhone or whatever. The workflow of the video data through the system is managed as a series of messages that are passed from one component to each other. Components take messages out of their input queue, perform their part of the process on the message and pass it on to the next component. If things get busy and queues start to fill up the system can "spin up" extra processing resource in the form of further servers in the cloud which can run components to work on the extra elements.
And because each component has very well defined inputs and outputs this makes testing much easier. Phil and Stuart talked about Behaviour Driven Development, which is where you express what you want to achieve in terms of business outcomes. They used a language called Cucumber which lets you express your requirements in a really easy to understand way, and will also produce a whole framework of tests you can use to prove the solution. This is very, very interesting stuff.
It was great to see Phil and Stuart again. It looks like they are doing very well. It was also really nice to hear them echo a lot of the things that we tell our students during the course. They had some very useful things to say about the craft of development. You can find the slide deck of their presentation here. I strongly advise you to take a look at the last few slides, there are some lovely references there that you can use to find out more.
