Find out about the problems and challenges of delivering high-quality audio for a wide range of platforms in an automated and cost-effective manner. Get the inside scoop on real use case studies that show how Emotion have worked with customers to provide solutions that integrate into their workflows.
You can download a free trial of Engine here.
Webinar transcript:
Cindy:
Welcome everyone. I’m so glad you’re here for our webinar today on one master, multiple deliverables. And I have MC Patel, CEO at Emotion Systems. Hey MC, how are you?
MC:
I’m good, Cindy.
Cindy:
Hey, let’s dig right in and talk about one master, many deliverables. What do you mean? Tell us all about it.
MC:
So let me just go back a little bit in history. If you made a piece of content, let’s say a movie, what you would have done 20 years ago is run off of a number of film prints. And then they would go on a jumbo and get distributed around to all the theaters. And similarly, if you made a episodic series, the tape would go to the network and it will be transmitted.
MC:
Now, obviously, the means of distribution have expanded dramatically. Film has demised. So now there is a great interest in saying every time you generate a piece of content, you want to monetize it as quickly as possible in as many markets as possible. And so what the industry is looking for and has to do is to create a number of masters out of that one original.
MC:
Now, there are plenty of solutions for video that allow you to do this. And for cost-effectiveness, these solutions are automated. There aren’t many solutions for audio. And so what we mean by one master many deliverables is really that we get one piece of audio and we regenerate audio that’s suitable for a number of platforms, a number of countries and so on. So that’s one master, many deliverables.
Cindy:
Got it. And can you tell us a little bit about how Emotion Systems got into this space originally?
MC:
Sure. So we’re 10 years old and we started off with an idea which was basically we wanted to make software modules. We recognized that the industry was moving away from tape and into files. And so we wanted to create software modules that would solve specific problems in the final domain. And we started off by examining the market and saying, what sort of things people are looking for?
MC:
And talking to a number of people, they said that they use their edit suites, which really are for creative purposes, to do a lot of mundane things. And we had a range of ideas, inserting bars in tone, clocks and so on. But at that time, what was very topical was loudness. The industry was changing from peak-based measurement to program loudness. And a lot of people didn’t understand what loudness was.
MC:
So what happened is people came to us and said, “Tell us about loudness. How do we solve that?” Now, instead of trying to solve an old problem in a different way, we saw the opportunity to solve a new problem and use our technology. So we developed a product that would measure loudness in a file. But the real challenge that people had is they were very familiar with what to do with the old audio if it was wrong and so on. They weren’t with this.
MC:
So they said, “Well, this is great, you telling me what’s wrong. What do I do about it?” And so that was the germ of the idea that started Emotion off. And in essence, what we recognized is that once you’ve married the video and audio together in a file, it’s very difficult to operate on one and do something with it. Normally you’d go into an edit suite, then you have access to the video or the audio or the metadata, and you do something with it.
MC:
So we wrote a piece of software that allowed us to read the file, take the audio out, do something with it, copy the file, copy modified audio back in, and we wouldn’t have touched the video or the metadata. So Emotion started off doing loudness in this manner with a manual solution. And very quickly the market came back to us and said, “This is really interesting.”
MC:
But what they said is, “If you can take the audio out, you can do more than loudness.” I know our friends that Dolby threw a challenge to us and said, “Imagine we had a fire with Dolby E,” so encoded for carrying more channels. “What if I needed to do loudness measurement and correction in that?” So we licensed the encoders, the decoders, and came up with an algorithm, which said, “If I detect Dolby E, I will decode it, I will measure the loudness or correct it. I will modify the metadata to reflect the changes I have made. I’ll end code that, put that back into the file.”
MC:
And the operator now doesn’t know that we’ve just carried out a measurement and correction in a file, the Dolby E. Now, clearly they do want to know what we’ve done, so we generate a report. When we do the analysis, we write a report. We said, “When opened the file, we found this was the loudness. These were the parameters that failed. These were the parameters that were okay, and that we corrected them.”
MC:
So that was the process some six years ago or so. And then a company that had a slightly inverse problem to what we’re trying to solve, but I’m just telling you the story now. The company said, “Well, we get from our clients video files with 14 variations of audio.” And I’ll tell you about that in a second. But what they said is, “We want to… Because we need to loudness correct. We need to Dolby encode. And some of the tracks are not in the right place so we need to map the tracks. And there are 14 different workflows.”
MC:
So for example, their normal expectation would be stereo and 5.1. And what they would do is they would loudness correct both of those, Dolby encode the 5.1. And then the output would be stereo Dolby E. And the stereo would be replicated as would the Dolby E and the stereo three and four. And they could then play that out for their transmission service. So we recognized that this is doable. We can encode, we can decode, we can do the track manipulation.
MC:
But they said, “14 workflows, we need to automate it.” So we wrote an API for the product so that you could address it from an external device like a MAM. And we then allowed you with a user interface to program the workflow. So if my file has just stereo, do they see if it just has 5.1 do this, et cetera. And then the MAM says, “For this file, apply this workflow and place the result in this location.”
MC:
So it’s a REST API. It’s very, very straightforward. Now, when we build this, this is what product we call the engine. And this is where the story of the one master many deliverables really begins. Because now people are coming to us and saying, “Well, actually, for this deliverable, we need this deliverable, we need this.” Now, as I said, in this particular instance, it was many masters, if you could say it that way, in one deliverable, because you’re delivering to a broadcast house.
MC:
Many deliverables was that they had more channels. They had 13, 14 channels. So the challenge there is to say, well, actually, I need to process 2000 hours a month say. And so I can’t have just one file at a time. So one of the things we built into engine was the ability to process more than one file at a time. So you could license the product to either you pick and choose your modules. I don’t need Dolby E, I need loudness, I need track mapping, et cetera.
MC:
And then you say, “I need to do so many hours, therefore I need to process three files at a time, four files at a time,” or generally the content that needs to be transmitted next week arrives on a Friday and we have 48 hours to process. So there’s all those variations that you have to put in. So that’s who we are. And one of the things we did is we decided that we would focus our effort and energy on being the best at audio.
MC:
There’s many people who do video, and we recognized that not many people did audio. So what that meant is really we went back to the customers and basically said, “Whatever your audio problems are, please talk to us and we will either talk to you about how we go about solving them and where possible and appropriate we will provide the solution for you.”
Cindy:
Nice. So we’re going to get into some examples here in just a minute. But just to recap what you said, if I heard all the important points there, it’s really interesting how you started out by dealing with audio and looking at a file, being able to pull that audio out and do something to it, and then put the audio back in. And then you end up working with people who need 14 variations of files in terms of their deliverables.
Cindy:
I like how you’re talking about the different modules and so people can kind of have a menu of the different pieces that they need. You kind of alluded to this. You’re going to talk about some examples and samples. Tell me more about the challenges that come up around the multiple deliverables.
MC:
So basically, the challenges are in this industry worldwide. You would think we have standards. I mean, we spend a lot of time on standards and deliverables. But standards move slowly and not everyone follows them and so on. So the first challenge is loudness. Even though it’s been around for 10 years, we still have conversations with customers about, “Tell us about loudness. We have this scenario, that scenario.”
MC:
And every country has a slightly different loudness standard. Now, that’s terrible news when you under distributed because there’s a standard in the UK, there’s a standard in Australia, there’s a standard in the U.S., there’s a standard in Japan. So by us specializing in it, the first challenge is, can we do the different standards worldwide? And three years or so back, Netflix came up with a variation of their own.
MC:
They did some analysis and they said the standards that are there for broadcasters aren’t the standards we want. The base is still the same. And so engine has a loudness configuration menu which allows you to set up any standard, any legacy. We put all the meters in when we put this thing in. So you can create a standard, give it a name, and then use that in any of your workflows.
MC:
So loudness is one of them. The other thing, signal processing tools. So you pull out your favorite episode from your archive. I like to use Fawlty Towers. It was shortly in the ’70s, mixed in stereo. You want to do a high definition deliverable, you need 5.1. So can we have an upmix please? Or you have a modern program that has been delivered to you in a country that’s still only transmits stereo.
MC:
So now you have to take your 5.1 into a done mix. The other thing that happens is you may get a master file from Hollywood, where they give you a 24 channel audio file, where you have the program stereo in one and two, the 5.1 in seven and eight. Then they’ll give you a second language in the next eight channels. And then they’ll give you mixing effects in the other eight.
MC:
And that’s all good, because what they’re saying is when we give you this, we’re giving you all the tools to create dubs inversions. But your deliverables will say it has to be eight channels. It has to be stereo in one and two and 5.1 in seven and eight. So that’s a challenge. You have to take… Now, for a simple process like that, even if you say I want to do the English, which is perfectly done in one and two and three to eight, you still have to go into an edit suite or use a tool to take the other channels off.
MC:
So we do loudness. We do signal processing, upmix, downmix. We do another form of signal processing where the master may have been a 24 frame, short origination. And very often for European deliveries, you’ll just speed up that the 25 frames. Of course, the video runs the program duration contracts because you speeded it up… Actually, in that case, speeded it up.
MC:
And the audio will be speeded up, but the pitch changes. So you need to correct the pitch and change the duration at the same time. So these are signal processing modules that we’ve developed, again, because customers ask for them. And then we have what we call file manipulation. Now, file manipulation is track mapping, adding channels, if your archive is two channels and your deliverable has to be eight channels, or removing channels.
MC:
As I said, if you had 24 channels coming in and eight need to go out. When you do all these manipulations, you have to label the tracks as left and right and the various surround components. And you also have to do language tagging. So in some play out platforms, you may have multiple languages so you need to tag each of the language. So you can see that the challenges of making these multiple masters are not insignificant.
MC:
And because bulk of the budget and attention goes into the creative process, this all has to be done on a budget and at great speed, without errors. And so us providing an automated or automateable platform is a very key part of it. Us about providing a scalable solution is a very key part of it. So I hope I’ve kind of given you a flavor of the challenges.
Cindy:
Yeah. I love the challenges. And Fawlty Towers, that’s perfect. 5.1 for Fawlty Towers. That’s great. And so you were talking about loudness and dealing with that, of course, and signal processing. And then looking at encoding and all of those are really clear and make sense. Those are challenges I can totally understand. Can you talk a little bit about some examples where those get used in real life?
MC:
Sure. So the first one I spoke about was the playout center. We have a lot of playout centers that use us for this. And the playout center has basically got… I mean, I can name a couple. Red Bee in France are big users. They do the playout for CANAL+. And we have a similar setup in the UK. We’ve got the National Playout Center in Australia, which is putting 60 plus channels.
MC:
And until recently, we had DMC in Holland that was doing 96 channels. And each one of them had a slightly different requirement because of the content they were being delivered or they were getting from their suppliers and also the internal arrangements as to how the architecture of the playout facility was done. So there was one that had a integration with TDL, another one where they did their own integration. So they worked with us.
MC:
Another one who works with just watch folders. They say, “We do not have an automation system. We just want to give you watch folders.” We also have people who are working with Telestream Vantage, and we have a customer in Viacom who uses all of the different methods. So Viacom have a MAM. They have Aspera Orchestrator triggering workflows. They have Vantage.
MC:
Their post-production use our e-client technology, which means every editor gets access to the engine and can post a job into it. So there’s a wide variety of this for playout and operations. And then we have places in Los Angeles who are essentially saying, “We will take a master from the Hollywood studios and then create deliverables for every country or as many countries as you like for theatrical, for broadcast and for online.” So iTunes, Netflix. They would do everything.
MC:
Now, for them, again, they have the challenge of saying, “Can we do loudness for Japan, for Australia, for Germany, Netflix, for Apple iTunes?” And also they want encoding for them or they may want stereo or 5.1, et cetera. So the idea here is that there is… It sounds very complex, but you set up a workflow for a particular deliverable, give it a name and the APR, or the API, you can call it, and say, “This file needs this workflow applied to it.”
MC:
The workflow can be very simple. It could be just one step, loudness only, or it could be six or seven steps. So six or seven steps, I’ll give you an example. I have stereo, loudness correct the stereo. Do an upmix. Loudness correctly the upmix. Dolby encode the upmix. Track map it, channel label it, and output it. That’s a multistage workflow, but you say this is a set up for Tx channel one. And the MAM calls up for a call workflow for Tx channel one and away you go.
Cindy:
And so how would, for example, the facility in Los Angeles, maybe if they’re using a Telestream Vantage and a MAM system. What would that look like if they were actually doing that on a day to day basis?
MC:
So you don’t see it is the first thing, because it’s a machine talking to a machine. But what I can do is give you some scenarios. It’s a good question. Recently, as we all have been working from home, we’ve had to… People are sitting there saying, “Oh my God, how do I do my job?” And we’ve had our customers calling us and saying, “Well, you know what? I can see what’s going on with your engine,” because I bring up the e-client and it’s monitoring all the jobs and so on.
MC:
I can go away into a TeamViewer or PC anywhere and initiate things and so on. But a versatile setup where you have clients posting from multiple devices, what happens inside engine is there is a job manager. We call this e-flow. So what e-flow says, “I will receive jobs from Vantage, from Orchestrator, from the MAM system, from the watch folders, from the e-client. I will put them in a queue and then I will look around and see what resources do I have and how many.”
MC:
So if I have the ability to process four engines, we call each processor an ESP, an Emotion Signal Processor. It says, number one is busy. Number two is free. So I will open number two, post a job into it. The next one that becomes free maybe number four. And it will do that. So it basically opens and shuts these processes as it does the jobs and posts them.
MC:
The jobs get queued inside e-flow in the first come first serve basis. So anybody can post it and they just join the queue. And then someone comes along and says, “I have a priority job.” So we give you the option of assigning a priority to the job. And what it will do is it will jump the queue and the next ESP that’s available will be used to process that job. So that’s kind of given you some idea of what the set up can look like.
Cindy:
Okay. So if somebody’s got a master and they need to get it out to multiple, I don’t know, stations, playout, channels, that’s the way to do it. And I like hearing about all the different ways really how you put the workflow together. And like you said, it’s just a machine talking to a machine. So once it’s set up, it’s off and going. And your examples are great, because you’re talking about deliverables for broadcast and for online platforms. We had a question earlier about social media or online distribution. Could you talk about that?
MC:
So this is a new thing. That the broadcast world has bodies that meet and discuss. They take a long time. CMT does it. EBU do it. And then agree some standards, to say in order to get the best user experience, this is a standard that we want you to adhere to. The social media platforms are essentially about anybody can post anything in any form. And they have some QC or they have some guidelines, but they’re not out and fast.
MC:
And so if a professional is trying to deliver to a social media platform, they have concern. They have spent a lot of money to make it sound good and look good. So they want to deliver to some level of quality and consistency. And so we often get asked, “What do we do if we want to do something for YouTube or for…” Well, iTunes is a platform as opposed to Apple TV, if you like. But the guidelines are quite vague.
MC:
Sometimes there’s a spec that is available which says, this is what we would like you to do. The key thing is it is always different to the way you make the master. So what we do is we are a very consultative company where we like to discuss the issues with our customers and then propose solutions and then see how they get on. And then we may iterate a couple of times to get it right.
MC:
Because at the end of the day, the customer, as I said, this expensively made content needs to be given the best treatment possible. So for social media, I’ll probably give you an example as a challenge that was sent to us. We were in Soho talking to an audio company that was just getting into feature films. And his issue was that he would have a 7.1 master.
MC:
Because it was a European or British company, the feature in question was shot at 25. So what he said to me is, “Look, for my deliverables, I’m going to start with this 7.1. And then what I want to do is I want to deliver a 5.1 that’s loudness corrected as a wave file because it was an audio house.” But the first wave file is a single file that has all six audio channels into interleaped.
MC:
Then I also want six mono files that represent the 5.1. I now want to do a downmix and get a stereo. I want to loudness correct that and give me the stereo as an interleaped file in the stereo three monos. So if you’re following this, we’ve got interleaped 5.1, six monos, a stereo, two monos. And then I wanted to do the same for a 24 frames a second so that I can do distribution to those countries.
MC:
So I said that’s great. We can do this for European loudness standard, the U.S. loudness standard, if you like. And I said, we can also do the same for social media. We can create the same set, but we would apply a Netflix setting, et cetera. Now, engine is capable of handling up to 64 channels. So the end of it, we had the ability to deliver I think it turned out to be 12 or 15 or some number of deliverables in one hit.
MC:
That was a challenge that is not necessarily a practical example of how you do it, because everything there will be subtle variation. But the idea is that you can… If there’s discipline and if there is a way you want to do this. But the way the 64 becomes useful and interesting is in a playout center, you may say my video master that is common to number of languages.
MC:
So what I really want to do is have a… If I do Dolby E encoding, I’ll say stereo one is stereo, my 5.1 is encoded as Dolby in stereo two. That’s English. The next two pairs could be French. The next two pairs could be German, et cetera. And you could end up with supporting 16 different languages in a 64 chanel package.
Cindy:
And that works for social media? That applies to social media users?
MC:
No. Social media, it doesn’t work in that parallelism. The important thing with social media is to say it’s different to the broadcast, and it’s a moving target. So you need to be able to program it. Now, the other thing is it needs to sound right. Now, the example you… What you have to think about is when you’re sitting in the cinema, the environment is darker and quieter, apart from some noisy popcorn eaters; compared to your living room where you may have a smaller room, a different ambient noise level.
MC:
And then you have to think about sitting on a plane with your iPad and listening to that. Now, if you think about it, these are all three different listening environments. And ideally you want audio to be presented as three choices so that you can say, here’s my cinema audio, here’s my living room audio. And that really is what the market is now waking up to. There is interest in saying if I’m a commuter, and I guess the present times are not that many commuters, but we want a good experience for for all of them.
Cindy:
So it’s the flexibility you’re talking about in terms of no matter what the deliverable is, you’ve got a way to meet the needs…
MC:
Yes. And that applies to evolving standards as well. The EBU are always discussing how to make audio better. And so we give you the flexibility and the programmability to give you whichever version you want. But the key thing is to keep it automated so that this isn’t costing as much as other solutions might.
Cindy:
Right. So we do have some questions. And also, if you have questions for us, you can always send them our way. And we’ll put the contact details in the show notes so you can do that. And one of the questions we have is, what about Atmos? How does it tie into all of this?
MC:
Now, we’d like to think of ourselves as an innovative company, but actually we ride behind the curve. What we say is innovation for us is, what can we do that is useful to the end user as opposed to a G-Well? Now, the reason why I say that is Atmos has been around for a while and is very successful. And if you ever had the pleasure of watching an Atmos movie, it’s absolutely stunning.
MC:
So our first brush with Atmost was a couple of years ago when a launch satellite broadcaster in the UK came to us and said, “We do an awful lot of real time Atmos and we have a workflow set up for that, but we are trying to do file-based Atmos. And the difficulty we’re having is our servers wouldn’t play it out.” And so we analyzed the thing. And what had happened is that the Atmos mix had been encoded into a format called ED2, which is Dolby E twice.
MC:
Because Atmos for transmission is about 16 channels of audio. And so the timing of the Dolby ED2 wasn’t right. We analyzed it. We found out what the problem was. We came up with a fix and we gave them a solution and it played out and they got what they wanted. Now, that’s probably not the answer people want to hear about Atmos. And the reality is what we do is very boring.
MC:
The creative aspects of Atmos are kept creative in the edit suite. We come into play when you want to distribute it. And for whatever reason, the Atmos that you’ve been given won’t let you transmit or won’t let you archive or store. That’s the other area where people are interested in packaging it. And so we are talking to a number of people. We’re really talking to them and said, “What is the problem? What is the challenge that you have?”
MC:
And right now the workflows are simple enough and infrequent enough to be sold in post or to be sold by a narrow line of workflows. Now, I can’t name some names. And also, we’re relying on two things: we’re relying on a partnership with Dolby. We have very good relationship. And we’re relying on the customer saying, “I have so much content that I can’t do this manually and I have these different workflows.”
MC:
So basically, if we go back again to the discipline of when we had the movie and we made the prints, or when we had a piece of content, we did very little with it. We restricted our workflows so it became an efficient operation. The challenge we have now is the efficiency is expected, but the discipline in the workflows has gone. This is where the issue is.
MC:
And in Atmos, we’re still in that early stage where we’ve being very careful about how we produce it, how we transmit it. But that lead is about to go off. And I’ll give you a little trailer. I don’t want to make out that we’ve got something magic around the corner, but we’re monitoring the challenges as they occur, and we will be providing the solutions that work.
MC:
So one of the issues, as an example, is that you get three files for Atmos: the one for the beginning titles, the one for the program and the one from the end credits. And you have stitch these together. At the moment, it’s done in a very tedious manner, in an edit suite and so on. And we’ve been asked to look at that. We are still looking at it because the requirement hasn’t crystallized enough to be a pain for the customer to say, “I can’t bear this anymore.”
MC:
It’s a careful market. If they can get away with it and do it, they don’t want to spend the money. And it’s when people come along and say, “Yeah, when I get this one feature a week, I can deal with it. But if I have to do 15 features, then I’m going scream.” It is a volume… As I said, we do best when we automate the solutions where they’re not frequent enough so that careful pruning and tweaking will take care of it. It’s still best done manually.
Cindy:
That’s right. It’s all about the mundane tasks and when that repetitive work hits that tipping point.
MC:
Yeah. And then the mundane tasks still need to be done really, really well so that you haven’t ended up ruining the quality.
Cindy:
Nice. All right. More questions here in our webinar on one master, multiple deliverables. Here’s a question. Is there an API? And if so, how does it work?
MC:
Okay. So yes, there is an API. It’s a REST API. So basically REST API is a very simple in their construction. And what we say is you preplan the workflows and program them in the engine. We have a very good tool for it. It’s something that really wants to be done with and attention. And then what you’re asking your control system to do is talk over the API.
MC:
And all you have to say is, “This is my file. This is the workflow. This is where I want you to place the file with this name,” if you like. And then post the job. And then you can request progress from it. So you can say, “Where am I with this?” And we will say 60%, 70%, et cetera. We’ve had people who integrated to us in a matter of days. It’s not a very complicated process.
Cindy:
Great. So we know we have people right now using the API every day and obviously it’s a huge part of the solution here. Another question. This is good. Can I just do all of this in my Pro Tools room? Over to you MC on that one.
MC:
Okay. I think if you’re doing a few hours a day or a week, absolutely. The challenge really becomes when you have to do this day in, day out with hundreds of hours. We recently quoted on the system where the requirement was for 200 hours a day. So for that, you need a lot of people and I think the client won’t pay for that. Now, again, in our lockdown mode, we can’t always get access to the edit suite.
MC:
The suite is there for a good reason. Again, if I go back to why we started the company 10 years ago, it’s because people are saying we have these suites and they’re great and they’ve brought book everything in it, but we can’t take an hour of the sweet time to do loudness correction. We should be doing it for more creative, more billable work. And also a suite requirements talent and it requires a room. It requires a huge amount of equipment. So if you’re able to build that, 10,000 hours a month, I’m sure a lot of people would love to do that, but they can’t.
Cindy:
That’s so good. I’ve had the pleasure of working with some people who tried out engine themselves and it was super fun demoing and working with… MC, let me just throw this one over to you. If someone wants to try out all the cool things you’ve been talking about for making multiple deliverables, what steps to take next?
MC:
Oh, okay. So you go to a website. Every single page on our website there’s a little blue button which say’s trial. Click on it and fill in a simple form. You can download the engine and we give you a 10 day trial. The product works in Windows, Mac and Linux. And so that’s how people work. We encourage them to engage with us. We have also got some useful guides. There’s a loudness factbook, Adobe factbook. We have a Netflix one.
MC:
So we also encourage them to talk to us about any issues they have. As I said, when we focused on audio, one of our missions was to be the best in audio file based processing. And one way for us to do that is to engage with the customers and actually share either what we’ve learned from our experiences or listen to what their pain is. And sometimes shrug our shoulders and say, “We can’t help you,” Or we can look at it and say, “Hey, we’ll take a look at this for you.”
MC:
In some instances, we may charge you for it. In most instances, we’ll at least do the investigation and come back to you with some ideas and solutions. We like to work with our customers closely and work as a team. We are small and flexible and agile and keen to be the best. So that’s the way we’re going to deal with it.
Cindy:
All right. I’ll make sure all those links that you just mentioned, MC, are the show notes for you. So if you want to take advantage of those resources MC just mentioned. And thank you so much. If you’ve got one master and need to have multiple deliverables, we look forward to working with you. Thank you.
MC:
Thank you very much.
