StreamOn recap | Immerse, monetize, and personalize every live sports moment

00:24:56:08 — 00:26:04.522

So this is a look at the, you know, product itself. Again Dolby’s built the back end. So we are in real demo now. We’re in real demo. This is the exciting part. All the fluffy words are over now. You’re like oh that’s what they meant. So, so yeah. So, this is a view of our Sports Intelligence product. This is focused on the content intelligence.

And so what we’ve got right now is we’re simulating several live games that are running so that we can actually show you how the product works. So imagine this is real time. We’ve got, you know, a whole bunch of games running. So Sander’s spun up a bunch of content. And so this dashboardis just an initial overview.

It’s, you know what alerts are firing. How much excitement is happening in a match? Again, we’ll get into some of these things that we can actually configure. You get a quick preview of heads up of what’s going on in the games. You can see the recent events, right? Events being things like an injuryor a corner kick or, kickoff, or a block or a foul.

These types of game events. Right. The first sport that we’ve implemented together with Sandor is, you know, as an American soccer, sorry, football for the rest of the world.

00:26:04.522 — 00:27:24.158Although American football does kick off this weekend. And so and then you get a quick system status, so you get a overview of what’s happening from the dashboard, but from the live games, we’ll start to drill in a little bit. And so you can see there’s, you know, we’re in four minutes of, overtime or, extra time on this match.

So maybe I’ll, I’ll scroll down here, we get another one that’s, going on here, and I’ll just open up the extra panel. You can get a preview of what’s happening in the game. You’ve got the score, you’ve got the time, you got the stream time, and then you’ve got the game events. So that’s kind of interesting.

Let’s go ahead and drill into this match here. And so what you can see is we’ve got this the game has just ended of course. So maybe I’ll go back and pick one that’s in progress. Let’s pick this one and this one. This one’s a little earlier. All right. The game’s still going on, so that’s great. And you can see, a history of the events.

You can see the scrub bar polishing. So if you need to go back and just quick, say, hey, what happened? You can jump back in time andand go back in time. You can also jump back to the live edge. So again, quick review. The whole thing is to make it faster for humans. But humans can either be in a loop or out of the loop.

00:27:26.202 — 00:28:23.598Once you’re here, you can actually see all the game events that have happened. Of course. And then, you can look at the transcriptsto be able to jump back in time as well. Now, I’m not a native Dutch speaker, so I’m going to translate to English and I’ll just jump back to here. Right. See, the time codes are associated with it.

And really what we’re showing is the power of all the metadata that we’re generating. By having all of this rich data about the game, we’re taking in data feeds, we’re generating subtitles, we’re understanding events based on analyzing the video and the audio of the game. To make additional meaning to it.

So that’s pretty exciting. So I can jump around. And even from this view, what I can do is I can mark in and I can mark out and say, that was a really exciting moment. I can either download this clip, I can save this clip to the library. Or I can go ahead and publish it. And publishing it is an automation step down the line.

00:28:25.002 — 00:29:29.438But what else have we got going on here? We’ve got the stream itself, and there’s a whole bunch of metadata with this stream. As Dolby, we know that people are going to build products on top of us that are going to blow our minds, things we never even thought of. And that really starts with that data. And so what we have here is an example of the actual data that we’re generating.

Right?This comes from a whole bunch of sources. But we can aggregate this together. And that term with the maybe the best way to kind of visualize this is here in the timeline view. So as this game is going on, events are being added to this view. And so you can see, hey here were the goals. Here were the shots on goal.

Here were the saves, I like to watch to saves. So I’m going to go ahead and click on this one. And there we go. There’s the save, really exciting moment. And I can just arrow over to the next one and there’s another save. So, it’s kind of cool that you can just quickly get a visual representation of the game, like, hey, how exciting was the game?

And the more color you see on here, probably the more exciting the game is. I’ll go ahead and click on goals as well, so this takes us into a goal.

00:29:32.442 — 00:30:13.838And so this is the timeline view, and each of these markers in here is an event in the game. And what do we mean by event. It’s kind of the, the moments that happen in the game and make it exciting. So we’ve got a whole bunch of events that happened. I’ve sorted it by excitement score. So I can either look at the least exciting things, like kick off, that wasn’t very exciting but the game started.

And then the most exciting things generally that’s probably going to be the goal or the near misses. So, I’m going to go aheadand just pick on, an event here. We can go into the details. So this is the detail view of this goal. Right, we saw it before in the timeline view. But again this is a lot about different ways to access the actual metadata.

00:30:15.642 — 00:30:54.598And if I show the breakdown, I think this is one of my favorite parts, you can actually see what we’ve identified in the content, so again, what is a goal? That could mean different things: Do I want to have a whole bunch of lead up with the kick, the attack, the celebration? Or do I want to just do the shot?

So if I can now jump back hereand I can see where the shot happens, I can mark in. I can, there’s the shot itself, and I’ll get a little bit of celebrationhere, and I’ll trim it right there. So there’s an out. And now again, I can save this clip and call this an exciting goal.

00:30:56.322 — 00:31:43.598and save that to my library. And so that’s been added. You can see, from this event, right. The event being the higher order, entity, I can have clips that are derived from it, and you have it, you’ve got your excitement, you’ve got when it occurred. And let me go ahead and look at the, we’ve got the 16×9 that we’ve seen, but so many companies are trying to get this content quickly to social media or to their own apps that you can add to a user scroll togive them something that they want to see.

And so we’ve taken that into account too. So if I click here, I can see this is the 9×16 version of the same thing. And if I go back to the event and go to the 9×16 variable.

00:31:45.202 — 00:32:17.638Here’s the actual tracking and you can see where it is zooming in and out dynamically as you’re doing this. But by adding the letter box at the top and bottom, and we’re zooming in and out because when you’re getting all this fast action, you don’t want to be cropped way in and you want to follow things,a logical way.

And so we’ll kind of dynamically zoom in and out there at the the cut, we saw, we just went to the full vertical. And now let’s say you don’t like thatcrop, right. So if I hit edit clip it’s going to say, hey, we’ve already published this because with automation we’ve already sent this out. But maybe I wantto generate a new version.

00:32:19.042 — 00:33:03.678What I can do is come in here and I can edit crop and this takes me to our crop editor, and I can move this around. You can see on the right I’m getting the representation of the visual of where the crop is. And down here I’ve got all the keyframes that make up this crops. This is the automated tracking that we determined.

So if I delete all 34 keyframes, oh sorry, I gotta highlight them all. If I delete all the keyframes and I now I can manually adjust this. So where is the ball. Here. It’s right there. It’s going to quickly follow the ball. I can make this one a little bit more zoomed in. And see, you can see on the right it’s changing again I’m not going to try to do anything too crazy here.

But let’s say that we’re getting even more narrow.

00:33:04.762 — 00:33:45.398And we’ll get all the way to the goal will come out wider. And if I just play this through, you can watch on the right side the composite of the crop. So we’re following the ball. And again, it’s just an easy way with the human in the loop to actually override the automated crop that was determined. It looks like I did a worse job than the computer at this point, but it’s got that nice kind of dynamic zoom in and out.

And of course, I didn’t adjust anything here, so the crop is going to be stuck. But these are the kinds of things that we’re working to make really easy. You know, Sander talked about how, for years he’s had the editorial team kind of doing this work very manually. The goal is to speed them up.

00:33:46.962 — 00:34:39.878So that is what we’ve got for the events. And Dan, sorry, I got a question from the audience from Martin. Yeah. Which sports are supported? Is it football only for now? Yeah. Our launch sport is football. But the thing that we’ve been building at Dolby, we’ve had in research for a long time, a bunch of this analysis capability that’s been being worked by our advanced technology group.

But what we’re doing is, we’ve built all the pipelines and all the infrastructure to be able to add sports very rapidly. So, you know, if somebody wantedto talk about another sport, we can add one relatively quickly. So as Sander can tell you, the way that Dolby likes to work is, is, you know, hip to hip with the customer so that we can make sure we’re building the thing that is actually meaningful to the customer.

So Sander, like I said, has had a lot of input on what we need the API to expose and how we can actually utilize this.

00:34:41.282 — 00:35:53.318And if I may, I also have another question. Sure. Let me see if I, it’s from Yonas. What accuracy can we expect for the live events? Is there a chance that we missed a goal or asave, for example? Oh great question, right? Especially, hey, we’re adding artificial intelligence to do something intelligent used to do for us, right?

It’s really easy as a human to say: Yep, ball went in the net, that’s a goal! So we’ve got a couple of things. We’ve also taken the data feedfrom one of the sports data providers, right. That generally is: hey, 100% confirmation the goal happened. When we’ve got that in, we can do things much more quickly.

When we don’t have that, we can still do all of the same analysis, do all the same work to surface, goals and things. But we tend to take a little bit longer because we want to validate that it actually happened. I mean, think about a VAR review or something gets overturned. We don’t want to keep that goal.

As counted once the VAR overturned it. So it’s one of those things where like yeah, it’s like I would say generally it’s very accurate, especially with validating that we’re we’re invincible. But with, when we’re doing analysis without that data feed, we tend to just take a little bit longer on delivery.

So I hope that makes sense.

00:35:54.562 — 00:36:37.358Yeah. Okay. Going back to so this is a clip, right. And we’ve clips are all tied to the events. The event also has a bunch of metadata. So we’ve got the actual clips that were generating themselves. We can generate a whole bunch of versions of the clip from this, by the way. We don’t have to generate just one.

We’ve got this really powerful virtualization engine under the hood that can, say that a clip exists and not until somebody hits play does it actually get rendered. So, again, as we think about personalization, we’re working with our customers to figure out what is the right level of personalization.

Let me know if I’m going too fast, guys. I’m going to keep moving on for the sake of time.

00:36:38.842 — 00:37:40.038I’m sorry for people that see this thing and get scared. This is, I think, a JSON data feed. So API, sorry, API are based on these kind of things. When we want computer to talk to each other, we send these bunch of text that is called JSON. And that’s the way they communicate. So this is the content of the APIs. Just to explain.

Yes. Thank you. Thank you Carlo. So exactly like as we talk machine to machine, this is JavaScript Object Notation or JSON And it is that data exchange format. And so again we know people like Sander are gonna build things on top of our platform. They’re just going to want the data. Right? I’ve heard so many times oh I just want as much data as you can give me.

What can you surface? Well, here it is, guys. Right. Here’s your entire blob of JSON that represents the entire event, right. Or even the entire game. But we’re making APIs on top of it to be able to pull out individual pieces as well. So, the end of the day, a sports event is just a bunch of data, and we’re representing it with all of these niceties on top to be able to pull out the most relevant things.

00:37:43.602 — 00:38:33.078All right. And Dan posing another question: what type of typical customer bias of the platform are in your view? Yeah. So it’s generally B2B that well sorry that was a question from Cormac. Sorry. Okay. Thanks, Cormac. I mean we deal with individual teams, we deal with leagues, we deal with streamers and broadcasters.

So I think that it can be any, I mean, maybe I’ll turn it over to Sander and see, you know, as you’re out talking to your customers. Yeah. Well, we see several things. So, what we see is that there are a lot of sporting clubs or rightsholders itself, talking to us, and then, do a license from our platform.

Then we are a customer of Dolby, obviously,

00:38:34.082 — 00:40:19.078So it depends. They are broadcasters, clubs itself or rights holders and even leagues. So, as all different kinds, but most of the time is always B2B. Yeah. I mean, I think would be kind of fun as an end viewer to be able to like, tinker around with some of these things. But like, that’s why the sports data companies make these APIs to sell.

Right? And, and if you look at any of the big streaming platforms, they’ve got clips. So the output of something like this already gets put into the experience. So I definitely think that this is squarely in the B2B realm. But, you know, come talk to us if you have other ideas. Yeah. All right. Here I am in the demo.

So we’ve got the library. So this is basically anything that’s, not live. So that can be a VOD game. It can also or within the game. Right. If I click on one of these, I’ve got all the same things. It’s just no longer live. So we move from live to library for VOD, or you can upload VOD, if you’ve got a history, because, again, if you’re doing personalization, it may not matter as much what happened in this game when you’ve got someone who’s a fan of number 12.

And so number 12, you want the history from the entire season. We can run this process against, games that have already happened in the past. And then in our automation stage, you can say, hey, give me all the clips from number 12 when he was, making a goal or a near miss, right. So you can really getthis powerful ability to put these things.

Excuse me, into, get the rubber to meet the road and actually, make something meaningful for a fan, as an example. So that’s library. Let me, let’s lookat automation for a second.

00:40:21.842 — 00:41:00.878Let’s see, we did events, right? Yeah. We did. Okay, so we go to automation and we’ve got, two different kinds of automation. And you can see that they’re already we have two others go into this one, already running. So this is the automation that runs on every game. And it basically says, hey, for everything that happens in the sport, detect it and make a clip.

If you wanted to make an automation that only pulled out goals, you can do that. And let’s look at some of the, rulesthat we have here. So you say, yeah, I want to make a clip or I want to make a sequence. A sequence is a collection of multiple clips you can think about, like, premiere or non-linear editor timeline, where we’ve got multiple shots together.

I can show you what some of these things look like.

00:41:02.282 — 00:41:14.838And you can then take the triggering thing. So do I want to goal yes or no? Do I want to get the shots? Yes or no? And again, you can pick the differentevents that exist in the sport

00:41:15.882 — 00:42:32.918and you can pick. Yes, I want this for every game. Or you can set one of these up for each team, because maybe each team wants to have a separate delivery or only wants to get certain things delivered. Again, it kind of depends on who’s using the clips. Are you selling this for marketing purposes? Are you sending it to players to post on social?

Are you posting this to the club website? Are you putting this upstream into production? Maybe you want to then deliver to like an S3 bucket. So we’ve tried to account for all of these possibilities. And of course you can figure out, you know, for the overview of the automation, what is the output going to be.

So we’re going to do a regular horizontal output. Or we’re going to do a variable vertical. Do I want to review? So part of our automation step accountsfor what Sander’s doing with human in the loop. So maybe for the marketing team, we give you ten minutes to review it, and after ten minutes,we’re going to publish it anyway.

But within that ten minutes, somebody can come in and modify the clip before they post it. We can also append a pre-roll or a post-roll, and we can put an overlay. So all the things that, you might want to do for a really basic edit, to be able to get something very quickly or are built in to this tool as well.

00:42:34.282 — 00:43:10.678All right. And then configuration when you’ve got like your outputs or your destinations, where do you want to write content to? Where do you want us to send the content to? If you want to put an overlay and we can have a configuration for that. Think of this.I like to think of this as a symlink. So if you’ve got a logo in the team’s logo changes because they got a new mascot.

If you make that change here, any automation that uses that logo will get the update. So immediately everything’s updated. Instead of steppingthrough every single automation and having to update a logo file, it’s really just this reference to it here.

00:43:12.802 — 00:43:34.598And then let me go back to library for a second and we’ve got sequences. So I talked about this in the automation, but I want to show you what it looks like. So we’ve got the clips that we extract which are the 9×16, 16×9. We can change the cropping. But sequences is another thing of automation where you can actually generate match highlights, where you’re concatenating,

00:43:36.522 — 00:44:25.958Multiple things together. And so it looks like this. You’ve got your timeline view. And if I hit edit sequence here, I can actually say, yeah, I’m gonna shorten this one down. Again. You would do this with much more finesse than I’m doing here for the live demo. And you can even reorder. So I’m going to just change the order of these things, because this one was more exciting and I wanted that to come first.

And so again, from here you can decide what do you wanna cut? Do I wanna Disolve? Do I want to fade between and then I’ve got my sequence. And I can just like, I can with clips. I can then ship that out as an edited, together sequence. And so, again, a really powerful way to take not just individual clips, but to put things together.

And, I think this is an area of the product we will be continuing to evolve.

00:44:27.162 — 00:45:43.238So, Carlo, I’ve gone on for a while. You’ve asked a couple of questions. Yeah. And one more question for you, it’s from Giovanni, which, if I remember well, is in Washington by the way, it’s an Italian name, but in Washington, if I remember well. Well, international soccer leagues in Europe, South American, Middle East are there in different languages obviously.

Will Dolby’s solution for live sport, support all languages from different geographical areas. So yeah, I think, I can, but I showed before was the translation from Dutch toEnglish. I think that this is the kind of thing. But again, in Sander’s product, he’s already got the captioning side fully covered.

From the Dolby side, this is definitely the kind of functionality we would add. I think an area that’s really interesting, and I’m really interested to see atIBC is text-to-speech, so when you go from a script that maybe is AI written and then having an AI voice narrated, again, we’ve got some technology we’re working on that.

If you come to a booth, we can start to show you, but this is an area where I think as you think about what I have on screen, where I’ve got a sequence of clips and I’m personalizing it per person. Like, it can be jarring as you cut from one to the next without any context. And I think both AI voicesand on screen graphics are going to be a really important way to add some of that context.

So that’s an area you can look for us, for future development.

00:45:45.082 — 00:45:54.118But yeah, I mean, Sander, did I miss anything? Is that all make sense? Anything you want to highlight? No. Makes sense from my perspective

00:45:55.522 — 00:46:52.718I was thinking, Sander, where do you see this going from hereonwards? Yeah. Well,I’d like to add on the localization part, regarding the last question is, that’ also happening faster and faster now. So from Southfields’ perspective, we also we’re already collecting moments, and then, put it in workflow with, for example, the 11 laps, the part of dubbing.

So they just that was the feature, part of dubbing. And then we dub the whole part and do a little personalization. So that’s where it evolves. So for Rights holders like IMG and all those kind of distribution parties, localization is also a very important topic within the industry. So that’s the next big thing, from Southfields perspective.

And we embrace all the technology from Dolby with that.

00:46:54.362 — 00:48:12.358And so what’s the in the next phase, Dan. Ah, Sander, sorry. Yeah. Well, the next phase. So, Dan already mentioned the excitement score. The excitement score is becoming more and more important because, if you have also the viewer intelligence is not only the content intelligence part of the sports intelligence platform, but also the viewer intelligence.

You can literally have a feeling about not a feeling data points from the excitement score of the audience at the viewer. And then you can use that eventually, in a control room as well. So you might even based upon the collecting audience, the excitement scores, make decision with decisions within the upstream control room.

So, for example, we’re currently making several super switch programs. That super switch program switches between all kind of live matches at the same time. But it’s quite hard for directors to select moments out there if you have that in the full chain and you can use the data generated from viewers’ perspective in the control room.

So that’s where we going to bring it further and further. So it’s

00:48:14.482 — 00:48:39.398not that far away. So as I’ve already said before, it’s the sports production chain is evolving very, very fast. So I have another question from Cormac: For those switching scenarios, I think he means, when you were editing the slide, a customer would need to send the clip to another software to add graphical overlay or that would be something that we can integrate?

See you there. Thank you, Sander.

Thanks for having me.

Gracia. Chao.