During a workshop with the European Broadcasting Union (EBU), the Danish Broadcasting Corporation (DR) and Media Distillery presented the results of a Proof-of-Concept on Binge Markers: the technology that automatically detects the end credits in videos. This enables which viewers may utilize to skip the end credits of a show while binge-viewing to enable a smooth, quick transition between programs. DR has identified this aspect of television viewership as an area to improve upon within their video platform, and discussed with Media Distillery both the challenges and possible solutions that lay ahead.
Read below as Birgitte Stannius, Architecture Consultant of DR, and Martin Prins, Head of Product of Media Distillery answer questions regarding the use of Binge Markers, the preliminary results of working together, the current state of the technology, business considerations, and more.
Birgitte Stannius, DR:
For a public broadcaster, the transformation to public service media is a challenge. Not only program formats, but also the presentation, the way it’s disseminated and consumed, and marketing are aspects that must be rethought.
Birgitte Stannius, DR:
We relaunched DR’s streaming service – DRTV – in 2019 as part of a larger strategic effort. DR´s goal with this and other actions is to increase the usage of DRTV. At the moment, 4 out of 10 Danes use DRTV every week. However, the goal is that 6 out of 10 Danes use the service. One of the actions we are looking for is that the viewer user remains engaged in watching another program after the program the viewer is currently watching comes to an end. Basically, we need solutions to retain the viewer on our service by offering the viewer programs when he or she is most receptive to suggestions. A Binge Marker identifies a point in time where the viewer is receptive to suggestions, and is most likely to remain tuned in.
Birgitte Stannius, DR:
Currently, we are using a solution where the view of the program is shrunk 5 seconds before it ends – while offering a new program. This is a safe way – the user will presumably not miss any of the content. However, it is not a particularly effective way to do it – especially not if you compare to the average duration of an end credit.
Birgitte Stannius, DR:
Binge Markers could be created manually as part of different production and application processes; e.g. in the editing process or in the quality control of the program before publishing. You could also consider making a separate process – where you update your catalogue of active programs manually with Binge Markers. However, this means changes in current processes and development of technical support of this new task.
Martin Prins, Media Distillery:
While you could create Binge Markers manually, that would require people to view all videos, scroll until the end, and note the time at which the credits start. This does not scale well for large collections of video or where fast processing is needed, and is quite tedious and relatively expensive.
But with advances in Artificial Intelligence and Computer Vision, it is now feasible to automate this by training an AI model, or a combination of AI models, to detect end credit sequences. This can be done by looking at several features of a video. For example, the appearance and amount of text, what the text says, or by looking at the audio, the start of music, or repetition of a sequence. We opted to go for the visual aspects. The main reason being we are already using this for other products, but another being: not all videos have audio during the credits, so it may not be a reliable source.
Birgitte Stannius, DR:
You don’t have to incorporate a new task with existing processes or create a new process. This can be complicated e.g. if you choose to get data from the editing process you have to bear in mind that you have different sources – internal/external production – and it could be difficult to build a streamlined production chain for the Binge Markers.
Martin Prins, Media Distillery:
We see several advantages: Because of automation, it can be scaled easily. It doesn’t matter if you need to create end credits for hundreds of programs or thousands. Additionally, it can be done really quickly and done at any time.
Birgitte Stannius, DR:
We have really gained valuable insight into the preliminary quality, the potential, and pitfalls for AI/ML. We have become more aware of the way programs are designed, specifically the way they end and provided substance for reflection on how Binge Markers should be used in relation to different types of end credit.
Martin Prins, Media Distillery:
In the PoC, DR asked us to evaluate end credit detections on a variety of content. What we discovered was that we needed a large pool of examples to evaluate accuracy – where the types of end credits varied. Looking at it from a user experience perspective, any detections before the start of the end credits could lead to a “skip credits” appearing during the cliffhanger of a program, and ideally this is something we want to prevent.
Looking at all the videos, our end credit detection was successful 85% of the time. This is pretty good, but also shows there is room for improvement. For example, we ran into some challenges with our AI model identifying text in the background of the program as an end credit sequence, mistakenly identifying a textual epilogue as an end credit sequence, detecting end credits when there were none, and identifying a proper end credit when there was additional content playing subsequent to the initial end credits. Think of a blooper reel that begins shortly after a movie ends, and after the credits start rolling.
Now we are looking to improve the results and expect to be able to increase the accuracy to 95%, especially if we have more data about the content. As it relates to DR’s current solution with the shrunken view five seconds before the end of the program, Media Distillery’s Binge Marker solution can solve DR’s viewer retention issue by identifying the exact point to run a new program, thereby increasing the likelihood the viewer remains engaged with the content and tuned in to the channel.
Of course, implementing this advanced solution on a scalable level, and at pace, comes with its challenges, but Media Distillery has always accepted these challenges to innovate and improve, while helping TV operators and broadcasters enhance their consumer’s experience. We have already enabled this for catch-up and replay tv for millions of TV viewers with our EPG Correction product; now we want to do the same for Video on Demand.
June 26, 2020
