NHK develops world's first real-time multi-layer VVC

NHK’s Science and Technology Research Laboratories (STRL) has succeeded in developing the world’s first real-time multi-layer video encoder which is compatible to the international video coding standard VVC (Versatile Video Coding). This will enable people to watch two (or more) different types of content through one broadcasting channel. This is the first time a real-time VVC encoder will make possible the multi-layering of video content without sacrificing video quality.

Looking to the broadcasting service for the next generation, NHK’s STRL has been researching for ways to offer viewers a personalized viewing experience, and one of these is the broadcasting of sub-content. Sub-content broadcasting is a service that allows viewers to overlay additional video content. Traditionally, broadcasting sub-content alongside the main content required multiple channels due to the limits of bandwidth and compression.

However, with this newly developed encoder, sub-content can be compressed in real-time using multi-layer encoding, allowing two videos to play smoothly through a single broadcast channel. This encoder will allow viewers more freedom in what they want to watch without degradation in visual quality by compression.

For example, think about watching a live football match on TV. What if fans could follow their favorite players or perhaps check in on what is happening in the dugout, while watching the actual game. Or perhaps a music concert where fans could choose to see their favorite artist from different angles.

Using this technology, broadcasters will be able to transmit several different types of visual content all on a single channel. Viewers will be able to choose what live sub-content they would like to watch together with the actual main live show, at the same time, all on one screen. And of course, they will also be able to choose not to show any sub-content as well.

For example, let’s take a look at the offering of sign-language interpretation. Until now, to enable viewers to choose whether or not to have sign-language interpretations, broadcasters had to transmit two separate channels. This is what NHK did at the time of the Tokyo Olympics/Paralympics in 2021 when it broadcast the Opening and Closing Ceremonies. In order to cater to the Deaf or hard-of- hearing, it aired the ceremony on two channels, one with additional sign-language and one without.

With this technology, broadcasters will only need to transmit one channel, and viewers will be able to choose whether they want sign-language layered on the screen or not, without having to switch to a different channel. As the multi-layer function can allow the broadcaster to transmit any kind of content, for example, for those who prefer captions rather than sign-language, broadcasters can further add captions to the sign-language, making the content even more accessible. And if the viewer prefers not to have these additional elements, they can simply switch them off, in which case, only the main program will remain. All this can be done through the broadcast of one channel

LC