Yandex has learned to translate online broadcasts on YouTube on the fly. Video

Posted by

A function of simultaneous translation of live broadcasts appeared in the Yandex browser. So far, the feature only works on YouTube and only on a few channels, but the developers are working on its development. The plans include the implementation of support for various streaming platforms. The service is available exclusively in Yandex Browser for desktop PCs.

Foreign languages ​​can not be taught

Yandex representatives informed CNews about the launch of a new simultaneous translation service for live online broadcasts. It allows you to watch live broadcasts on foreign channels and understand what they say without knowing a single word in the original broadcast language.

At the time of publication of the material, the new Yandex service was in beta testing, but at the same time open. This means that any user can experience its capabilities. In this regard, there is only one limitation – the function works exclusively in Yandex Browser version 22.7.2 or later for Windows and macOS. Third-party browsers, like the mobile version of the Yandex browser, are not yet supported.

Watching foreign broadcasts on the Internet has become ten times more convenient

When asked by CNews about the timing of the completion of beta testing, Yandex representatives replied: “It is difficult to say how long open beta testing will last at this stage, this is a rather unpredictable process. Now our task is to understand how the technology works “in combat”.

The innovation, by the way, may come in handy as early as the beginning of autumn 2022. For this period, Apple is scheduled to present new products, which, with a high degree of probability, will not receive Russian subtitles – the company fled Russia in a hurry back in March 2022.

How it all works

The new Yandex service is based on the same neural network that is used to translate pre-recorded online videos. As CNews reported, this feature appeared in the Yandex browser in the summer of 2021.

Service example

To translate live broadcasts, the developers had to significantly upgrade their neural network. “In the case of recorded videos, the neural network receives the entire audio track, which means it has the full context, which makes the task easier,” Yandex representatives told CNews. – Working with the translation of a live broadcast is built in a completely different way: one neural network recognizes audio and turns it into text literally on the fly, the other determines the gender of the speaker by biometrics. But the hardest part is the next step. The third neural network places punctuation marks and extracts semantic fragments from the text – parts that contain a complete thought. It is they who are taken by another neural network responsible for the translation, which is immediately synthesized in Russian.

Everything ahead

At the time of the launch of the Yandex service for simultaneous translation of webcasts, it supported exclusively live broadcasts on YouTube. Representatives of Yandex told CNews that their plans include training the service to work with other streaming sites, including Twitch, which is very popular among gamers, but did not specify when exactly these innovations should be expected. “We plan to launch translation of streams on the Twitch platform, but this requires a lot of development time. This will not happen until next year,” Yandex told CNews.

Revolution in DLP: how does AI protect sensitive data from leaks?

Data protection

Currently, the Yandex neural network, along with English, supports several other major languages ​​of the world. These are Italian, Spanish, German and French. In the foreseeable future, support for several more European languages ​​u200bu200bwill appear, plus developers are also looking towards Asia – the neural network will be trained to translate at least from Chinese and Japanese. Representatives of “Yandex” CNews disclosed the timing of the implementation of these plans. “We will start testing new language pairs (Asian) soon, but we cannot say exactly when they will be available to all users,” they said.

It should also be noted that there are restrictions on the number of supported channels. Currently, it is impossible to connect to the live broadcast of a beginner and yet unknown foreign streamer and immediately receive a simultaneous translation of his speech.

List of supported channels

So far, the new Yandex service is working with 17 channels, mostly American or English. They are divided into Celebrity Interviews, Technology, Business, Education, Space, and Gaming News categories.

Symbiosis of Yandex services

By itself, the new service can be considered a new generation of Yandex Translator, familiar to many. This project has existed since 2011. 11 years ago, he understood only Russian, Ukrainian and English, but now his “portfolio” includes more than 90 languages ​​of the world.

UBA: Predictive analytics technologies can reduce the load on information security

Data protection

At the age of five, “Translator” mastered the skill of recognizing text in pictures using computer vision. In April 2021, Yandex integrated this feature into its own browser.

The Yandex browser itself will soon celebrate its tenth anniversary. Its premiere took place on October 1, 2012. Over the years, it has become the second most popular in Russia – in July 2022, its share of the Russian market was 16.28% against 55.9% for Google Chrome (StatCounter data). Behind it were Apple Safari (10.43%), Opera (6.99%) and Firefox (4.62%).

Source link

Leave a Reply

Your email address will not be published.