Facebook is going to use Rosetta to issue more personalized news feeds.
The Facebook company presented a machine learning system Rosetta is able in real time to extract the text from the published in social networks pictures and videos in different languages and to recognize their semantic content.
“To meet our specific needs we have built and deployed large-scale machine learning system called Rosetta . It extracts text from more than a billion public images Facebook and Instagram and videos (in different languages) in real time and enter it in the model text recognition, which was trained classifiers to understand the context of text and image together”, – says the publication on the website.
During discovery of the Rosetta text simultaneously performs two independent from each other process: detection of the regions that may contain text, and word recognition using convolutional neural networks Faster R-CNN architecture ResNet18.
The algorithm recognizes English, Arabic, Hindi, German, Spanish and other languages, including those that have horizontal right-to-left writing, diacritics and other special characters.
In the future, the Corporation will try to teach the system to recognize more languages, types of text and picture templates.
Facebook is going to use Rosetta to issue more personalized news feeds, and track spam, offensive or inappropriate content. Now it is sorted by the operators and it takes a long time.
In June 2018, researchers from Stanford spoke about a machine learning model capable of creating memes in the style of “advice animals”. The authors noted that the average “faux” meme is difficult to distinguish from the “real” in the context of quality jokes in it.