A group of researchers from Carnegie Mellon University (United States)
has developed a technology that is able to identify the
memes published on the internet and that translates them to help people with visual impairments.
Blind or visually impaired people have technologies that allow them to access the text of social networks, but in the case of memes, if they do not understand humorous images, they lose an important part of the conversation, as explained by the authors of the study.
To extend accessibility, they have developed
a method that identifies memes through the images they use and that includes previously written content with which they provide an alternative text with additional context to understand them.
Through computer vision techniques, the system first describes the image of the meme and then uses the optical character recognition to read the text that has been superimposed on the image, also indicating whether it is located at the top or bottom of the image. meme
The Carnegie Mellon University system is capable of translating the memes into text but also in the form of sound, so in addition to reading the text other elements such as music and sound effects have been added to help describe the sense of the meme. However, they have explained that in their tests people prefer only the text due to custom.
Researchers have also drawn attention to the difficulty of introducing this system on the Internet, since when users create a meme they copy the image, but not the alternative text that could help blind people. Therefore, they are currently working on a Twitter extension that deals with this task or even in the photo metadata.
Create a technology that translates memes to help visually impaired people – LA NACION