A Multimodal Approach to Image Sentiment Analysis

Multimodal sentiment analysis is a process for the classi- cation of the content of composite comments in social media at the sentiment level that takes into consideration not just the textual content but also the accompanying images. A composite comment is normally represented by the union of text...

ver descrição completa

Detalhes bibliográficos
Autor principal: Gaspar, António (author)
Outros Autores: Alexandre, Luís (author)
Formato: conferenceObject
Idioma:eng
Publicado em: 2020
Assuntos:
Texto completo:http://hdl.handle.net/10400.6/8162
País:Portugal
Oai:oai:ubibliorum.ubi.pt:10400.6/8162
Descrição
Resumo:Multimodal sentiment analysis is a process for the classi- cation of the content of composite comments in social media at the sentiment level that takes into consideration not just the textual content but also the accompanying images. A composite comment is normally represented by the union of text and image. Multimodal sentiment analysis has a great dependency on text to obtain its classi cation, because image analysis can be very subjective according to the context where the image is inserted. In this paper we propose a method that reduces the text analysis dependency on this kind of classi cation giving more importance to the image content. Our method is divided into three main parts: a text analysis method that was adapted to the task, an image classi er tuned with the dataset that we use, and a method that analyses the class content of an image and checks the probability that it belongs to one of the possible classes. Finally a weighted sum takes the results of these methods into account to classify content according to its sentiment class. We improved the accuracy on the dataset used by more than 9%.