This paper addresses nonverbal behavior analysis for the classification of perceived personality traits using novel deep visual activity (VA)-based features extracted only from key-dynamic images. Dynamic images represent short-term VA. Key-dynamic images carry more discriminative information i.e., nonverbal features (NFs) extracted from them contribute to the classification more than NFs extracted from other dynamic images. Dynamic image construction, learning long-term VA with CNN+LSTM, and detecting spatio-temporal saliency are applied to determine key-dynamic images. Once VA-based NFs are extracted, they are encoded using covariance, and resulting representation is used for classification. This method was evaluated on two datasets: small group meetings and vlogs. For the first dataset, proposed method outperforms not only the state-of-the-art VA-based methods but also multi-modal approaches for all personality traits. For extraversion classification, it performs better than i) the most popular key-frames selection algorithm, ii) random and uniform dynamic image selection, and iii) NFs extracted from all dynamic images. Furthermore, the ablation study proves the superiority of proposed method. For the further dataset, it performs as well as the state-of-the-art visual-NFs on average, while showing improved performance for agreeableness classification. Proposed method can be adapted to any application based on nonverbal behavior analysis, thanks to being data-driven.

(2021). Personality Traits Classification Using Deep Visual Activity-Based Nonverbal Features of Key-Dynamic Images [journal article - articolo]. In IEEE TRANSACTIONS ON AFFECTIVE COMPUTING. Retrieved from https://hdl.handle.net/10446/260535

Personality Traits Classification Using Deep Visual Activity-Based Nonverbal Features of Key-Dynamic Images

Beyan, Cigdem;
2021-01-01

Abstract

This paper addresses nonverbal behavior analysis for the classification of perceived personality traits using novel deep visual activity (VA)-based features extracted only from key-dynamic images. Dynamic images represent short-term VA. Key-dynamic images carry more discriminative information i.e., nonverbal features (NFs) extracted from them contribute to the classification more than NFs extracted from other dynamic images. Dynamic image construction, learning long-term VA with CNN+LSTM, and detecting spatio-temporal saliency are applied to determine key-dynamic images. Once VA-based NFs are extracted, they are encoded using covariance, and resulting representation is used for classification. This method was evaluated on two datasets: small group meetings and vlogs. For the first dataset, proposed method outperforms not only the state-of-the-art VA-based methods but also multi-modal approaches for all personality traits. For extraversion classification, it performs better than i) the most popular key-frames selection algorithm, ii) random and uniform dynamic image selection, and iii) NFs extracted from all dynamic images. Furthermore, the ablation study proves the superiority of proposed method. For the further dataset, it performs as well as the state-of-the-art visual-NFs on average, while showing improved performance for agreeableness classification. Proposed method can be adapted to any application based on nonverbal behavior analysis, thanks to being data-driven.
articolo
2021
Beyan, Cigdem; Zunino, Andrea; Shahid, Muhammad; Murino, Vittorio
(2021). Personality Traits Classification Using Deep Visual Activity-Based Nonverbal Features of Key-Dynamic Images [journal article - articolo]. In IEEE TRANSACTIONS ON AFFECTIVE COMPUTING. Retrieved from https://hdl.handle.net/10446/260535
File allegato/i alla scheda:
File Dimensione del file Formato  
IJ15_Personality Traits Classification Using Deep.pdf

Solo gestori di archivio

Versione: publisher's version - versione editoriale
Licenza: Licenza default Aisberg
Dimensione del file 1.99 MB
Formato Adobe PDF
1.99 MB Adobe PDF   Visualizza/Apri
Pubblicazioni consigliate

Aisberg ©2008 Servizi bibliotecari, Università degli studi di Bergamo | Terms of use/Condizioni di utilizzo

Utilizza questo identificativo per citare o creare un link a questo documento: https://hdl.handle.net/10446/260535
Citazioni
  • Scopus 16
  • ???jsp.display-item.citation.isi??? 11
social impact