Multimodal Ambivalence and Hesitancy Recognition with Bidirectional Audio--Text Cross-Attention

A new audio-text framework uses pre-trained multilingual BERT and wav2vec 2.0 with bidirectional cross-attention blocks to recognize ambivalence and hesitancy in videos, achieving 72.19% accuracy and 77.95% A/H F1-score on the BAH dataset.