论文标题
CTA-RNN：渠道和时间关注RNN RNN利用预先训练的ASR嵌入语音情感识别
CTA-RNN: Channel and Temporal-wise Attention RNN Leveraging Pre-trained ASR Embeddings for Speech Emotion Recognition
论文作者
论文摘要
先前的研究已经研究了通过利用语音和语言提示来改善语音情绪识别（SER）的方法。但是，最先进的ASR模型与SER任务之间的潜在关联尚未研究。在本文中，我们提出了一种基于预训练的ASR模型的中间表示形式的新型通道和时间关注RNN（CTA-RNN）结构。具体而言，大规模预训练的端到端ASR编码器的嵌入包含声学和语言信息，以及能够推广到不同扬声器的能力，使其非常适合下游SER任务。为了进一步利用ASR编码器不同层的嵌入，我们提出了一种新颖的CTA-RNN架构，以捕获通道和时间方向上嵌入的情感显着部分。我们使用Corpus内部和交叉孔口设置在两个流行的基准数据集Iemocap和MSP-Improv上评估了我们的方法。实验结果表明，我们提出的方法可以在准确性和鲁棒性方面实现出色的性能。
Previous research has looked into ways to improve speech emotion recognition (SER) by utilizing both acoustic and linguistic cues of speech. However, the potential association between state-of-the-art ASR models and the SER task has yet to be investigated. In this paper, we propose a novel channel and temporal-wise attention RNN (CTA-RNN) architecture based on the intermediate representations of pre-trained ASR models. Specifically, the embeddings of a large-scale pre-trained end-to-end ASR encoder contain both acoustic and linguistic information, as well as the ability to generalize to different speakers, making them well suited for downstream SER task. To further exploit the embeddings from different layers of the ASR encoder, we propose a novel CTA-RNN architecture to capture the emotional salient parts of embeddings in both the channel and temporal directions. We evaluate our approach on two popular benchmark datasets, IEMOCAP and MSP-IMPROV, using both within-corpus and cross-corpus settings. Experimental results show that our proposed method can achieve excellent performance in terms of accuracy and robustness.
