论文标题
λ尺度注意：一种新型的快速注意机制，用于有效建模蛋白质序列
λ-Scaled-Attention: A Novel Fast Attention Mechanism for Efficient Modeling of Protein Sequences
论文作者
论文摘要
基于注意力的深网已成功应用于NLP领域的文本数据。但是，由于蛋白质单词的语义弱，与纯文本单词不同，它们在蛋白质序列上的应用构成了其他挑战。标准注意技术面临的这些未开发的挑战包括（i）消失的注意力评分问题和（ii）注意力分布的较高差异。在这方面，我们引入了一种新型的λ刻度注意力技术，以快速有效地建模蛋白质序列，以解决上述两个问题。这用于开发λ标准的注意网络，并评估了在蛋白质子序列水平上实施的蛋白质功能预测的任务。基于标准注意力技术的λ量表量表的注意力（MF）的生物过程（BP）和分子功能（MF）的实验表明，基于标准注意技术（BP的 +2.01％）的F1得分值显着提高，MF的方法为 +2.01％，而MF的4.67％）和对ART-ART PROTVECGEN-PORTVECGEN-PLEVECGEN-PLUS方法（ +2.61％）（ +2.61％）（ +2.61％）。此外，在训练过程中，还观察到快速收敛（在训练和验证损失之间的差异和验证损失之间的差异）和有效学习（在训练和验证损失之间的差异很低）中。
Attention-based deep networks have been successfully applied on textual data in the field of NLP. However, their application on protein sequences poses additional challenges due to the weak semantics of the protein words, unlike the plain text words. These unexplored challenges faced by the standard attention technique include (i) vanishing attention score problem and (ii) high variations in the attention distribution. In this regard, we introduce a novel λ-scaled attention technique for fast and efficient modeling of the protein sequences that addresses both the above problems. This is used to develop the λ-scaled attention network and is evaluated for the task of protein function prediction implemented at the protein sub-sequence level. Experiments on the datasets for biological process (BP) and molecular function (MF) showed significant improvements in the F1 score values for the proposed λ-scaled attention technique over its counterpart approach based on the standard attention technique (+2.01% for BP and +4.67% for MF) and state-of-the-art ProtVecGen-Plus approach (+2.61% for BP and +4.20% for MF). Further, fast convergence (converging in half the number of epochs) and efficient learning (in terms of very low difference between the training and validation losses) were also observed during the training process.
