论文标题
基于堆栈转换器的基于过渡的解析
Transition-based Parsing with Stack-Transformers
论文作者
论文摘要
建模解析器状态是基于过渡的解析的良好性能的关键。复发性神经网络通过对全球状态进行建模，例如stack-lstm解析器或上下文化特征的局部状态建模，例如BI-LSTM解析器。鉴于变压器体系结构在最近的解析系统中的成功，这项工作探讨了序列到序列变压器体系结构的修改，以模拟基于过渡的解析中的全球或局部解析器状态。我们表明，变压器的交叉注意机制的修改大大增强了依赖性和抽象意义表示（AMR）解析任务的性能，尤其是对于较小的模型或有限的培训数据。
Modeling the parser state is key to good performance in transition-based parsing. Recurrent Neural Networks considerably improved the performance of transition-based systems by modelling the global state, e.g. stack-LSTM parsers, or local state modeling of contextualized features, e.g. Bi-LSTM parsers. Given the success of Transformer architectures in recent parsing systems, this work explores modifications of the sequence-to-sequence Transformer architecture to model either global or local parser states in transition-based parsing. We show that modifications of the cross attention mechanism of the Transformer considerably strengthen performance both on dependency and Abstract Meaning Representation (AMR) parsing tasks, particularly for smaller models or limited training data.
