论文标题
长篇小说中的可区分推理 - 评估神经模型中的系统概括
Differentiable Reasoning over Long Stories -- Assessing Systematic Generalisation in Neural Models
论文作者
论文摘要
当代神经网络在许多方面都取得了一系列发展和成功。但是，当暴露于培训分布之外的数据时，他们可能无法预测正确的答案。在这项工作中，我们担心这个概括问题，因此在长篇小说中系统地和牢固地分析了一系列广泛的模型。相关实验是基于ClutRR进行的，该实验是一个诊断基准套件，可以通过对小故事图进行训练并对较大的基础进行测试来分析自然语言理解（NLU）系统的概括。为了处理多关系故事图，我们考虑了两个类别的神经模型：“ e-gnn”，基于图的模型可以处理图形结构化的数据并同时考虑边缘属性；和“ L-Graph”，基于序列的模型，可以处理图形的线性化版本。我们进行了广泛的经验评估，我们发现修改后的复发性神经网络在每个系统的概括任务中产生了令人惊讶的准确结果，这些任务表现优于修改后的图神经网络，而后者产生了更强大的模型。
Contemporary neural networks have achieved a series of developments and successes in many aspects; however, when exposed to data outside the training distribution, they may fail to predict correct answers. In this work, we were concerned about this generalisation issue and thus analysed a broad set of models systematically and robustly over long stories. Related experiments were conducted based on the CLUTRR, which is a diagnostic benchmark suite that can analyse generalisation of natural language understanding (NLU) systems by training over small story graphs and testing on larger ones. In order to handle the multi-relational story graph, we consider two classes of neural models: "E-GNN", the graph-based models that can process graph-structured data and consider the edge attributes simultaneously; and "L-Graph", the sequence-based models which can process linearized version of the graphs. We performed an extensive empirical evaluation, and we found that the modified recurrent neural network yield surprisingly accurate results across every systematic generalisation tasks which outperform the modified graph neural network, while the latter produced more robust models.
