论文标题
适用语言模型的伪-OOD培训
Pseudo-OOD training for robust language models
论文作者
论文摘要
虽然预先训练的大规模深层模型吸引了许多下游自然语言处理（NLP）任务的重要主题，但这种模型通常会对分布式（OOD）输入进行不可靠的预测。因此，OOD检测是任何行业规模应用的可靠机器学习模型的关键组成部分。常见方法通常假设在训练阶段访问其他OOD样本，但是，离群分布通常未知。取而代之的是，我们提出了一个称为poore的事后框架 - poore-thoc pseudo-ood正则化，该框架使用分布式（IND）数据生成伪-OON样品。该模型通过引入新的正则损失来微调，该损失将IND和OOD数据的嵌入分开，从而在测试过程中导致OOD预测任务的显着增长。我们对三个现实世界对话系统进行了广泛的评估框架，从而实现了OOD检测的新最新。
While pre-trained large-scale deep models have garnered attention as an important topic for many downstream natural language processing (NLP) tasks, such models often make unreliable predictions on out-of-distribution (OOD) inputs. As such, OOD detection is a key component of a reliable machine-learning model for any industry-scale application. Common approaches often assume access to additional OOD samples during the training stage, however, outlier distribution is often unknown in advance. Instead, we propose a post hoc framework called POORE - POsthoc pseudo-Ood REgularization, that generates pseudo-OOD samples using in-distribution (IND) data. The model is fine-tuned by introducing a new regularization loss that separates the embeddings of IND and OOD data, which leads to significant gains on the OOD prediction task during testing. We extensively evaluate our framework on three real-world dialogue systems, achieving new state-of-the-art in OOD detection.
