论文标题
对象表示作为固定点：培训具有隐式差异化的迭代改进算法
Object Representations as Fixed Points: Training Iterative Refinement Algorithms with Implicit Differentiation
论文作者
论文摘要
迭代精致 - 从随机的猜测开始，然后迭代地改善猜测 - 是表示学习的有用范式，因为它提供了一种在数据中同样合理的解释之间打破对称性的方法。此属性使此类方法的应用可以推断实体集的表示形式，例如物理场景中的对象，在结构上类似于潜在空间中的聚类算法。但是，大多数先前的工作通过展开的完善过程进行区分，这可能会使优化具有挑战性。我们观察到，可以通过隐式函数定理使这种方法可区分，并开发一种隐含的分化方法，从而通过解耦来向前和向后传递来提高训练的稳定性和障碍。该连接使我们能够在优化隐式层时应用进步，不仅可以改善Slate中的插槽注意模块的优化，Slate是一种学习实体表示的最新方法，而且要在反向传播中持续的空间和时间复杂性，也只有一条额外的代码。
Iterative refinement -- start with a random guess, then iteratively improve the guess -- is a useful paradigm for representation learning because it offers a way to break symmetries among equally plausible explanations for the data. This property enables the application of such methods to infer representations of sets of entities, such as objects in physical scenes, structurally resembling clustering algorithms in latent space. However, most prior works differentiate through the unrolled refinement process, which can make optimization challenging. We observe that such methods can be made differentiable by means of the implicit function theorem, and develop an implicit differentiation approach that improves the stability and tractability of training by decoupling the forward and backward passes. This connection enables us to apply advances in optimizing implicit layers to not only improve the optimization of the slot attention module in SLATE, a state-of-the-art method for learning entity representations, but do so with constant space and time complexity in backpropagation and only one additional line of code.
