论文标题
潜在的神经氧和稀疏的贝叶斯多射击
Latent Neural ODEs with Sparse Bayesian Multiple Shooting
论文作者
论文摘要
长轨迹上的训练动态模型（例如神经odes）是一个困难的问题，需要使用各种技巧，例如轨迹裂片，以使模型训练在实践中起作用。这些方法通常是启发式方法，理论理由差，需要迭代手动调整。我们为神经odes提出了一种原则上的多重拍摄技术，将轨迹分解为可管理的短段，并并行优化，同时确保对连续段的连续性进行概率控制。我们针对基于拍摄的潜在神经ode模型得出了变异推断，并提出了具有时间关注和相对位置编码的基于变压器的识别网络的不规则采样轨迹的摊销编码。我们在多个大规模基准数据集上展示了高效且稳定的培训以及最先进的性能。
Training dynamic models, such as neural ODEs, on long trajectories is a hard problem that requires using various tricks, such as trajectory splitting, to make model training work in practice. These methods are often heuristics with poor theoretical justifications, and require iterative manual tuning. We propose a principled multiple shooting technique for neural ODEs that splits the trajectories into manageable short segments, which are optimised in parallel, while ensuring probabilistic control on continuity over consecutive segments. We derive variational inference for our shooting-based latent neural ODE models and propose amortized encodings of irregularly sampled trajectories with a transformer-based recognition network with temporal attention and relative positional encoding. We demonstrate efficient and stable training, and state-of-the-art performance on multiple large-scale benchmark datasets.
