论文标题
加强了具有自动交易应用程序的Markov模型
Reinforced Deep Markov Models With Applications in Automatic Trading
论文作者
论文摘要
受深生成模型中的发展的启发，我们提出了一种基于模型的RL方法，它提出了增强的Deep Markov模型（RDMM），该模型旨在整合充当自动交易系统的增强学习算法的理想属性。网络体系结构允许市场动态部分可见，并可能通过代理商的行动来修改。 RDMM过滤不完整和嘈杂的数据，以创建行为更好的输入数据，以用于RL计划。政策搜索优化还适当说明状态不确定性。由于RKDF模型体系结构的复杂性，我们进行了消融研究，以更好地了解该方法的各个组件的贡献。为了测试RDMM的财务业绩，我们使用Q-学习，Dynaq-Arima和Dynaq-LSTM算法的变体实施策略。实验表明，与最佳执行问题中的基准相比，RDMM具有数据效率，并提供了财务上的收益。当价格动态更为复杂时，性能的改进会变得更加明显，并且使用Facebook，Intel，Vodafone和Microsoft的“限制顺序”簿中的真实数据集证明了这一点。
Inspired by the developments in deep generative models, we propose a model-based RL approach, coined Reinforced Deep Markov Model (RDMM), designed to integrate desirable properties of a reinforcement learning algorithm acting as an automatic trading system. The network architecture allows for the possibility that market dynamics are partially visible and are potentially modified by the agent's actions. The RDMM filters incomplete and noisy data, to create better-behaved input data for RL planning. The policy search optimisation also properly accounts for state uncertainty. Due to the complexity of the RKDF model architecture, we performed ablation studies to understand the contributions of individual components of the approach better. To test the financial performance of the RDMM we implement policies using variants of Q-Learning, DynaQ-ARIMA and DynaQ-LSTM algorithms. The experiments show that the RDMM is data-efficient and provides financial gains compared to the benchmarks in the optimal execution problem. The performance improvement becomes more pronounced when price dynamics are more complex, and this has been demonstrated using real data sets from the limit order book of Facebook, Intel, Vodafone and Microsoft.
