论文标题
处理粒子数据用智能数据流动
Processing Particle Data Flows with SmartNICs
论文作者
论文摘要
许多分布式应用程序实现了复杂的数据流，并且需要一种灵活的机制，以在生产者和消费者之间进行路由数据。可编程网络接口卡（智能卡）的最新进展代表了将数据流任务卸载到网络结构中的机会，从而使主机释放了执行其他工作的机会。该空间中的系统架构师面临有关利用智能智能作为数据流中的处理元素的最佳方法的多个问题。在本文中，我们主张使用Apache Arrow作为实施Smartnics的数据流任务的基础。我们报告了我们的经验，以适应粒子数据的分区算法以apache Arrow并测量Bluefield-2智能NIC的卡上处理性能。我们的实验证实，Bluefield-2（DE）压缩硬件可以对必须打开，处理，处理和重新包装的数据的横向工作流程产生重大影响。
Many distributed applications implement complex data flows and need a flexible mechanism for routing data between producers and consumers. Recent advances in programmable network interface cards, or SmartNICs, represent an opportunity to offload data-flow tasks into the network fabric, thereby freeing the hosts to perform other work. System architects in this space face multiple questions about the best way to leverage SmartNICs as processing elements in data flows. In this paper, we advocate the use of Apache Arrow as a foundation for implementing data-flow tasks on SmartNICs. We report on our experiences adapting a partitioning algorithm for particle data to Apache Arrow and measure the on-card processing performance for the BlueField-2 SmartNIC. Our experiments confirm that the BlueField-2's (de)compression hardware can have a significant impact on in-transit workflows where data must be unpacked, processed, and repacked.
