论文标题
下一代ML模型服务的Desiderata
Desiderata for next generation of ML model serving
论文作者
论文摘要
推论是ML软件基础架构的重要组成部分。尽管有各种各样的推理框架，但整个领域都可以在其早期考虑。该立场论文提出了下一代推理平台应追求的一系列重要素质。我们为每种质量的重要性提供了理由，并讨论了实践中实现它的方法。我们建议将重点放在数据中心作为总体设计模式上，该模式可以大规模地进行ML系统部署和操作。
Inference is a significant part of ML software infrastructure. Despite the variety of inference frameworks available, the field as a whole can be considered in its early days. This position paper puts forth a range of important qualities that next generation of inference platforms should be aiming for. We present our rationale for the importance of each quality, and discuss ways to achieve it in practice. We propose to focus on data-centricity as the overarching design pattern which enables smarter ML system deployment and operation at scale.
