论文标题
在手术动作三重盘数据集上进行方法基准测试的数据拆分和指标
Data Splits and Metrics for Method Benchmarking on Surgical Action Triplet Datasets
论文作者
论文摘要
除了生成数据和注释外，设计明智的数据拆分策略和评估指标对于创建基准数据集至关重要。这种做法确保了对数据的使用，均质评估以及数据集研究方法的统一比较的共识。这项研究的重点是Cholect50，这是一个50个视频外科数据集，将手术活动形式化为<仪器，动词，目标>的三胞胎。在本文中，我们介绍了Cholect50和Cholect45数据集的标准拆分，并展示了它们与数据集的现有使用方式。 Cholect45是45个Cholect50数据集的第一个公开版本。我们还开发了一个指标库，即IVTmetrics，用于手术三胞胎的模型评估。此外，我们通过在最主要使用的深度学习框架（Pytorch和Tensorflow）中复制基线方法来进行基准研究，以使用拟议的数据拆分和度量标准对其进行评估，并公开释放它们以支持未来的研究。提出的数据拆分和评估指标将使数据集上的研究进度进行全球跟踪，并促进最佳模型选择以进行进一步部署。
In addition to generating data and annotations, devising sensible data splitting strategies and evaluation metrics is essential for the creation of a benchmark dataset. This practice ensures consensus on the usage of the data, homogeneous assessment, and uniform comparison of research methods on the dataset. This study focuses on CholecT50, which is a 50 video surgical dataset that formalizes surgical activities as triplets of <instrument, verb, target>. In this paper, we introduce the standard splits for the CholecT50 and CholecT45 datasets and show how they compare with existing use of the dataset. CholecT45 is the first public release of 45 videos of CholecT50 dataset. We also develop a metrics library, ivtmetrics, for model evaluation on surgical triplets. Furthermore, we conduct a benchmark study by reproducing baseline methods in the most predominantly used deep learning frameworks (PyTorch and TensorFlow) to evaluate them using the proposed data splits and metrics and release them publicly to support future research. The proposed data splits and evaluation metrics will enable global tracking of research progress on the dataset and facilitate optimal model selection for further deployment.
