论文标题
着装：动态实时稀疏子网
DRESS: Dynamic REal-time Sparse Subnets
论文作者
论文摘要
边缘设备上有限且动态的资源促使我们部署了一个优化的深神经网络，该网络可以使其子网络适应不同的资源约束。但是，现有作品通常通过在手工制作的采样空间中搜索不同的网络体系结构来构建子网络，这不仅可以导致低标准的性能，而且可能导致设备上的重新配置开销。在本文中，我们提出了一种新颖的培训算法，动态的实时稀疏子网（着装）。着装通过基于行的非结构化稀疏度从相同的骨干网络进行多个子网络样本，并与加权损失并行训练这些子网络。着装还利用策略，包括参数重复使用和基于行的细粒抽样，以进行有效的存储消耗和有效的机上适应。公众视觉数据集的广泛实验表明，与最先进的子网络相比，着装的准确性明显更高。
The limited and dynamically varied resources on edge devices motivate us to deploy an optimized deep neural network that can adapt its sub-networks to fit in different resource constraints. However, existing works often build sub-networks through searching different network architectures in a hand-crafted sampling space, which not only can result in a subpar performance but also may cause on-device re-configuration overhead. In this paper, we propose a novel training algorithm, Dynamic REal-time Sparse Subnets (DRESS). DRESS samples multiple sub-networks from the same backbone network through row-based unstructured sparsity, and jointly trains these sub-networks in parallel with weighted loss. DRESS also exploits strategies including parameter reusing and row-based fine-grained sampling for efficient storage consumption and efficient on-device adaptation. Extensive experiments on public vision datasets show that DRESS yields significantly higher accuracy than state-of-the-art sub-networks.
