Page 28 - 《软件学报》2020年第9期

葛道辉 等:轻量级神经网络架构综述 2649

要有减少卷积核的数量、减少特征的通道数以及设计更高效的卷积操作等关键技术,但是非常依赖设计者的经
验.如何有效地将针对特定问题的先验知识加入到模型构建过程中,是未来研究的重点方向.通过网络剪枝、权
重压缩和低秩分解是对已有的网络进行压缩,但是压缩算法需要设计者探索较大的设计空间以及在模型大小、
速度和准确率之间权衡.为了减少人为因素的干扰,自动机器学习技术是未来研究的热点,联合优化深度神经网
络流程的所有模型参数.神经网络架构搜索的研究主要集中在深度神经网络上,许多搜索架构都源自 NASNet [6]
搜索空间,通过各种搜索算法在定义的搜索空间内自动生成的,广泛应用于解决图像识别、图像分割和语言建
模等任务 [6,7,98,99] ,但是只能针对某一特定或同一类型的数据集.如何使用跨不同数据集的知识来加速优化过程,
是未来研究的热点.其他的挑战是联合优化深度神经网络流程的所有模型参数.到目前为止,深度神经网络的通
用自动化仍处于起步阶段,许多问题尚未得到解决.然而,这仍然是一个令人兴奋的领域,并且未来的工作的方
向需要强调其突出的实用性.
轻量级模型的发展使得神经网络更加高效,从而能够广泛地应用到各种场景任务中.一方面,轻量级神经网
络有更小的体积和计算量,降低了对设备存储能力和计算能力的需求,既可以装配到传统家电中使其更加智能
化,也可以将深度学习系统应用在虚拟现实、增强现实、智能安防和智能可穿戴设备等新兴技术中;另一方面,
轻量级神经网络具有更快的运行速度和更短的延时,能够对任务进行实时处理,对于在线学习、增量学习和分
布式学习有重大意义;另外,实时处理的神经网络能够满足自动驾驶技术的需求,提高自动驾驶的安全性.轻量
级神经网络将对于人工智能技术的普及、建立智能化城市起不可或缺的作用.

References:
[1] Simonyan K, Zisserman A. Very deep convolutional networks for large-scale image recognition. In: Proc. of the 3rd Int’l Conf. on
Learning Representations. 2015.
[2] He K, Zhang X, Ren S, Sun J. Deep residual learning for image recognition. In: Proc. of the IEEE Conf. on Computer Vision and
Pattern Recognition. 2016. 770−778.
[3] Huang G, Liu Z, Maaten LVD, Weinberger K. Densely connected convolutional networks. In: Proc. of the IEEE Conf. on
Computer Vision and Pattern Recognition. 2017. 4700−4708.
[4] Han S, Mao H, Dally WJ. Deep compression: Compressing deep neural network with pruning, trained quantization and huffman
coding. In: Proc. of the 4th Int’l Conf. on Learning Representations. 2016.
[5] He Y, Lin J, Liu Z, Wang H, Li L, Han S. AMC: Automl for model compression and acceleration on mobile devices. In: Proc. of
the European Conf. on Computer Vision. 2018. 784−800.
[6] Zoph B, Vasudevan V, Shlens JV, Le Q. Learning transferable architectures for scalable image recognition. In: Proc. of the IEEE
Conf. on Computer Vision and Pattern Recognition. 2018. 8697−8710.
[7] Tan M, Chen B, Pang R, Vasudevan V, Sandler M, Howard AV, Le Q. Mnasnet: Platform-aware neural architecture search for
mobile. In: Proc. of the IEEE Conf. on Computer Vision and Pattern Recognition. 2019. 2820−2828.
[8] Howard AG, Zhu M, Chen B, Kalenichenko D, Wang W, Weyand T, Andreetto M, Adam H. Mobilenets: Efficient convolutional
neural networks for mobile vision applications. In: Proc. of the IEEE Conf. on Computer Vision and Pattern Recognition. 2017.
432−445.
[9] Zhang X, Zhou X, Lin M, Sun J. Shufflenet: An extremely efficient convolutional neural network for mobile devices. In: Proc. of
the IEEE Conf. on Computer Vision and Pattern Recognition. 2018. 6848−6856.
[10] Sandler M, Howard A, Zhu M, Zhmoginov A, Chen L. MobileNetV2: Inverted residuals and linear bottlenecks. In: Proc. of the
IEEE Conf. on Computer Vision and Pattern Recognition. 2018. 4510−4520.
[11] Qin Z, Li Z, Zhang Z, Bao Y, Yu G, Peng Y, Sun J. ThunderNet: Towards real-time generic object detection on mobile devices. In:
Proc. of the IEEE Int’l Conf. on Computer Vision. 2019. 6718−6727.
[12] Ma N, Zhang X, Zheng H, Sun J. ShuffleNet V2: Practical guidelines for efficient cnn architecture design. In: Proc. of the
European Conf. on Computer Vision. 2018. 116−131.
[13] Landola FN, Han S, Moskewicz MW, Ashraf K, Han S, Dally WJ, Keutzer K. SqueezeNet: AlexNet-level accuracy with 50x fewer
parameters and <0.5MB model size. In: Proc. of the 5th Int’l Conf. on Learning Representations. 2017.
[14] Bergstra JS, Bardenet R, Bengio Y, Kégl B. Algorithms for hyper-parameter optimization. In: Proc. of the Advances in Neural
Information Processing Systems. 2011. 2546−2554.