Page 142 - 《软件学报》2021年第11期

软件学报 ISSN 1000-9825, CODEN RUXUEW E-mail: jos@iscas.ac.cn
Journal of Software,2021,32(11):3468−3481 [doi: 10.13328/j.cnki.jos.006057] http://www.jos.org.cn
©中国科学院软件研究所版权所有. Tel: +86-10-62562563

∗
自动化张量分解加速卷积神经网络

2,3
2,3
1
2,3
2,3
宋冰冰 , 张 浩 , 吴子锋 , 刘俊晖 , 梁 宇 , 周 维 2,3
1
(云南大学 信息学院,云南 昆明 650504)
2
(云南大学 软件学院,云南 昆明 650504)
3 (云南大学 跨境网络空间安全工程研究中心,云南 昆明 650504)
通讯作者: 周维, E-mail: zwei@ynu.edu.cn

摘 要: 近年来,卷积神经网络(CNN)展现了强大的性能,被广泛应用到了众多领域.由于 CNN 参数数量庞大,且存
储和计算能力需求高,其难以部署在资源受限设备上.因此,对 CNN 的压缩和加速成为一个迫切需要解决的问题.随
着自动化机器学习(AutoML)的研究与发展,AutoML 对神经网络发展产生了深远的影响.受此启发,提出了基于参数
估计和基于遗传算法的两种自动化加速卷积神经网络算法.该算法能够在给定精度损失范围内自动计算出最优的
CNN 加速模型,有效地解决了张量分解中,人工选择秩带来的误差问题,能够有效地提升 CNN 的压缩和加速效果.
通过在 MNIST 和 CIFAR-10 数据集上的严格测试,与原网络相比,在 MNIST 数据集上准确率稍微下降了 0.35%,模
型的运行时间获得了4.1倍的大幅提升;在CIFAR-10数据集上,准确率稍微下降了5.13%,模型的运行时间获得了0.8
倍的大幅提升.
关键词: 张量分解;卷积神经网络;自动化机器学习;神经网络压缩;神经网络加速
中图法分类号: TP183

中文引用格式: 宋冰冰,张浩,吴子锋,刘俊晖,梁宇,周维.自动化张量分解加速卷积神经网络.软件学报,2021,32(11):3468−
3481. http://www.jos.org.cn/1000-9825/6057.htm
英文引用格式: Song BB, Zhang H, Wu ZF, Liu JH, Liang Y, Zhou W. Automated tensor decomposition to accelerate
convolutional neural networks. Ruan Jian Xue Bao/Journal of Software, 2021,32(11):3468−3481 (in Chinese). http://www.jos.org.
cn/1000-9825/6057.htm
Automated Tensor Decomposition to Accelerate Convolutional Neural Networks

1
2,3
2,3
2,3
2,3
SONG Bing-Bing , ZHANG Hao , WU Zi-Feng , LIU Jun-Hui , LIANG Yu , ZHOU Wei 2,3
1
(School of Information Science and Engineering, Yunnan University, Kunming 650504, China)
2
(National Pilot School of Software, Yunnan University, Kunming 650504, China)
3
(Engineering Research Center of Cyberspace, Yunnan University, Kunming 650504, China)
Abstract: Recently, convolutional neural network (CNN) have demonstrated strong performance and are widely used in many fields.
Due to the large number of CNN parameters and high storage and computing power requirements, it is difficult to deploy on
resource-constrained devices. Therefore, compression and acceleration of CNN models have become an urgent problem to be solved. With
the research and development of automatic machine learning (AutoML), AutoML has profoundly impacted the development of neural
networks. Inspired by this, this study proposes two automated accelerated CNN algorithms based on parameter estimation and genetic
algorithms, which can calculate the optimal accelerated CNN model within a given accuracy loss range, effectively solving the error
caused by artificially selected rank in tensor decomposition. It can effectively improve the compression and acceleration effects of the
convolutional neural network. By rigorous testing on the MNIST and CIFAR-10 data sets, the accuracy rate on the MNIST dataset is

∗ 基金项目: 国家自然科学基金(61762089, 61863036, 61663047)
Foundation item: National Natural Science Foundation of China (61762089, 61863036, 61663047)
收稿时间: 2019-11-01; 修改时间: 2020-02-05; 采用时间: 2020-04-16