Page 483 - 《软件学报》2025年第10期

软件学报 ISSN 1000-9825, CODEN RUXUEW E-mail: jos@iscas.ac.cn
2025,36(10):4880−4892 [doi: 10.13328/j.cnki.jos.007363] [CSTR: 32375.14.jos.007363] http://www.jos.org.cn
©中国科学院软件研究所版权所有. Tel: +86-10-62562563

*
用于二值神经网络的加宽和收缩机制

韩 凯 1,2 , 刘传建 3 , 吴恩华 1,4

1
(计算机科学国家重点实验室 (中国科学院 软件研究所), 北京 100190)
2
(中国科学院大学, 北京 100049)
3
(华为诺亚方舟实验室, 北京 100085)
4
(澳门大学 科技学院, 澳门 999078)
通信作者: 吴恩华, E-mail: ehwu@um.edu.mo
摘 要: 二值神经网络 (binary neural network, BNN) 因其较少的计算和存储开销而对业界非常有吸引力, 但其准确
率仍然比全精度参数的网络差. 大多数现有方法旨在通过利用更有效的训练技术来提高二值神经网络的性能. 然
而, 通过实验发现量化后特征的表示能力远弱于全精度的特征. 因此, 提出一种加宽和收缩机制来构建高精度而紧
凑的二值神经网络. 首先, 通过将原始全精度网络中的特征投影到高维量化特征来解决量化特征表示能力弱的问
题. 同时, 冗余的量化特征将被消除, 以避免某些特征维度的过度增长. 进而建立一个紧凑但具有足够表示能力的
量化神经网络. 基准数据集上的实验结果表明, 该方法能够以更少的参数量和计算量建立高精度二值神经网络, 其
准确率与全精度基线模型几乎相同, 例如, 二值量化的 ResNet-18 在 ImageNet 数据集上达到了 70% 的准确率.
关键词: 神经网络; 模型量化; 图像分类; 目标检测
中图法分类号: TP301

中文引用格式: 韩凯, 刘传建, 吴恩华. 用于二值神经网络的加宽和收缩机制. 软件学报, 2025, 36(10): 4880–4892. http://www.jos.
org.cn/1000-9825/7363.htm
英文引用格式: Han K, Liu CJ, Wu EH. Widening and Squeezing Mechanism for Binary Neural Network. Ruan Jian Xue Bao/Journal
of Software, 2025, 36(10): 4880–4892 (in Chinese). http://www.jos.org.cn/1000-9825/7363.htm

Widening and Squeezing Mechanism for Binary Neural Network
3
1,2
HAN Kai , LIU Chuan-Jian , WU En-Hua 1,4
1
(State Key Laboratory of Computer Science (Institute of Software, Chinese Academy of Sciences), Beijing 100190, China)
2
(University of Chinese Academy of Sciences, Beijing 100049, China)
3
(Huawei Noah’s Ark Lab, Beijing 100085, China)
4
(Faculty of Science and Technology, University of Macau, Macao 999078, China)
Abstract: Binary neural networks (BNNs) are highly appealing to the industry due to their significantly reduced computation and storage
requirements. However, their accuracy still lags behind that of networks with full-precision parameters. Most existing methods focus on
improving the performance of BNNs through advanced training techniques. Empirical findings reveal that the representation capability of
quantized features is considerably weaker than that of full-precision features. To address this limitation, a widening and squeezing
mechanism is proposed to construct high-accuracy yet compact BNNs. Specifically, features from the original full-precision networks are
projected into high-dimensional quantized features to mitigate the representation gap. Meanwhile, redundant quantized features are pruned
to prevent the over growth of feature dimensions. As a result, a compact yet sufficiently expressive quantized neural network is
constructed. Experimental results on benchmark datasets demonstrate that the proposed method achieves high-accuracy BNNs with
significantly fewer parameters and computations while delivering performance comparable to full-precision baseline models. For instance,

* 基金项目: 国家自然科学基金 (62332015)
收稿时间: 2023-11-09; 修改时间: 2024-06-17, 2024-09-18; 采用时间: 2024-11-26; jos 在线出版时间: 2025-07-17
CNKI 网络首发时间: 2025-07-18