Page 138 - 《软件学报》2021年第12期

软件学报 ISSN 1000-9825, CODEN RUXUEW E-mail: jos@iscas.ac.cn
Journal of Software,2021,32(12):3802−3813 [doi: 10.13328/j.cnki.jos.006126] http://www.jos.org.cn
©中国科学院软件研究所版权所有. Tel: +86-10-62562563

∗
基于实值 RBM 的深度生成网络研究

3
1,2
1,2
张 健 , 丁世飞 , 丁 玲 , 张成龙 1
1
(中国矿业大学 计算机科学与技术学院,江苏 徐州 221116)
2 (矿山数字化教育部工程研究中心,江苏 徐州 221116)
3 (中国矿业大学 徐海学院,江苏 徐州 221008)
通讯作者: 丁世飞, E-mail: dingsf@cumt.edu.cn; 张健, E-mail: 597409675@qq.com

摘 要: 受限玻尔兹曼机(restricted Boltzmann machine,简称 RBM)是一种概率无向图,传统的 RBM 模型假设隐藏
层单元是二值的,二值单元的优势在于计算过程和采样过程相对简单,然而二值化会对基于隐藏层单元的特征提取
和数据重构过程带来信息损失.因此,将 RBM 的可见层单元和隐藏层单元实值化并保持模型训练的有效性,是目前
RBM 理论研究的重点问题.为了解决这个问题,将二值单元拓展为实值单元,利用实值单元建模数据并提取特征.具
体而言,在可见层单元和隐藏层单元之间增加辅助单元,然后将图正则化项引入到能量函数中,基于二值辅助单元和
图正则化项,流形上的数据有更高的概率被映射为参数化的截断高斯分布;同时,远离流形的数据有更高的概率被映
射为高斯噪声.由此,模型的隐层单元可以被表示为参数化截断高斯分布或高斯噪声的采样实值.该模型称为基于辅
助单元的受限玻尔兹曼机(restricted Boltzmann machine with auxiliary units,简称 ARBM).在理论上分析了模型的有
效性,然后构建了相应的深度模型,并通过实验验证模型在图像重构任务和图像生成任务中的有效性.
关键词: 受限玻尔兹曼机;神经网络;概率图模型;深度学习
中图法分类号: TP18

中文引用格式: 张健,丁世飞,丁玲,张成龙.基于实值 RBM 的深度生成网络研究.软件学报,2021,32(12):3802−3813. http://
www.jos.org.cn/1000-9825/6126.htm
英文引用格式: Zhang J, Ding SF, Ding L, Zhang CL. Deep generative neural networks based on real-valued RBM with auxiliary
hidden units. Ruan Jian Xue Bao/Journal of Software, 2021,32(12):3802−3813 (in Chinese). http://www.jos.org.cn/1000-9825/6126.htm
Deep Generative Neural Networks Based on Real-valued RBM with Auxiliary Hidden Units

3
1,2
ZHANG Jian 1,2 , DING Shi-Fei , DING Ling , ZHANG Cheng-Long 1
1 (School of Computer Science and Technology, China University of Mining and Technology, Xuzhou 221116, China)
2 (Engineering Research Center of Mine Digitization of Ministry of Education, Xuzhou 221116, China)
3 (Xuhai College, China University of Mining and Technology, Xuzhou 221008, China)

Abstract: Restricted Boltzmann machine (RBM) is a probabilistic undirected graph, and most traditional RBM models assume that their
hidden layer units are binary. The advantage of binary units is their calculation process and sampling process are relatively simple.
However, binarized hidden units may bring information loss to the process of feature extraction and data reconstruction. Therefore, a key
research point of RBM theory is to construct real-valued visible layer units and hidden layer units, meanwhile, maintain the effectiveness
of model training. In this study, the binary units are extended to real-valued units to model data and extract features. To achieve this,
specifically, an auxiliary unit is added between the visible layer and the hidden layer, and then the graph regularization term is introduced
into the energy function. Based on the binary auxiliary unit and graph regularization term, the data on the manifold has a higher
probability to be mapped as a parameterized truncated Gaussian distribution, simultaneously, the data far from the manifold has a higher

∗ 基金项目: 国家自然科学基金(61976216, 61672522)
Foundation item: National Natural Science Foundation of China (61976216, 61672522)
收稿时间: 2020-04-14; 修改时间: 2020-06-05; 采用时间: 2020-08-07