Page 476 - 《软件学报》2024年第6期

软件学报 ISSN 1000-9825, CODEN RUXUEW E-mail: jos@iscas.ac.cn
Journal of Software,2024,35(6):3052−3068 [doi: 10.13328/j.cnki.jos.006921] http://www.jos.org.cn
©中国科学院软件研究所版权所有. Tel: +86-10-62562563

*
高清几何缓存多尺度特征融合的渲染超分方法

张浩南, 过 洁, 覃浩宇, 傅锡豪, 郭延文

(计算机软件新技术国家重点实验室 (南京大学), 江苏 南京 210023)
通信作者: 过洁, E-mail: guojie@nju.edu.cn

摘 要: 人们对图像显示设备高分辨率和逼真视觉感知的需求随着现代信息技术的发展日益增长, 这对计算机软
硬件提出了更高要求, 也为渲染技术在性能与工作负载上带来更多挑战. 利用深度神经网络等机器学习技术对渲
染图像进行质量改进和性能提升成为了计算机图形学热门的研究方向, 其中通过网络推理将低分辨率图像进行上
采样获得更加清晰的高分辨率图像是提升图像生成性能并保证高清细节的一个重要途径. 而渲染引擎在渲染流程
中产生的几何缓存 (geometry buffer, G-buffer) 包含较多的语义信息, 能够帮助网络有效地学习场景信息与特征, 从
而提升上采样结果的质量. 设计一个基于深度神经网络的低分辨率渲染内容的超分方法. 除了当前帧的颜色图像,
其使用高分辨率的几何缓存来辅助计算并重建超分后的内容细节. 所提方法引入一种新的策略来融合高清缓存与
低清图像的特征信息, 在特定的融合模块中对不同种特征信息进行多尺度融合. 实验验证所提出的融合策略和模
块的有效性, 并且, 在和其他图像超分辨率方法的对比中, 所提方法体现出明显的优势, 尤其是在高清细节保持方面.
关键词: 神经网络; 渲染; 图像超分; 几何缓存; 特征融合
中图法分类号: TP391

中文引用格式: 张浩南, 过洁, 覃浩宇, 傅锡豪, 郭延文. 高清几何缓存多尺度特征融合的渲染超分方法. 软件学报, 2024, 35(6):
3052–3068. http://www.jos.org.cn/1000-9825/6921.htm
英文引用格式: Zhang HN, Guo J, Qin HY, Fu XH, Guo YW. Super-resolution Method for Rendered Contents by Multi-scale Feature
Fusion with High-resolution Geometry Buffers. Ruan Jian Xue Bao/Journal of Software, 2024, 35(6): 3052–3068 (in Chinese). http://
www.jos.org.cn/1000-9825/6921.htm

Super-resolution Method for Rendered Contents by Multi-scale Feature Fusion with
High-resolution Geometry Buffers

ZHANG Hao-Nan, GUO Jie, QIN Hao-Yu, FU Xi-Hao, GUO Yan-Wen
(State Key Laboratory for Novel Software Technology (Nanjing University), Nanjing 210023, China)
Abstract: With the development of modern information technology, people’s demand for high resolution and realistic visual perception of
image display devices has increased, which has put forward higher requirements for computer software and hardware and brought many
challenges to rendering technology in terms of performance and workload. Using machine learning technologies such as deep neural
networks to improve the quality and performance of rendered images has become a popular research method in computer graphics, while
upsampling low-resolution images through network inference to obtain clearer high-resolution images is an important way to improve
image generation performance and ensure high-resolution details. The geometry buffers (G-buffers) generated by the rendering engine in
the rendering process contain much semantic information, which help the network learn scene information and features effectively and then
improve the quality of upsampling results. In this study, a super-resolution method for rendered contents in low resolution based on deep
neural networks is designed. In addition to the color image of the current frame, the method uses high-resolution G-buffers to assist in the
calculation and reconstruct the high-resolution content details. The method also leverages a new strategy to fuse the features of high-
resolution buffers and low-resolution images, which implements a multi-scale fusion of different feature information in a specific fusion

* 基金项目: 国家自然科学基金 (61972194, 62032011); 江苏省自然科学基金 (BK20211147)
收稿时间: 2022-08-20; 修改时间: 2022-10-08; 采用时间: 2023-02-15; jos 在线出版时间: 2023-08-09
CNKI 网络首发时间: 2023-08-10