Page 249 - 《软件学报》2026年第4期

软件学报 ISSN 1000-9825, CODEN RUXUEW E-mail: jos@iscas.ac.cn
2026,37(4):1690−1714 [doi: 10.13328/j.cnki.jos.007474] [CSTR: 32375.14.jos.007474] http://www.jos.org.cn
©中国科学院软件研究所版权所有. Tel: +86-10-62562563

*
基于大语言模型的故障复现测试用例生成方法

汪 莹, 字千成, 彭 鑫, 娄一翎

(复旦大学 计算机科学技术学院, 上海 200438)
通信作者: 娄一翎, E-mail: yilinglou@fudan.edu.cn

摘 要: GitHub 是目前最流行的开源项目管理平台之一. 由于团队协作的需要, GitHub 引入了问题报告跟踪功能
以方便项目使用者提交和追踪项目中出现的问题或新功能请求. 问题报告贡献者在解决问题报告时, 通常需要执
行故障复现测试用例来复现问题报告中提到的问题并验证问题报告是否解决. 然而, 在 SWE-bench Lite 数据集上
进行实证研究发现, 有近 90% 的问题报告在用户提交时没有附带故障复现测试用例, 这导致问题报告贡献者在解
决问题报告时还需额外编写故障复现测试用例, 带来了额外的工作负担. 现有的故障复现测试用例生成技术通常
依赖错误栈信息, 然而 GitHub 问题报告中并未明确要求有这类信息. 因此, 提出基于大语言模型的故障复现测试
用例生成方法, 旨在自动化地为 GitHub 问题报告生成故障复现测试用例, 帮助问题报告贡献者复现、理解并验证
问题报告, 提升问题报告解决效率. 该方法首先通过检索与问题报告相关的多样化代码上下文信息, 包括报错根函
数、import 语句和测试用例样本, 随后构建精确的 prompt, 以引导大语言模型生成有效的故障复现测试用例. 开展
对比实验和消融实验, 验证所提方法在面向 GitHub 问题报告的故障复现测试用例生成任务上的有效性.
关键词: 故障复现; 测试用例生成; 大语言模型; 检索增强生成
中图法分类号: TP311

中文引用格式: 汪莹, 字千成, 彭鑫, 娄一翎. 基于大语言模型的故障复现测试用例生成方法. 软件学报, 2026, 37(4): 1690–1714. http://
www.jos.org.cn/1000-9825/7474.htm
英文引用格式: Wang Y, Zi QC, Peng X, Lou YL. Failure Reproducing Test Case Generation Method Based on Large Language
Model. Ruan Jian Xue Bao/Journal of Software, 2026, 37(4): 1690–1714 (in Chinese). http://www.jos.org.cn/1000-9825/7474.htm

Failure Reproducing Test Case Generation Method Based on Large Language Model
WANG Ying, ZI Qian-Cheng, PENG Xin, LOU Yi-Ling
(School of Computer Science, Fudan University, Shanghai 200438, China)
Abstract: GitHub is one of the most popular open-source project management platforms. Due to the need for team collaboration, GitHub
introduced an issue tracking function to facilitate project users in submitting and tracking problems or new feature requests. When
resolving issues, contributors of open-source projects typically need to execute failure reproducing test cases to reproduce the problems
mentioned in the issue and verify whether the issue has been resolved. However, empirical research conducted on the SWE-bench Lite
dataset reveals that nearly 90% of issues are submitted without failure reproducing test cases, leading contributors to write additional
failure reproducing test cases when resolving the issues, bringing additional work burden. Existing failure reproducing test case generation
methods usually rely on stack trace information, but GitHub issues do not explicitly require such information. Therefore, this study
proposes a failure reproducing test case generation method based on a large language model, aimed at automatically generating failure
reproducing test cases for GitHub issues, assisting issue contributors in reproducing, understanding, and verifying issues, and improving the
efficiency of issue resolution. This method first retrieves diverse code context information related to the issue, including error root
functions, import statements, and test case examples, then constructs precise prompts to guide the large language model in generating
effective failure reproducing test cases. This study conducts comparative and ablation experiments to verify the effectiveness of this
method in generating failure reproducing test cases for GitHub issues.
Key words: failure reproducing; test case generation; large language model (LLM); retrieval-augmented generation (RAG)

* 收稿时间: 2024-11-09; 修改时间: 2025-03-25, 2025-05-06; 采用时间: 2025-05-26; jos 在线出版时间: 2025-09-28
CNKI 网络首发时间: 2025-09-30