Page 198 - 《软件学报》2025年第4期

软件学报 ISSN 1000-9825, CODEN RUXUEW E-mail: jos@iscas.ac.cn
2025,36(4):1604−1619 [doi: 10.13328/j.cnki.jos.007215] [CSTR: 32375.14.jos.007215] http://www.jos.org.cn
©中国科学院软件研究所版权所有. Tel: +86-10-62562563

*
基于可控性解释的混合数据增强框架

孙泽辰, 肖义胜, 李俊涛, 张 民, 周国栋

(苏州大学 计算机科学与技术学院, 江苏 苏州 215008)
通信作者: 李俊涛, E-mail: ljt@suda.edu.cn

摘 要: 先前的预训练语言模型已在众多自然语言理解任务中展现了其卓越的性能. 然而, 它们常表现出捷径学习
的问题, 即学习了非鲁棒性特征与标签之间的虚假关联, 导致模型在不同于训练分布的测试场景中的泛化能力不
佳. 近期, 生成式预训练大模型在理解任务中的出色表现引起了广泛的关注, 但它们是否受到捷径学习的影响尚未
被充分研究. 以 LLaMA 系列模型与 FLAN-T5 模型为代表, 探究生成式预训练大模型在多个自然语言理解任务中
的捷径学习现象. 研究结果表明, 近期流行的生成式大模型仍然存在捷径学习的问题. 进而, 提出针对生成式预训

练大模型的捷径学习问题的缓解策略——基于可控性解释的混合数据增强框架. 该框架以数据为中心, 基于模型
生成的可控性解释数据与部分原始提示性数据构造小规模混合数据集, 开展模型微调. 在 3 个具有代表性的自然
语言理解任务中的大量实验结果表明, 使用该框架所构造的数据集训练模型能够有效缓解模型的捷径学习问题,
提升模型在分布外测试场景中的鲁棒性与泛化能力, 同时不牺牲甚至提升模型在分布内测试场景中的性能. 代码
已公开发布在 https://github.com/Mint9996/HEDA.
关键词: 捷径学习; 生成式预训练语言模型; 自然语言理解
中图法分类号: TP18

中文引用格式: 孙泽辰, 肖义胜, 李俊涛, 张民, 周国栋. 基于可控性解释的混合数据增强框架. 软件学报, 2025, 36(4): 1604–1619.
http://www.jos.org.cn/1000-9825/7215.htm
英文引用格式: Sun ZC, Xiao YS, Li JT, Zhang M, Zhou GD. Hybrid Data Augmentation Framework Based on Controllable
Explanation. Ruan Jian Xue Bao/Journal of Software, 2025, 36(4): 1604–1619 (in Chinese). http://www.jos.org.cn/1000-9825/7215.htm
Hybrid Data Augmentation Framework Based on Controllable Explanation

SUN Ze-Chen, XIAO Yi-Sheng, LI Jun-Tao, ZHANG Min, ZHOU Guo-Dong
(School of Computer Science and Technology, Soochow University, Suzhou 215008, China)
Abstract: Previous pre-trained language models (PLMs) have demonstrated excellent performance in numerous tasks of natural language
understanding (NLU). However, they generally suffer shortcut learning, which means learning the spurious correlations between non-robust
features and labels, resulting in poor generalization in out-of-distribution (OOD) test scenarios. Recently, the outstanding performance of
generative large language models (LLMs) in understanding tasks has attracted widespread attention, but the extent to which it is affected
by shortcut learning has not been fully studied. In this paper, the shortcut learning effect of generative LLMs in three NLU tasks is
investigated for the first time using the LLaMA series models and FLAN-T5 models as representatives. The results show that the shortcut
learning problem still exists in generative LLMs. Therefore, a hybrid data augmentation framework is proposed based on controllable
explanations as a mitigation strategy for the shortcut learning problem in generative LLMs. The framework is data-centric, constructing a
small-scale mix dataset composed of model-generated controllable explain data and partial original prompting data for model fine-tuning.
The experimental results in three representative NLU tasks show that the framework can effectively mitigate shortcut learning, and
significantly improve the robustness and generalization of the model in OOD test scenarios while avoiding sacrifice of or even improving

* 基金项目: 国家自然科学基金 (62206194); 江苏省自然科学基金 (BK20220488)
孙泽辰和肖义胜为共同第一作者.
收稿时间: 2023-10-18; 修改时间: 2024-02-03, 2024-03-27; 采用时间: 2024-04-15; jos 在线出版时间: 2024-06-20
CNKI 网络首发时间: 2024-06-21