Page 270 - 《软件学报》2026年第5期

曲慕子 等: CodeLLMTuner: 基于样本重用的代码大模型选择与解码参数调优框架 2149

4 总 结

针对代码大模型选择与解码参数调优问题, 现有组合方法在模型选择与参数调优两个阶段所采样的数据无法
重用, 导致两个阶段无法共享样本数据, 计算成本过大. 因此, 本文提出了一个基于样本重用的代码大模型选择与
解码参数调优框架 CodeLLMTuner, 该框架利用性能期望选择最优模型, 同时采用倾向评分匹配方法实现样本重
用以降低采样成本. 本文在不同类型代码任务上对参与实验的大模型本身的性能表现都进行了实验, 作为本文的
基线方法效果. 实验结果表明, 针对代码生成、代码摘要和测试用例生成这 3 项任务, CodeLLMTuner 相比基线方
法能够在相同成本下提高 10%–15% 的性能, 或在相同性能下节省超过 20% 的成本, 证明了本文所提出方法的有效性.

References
[1] Chen M, Tworek J, Jun H, et al. Evaluating large language models trained on code. arXiv:2107.03374, 2021.
[2] Li R, Allal LB, Zi YT, et al. StarCoder: May the source be with you! arXiv:2305.06161, 2023.
[3] Roziere B, Gehring J, Gloeckle F, et al. Code Llama: Open foundation models for code. arXiv:2308.12950, 2023.
[4] GitHub Copilot. Your AI pair programmer. 2025. https://github.com/features/copilot
[5] Yang ZZ, Chen SR, Gao CY, Li ZH, Li G, Lü RC. Deep learning based code generation methods: Literature review. Ruan Jian Xue
Bao/Journal of Software, 2024, 35(2): 604–628 (in Chinese with English abstract). http://www.jos.org.cn/1000-9825/6981.htm [doi: 10.
13328/j.cnki.jos.006981]
[6] Wang Y, Wang WS, Joty S, Hoi SCH. CodeT5: Identifier-aware unified pre-trained encoder-decoder models for code understanding and
generation. In: Proc. of the 2021 Conf. on Empirical Methods in Natural Language Processing. Punta Cana: ACL, 2021. 8696–8708. [doi:
10.18653/v1/2021.emnlp-main.685]
[7] Luo ZY, Xu C, Zhao P, Sun QF, Geng XB, Hu WX, Tao CY, Ma J, Lin QW, Jiang DX. WizardCoder: Empowering code large language
models with evol-instruct. In: Proc. of the 12th Int’l Conf. on Learning Representations. Vienna: OpenReview.net, 2024.
[8] Guo DY, Xu CW, Duan N, Yin J, McAuley J. LongCoder: A long-range pre-trained language model for code completion. In: Proc. of the
40th Int’l Conf. on Machine Learning. Honolulu: ACM, 2023. 486.
[9] Wu QY, Bansal G, Zhang JY, Wu YR, Zhang SK, Zhu EK, Li BB, Jiang L, Zhang XY, Wang C. AutoGen: Enabling next-gen LLM
applications via multi-agent conversation framework. arXiv:2308.08155, 2023.
[10] Jiang N, Lutellier T, Tan L. CURE: Code-aware neural machine translation for automatic program repair. In: Proc. of the 43rd
IEEE/ACM Int’l Conf. on Software Engineering (ICSE). Madrid: IEEE, 2021. 1161–1173. [doi: 10.1109/ICSE43902.2021.00107]
[11] Olausson TX, Inala JP, Wang CL, Gao JF, Solar-Lezama A. Is self-repair a silver bullet for code generation? In: Proc. of the 12th Int’l
Conf. on Learning Representations. Vienna: OpenReview.net, 2024.
[12] Li HN, Hao Y, Zhai YZ, Qian ZY. Enhancing static analysis for practical bug detection: An LLM-integrated approach. Proc. of the ACM
on Programming Languages, 2024, 8(OOPSLA1): 474–499. [doi: 10.1145/3649828]
[13] Phan L, Tran H, Le D, Nguyen H, Anibal JT, Peltekian A, Ye YF. CoTexT: Multi-task learning with code-text Transformer. arXiv:
2105.08645, 2021.
[14] van Aken D, Pavlo A, Gordon GJ, Zhang BH. Automatic database management system tuning through large-scale machine learning. In:
Proc. of the 2017 ACM Int’l Conf. on Management of Data. Chicago: ACM, 2017. 1009–1024. [doi: 10.1145/3035918.3064029]
[15] Wang C, Liu XQ, Awadallah AH. Cost-effective hyperparameter optimization for large language model generation inference. In: Proc. of
the 2023 Int’l Conf. on Automated Machine Learning. Potsdam: PMLR, 2023. 21.
[16] Wang SY, Li SM, Sun TX, Fu JL, Cheng QY, Ye JS, Ye JJ, Qiu XP, Huang XJ. LLM can achieve self-regulation via hyperparameter
aware generation. In: Proc. of the 2024 Findings of the Association for Computational Linguistics. Bangkok: ACL, 2024. 6632–6646.
[doi: 10.18653/v1/2024.findings-acl.396]
[17] Fu J, Qin XT, Yang FK, Wang L, Zhang J, Lin QW, Chen YB, Zhang DM, Rajmohan S, Zhang Q. AutoRAG-HP: Automatic online
hyper-parameter tuning for retrieval-augmented generation. In: Proc. of the 2024 Findings of the Association for Computational
Linguistics. Miami: ACL, 2024. 3875–3891. [doi: 10.18653/v1/2024.findings-emnlp.223]
[18] Guo QY, Wang R, Guo JL, Li B, Song KT, Tan X, Liu GQ, Bian J, Yang YJ. Connecting large language models with evolutionary
algorithms yields powerful prompt optimizers. In: Proc. of the 12th Int’l Conf. on Learning Representations. Vienna: OpenReview.net,
2024.
[19] Wang Z, Panda R, Karlinsky L, Feris R, Sun H, Kim Y. Multitask prompt tuning enables parameter-efficient transfer learning. In: Proc. of
the 11th Int’l Conf. on Learning Representations. Kigali: OpenReview.net, 2023.
[20] Wang YD, Yu ZH, Yao WJ, Zeng ZR, Yang LY, Wang CX, Chen H, Jiang CY, Xie R, Wang JD, Xie X, Ye W, Zhang SK, Zhang Y.
PandaLM: An automatic evaluation benchmark for LLM instruction tuning optimization. In: Proc. of the 12th Int’l Conf. on Learning
Representations. Vienna: OpenReview.net, 2024.
[21] Tribes C, Benarroch-Lelong S, Lu P, Kobyzev I. Hyperparameter optimization for large language model instruction-tuning. arXiv:
2312.00949, 2023.