Page 457 - 《软件学报》2024年第6期

朱光辉 等: 基于自引导进化策略的高效自动化数据增强算法 3033

略的自动化数据增强算法 SGES AA. 首先, 研究设计了一种有效的数据增强策略连续化向量表示方法, 将自动化
数据增强问题转换为连续化策略向量的搜索问题. 其次, 研究提出了一种基于自引导进化策略的策略向量搜索方
法, 通过引入历史估计梯度信息指导探索点的更新. 大量的实验结果表明, SGES AA 在不显著增加搜索耗时的同
时, 预测准确率优于或匹配目前最优的方法. 而且, SGES AA 能够有效支持图像分类、语音分类及文本分类任务
的自动化数据增强.
未来工作中, 将尝试改进自引到进化策略, 使其能够更好地适应于数据增强策略的搜索中, 另外将进一步扩展
自动化数据增强的应用场景, 探索针对图结构数据的自动化数据增强. 同时, 也将尝试将自动化数据增强与对比学
习结合, 通过自动化数据增强提升对比学习性能.

References:
[1] Shorten C, Khoshgoftaar TM. A survey on image data augmentation for deep learning. Journal of Big Data, 2019, 6(1): 60. [doi: 10.1186/
s40537-019-0197-0]
[2] He X, Zhao KY, Chu XW. AutoML: A survey of the state-of-the-art. Knowledge-based Systems, 2021, 212: 106622. [doi: 10.1016/j.
Representations (ICLR). Toulon: OpenReview.net, 2017.
knosys.2020.106622]
[3] Cubuk ED, Zoph B, Mané D, Vasudevan V, Le QV. AutoAugment: Learning augmentation strategies from data. In: Proc. of the 2019
IEEE/CVF Conf. on Computer Vision and Pattern Recognition (CVPR). Piscataway: IEEE, 2019. 113–123. [doi: 10.1109/CVPR.2019.
00020]
[4] Ho D, Liang E, Chen X, Stoica I, Abbeel P. Population based augmentation: Efficient learning of augmentation policy schedules. In:
Proc. of the 36th Int’l Conf. on Machine Learning. Long Beach: PMLR, 2019. 2731–2741.
[5] Lim S, Kim I, Kim T, Kim C, Kim S. Fast AutoAugment. In: Proc. of the 2019 Annual Conf. on Neural Information Processing Systems.
Vancouver, 2019. 6662–6672.
[6] Hataya R, Zdenek J, Yoshizoe K, Nakayama H. Faster AutoAugment: Learning augmentation strategies using backpropagation. In: Proc.
of the 16th European Conf. on Computer Vision. Glasgow: Springer, 2020. 1–16. [doi: 10.1007/978-3-030-58595-2_1]
[7] Li YG, Hu GS, Wang YT, Hospedales T, Robertson NM, Yang YX. Differentiable automatic data augmentation. In: Proc. of the 16th
European Conf. on Computer Vision. Glasgow: Springer, 2020. 580–595. [doi: 10.1007/978-3-030-58542-6_35]
[8] Cubuk ED, Zoph B, Shlens J, Le QV. Randaugment: Practical automated data augmentation with a reduced search space. In: Proc. of the
2020 IEEE/CVF Conf. on Computer Vision and Pattern Recognition Workshops (CVPRW). Seattle: IEEE, 2020. 3008–3017. [doi: 10.
1109/cvprw50498.2020.00359]
[9] Liu FY, Li ZN, Qian C. Self-guided evolution strategies with historical estimated gradients. In: Proc. of the 29th Int’l Joint Conf. on
Artificial Intelligence. Yokohama: IJCAI, 2020. 1474–1480. [doi: 10.24963/ijcai.2020/205]
[10] Moritz P, Nishihara R, Wang S, Tumanov A, Liaw R, Liang E, Elibol M, Yang ZH, Paul W, Jordan MI, Stoica I. Ray: A distributed
framework for emerging AI applications. In: Proc. of the 13th USENIX Symp. on Operating Systems Design and Implementation.
Carlsbad: USENIX Association, 2018. 561–577.
[11] Kaelbling LP, Littman ML, Moore AW. Reinforcement learning: A survey. Journal of Artificial Intelligence Research, 1996, 4: 237–285.
[doi: 10.1613/jair.301]
[12] Krizhevsky A. Learning multiple layers of features from tiny images [MS. Thesis]. Toronto: University of Toronto, 2009.
[13] Jaderberg M, Dalibard V, Osindero S, Czarnecki WM, Donahue J, Razavi A, Vinyals O, Green T, Dunning I, Simonyan K, Fernando C,
Kavukcuoglu K. Population based training of neural networks. arXiv:1711.09846, 2017.
[14] Liu HX, Simonyan K, Yang YM. DARTS: Differentiable architecture search. In: Proc. of the 7th Int’l Conf. on Learning Representations
(ICLR). New Orleans: OpenReview.net, 2019.
[15] Jang E, Gu SX, Poole B. Categorical reparameterization with Gumbel-Softmax. In: Proc. of the 5th Int ’l Conf. on Learning

[16] Nesterov Y, Spokoiny V. Random gradient-free minimization of convex functions. Foundations of Computational Mathematics, 2017,
17(2): 527–566. [doi: 10.1007/s10208-015-9296-2]
[17] Wierstra D, Schaul T, Glasmachers T, Sun Y, Peters J, Schmidhuber J. Natural evolution strategies. The Journal of Machine Learning
Research, 2014, 15(1): 949–980.
[18] Choromanski K, Rowland M, Sindhwani V, Turner RE, Weller A. Structured evolution with compact architectures for scalable policy
optimization. In: Proc. of the 35th Int’l Conf. on Machine Learning. Stockholm: PMLR, 2018. 969–977.