Page 426 - 《软件学报》2026年第5期

唐昊 等: 基于视觉 Transformer 的双视图融合细粒度图像识别 2305

有挑战性背景的训练样本, 以提高模型的适应性和鲁棒性; (3) 域适应技术: 利用域适应方法来缩小训练数据与真
实应用场景之间的差异, 尤其是针对背景复杂度和样本分布的不同, 以增强模型对复杂环境的适应能力.
4.7.2 模型的可解释性
尽管本文提出的双视图细粒度识别框架在多个细粒度图像识别任务上表现出色, 但其决策过程的透明度与可
解释性仍存在一定局限. 当前模型基于深度视觉 Transformer 架构, 该架构能够通过分析大量图像数据捕捉复杂的
视觉特征. 然而, Transformer 内部的决策机制相对封闭, 使得模型的决策原理难以被直观理解. 图 12(c) 展示了模
型在识别过程中的注意力分布, 尽管背景复杂且目标物种间的差异微小, 模型依然能够准确识别并聚焦于关键特
征. 虽然能够观察到模型在处理图像时对特定区域的聚焦, 但这些区域的选择以及它们在模型分类决策中的具体
语义贡献尚缺乏明确的解释. 为了克服这一局限性, 未来的研究需致力于提升模型的可解释性, 进一步揭示模型内
部的决策逻辑. 一个可行的策略是引入先进的可解释性框架或算法, 如集成梯度或相关传播算法. 这些方法能够揭
示 Transformer 层间信息流动的具体模式及其对最终决策的影响, 从而使模型决策过程更加透明. 通过引入这些可
解释性方法, 可以定量地评估图像各区域在决策中的重要性, 并详细解释这些区域如何影响整体的识别结果.

5 总 结

本文提出了一种基于视觉 Transformer (ViT) 的细粒度图像识别框架, 该框架采用了双视图识别范式. 通过有
效整合全局视图和局部视图, 显著地克服了 ViT 在细粒度图像识别任务中的局限性, 尤其是在处理细节特征和背
景噪声方面. 在 4 个标准的细粒度图像识别数据集上进行的广泛实验结果证明了该双视图识别框架的有效性, 并
展现了其在解决细粒度图像识别领域实际挑战中的应用潜力, 为未来相关研究领域提供了新的视角和方法.

References
[1] Zhang S, Gong YH, Wang JJ. The development of deep convolution neural network and its applications on computer vision. Chinese
Journal of Computers, 2019, 42(3): 453–482 (in Chinese with English abstract). [doi: 10.11897/SP.J.1016.2019.00453]
[2] Luo JH, Wu JX. A survey on fine-grained image categorization using deep convolutional features. Acta Automatica Sinica, 2017, 43(8):
1306–1318 (in Chinese with English abstract). [doi: 10.16383/j.aas.2017.c160425]
[3] Wei XS, Song YZ, Aodha OM, Wu JX, Peng YX, Tang JH, Yang J, Belongie S. Fine-grained image analysis with deep learning: A
survey. IEEE Trans. on Pattern Analysis and Machine Intelligence, 2022, 44(12): 8927–8948. [doi: 10.1109/TPAMI.2021.3126648]
[4] Choudhury S, Laina I, Rupprecht C, Vedaldi A. The curious layperson: Fine-grained image recognition without expert labels. Int’l
Journal of Computer Vision, 2024, 132(2): 537–554. [doi: 10.1007/s11263-023-01885-9]
[5] Qi L, Yu PZ, Gao Y. Research on weak-supervised person re-identification. Ruan Jian Xue Bao/Journal of Software, 2020, 31(9):
2883–2902 (in Chinese with English abstract). http://www.jos.org.cn/1000-9825/6083.htm [doi: 10.13328/j.cnki.jos.006083]
[6] Wei XS, Cui Q, Yang L, Wang P, Liu LQ, Yang J. RPC: A large-scale and fine-grained retail product checkout dataset. Science China
Information Sciences, 2022, 65(9): 197101. [doi: 10.1007/s11432-022-3513-y]
[7] Dou H, Zhang LM, Han F, Shen FR, Zhao J. Survey on convolutional neural network interpretability. Ruan Jian Xue Bao/Journal of
Software, 2024, 35(1): 159–184 (in Chinese with English abstract). http://www.jos.org.cn/1000-9825/6758.htm [doi: 10.13328/j.cnki.jos.
006758]
[8] Tan M, Yuan F, Yu J, Wang GJ, Gu XL. Fine-grained image classification via multi-scale selective hierarchical biquadratic pooling.
ACM Trans. on Multimedia Computing, Communications, and Applications, 2022, 18(1s): 31. [doi: 10.1145/3492221]
[9] Chang DL, Ding YF, Xie JY, Bhunia AK, Li XX, Ma ZY, Wu M, Guo J, Song YZ. The devil is in the channels: Mutual-channel loss for
fine-grained image classification. IEEE Trans. on Image Processing, 2020, 29: 4683–4695. [doi: 10.1109/TIP.2020.2973812]
[10] He XT, Peng YX. Unsupervised fine-grained video categorization via adaptation learning across domains and modalities. Ruan Jian Xue
Bao/Journal of Software, 2021, 32(11): 3482–3495 (in Chinese with English abstract). http://www.jos.org.cn/1000-9825/6058.htm [doi:
10.13328/j.cnki.jos.006058]
[11] Dosovitskiy A, Beyer L, Kolesnikov A, Weissenborn D, Zhai XH, Unterthiner T, Dehghani M, Minderer M, Heigold G, Gelly S,
Uszkoreit J, Houlsby N. An image is worth 16x16 words: Transformers for image recognition at scale. In: Proc. of the 9th Int’l Conf. on
Learning Representations. OpenReview.net, 2021. 1–21.
[12] Vaswani A, Shazeer N, Parmar N, Uszkoreit J, Jones L, Gomez AN, Kaiser Ł, Polosukhin I. Attention is all you need. In: Proc. of the
31st Int’l Conf. on Neural Information Processing Systems. Long Beach: Curran Associates Inc., 2017. 6000–6010.