Page 72 - 《软件学报》2026年第5期

方承炀 等: 面向免训练视频问答的双重自适应冗余消除 1951

and local body parts, as non-target regions, introduce spatial feature bias, significantly affecting the reliability of answer generation. To
address these two issues, this study proposes a dual adaptive redundancy elimination (DARE) framework, which aims to systematically
improve the accuracy of video semantic understanding and answer quality in the training-free paradigm through a spatiotemporal
redundancy collaborative optimization mechanism. First, a dual-relation temporal sampling method is proposed, based on text-visual
alignment and inter-frame semantic consistency. This method selects key frame sequences through bidirectional interactive reasoning, while
simultaneously eliminating redundant frames that conflict with the text context. Next, a dynamic spatial sampling method is introduced,
which extracts the largest connected semantic region from candidate regions in the prompt-related heatmap, aiming to eliminate scattered
non-target regions and enhance the compactness of spatial feature representations. Experiments are conducted on widely used datasets,
including MSVD-QA, MSRVTT-QA, TGIF-QA, and ActivityNet-QA. The proposed method is evaluated in a zero-shot setting against 14
state-of-the-art models. The results show that the proposed approach achieves competitive performance with significantly fewer video
feature sequences. Visual analysis confirms that the proposed method exhibits more accurate spatiotemporal localization abilities in
challenging tasks, such as multi-person interactions and fine-grained action recognition in complex scenes. The proposed DARE-VQA
framework achieves significant improvements in video question answering performance by collaboratively optimizing spatiotemporal
redundancy. It can generate accurate and high-quality answers within the training-free paradigm, demonstrating its potential in multimodal
video understanding.
Key words: training-free; video question answering; dual-relation temporal sampling; dynamic spatial sampling; zero-shot

视频问答技术 [1–3] 旨在赋予机器对多模态视频内容的语义理解与推理能力, 使其能够回答与视频内容相关的
问题. 这一任务已成为衡量多模态人工智能系统认知水平的核心标准之一. 尽管针对图像的视觉问答技术已经相
对成熟, 但针对视频的问答技术仍面临诸多独特挑战, 主要在于该技术需要同时建模视频中多帧图像的时间动态
[5]
变化与空间语义关联. 当前主流方法通常依赖于对大规模预训练视觉-语言模型 [4] (如 CLIP 模型) 的端到端微调
策略来应对上述问题, 虽然这些方法在特定基准测试上取得了一定进展, 但是这种计算密集型的优化范式存在两
大瓶颈: 1) 处理长视频序列需要高昂的训练成本 (需要在海量视频帧上优化复杂的时序关联模块), 严重限制了模
型的可扩展性; 2) 视频内容分布的高度异构性 (如不同的视频在运动节奏、场景切换等方面高度多样), 显著削弱
了模型的跨领域泛化能力.
近年来, 免训练的视频理解模型因其即插即用的特性成为新兴的研究方向. 该范式通过冻结预训练的视觉-语
言模型, 并设计轻量化适配策略, 将原始视频帧序列转换为与模型兼容的输入表示, 从而避免了高资源开销的训练
[7]
过程. 例如 IG-VLM 和 [6] FreeVA 等模型展示了将视频帧转换为符合高性能视觉-语言模型兼容输入格式并应用大
模型的潜力. 这些方法通过重构视频帧表示并结合轻量的时间推理模块, 成功减少了传统视频问答模型的计算开
销, 同时保持了与之相当的语义理解能力. 然而, 现有的免训练视频问答方法仍面临两个关键挑战.
挑战 1: 时序对齐失准 (temporal misalignment). 高帧率视频的冗余使得时间采样策略易受噪声干扰. 传统时间
采样方法, 如均匀采样或双流特征提取 (如 SF-LLaVA ), 无法有效处理视频段落中非均匀的语义分布. 近期如
[8]
[9]
Free Video-LLM 虽引入了基于提示引导的帧筛选方法, 但其未能建立文本语义与视频帧间的关系协同推理机制,
尽管提升了一定的帧选择相关性, 但其缺乏对文本语义与视频帧之间关系的协同建模与联合推理, 难以准确捕捉
问题所对应的关键帧. 以图 1 中的场景 1 为例, 面对问题“What is a family having?”, Free Video-LLM 能够筛选出 3
张与问题较相关的帧图, 但其中第 3 张帧明显与语义无关, 表现出帧选择偏差. 这种偏差正是由于缺乏对问题上下
文与视频语义之间一致性的建模, 导致模型可能选取非关键帧或遗漏关键动作片段, 从而影响最终的问答准确性.
挑战 2: 空间噪声干扰 (spatial noise interference). 在复杂视频场景中, 背景杂物和局部身体部位等无关区域可
能污染视觉特征表示. 现有方法通常采用全局池化的方式应对以上问题, 但是这会导致严重的信息损失, 或使用固
定尺寸的提示引导裁剪视频帧区域 (如 Free Video-LLM), 难以动态聚焦于与任务相关的语义区域. 这种缺陷在需
要精确空间定位的问题中尤为突出. 如图 1 中的场景 2 所示, 面对问题“How many higher officials meet in a room?”,
Free Video-LLM 根据问题与图像标记的相关性选取区域中心, 并在其周围构建固定长宽的矩形框. 然而, 该策略
将低相关甚至无关的图像区域一并选入, 另外问题相关的区域形状固定尺寸也并不合理, 导致模型生成错误答案.
这表明缺乏精细空间感知机制会使模型在处理目标计数、关系推理等任务时易受干扰区域误导.
本文提出了双重自适应冗余消除框架来应对上述挑战: 1) 针对时序对齐失准问题, 框架设计双重关联采样策