Page 9 - 《软件学报》2026年第5期

1888 软件学报 2026 年第 37 卷第 5 期

Abstract: Symbolic music understanding (SMU) is a crucial task in multimedia content understanding, aiming to extract multi-dimensional
musical attributes such as melody, dynamics, compositional style, emotion, and genre from symbolic representations. Although existing
approaches have substantially advanced dependency modeling in musical sequences, two critical challenges remain: (1) Simplified
representation: Current methods typically flatten complex musical structures into linear symbolic sequences, overlooking the inherent multi-
dimensional hierarchical information; (2) Lack of music-theory integration: Purely data-driven sequence models struggle to incorporate
structured music-theory knowledge, limiting deep semantic understanding of music. To address these issues, this study proposes CNN-
Midiformer, a high-precision symbolic music understanding model that integrates structured representations of musical knowledge. First,
the model constructs structured representations for music theory and musical sequences based on domain knowledge. Second, a
complementary music-feature extraction module is devised to employ convolutional neural networks (CNN) for capturing deep local
features from structured musical-knowledge representations, while a Transformer encoder with self-attention captures deep semantic
features from musical sequences. Finally, a music-knowledge adaptive-enhancement feature-fusion module dynamically integrates the deep
musical-knowledge features extracted by CNN with the deep semantic features of the Transformer via an efficient cross-attention
mechanism, thus enhancing contextual sequence understanding and representation learning. Comparative experiments conducted on six
public symbolic-music datasets (Pop1K7, ASAP, POP909, Pianist8, EMOPIA, and ADL) demonstrate that CNN-Midiformer surpasses state-
of-the-art methods across five benchmark downstream tasks: melody recognition, dynamics prediction, composer classification, emotion
classification, and genre classification, achieving a precision gain of 0.21–7.14 percentage points over baseline models.
Key words: symbolic music understanding (SMU); structured representation of music knowledge; convolutional neural network (CNN);
Transformer; feature fusion

符号音乐理解 (symbolic music understanding, SMU) 是多媒体内容理解的重要任务之一, 不同于直接处理 wav、
mp3 等音频格式的音乐理解任务, SMU 旨在从乐器数字接口 (musical instrument digital interface, MIDI)、
MusicXML 等符号化音乐表示中提取旋律 [1,2] 、力度 [3,4] 、作曲家风格 [5,6] 与情感 [7,8] 等多维音乐属性, 如图 1 所示.
符号化音乐表示一方面能够精确记录音高、时值、力度、音轨和乐器等构成音乐骨架的基础要素, 为深层计算分
析提供了结构化数据基础 [9] ; 另一方面避免了在处理音频时, 自动音乐转录 (automatic music transcription, AMT) 可
能引入的识别误差, 使模型能更专注于音乐内容本身的深层理解. 因此, SMU 在音乐自动作曲与编曲、旋律补全、
智能乐谱分析等领域有着广泛的应用.

旋律识别 力度预测 作曲家分类 情感分类 流派分类

图 1 符号音乐理解任务概览图

基于 Transformer 架构及其变体 (BERT [10] 、GPT [11] 等) 的 SMU 方法通过大规模预训练模型和自注意力机制,
能有效地捕捉符号音乐序列中的长距离依赖关系, 在旋律识别、力度预测、作曲家分类与情感分类等任务上实现
了显著性能提升, 极大地推动了符号音乐理解领域的发展 [12,13] . 然而, 音乐并非简单的符号序列, 还具有复杂的多
维层级信息 [14] . 基于 Transformer 架构的模型在处理序列数据方面表现出色, 但仅以线性序列视角处理音乐, 忽略
了音乐的结构和层级, 导致模型难以有效建模音乐和声、旋律和节奏等基本元素间的关系, 限制了对音乐内在规
律与模式的理解. 因此, 当前方法仍主要存在两方面问题: (1) 表示单一化: 将复杂的音乐结构简化为线性符号序
列, 忽略了音乐固有的多维层级信息; (2) 乐理知识缺乏: 基于序列数据驱动的模型难以融入系统化乐理知识, 限制
了对音乐深层语义的理解.
音乐学研究表明, 许多关键的音乐结构特征可以通过数学建模进行有效量化和表征 [15] . 例如, 音高类别转移