Page 95 - 《软件学报》2026年第3期
P. 95

软件学报 ISSN 1000-9825, CODEN RUXUEW                                        E-mail: jos@iscas.ac.cn
                 2026,37(3):1058−1083 [doi: 10.13328/j.cnki.jos.007513] [CSTR: 32375.14.jos.007513]  http://www.jos.org.cn
                 ©中国科学院软件研究所版权所有.                                                          Tel: +86-10-62562563



                                                                         *
                 LSMDiskANN: 更新友好型磁盘向量索引框架

                 邱海浪  1 ,    彭煜玮  1 ,    彭智勇  1,2


                 1
                  (武汉大学 计算机学院, 湖北 武汉 430072)
                 2
                  (武汉大学 大数据研究院, 湖北 武汉 430072)
                 通信作者: 彭煜玮, E-mail: ywpeng@whu.edu.cn

                 摘 要: 在大模型时代, 向量数据库的广泛应用推动了向量索引规模的急剧膨胀. 如何在磁盘级向量索引中高效支
                 持大规模向量的更新操作, 并同时提供高性能的查询服务, 已成为近年来的重要研究课题. 针对当前领先算法
                 FreshDiskANN 在查询与更新混合负载场景中面临的查询吞吐瓶颈和极端查询延迟过高等问题, 受到日志合并思
                 想在次级索引中成功应用的启发, 提出了一种基于                LSM (log-structured merge) 思想的更新友好型磁盘向量索引框
                 架  LSMDiskANN. 在继承 FreshDiskANN  架构的基础上, 设计并实现了包含磁盘中间层的              3  层架构, 同时引入了磁
                 盘组件搜索参数的动态确定机制以及面向合并操作删除阶段的重布局算法, 从而进一步降低查询延迟和合并过程
                 中的 I/O 开销. 实验结果表明, 在多个经典大规模高维向量数据集上, LSMDiskANN                    系统查询吞吐量最高提升
                 35.5%, 更新吞吐量最高提升      14.24%, 极端查询延迟最多降低       73.45%, 所提出的框架和策略能够有效提升系统在混
                 合负载场景下的整体性能与稳定性.
                 关键词: 向量数据库; 磁盘向量索引; 动态向量索引; 日志合并
                 中图法分类号: TP311

                 中文引用格式: 邱海浪, 彭煜玮, 彭智勇. LSMDiskANN: 更新友好型磁盘向量索引框架. 软件学报, 2026, 37(3): 1058–1083. http://
                 www.jos.org.cn/1000-9825/7513.htm
                 英文引用格式: Qiu HL, Peng YW, Peng ZY. LSMDiskANN: Update-friendly Disk-resident Vector Index Framework. Ruan Jian Xue
                 Bao/Journal of Software, 2026, 37(3): 1058–1083 (in Chinese). http://www.jos.org.cn/1000-9825/7513.htm

                 LSMDiskANN: Update-friendly Disk-resident Vector Index Framework
                           1
                                        1
                 QIU Hai-Lang , PENG Yu-Wei , PENG Zhi-Yong 1,2
                 1
                 (School of Computer Science, Wuhan University, Wuhan 430072, China)
                 2
                 (Big Data Institute, Wuhan University, Wuhan 430072, China)
                 Abstract:  In  the  era  of  large  models,  the  widespread  use  of  vector  databases  has  led  to  a  rapid  expansion  in  the  scale  of  vector  indexes.
                 How  to  efficiently  support  large-scale  vector  updates  in  disk-based  vector  indexes  while  maintaining  high  query  performance  has  become
                 an  important  research  problem  in  recent  years.  FreshDiskANN,  as  a  leading  algorithm,  suffers  from  query  throughput  bottlenecks  and  high
                 tail  latency  under  mixed  query-update  workloads.  Inspired  by  the  successful  application  of  log-structured  merge  (LSM)  in  secondary
                 indexes,  LSMDiskANN  is  proposed  as  an  update-friendly  disk-resident  vector  index  framework  based  on  the  LSM  paradigm.  Building  on
                 the  FreshDiskANN  architecture,  a  three-level  structure  including  a  disk  intermediate  level  is  designed  and  implemented.  In  addition,  a
                 dynamic  parameter  selection  mechanism  for  disk  component  search  and  a  re-layout  strategy  for  the  deletion  phase  of  compaction  are
                 introduced  to  further  reduce  query  latency  and  I/O  overhead  during  merges.  Experimental  results  show  that  on  multiple  large-scale,  high-
                 dimensional  datasets,  query  throughput  is  improved  by  up  to  35.5%,  update  throughput  by  up  to  14.24%,  and  tail  query  latency  is  reduced
                 by up to 73.45%. The proposed framework and strategies effectively enhance overall performance and stability under mixed workloads.


                 *    基金项目: 国家重点研发计划  (2023YFB4503604)
                  本文由“向量数据库及     DB4LLM  技术”专题特约编辑高宏教授、李国良教授、张蓉教授推荐.
                  收稿时间: 2025-05-06; 修改时间: 2025-06-30, 2025-08-14; 采用时间: 2025-08-20; jos 在线出版时间: 2025-09-02
                  CNKI 网络首发时间: 2026-01-15
   90   91   92   93   94   95   96   97   98   99   100