Skip to content

fix(memory): prevent duplicate records across pagination - #87

Open
openjiuwen-sync-bot[bot] wants to merge 3 commits into
openJiuwen-ai:developfrom
openjiuwenai:sync/pr-181
Open

fix(memory): prevent duplicate records across pagination#87
openjiuwen-sync-bot[bot] wants to merge 3 commits into
openJiuwen-ai:developfrom
openjiuwenai:sync/pr-181

Conversation

@openjiuwen-sync-bot

@openjiuwen-sync-bot openjiuwen-sync-bot Bot commented Aug 5, 2026

Copy link
Copy Markdown

Paired: GitHub #87GitCode !181

What type of PR is this?

/kind

Self-checklist:(请自检,在[ ]内打上x,我们将检视你的完成情况,否则会导致pr无法合入

    • 设计:PR对应的方案是否已经经过Maintainer评审,方案检视意见是否均已答复并完成方案修改
    • 测试:PR中的代码是否已有UT/ST测试用例进行充分的覆盖,新增测试用例是否随本PR一并上库或已经上库
    • 验证:PR描述信息中是否已包含对该PR对应的Feature、Refactor、Bugfix的预期目标达成情况的详细验证结果描述
    • 接口:是否涉及对外接口变更,相应变更已得到接口评审组织的通过,API对应的注释信息已经刷新正确
    • 文档:是否涉及官网文档修改,如果涉及请及时提交资料到Doc仓

@CLAassistant

Copy link
Copy Markdown

CLA assistant check
Thank you for your submission! We really appreciate it. Like many open source projects, we ask that you sign our Contributor License Agreement before we can accept your contribution.


LZF seems not to be a GitHub user. You need a GitHub account to be able to sign the CLA. If you have already a GitHub account, please add the email address used for this commit to your account.
You have signed the CLA already but the status is still pending? Let us recheck it.

@openjiuwen-collaboration-bot

openjiuwen-collaboration-bot Bot commented Aug 5, 2026

Copy link
Copy Markdown

head_sha: e8eec1cd2529c77f094b4e76f2883381435fa513

变更摘要

此 PR 修复了记忆分页查询中跨页出现重复记录的问题。根本原因在于原有的分页逻辑每次仅获取最新 N 条记录并将其反转排序,导致不同页的前缀集合重叠,已返回的记录在后续页面中再次出现。修复方案改为先获取完整的确定性快照(全量记忆),再对该快照统一排序和分页,同时在排序键中引入 mem_id 作为次级排序条件,确保相同时间戳的记录具有确定的先后顺序。

主要改动

  • SimpleMemoryIndex 排序键增加确定性:在 _list_memories_via_index_list_memories_via_kv_scan 两个方法中,排序键从 (type_order, -timestamp) 扩展为 (type_order, -timestamp, d.id),以 d.id 作为第三排序键,消除相同时间戳下排序结果的不确定性。
  • LongTermMemory.get_user_mem_by_page 分页策略重构:不再每次仅获取 fetch_size 条记录后排序,改为调用新增的 _list_all_indexed_memories_list_all_middle_memories 获取完整快照,统一排序后再进行切片分页,从根本上消除跨页重复。
  • 新增 _list_all_indexed_memories 方法:通过循环分页请求获取所有索引记忆,使用 dictmem_id 去重,返回完整快照。
  • 新增 _list_all_middle_memories 方法:通过倍增 limit 的方式逐步拉取所有中期记忆,直到返回数量不足 limit 时终止,同样以 mem_id 去重后返回完整快照。
  • 排序键函数 _mem_info_chronological_sort_key 升级:返回值从 float 改为 tuple[float, str],在时间戳之后增加 mem_id 作为次级排序键,确保排序结果完全确定性。

@openjiuwen-collaboration-bot

openjiuwen-collaboration-bot Bot commented Aug 5, 2026

Copy link
Copy Markdown

head_sha: e8eec1cd2529c77f094b4e76f2883381435fa513

代码审查

✅ 未发现问题

@openjiuwen-collaboration-bot

Copy link
Copy Markdown

head_sha: e8eec1cd2529c77f094b4e76f2883381435fa513

任务名称 结果 日志操作
静态检查 ✅SUCCESS 点此跳转
防投毒检查 ✅SUCCESS 点此跳转
开源合规检查 ✅SUCCESS 点此跳转
UT测试 ❌FAILED 点此跳转
ST测试 N/A N/A
build 编译包 N/A N/A
ruff codecheck ✅SUCCESS N/A

@openjiuwen-collaboration-bot

Copy link
Copy Markdown

head_sha: d3fc9e5a73b0e804ab6a2c127b7f63eea602532c

任务名称 结果 日志操作
静态检查 ✅SUCCESS 点此跳转
防投毒检查 ✅SUCCESS 点此跳转
开源合规检查 ✅SUCCESS 点此跳转
UT测试 ✅SUCCESS 点此跳转
ST测试 N/A N/A
build 编译包 N/A N/A
ruff codecheck ✅SUCCESS N/A

@openjiuwen-collaboration-bot

Copy link
Copy Markdown

head_sha: d3fc9e5a73b0e804ab6a2c127b7f63eea602532c

在分页修复版本上重新执行原始 test_simple_memory_search_019,测试通过,分页重复问题未再出现。
分页结果:
page=1; count=3; overlap_with_previous=0
page=2; count=3; overlap_with_previous=0
page=3; count=3; overlap_with_previous=0
page=4; count=1; overlap_with_previous=0
page=5; count=0
汇总结果:非空页 4 页,累计返回 10 条记录,唯一 mem_id 数量 10,页间重复 0,超出总页数返回空列表。
测试日志:
838f5460cde840bfb43a5b1386fb0874.log

openjiuwenai pushed a commit to openjiuwenai/bot0-branches-agent-memory that referenced this pull request Aug 6, 2026
…lzf2025/agent-memory_5353.git into 'develop'

# Conflicts:
#   conflict pyproject.toml
@openjiuwen-collaboration-bot

Copy link
Copy Markdown

head_sha: 128472f3bd9395be0c693fc27a50323c27434fed

任务名称 结果 日志操作
静态检查 ✅SUCCESS 点此跳转
防投毒检查 ✅SUCCESS 点此跳转
开源合规检查 ✅SUCCESS 点此跳转
UT测试 ✅SUCCESS 点此跳转
ST测试 N/A N/A
build 编译包 N/A N/A
ruff codecheck ✅SUCCESS N/A

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant