Skip to content

split_continuous_references: only split actual reference tags - #2447

Open
simpleqt wants to merge 1 commit into
MemTensor:mainfrom
simpleqt:sq/ref-split-shape
Open

simpleqt wants to merge 1 commit into
MemTensor:mainfrom
simpleqt:sq/ref-split-shape

Conversation

@simpleqt

Copy link
Copy Markdown

Fixes #2446

Summary

split_continuous_references in src/memos/mem_os/utils/reference_utils.py converts any single [...] block containing a comma into split tags, even when the content is not a reference list:

>>> split_continuous_references("The set is [apple, banana] here")
'The set is [apple][banana] here'    # plain prose corrupted

Fix

Require every item to match the reference-tag shape int:id before splitting:

The set is [apple, banana] here     -> unchanged
[1:92ff35fb, 4:bfe6f044]            -> [1:92ff35fb][4:bfe6f044]   (unchanged behavior)

This tightens the shape guard added in #2418 (which required a comma-separated list but did not check the item shapes).

Regression test

Added tests/mem_os/test_reference_split_shape.py:

  • red/green verified: the prose cases fail against the old implementation and pass with the fix
  • covers split/no-space reference lists (unchanged behavior) plus plain lists, number lists, and bracketless text
  • ruff check (repo-pinned 0.11.8) passes on both touched files

AI Disclosure

  • Tool(s): Claude (ZCode CLI)
  • Used for: debugging assistance, code suggestions, and drafting this PR description. All changes human-reviewed.

split_continuous_references treated any [a, b] block as a reference
list, corrupting plain prose like 'the set is [apple, banana]'. The
items must now match int:id reference-tag shapes.

Fixes MemTensor#2446
Copilot AI balanced review requested due to automatic review settings September 30, 2026 15:59
@Memtensor-AI Memtensor-AI added area:core MOS 编排层 / 框架底座 / 跨模块问题 status:in-progress Someone or AI is working on it | 人工或 AI 正在处理 labels Sep 30, 2026

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copilot was unable to review this pull request because the user who requested the review has reached their quota limit.

@Memtensor-AI

Copy link
Copy Markdown
Collaborator

🤖 Open Code Review

Target: PR #2447
Task: 5669ea117d4e38f7
Base: main
Head: sq/ref-split-shape

🔍 OpenCodeReview found 1 issue(s) in this PR.


1. src/memos/mem_os/utils/reference_utils.py (L46-L50)

The import re statement is placed mid-function, after the early-return guards. While Python caches module imports so this is not a runtime correctness issue, placing imports inside a function body (especially after executable statements) violates PEP 8 and makes the dependency harder to discover. Since re is already used in the sibling function process_streaming_references_complete via a local import, the cleanest fix is to hoist import re to the top of the file alongside the existing import.

💡 Suggested Change

Before:

    import re

    if not re.fullmatch(
        r"\s*\d+:[^,\s]+(?:,\s*\d+:[^,\s]+)*\s*", content_between_brackets
    ):

After:

# At the top of the file, alongside the existing import:
import re
from memos.memories.textual.item import (
    TextualMemoryItem,
)

# Then inside split_continuous_references, remove the local import re and keep:
    if not re.fullmatch(
        r"\s*\d+:[^,\s]+(?:,\s*\d+:[^,\s]+)*\s*", content_between_brackets
    ):

Generated by cloud-assistant via Open Code Review.

@Memtensor-AI

Copy link
Copy Markdown
Collaborator

✅ Automated Test Results: PASSED

All tests passed (7/7 executed). memos_python_core/changed-repo-python: 7/7. Duration: 8s [advisory, non-gating] AI-generated tests on branch test/auto-gen-5669ea117d4e38f7-20261001000157: 54/54 passed — these do NOT affect the PR verdict; review the branch manually.

Branch: sq/ref-split-shape

@Memtensor-AI Memtensor-AI added status:ready Ready for implementation; waiting for assignee or AI dispatch | 可进入实现,等待认领或派发 and removed status:in-progress Someone or AI is working on it | 人工或 AI 正在处理 labels Sep 30, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

area:core MOS 编排层 / 框架底座 / 跨模块问题 status:ready Ready for implementation; waiting for assignee or AI dispatch | 可进入实现,等待认领或派发

Projects

None yet

Development

Successfully merging this pull request may close these issues.

split_continuous_references corrupts plain bracketed lists (non-reference text)

4 participants