fix(scheduler): stop retaining retrieval payloads across mem_update b… - #2402
larryluozhang wants to merge 1 commit into
Conversation
…atches Added garbage collection to free memory and prevent leaks in long-lived scheduler threads. Removed unnecessary embeddings from search results to optimize memory usage.
🤖 Open Code ReviewTarget: PR #2402 🔍 OpenCodeReview found 3 issue(s) in this PR. 1.
|
✅ Automated Test Results: PASSEDAll tests passed (2/2 executed). memos_python_core/changed-python-source: 2/2. Duration: 8s [advisory, non-gating] AI-generated tests on branch test/auto-gen-73f8fc2583d8d6f5-20260922090738: 47/48 passed, 1 failed — these do NOT affect the PR verdict; review the branch manually. Branch: |
Problem
Each
mem_updatebatch retrieves candidate memories carrying full embeddings(~33KB/item as Python float lists) plus long source payloads. These result
lists stay referenced by the long-lived scheduler thread frames, so they
accumulate indefinitely.
Production evidence (tracemalloc + objgraph, RSS 4.98GB snapshot):
~3.4GB retained across ~900 batches → RSS grows ~660MB/h → watchdog restarts
(10-90/day in our deployment).
Fix
metadata.embeddingfrom candidates immediately after retrieval —vectors are retrieval artifacts; the working-memory replacement decision
only uses text.
delbatch references +gc.collect()afterreplace_working_memory,because scheduler threads are long-lived and locals otherwise linger.
No behavior change to outputs; purely memory hygiene.