Skip to content

fix: restore truncated identifiers and missing imports (NameError in process_document, prepare_dataset, plugin discovery) - #101

Open
Anai-Guo wants to merge 1 commit into
multimindlab:developfrom
Anai-Guo:fix/truncated-identifiers
Open

Anai-Guo wants to merge 1 commit into
multimindlab:developfrom
Anai-Guo:fix/truncated-identifiers

Conversation

@Anai-Guo

Copy link
Copy Markdown

What does this PR do?

Fixes a handful of identifiers that lost their last character, plus two missing stdlib imports. Each one makes the affected code path fail at runtime.

File Problem Effect
multimind/document_processing/document.py text = documen / text = document.tex DocumentProcessor.process_document always raises (NameError for str input, AttributeError for Document input), so process_file / process_text / process_url fail too
multimind/fine_tuning/peft_methods.py return tokenized_datase prepare_dataset raises NameError
multimind/fine_tuning/unified_peft.py (2脳) return tokenized_datase same
multimind/vector_store/vector_store_enhanced.py importlib.util used, never imported the NameError is caught by the except, so every plugin in discover_plugins gets logged as a failed load
multimind/document_loader/document_loader.py io.BytesIO used, never imported loading a PDF from a URL raises NameError

All of these were flagged by pyflakes as undefined name. The repo's ruff config doesn't catch them because it doesn't enable F821.

Before / after

p = DocumentProcessor()
p.process_document("hello world. second sentence.")
p.process_document(Document(text="obj text here.", metadata={"k": 1}))

Before:

ERR NameError name 'documen' is not defined
ERR AttributeError 'Document' object has no attribute 'tex'

After:

OK [('hello world. second sentence.', {'chunk_index': 0, 'total_chunks': 1})]
OK [('obj text here.', {'k': 1, 'chunk_index': 0, 'total_chunks': 1})]

Checklist

  • No breaking changes (7 lines added, 5 removed, no behaviour change beyond the fixes above)
  • ruff check on the changed files: no new findings

馃 Generated with Claude Code

Several identifiers lost their final character, and two modules use a
stdlib module they never import:

- DocumentProcessor.process_document: `documen` / `document.tex` made
  every call fail (NameError for str input, AttributeError for Document
  input), which also broke process_file/process_url/process_text.
- prepare_dataset in peft_methods.py and unified_peft.py (x2) returned
  the undefined `tokenized_datase`.
- vector_store_enhanced.py used `importlib.util` without importing it,
  so every plugin in discover_plugins was logged as a failed load.
- document_loader.py used `io.BytesIO` for PDFs loaded from URLs without
  importing `io`.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant