fix: resolve type hint mismatch in deduplicator and broaden embedder exceptions (#259)#279
Open
prasiddhi-105 wants to merge 2 commits into
Open
Conversation
|
🎉 Thank you @prasiddhi-105 for submitting a Pull Request! We're excited to review your contribution. Before Review✅ Ensure all CI checks pass ⚡ Want faster reviews and contributor support? Join our Discord community: 🔗 https://discord.gg/FcXuyw2Rs Maintainers and mentors are active there and can help resolve blockers quickly. Happy Contributing! 🚀 |
|
✅ PR template check passed! @arpit2006 this PR is ready for your review. 🚀 |
|
✅ PR template check passed! @arpit2006 this PR is ready for your review. 🚀 |
arpit2006
requested changes
Jul 6, 2026
arpit2006
left a comment
Collaborator
There was a problem hiding this comment.
@prasiddhi-105 , Resolve CI fails!
Collaborator
|
@prasiddhi-105 , Any Update. |
Author
Yeah on it, I'll give u an update tomorrow morning, please do consider ! |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Linked issue
Closes #259
What this PR does
Resolves a type system mismatch between the main scan workflow engine and the machine learning deduplication / embedding logic. Previously, type hints on the deduplicator signature led to a silent failure where Pydantic
Findingobjects were being processed with dictionary-stylegetattr()defaults, resulting in zero-information empty text embeddings. This PR introduces a robust runtime instance-based type extractor, fixes the functional type hints, adds a type assertion guard inside the scanning controller loop, and broadens initialization exception wrappers forSentenceTransformerto protect the server boot process from third-party runtime failures (e.g., PyTorch out-of-memory or HuggingFace timeouts).Type of change
ML tier (if applicable)
Stack affected
Changes
Backend
backend/app/ml/deduplicator.py: Upgraded data type signatures and annotations to explicitly accept structural collections ofUnion[Finding, Dict[str, Any]].backend/app/ml/embedder.py: Introduced a localized polymorph conditional parser_extract_textto handle extraction paths natively using string parsing via.get()for raw dictionaries orgetattr()for model properties.backend/app/ml/embedder.py: Widen the initialization block check statement fromexcept ImportErrortoexcept Exceptionto capture connection drops or memory faults gracefully.backend/app/main.py: Deployed a clear static type check assertion boundary condition wrapper on the engine array loop directly before calculating cosine similarity.Frontend
New dependencies
Database / schema changes
Testing
How did you test this?
Checklist
console.erroror unhandled Python exceptions introducedrequirements.txt/package.jsonupdated if new dependencies added.pkl,.pt, etc.) are gitignored, not committedAnything reviewers should focus on
Please check the structural implementation of the new private helper
_extract_textwithinembedder.pyto ensure it seamlessly toggles between dictionary-like and object-like item evaluations without degrading execution processing loops during full scan passes.