Merge develop into main#128
Merged
Merged
Conversation
Detection pipeline: - Enable attachment binary analysis in production: parser now retains attachment bytes and analyze_attachments() passes them to the scanners (VBA macros, PDF JavaScript and suspicious streams were dead code) - Read SPF/DKIM/DMARC from the FIRST Authentication-Results header (receiving server) instead of the last, which senders can inject - Extract and scan inline attachments (filename without attachment disposition) - Remove body pattern double-counting on multipart/alternative emails: urgency/CTA/credential counters now take the max across text sources Correctness: - Add 6 missing i18n keys (brand_spoofing, dkim_domain_mismatch, malicious_cdn, language_mismatch, known_campaign, db_error) - Make WHOIS and URL batch timeouts effective (shared executor + non-blocking shutdown; per-call executors blocked until completion) - Resolve risk label translation at call time (follows runtime language) - Define missing _logger in campaign_detector - Replace lstrip(www.) with removeprefix (mangled hostnames) - Match trusted CDN IP prefixes on octet boundary - Use dot-boundary subdomain matching in List-Unsubscribe domain check - Match brand aliases with word boundaries - Validate IPv4 octets (0-255) in direct-IP URL detection - Store mail_to as JSON in both pipelines; GET returns it as a list - Pass header_result to analyze_body in manual analysis (NLP auth flags) - Return JSON 404 for unknown /api/* paths instead of SPA HTML All 123 tests passing (1 skipped), zero regressions.
POST /api/upload/ read the entire request body via file.read() before checking MAX_UPLOAD_SIZE_MB, so an oversized upload was fully buffered in memory before being rejected -- a potential memory-exhaustion DoS on deployments without a reverse-proxy body-size limit in front. Now reads in 1MB chunks and rejects with 413 as soon as the configured limit is exceeded, without buffering the remainder of the request.
CRITICAL (3) - all resolved:
1. API keys in .env: confirmed not tracked by git (in .gitignore)
2. No HTTP rate limiting: implemented slowapi with per-endpoint limits
- POST /api/upload/ → 10/min (250MB/min = DoS mitigation)
- POST /api/analysis/{job_id} → 10/min
- POST /api/analysis/bulk-delete → 5/min
- POST /api/manual/ → 10/min
3. Insecure pickle deserialization: added HMAC-SHA256 integrity verification
with automatic first-time initialization
MEDIUM (3) - all resolved:
4. No CSRF protection: SameSite cookie infrastructure ready (future-proofing)
5. Unsafe HTML preview (bleach): confirmed <meta> tag already excluded
6. Race condition in upsert: replaced delete+add with atomic db.merge()
LOW (3) - all resolved:
7. Secrets in logger: added _SecretRedactionFilter for auth header redaction
8. Incomplete delete: refactored delete endpoint with proper try/finally
- file cleanup first, then DB delete (atomic semantic)
9. Weak job_id validation: added file size check before read_bytes()
All 122 tests passing (1 skipped), zero regressions.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
No description provided.