integration: consolidated PDF.js parsing resilience fixes#809
Closed
vitormattos wants to merge 75 commits into
Closed
integration: consolidated PDF.js parsing resilience fixes#809vitormattos wants to merge 75 commits into
vitormattos wants to merge 75 commits into
Conversation
Signed-off-by: Vitor Mattos <1079143+vitormattos@users.noreply.github.com>
Signed-off-by: Vitor Mattos <1079143+vitormattos@users.noreply.github.com>
Signed-off-by: Vitor Mattos <1079143+vitormattos@users.noreply.github.com>
Signed-off-by: Vitor Mattos <1079143+vitormattos@users.noreply.github.com>
Some PDFs include bytes before the %PDF- header while still using absolute xref offsets from the beginning of the file. The parser trimmed data before %PDF-, which shifted offsets and caused xref lookup failures. This manifested as an Invalid object reference error in the veraPDF corpus header case. Changes: - Keep original byte layout in RawDataParser::parseData - Add stricter trailer key matching for /Size /Root /Encrypt /Info /Prev - Add defensive handling in xref stream resolution when startxref is near, but not exactly at, the xref stream object - Add regression fixture and integration test Regression fixture: - samples/bugs/PullRequestInvalidObjectReference.pdf Test: - DocumentIssueFocusTest::testParseFileWithCompressedObjRefInXrefStream Signed-off-by: Vitor Mattos <1079143+vitormattos@users.noreply.github.com>
Signed-off-by: Vitor Mattos <1079143+vitormattos@users.noreply.github.com>
Signed-off-by: Vitor Mattos <1079143+vitormattos@users.noreply.github.com>
Signed-off-by: Vitor Mattos <1079143+vitormattos@users.noreply.github.com>
vitormattos
force-pushed
the
integration/pdfjs-fixes
branch
from
ff2abc5 to
e786ad9
Compare
Some PDFs set startxref to the whitespace immediately before the xref keyword instead of the first letter of xref. The parser required an exact match and incorrectly switched to xref stream decoding, which then failed with Invalid object reference. Changes: - Skip PDF whitespace before checking startxref position - Use adjusted offset when decoding classic xref - Apply same whitespace tolerance for Unix line-ending detection - Tighten trailer key regexes to match /Size /Root /Encrypt /Info /Prev - Add regression fixture and integration test Regression fixture: - samples/bugs/PullRequestXrefWhitespaceStart.pdf Test: - DocumentIssueFocusTest::testParseFileWhenStartxrefPointsToLeadingWhitespace Signed-off-by: Vitor Mattos <1079143+vitormattos@users.noreply.github.com>
Signed-off-by: Vitor Mattos <1079143+vitormattos@users.noreply.github.com>
Signed-off-by: Vitor Mattos <1079143+vitormattos@users.noreply.github.com>
vitormattos
force-pushed
the
integration/pdfjs-fixes
branch
from
e786ad9 to
b1684d5
Compare
vitormattos
force-pushed
the
integration/pdfjs-fixes
branch
from
09d17df to
5779af8
Compare
vitormattos
force-pushed
the
integration/pdfjs-fixes
branch
from
a624421 to
7290bc9
Compare
vitormattos
force-pushed
the
integration/pdfjs-fixes
branch
from
81cd214 to
cc984c1
Compare
… to recover-invalid branch
vitormattos
force-pushed
the
integration/pdfjs-fixes
branch
from
be82edc to
30d369f
Compare
vitormattos
force-pushed
the
integration/pdfjs-fixes
branch
from
30d369f to
3a6f1ac
Compare
# Conflicts: # src/Smalot/PdfParser/Document.php
# Conflicts: # src/Smalot/PdfParser/RawData/RawDataParser.php # tests/PHPUnit/Integration/RawData/RawDataParserTest.php
# Conflicts: # src/Smalot/PdfParser/Parser.php # src/Smalot/PdfParser/RawData/RawDataParser.php # tests/PHPUnit/Integration/RawData/RawDataParserTest.php
vitormattos
force-pushed
the
integration/pdfjs-fixes
branch
from
3a6f1ac to
16f2e3f
Compare
Author
|
Consolidated into #795 to avoid repeated cross-PR conflicts and keep a single review lane. Closing this PR as part of branch unification strategy. |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
PDF.js validation: aggregated fixes
Objective: Validate smalot/pdfparser against the Mozilla PDF.js corpus by comparing parser output with pdfinfo (Poppler reference implementation).
Validation approach: Run parser on each PDF, extract page count. Compare against pdfinfo page count. Files match = ok. Mismatch or parser error = issue to fix.
Results (PDF.js corpus, 930 files / 929 unique hashes):
Success rate: 93.1% → 97.8% (+4.7pp)
Deduplication note: the validation state is keyed by SHA-256, so two byte-identical files (
empty.pdfandempty#hash.pdf) collapse into one effective corpus entry. The directory contains 930 PDFs, but the aggregate counts operate on 929 unique hashes.Note on
pdfinfo_error(+13): this is a classification shift, not a parser regression. These files are cases where parser-side failures were reduced while pdfinfo still reports malformed-input errors (e.g. Poppler syntax or xref failures). The parser now successfully reads more PDFs than pdfinfo can process.Additional validation:
Review options:
Maintainer can choose either path:
Both paths achieve the same end state. Choose based on review capacity and preference.