Skip to content

Phase 6 follow-up: GROBID processFulltext fallback for TOC-less book PDFs #14

Description

@deOliveira-R

`extract.book_structure.detect` splits books using the PDF's embedded TOC (`pymupdf.get_toc()`) or the EPUB spine. When a PDF has neither — scanned or badly-produced files — the book stays as a single item and the "chapters as first-class items" feature (plan §6 Phase 6a) silently skips it.

Plan hint at the time:

PDF second (pymupdf bookmark tree; fall back to GROBID's `processFullText` when bookmarks are missing)

Proposal

  1. When `detect()` returns `None` for a PDF, optionally call GROBID's `/api/processFulltextDocument` (already wrapped via `extract.grobid` — only the header endpoint is used today).
  2. Parse the TEI response for `<div type="chapter">` or top-level `` structures.
  3. Convert to `ChapterSpec` entries.
  4. Opt-in via a config flag (`GRIMOIRE_GROBID_FULLTEXT_FALLBACK=true`) — the endpoint is 3–10 s per PDF and not always reliable.

Acceptance

  • `detect()` honors the opt-in config flag and falls back to GROBID only when TOC + spine detection both returned `None`.
  • Oracle: add a TOC-less book PDF fixture (or synthetic) and assert chapters are created when the flag is on.
  • Failure modes (GROBID down, malformed TEI, no `<div type="chapter">` structure) degrade gracefully — book stays as a single item, no partial chapters.

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions