Skip to content

Prefetch research: shared prefetch() helper API in plone.base #4352

Description

@jensens

Part of the research plan for #4350 (see anchor comment there for the full decomposition).

All call sites identified in the sibling sub-issues should go through one shared helper, so semantics, chunking, and cross-DB handling are implemented once. Proposed home: plone.base (it already depends on the relevant stack; five.intid is only needed lazily for relation input).

Sketch

def prefetch(items, context=None) -> int:
    """Batch-hint the storage about soon-needed objects.

    Safe no-op without storage support. Returns the number of oids hinted.
    """

Accepted inputs (mixed iterable, dispatched per item):

  • RelationValue — wake-free oid via intids.refs[to_id].oid; five.intid's KeyReferenceToPersistent stores the oid as a plain attribute (five/intid/keyreference.py:103, five.intid 3.1.0). Fallback for plain zope.keyreference keyrefs: keyref.object._p_oid — also wake-free, since _p_oid access does not activate a ghost.
  • persistent objects / ghosts — via _p_oid
  • raw oids (bytes)
  • intid tokens (int) — resolved through intids.refs

Design points to settle

  • Where the jar comes from: context._p_jar, falling back to the site's jar. Document that prefetch is connection-bound (MVCC tid, see ZODB/mvccadapter.py:192).
  • Cross-DB mounts: five.intid keyrefs carry dbname (five/intid/keyreference.py:101-107); skip oids whose dbname differs from the target connection's.
  • Chunking: call jar.prefetch in windows (issue Research: use ZODB Connection.prefetch to batch object loads (relations, contentlisting, catalog results) — eliminate common N+1 patterns #4350's helper uses 500; plone-pgcatalog uses a window of 100 — pick and document a default).
  • Catalog brains deliberately not accepted in v1 — standard ZCatalog brains carry no oid (see the result-set sub-issue).
  • Re-export as plone.api.prefetch? (discoverability vs. keeping plone.api slim)
  • Error handling: helper must never raise on odd input in production paths — it is a hint.

Deliverable: implementation + tests in plone.base, used by the relations and folder sub-issues.

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Fields

    No fields configured for issues without a type.

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions