Skip to content

Content-hash media on upload (dedup + find-identical-by-content) #69

Description

@parisek

What

Compute and store a content hash (sha256) of every uploaded attachment, so media can be identified by its bytes — not just by filename. This enables (a) not storing bit-for-bit identical duplicates, and (b) finding an existing identical image regardless of the name it was saved under.

Why WordPress core can't do it for us (verified on WP 7.0)

Core has no native content hashing for media:

  • No wp_hash_file / wp_get_attachment_hash / wp_generate_attachment_hash functions.
  • wp_get_attachment_metadata() returns only sizes, filesize — no hash.
  • Attachment postmeta is _wp_attached_file + _wp_attachment_metadata — no hash/md5/sha meta.
  • Core's md5_file/sha1_file usage is for update-package verification (wordpress.org checksums) and the getID3 media-analysis library — not a stored, queryable content hash.

Core's only native media dedup is by filenamewp_unique_filename() renames a colliding upload to name-1.ext, so a different file never overwrites an existing one, but an identical file uploaded under a different name still creates a duplicate attachment. There is no content-level dedup.

Why this belongs in timber-kit (not in a single consumer)

timber-kit already owns the fleet's media layer (Resizer, dev media proxy, upload/filename handling), so it's the natural place to hook uploads and compute the hash once, for every attachment, regardless of who created it (WP admin, an importer, a plugin). Any consumer can then read the hash. The first consumer is portadesign/portadesign-mcp (the MCP/AI-agent connector) — its upload-media ability wants to dedup by content and answer "is this image already here?" — but reimplementing hashing per-plugin would be redundant and would only cover that plugin's own uploads.

Proposed shape (sketch)

  • On add_attachment (or the sideload/upload hook), sha256_file() the attached file and store it in a namespaced post meta (e.g. _timberkit_sha256), indexed for lookup.
  • A backfill for the existing library (WP-CLI command, chunked) — because a stored-ahead hash is the only cheap way to look media up by content; hashing the whole library on-demand per query is O(N × filesize) and doesn't scale.
  • A small lookup helper/API: findAttachmentByHash(string $sha256): ?int (fast indexed meta query), and optionally expose the hash where media is formatted.
  • Byte size as a cheap pre-filter, sha256 as the definitive bit-for-bit test.
  • Opt-in via a StarterBase flag, consistent with timber-kit's config style.

Consumers / follow-up

  • portadesign-mcp upload-media: on upload, look up by hash → reuse the existing attachment instead of creating a duplicate (deduplicated: true); expose the hash in its media output; a future find-media by content.
  • General "do we already have this image?" for editors/importers.

Deferred out of portadesign-mcp deliberately (see its media-abilities design), to be built here and consumed there once available.

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions