feat: v0.6.1 — pull-wordpress adapter#13
Merged
Conversation
These two modules were named with a `sanity-` prefix in v0.6.0 because the only caller at the time was `pull-sanity`. Their logic is fully source-agnostic — `inferSchemas` walks Document samples by content type and recognizes both Sanity-shaped (`_type: 'reference'`/`'image'`) and Contentful Link-sys values, and `detectLocales` / `expandDocumentsByLocale` only inspect field-value shape. Promoting them to `src/shared/` (sibling of `src/sanity/`) unblocks `pull-wordpress` reusing them without an awkward import path like `../sanity/sanity-schema-inference.js` from WordPress code. Renames (no behavior change): src/sanity/sanity-schema-inference.ts → src/shared/schema-inference.ts src/sanity/sanity-locale-expander.ts → src/shared/locale-expander.ts + their .test.ts companions Import path updates in `pull-sanity.ts` and `index.ts`. The public type alias `SanityInferenceWarning` (etc.) is removed in favor of the unprefixed names — the `inferSchemas` export is now top-level. 601 tests pass (same count as v0.6.0 — no regressions). 5 examples (`example-from-scratch`, `-sanity-migration`, `-wp-migration`, `-pull-contentful`, `-sanity-pull`) all produce identical counters.
…leton)
Adds `cms-sim pull-wordpress` and the public `pullWordpress()` API.
Reads a WordPress WXR XML export (offline, from wp-admin's
Tools → Export → All content) and writes the same on-disk shape as
`cms-sim pull` for Contentful spaces, so the rest of the pipeline
(simulate / to-import / diff) consumes the result unchanged.
Output mirrors pull-contentful + pull-sanity exactly:
schemas/<id>.js one .js per surviving WP content type
data/entries.ndjson one JSON per line
contentful-space.json + WP site metadata (title / url / language)
Reuses the existing `parseWXR` (already shipped in v0.5.0) plus the
shared `inferSchemas` / `detectLocales` / `expandDocumentsByLocale`
modules promoted to `src/shared/` in the previous commit.
Stable id prefixes per content type so refs (next commit) have
predictable targets:
- post / page → wp_<postId> e.g. wp_42
- author → wp_author_<login> e.g. wp_author_jdoe
- category → wp_category_<slug> e.g. wp_category_tech
- tag → wp_tag_<slug> e.g. wp_tag_javascript
Default skip set drops content WordPress emits but Contentful has no
equivalent for: nav_menu_item, wp_navigation, wp_template,
wp_template_part, wp_global_styles, wp_block, oembed_cache,
customize_changeset, custom_css, revision, and (for now) attachment.
Attachments will be reshaped into assets/assets.json in commit 4 —
`--include-attachments` opts back into emitting them as entries until
then.
Status: skeleton + output contract. The following land in follow-up
commits:
- ref rewriting: post.categories/tags (string slugs) → Array<Link Entry>,
post.author (login string) → Link Entry + linkContentType validations
- asset linking: attachment posts → assets.json + featured_image / inline
images → Link Asset
- Gutenberg / Classic HTML → Contentful RichText (uses existing
htmlToRichText from transform/rich-text.ts)
- Polylang locale detection + fan-out
+10 unit tests (611 total). Pre-commit smoke against the five
`file:`-linked examples regression-free (5/25, 6/20, 5/18, 23/69, 7/21).
npm audit: 0 vulnerabilities.
`mapWordpressRefs` walks every WordPress document and rewrites cross-document references into the Contentful Entry-link sys shape: post.author (login) → Link Entry wp_author_<login> post.categories[] (slugs) → Array<Link Entry> wp_category_<slug> post.tags[] (slugs) → Array<Link Entry> wp_tag_<slug> category.parent (slug) → Link Entry wp_category_<slug> The same pass harvests a `linkTargets` map keyed by `contentTypeId.fieldId`, mirroring `pull-sanity`'s contract. The shared `inferSchemas` consumes it to attach real `linkContentType` validations to the inferred fields. Orphan refs (target id missing from the corpus — e.g., a post tagged with a slug whose `<wp:term>` was filtered by the skip set) stay rewritten as dangling links but don't pollute the validation list. `pullWordpress` runs the mapper between id assignment and schema inference, so by the time `inferSchemas` sees the data, every ref is already Contentful-shaped and the shared shape detector picks it up without needing WordPress-specific knowledge. ### Breaking (pre-1.0.0) `parseWXR` / `readWXR` now emit slugs (the `nicename` attribute) in `data.categories` and `data.tags` instead of display names (CDATA content). The slug is WordPress's canonical taxonomy identifier — what any cross-reference resolution would expect. The display name remains available on the matching `<wp:category>` / `<wp:term>` document. Public test fixture in `wxr-reader.test.ts` updated to assert slugs. +14 unit tests (625 total). 5 `file:`-linked examples regression-free (5/25, 6/20, 5/18, 23/69, 7/21).
…nk Asset
`mapWordpressAssets` extracts every WordPress `attachment` document into
the Contentful-shaped asset metadata that pull-contentful writes to
`assets/assets.json`:
{ id, title, fileName, contentType, url, size }
Featured-image references on posts (the `_thumbnail_id` post-meta key
that WordPress uses to point at an attachment by post_id) are rewritten
into `featuredImage: { sys: Link Asset }` on the parent document, with
the asset id matching the attachment's prefixed entry id (`wp_<postId>`)
so `assets/assets.json` and the in-entry Link sys share one identifier.
The shared `inferSchemas` shape detector picks up `featuredImage` as
`Link Asset` automatically — no WP-specific knowledge added to inference.
Dangling featured-image refs (target attachment missing from the corpus)
leave the original meta string in place rather than emit a bogus link.
`pullWordpress`:
- runs asset extraction BEFORE filtering (so attachments always reach
the assets array even though they're dropped from `entries.ndjson`)
- writes `<output>/assets/assets.json` only when at least one
attachment was present
- still honors `--include-attachments` (attachments are re-added to
`entries.ndjson` in addition to the assets index)
WXR reader change: extractPostMeta gains an `INCLUDED_INTERNAL_META_KEYS`
whitelist so `_thumbnail_id`, `_wp_attached_file`, `_wp_attachment_metadata`,
and `_wp_page_template` flow through. Non-breaking — only adds keys.
+19 unit tests (644 total, was 625). 5 `file:`-linked examples
regression-free (5/25, 6/20, 5/18, 23/69, 7/21). npm audit: 0
vulnerabilities.
`mapWordpressBodies` walks every WordPress document and converts the
`body` and `excerpt` fields (raw HTML strings, already Gutenberg-stripped
by `parseWXR`) into Contentful RichText documents via the existing
`htmlToRichText` walker from `transform/rich-text.ts`.
Block-element mapping:
<p> → paragraph
<h1>..<h6> → heading-1..heading-6
<blockquote> → blockquote
<ul>/<ol>/<li> → unordered-list / ordered-list / list-item
<pre> → paragraph (preserved as plain text)
<table>/<tr>/<td>/<th> → table / table-row / table-cell / table-header-cell
Inline marks:
<strong>/<b> → bold
<em>/<i> → italic
<u> → underline
<code> → code
<s>/<del>/<strike> → strikethrough
<sup>/<sub> → superscript / subscript
<a href> → hyperlink (data.uri preserved)
<img> → embedded-asset-block placeholder
Empty / whitespace-only body/excerpt fields are dropped entirely instead
of being emitted as empty RichText documents — keeps schema inference
from carrying a stale `Text` field that every entry leaves null. After
conversion, the shared shape detector classifies the field as
`RichText` on the inferred schema.
Why convert at pull-time (not let the simulator do it):
(a) the inferred schema gets a typed `RichText` field
(b) `cms-sim to-import` emits a valid RichText payload for
`contentful-import` directly
(c) the pull-* adapters' output shape stays uniform across
Sanity (Portable Text → RichText) and WordPress (HTML → RichText)
+13 unit tests (656 total, was 644). 5 `file:`-linked examples
regression-free (5/25, 6/20, 5/18, 23/69, 7/21).
`parseWXR` now extracts the Polylang `language` taxonomy alongside
categories and tags. When a WordPress `<item>` carries
`<category domain="language" nicename="fr">`, the document's `locale`
field is set to that slug (overriding the channel-level fallback).
Multiple posts in different languages each get their own per-doc locale.
`pullWordpress` collects all distinct per-doc locales into
`contentful-space.json` (`locales[]`), so a bilingual / trilingual
WordPress site exports as multi-locale automatically — no transforms
needed.
`--force-locale` overrides Polylang (and the channel language) for
callers that want to flatten everything to one locale.
### Shared expand-by-locale fix
The shared `expandDocumentsByLocale` previously fanned every doc out
to every detected locale, regardless of whether the doc actually
carried locale-shaped field values. That's the right behavior for
Sanity (where one doc with `{en, es}` field values represents N
locale variants), but wrong for WordPress+Polylang (where each post
already represents one locale and shouldn't be duplicated).
Fix: per-doc decision. If at least one of the doc's fields has a
`{en, es}` shape, fan out across all locales picking each locale's
value. Otherwise emit a single variant with the doc's own `locale`
tag. Sanity behavior unchanged; WP+Polylang no longer duplicates
content.
+3 unit tests covering: per-item language tagging, all-locales
captured in `contentful-space.json`, and `--force-locale` overriding
Polylang. 659 tests total (was 656).
5 `file:`-linked examples regression-free (5/25, 6/20, 5/18, 23/69,
7/21 — all locales 1 or 2 as expected). npm audit: 0 vulnerabilities.
Adds a "Pull from WordPress" row to the Core capabilities table and a `cms-sim pull-wordpress — zero-config WordPress → Contentful` subsection under Real workflows § 3, alongside the existing `pull-sanity` documentation. Lists what the adapter handles automatically: default content-type filtering, stable prefixed ids, reference rewriting, attachments → assets.json, featured_image → Link Asset, Gutenberg / Classic HTML body → RichText, and Polylang locale detection. Points at the runnable walk-through in `example-wp-pull/`. Tests badge bumped from 510 → 659 (was stale since v0.5.0; the CLAUDE.md release process now lists this as a "Siempre" check). A companion runnable example lives at the workspace level in `example-wp-pull/` (file:-linked, not part of this repo) with a synthetic 12-document WXR fixture exercising authors, nested categories, tags, Gutenberg + Classic bodies, attachments, featured images, and the nav-menu filter. End-to-end against the sample: 5 CTs / 11 entries / 2 assets / 0 errors / 2 NULL_FIELD warnings (on optional fields like the About page's author and root-category parent).
Bumps version to 0.6.1 and closes the [Unreleased] CHANGELOG section with the 2026-05-10 date. v0.6.1 ships `cms-sim pull-wordpress`: a single-command path from a WordPress WXR XML export to the same on-disk shape that `cms-sim pull` produces for Contentful spaces, so simulate / to-import / diff consume it unchanged. Reference rewriting (taxonomies + author + parent), attachments → assets.json, featured-image → Link Asset, Gutenberg / Classic HTML body → Contentful RichText, and Polylang locale detection are all automatic. Also promotes `inferSchemas` and `detectLocales` / `expandDocumentsByLocale` from `src/sanity/` to `src/shared/` (source-agnostic), fixes the expand-by-locale fan-out so Sanity-style (per-field locales) and WordPress-style (per-doc locale tag) coexist correctly, and changes `parseWXR` to emit category/tag slugs (the `nicename` attribute) instead of display names — the canonical taxonomy identifier WordPress uses. 659 tests passing (was 510 in v0.5.0; +149 net). Zero new runtime dependencies. 6 file:-linked examples regression-free.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
cms-sim pull-wordpresssubcommand: single-command path from a WordPress WXR XML export to the same Contentful-shape output thatcms-sim pullandcms-sim pull-sanityproduce.inferSchemas+ locale detection tosrc/shared/(source-agnostic) so future adapters can reuse them without an awkwardsanity-import path.file:-linked examples regression-free.What's inside
1bdeb2fsrc/shared/665fb6312fe15a2f3ff58assets.json+ featured_image → Link Asset + 19 tests9cc5516013b519e7abdea149d630Breaking (pre-1.0.0)
parseWXR/readWXRnow emit category and tag slugs indata.categories/data.tags(thenicenameattribute), not display names (the CDATA content). The slug is WordPress's canonical taxonomy identifier — what cross-references resolve against. Display names remain on the matching<wp:category>/<wp:term>documents. Public API exportinferSanitySchemasis replaced by the unprefixedinferSchemas.Validation
file:-linked examples: from-scratch (5/25), sanity-migration (6/20/17W), wp-migration (5/18/24W), pull-contentful cached (23/69), sanity-pull (7/21/2L), wp-pull (5/11/2W against the new synthetic WXR fixture). Warning counts identical to v0.6.0 baseline.npm audit: 0 vulnerabilities (prod + dev).example-wp-pull/(workspace level,file:-linked, mirrorsexample-sanity-pull/) with a synthetic 12-document WXR fixture exercising authors, nested categories, tags, Gutenberg + Classic bodies, attachments, featured images, and the nav-menu filter.Test plan
npm test— 659 passing locallynpm run build— cleannpm auditcleanMerge note
Please merge commit rather than squash so the
v0.6.1tag (already on remote, pointing at149d630) stays reachable from main without retagging. Same pattern as v0.5.0 (222aabf) and v0.6.0 (281a6d8).