Validation on partition mappers for source partition formatting

[dstandish]

Body

Partition mappers map source partition keys to target partition keys.

I.e. they translate from a "producer" partition to the partition key that should be associated with the run that should be created for the "listening" dag.

But it also has to map backwards, so that for a given target partition key, it knows what source keys it should be looking for.

For example with the FiveMinuteMapper i wrote as an experiment, I found that I accidentally made it too permissive on the input side, and it mapped 20251201 05:01 (bad formatting) to 2025-12-01 05:01 (good formatting). Then when it inverse mapped, it mapped to 2025-12-01 05:01 -- and therefore did not see that the right partition was there (since it had the bad format).

We should probably validate that source partition key round trips properly. We could possibly add a check that is run in the asset manager at the time it creates the target events? I don't think there's really any other place / time we could validate, and we can't really "prove" that it will round trip for all values.

We could just not validate. But this will certainly could trip users up if they make custom mappers.

Committer

• I acknowledge that I am a maintainer/committer of the Apache Airflow project.

[desusaiteja]

Hello - I looked at the code and planning to work on this. But, would like to know if my approach sounds good.

My proposed approach is to add source-key validation on partition mappers before downstream mapping is recorded, with a backward-compatible default on the base mapper.

For temporal mappers, the validation would reject keys that are parseable but not canonical for the declared input format, which seems to be the failure mode described here.

I also plan to add regression coverage at the scheduler/asset-manager level so invalid source keys are logged and skipped instead of creating incorrect APDR state.

If that direction sounds right, I can open a PR.

[Lee-W]

Built-in mappers round-trip already; custom mappers are the author's responsibility. I'll take a look at #67119. but now closing it as not strickly needed