feat: Add {DataFrame, LazyFrame}.cast

[mkzung]

Description

Adds DataFrame.cast and LazyFrame.cast.

df.cast({"a": nw.Float64, "b": nw.Int32})

Takes a {column: dtype} mapping and casts those columns, leaving the rest unchanged. It uses each backend's native frame cast where there is one (pyarrow Table.cast, pandas/dask astype, polars cast); the SQL backends (duckdb, pyspark) cast per column through with_columns, and ibis inherits that path. Casting a column that is not in the frame raises ColumnNotFoundError, the same as select and drop.

I kept it to the mapping form, which is what was proposed in the issue. Polars also takes a single dtype to cast every column, so I can add that too if you'd like it for parity.

What type of PR is this? (check all applicable)

• 💾 Refactor
• ✨ Feature
• 🐛 Bug Fix
• 🔧 Optimization
• 📝 Documentation
• ✅ Test
• 🐳 Other

Related issues

• Closes [Enh]: Add support {DataFrame, LazyFrame}.cast(...) #3402

AI assistance

• No AI tools were used for this PR.
• AI tools were used.

Checklist

• Code follows style guide (ruff)
• Tests added
• Documented the changes

[FBruzzesi]

Thanks for the contribution @mkzung - I left a few inline comments. On top of those, I wonder about the rationale regarding:

the SQL backends (duckdb, pyspark) cast per column through with_columns, and ibis inherits that path

as listed in the issue, pyspark and ibis have their own casting method and I think we should used them instead of choosing for a user. DuckDB doesn't seem to have it, SQLFrame I didn't check.

---

Also, there are a few conflicts to solve with the main branch before being able to merge

[FBruzzesi]

Is # pyright: ignore[reportArgumentType] an upstream polars issue?

[mkzung]

Rebased, conflicts resolved. On the comments:

• Benchmarked the pyarrow path against with_columns(ns.col(name).cast(...)), same speed. Kept
  the schema version since field.with_type preserves nullability and metadata that the
  per-column cast drops (double not null becomes double).
• pandas now reads native.dtypes[name] instead of slicing.
• ibis has a native Table.cast, so it uses that. pyspark/sqlframe/duckdb have no frame-level
  cast, so they stay on the shared path.
• The polars ignore is just a stubs gap (the mapping key is typed wider than dict[str, ...]).
  Moved it to # type: ignore[arg-type] to match the rest of the polars backend.

Also moved the LazyFrame column check to collect_schema().names(), since self.columns now
warns on lazy polars.

[FBruzzesi]

Thanks for adjusting @mkzung

• Benchmarked the pyarrow path against with_columns(ns.col(name).cast(...)), same speed. Kept
  the schema version since field.with_type preserves nullability and metadata that the
  per-column cast drops (double not null becomes double).

Preserving mutability and metadata is a good point. I wonder if we should do it in any cast operation.
Regarding the benchmark, can you please share the script and the numbers with us?

pyspark/sqlframe/duckdb have no frame-level cast, so they stay on the shared path.

What about pyspark.sql.DataFrame.to

---

Can you also please run all the checks locally as mentioned in the contributing guide?

[mkzung]

Benchmark script and numbers (pyarrow 23, 500k rows x 8 int columns, 50 runs after warm-up):

import time
import numpy as np
import pyarrow as pa
import narwhals as nw

rng = np.random.default_rng(0)
tbl = pa.table({f"c{i}": rng.integers(0, 100, 500_000) for i in range(8)})
df = nw.from_native(tbl, eager_only=True)
targets = {f"c{i}": nw.Float64 for i in range(8)}

def schema_cast():
    return df.cast(targets)

def with_columns_cast():
    return df.with_columns(*(nw.col(n).cast(d) for n, d in targets.items()))

for fn in (schema_cast, with_columns_cast):
    fn()
    start = time.perf_counter()
    for _ in range(50):
        fn()
    print(f"{fn.__name__}: {(time.perf_counter() - start) / 50 * 1000:.2f} ms")

schema_cast: 1.73 ms
with_columns_cast: 1.83 ms

So no meaningful speed difference; the reason to keep the schema path is only the
nullability/metadata preservation.

On DataFrame.to: it reconciles the whole frame to the given schema, so a partial mapping
would mean rebuilding the full target schema from the current one plus the overrides, and
sqlframe does not implement .to at all (checked both its DuckDB and Spark flavors), so the
spark-like backend cannot rely on it across the board. I can add a pyspark-only path via a
constructed StructType if you think it is worth the special case; otherwise the shared
with_columns path keeps the three SQL backends uniform.

On doing the field-preserving cast everywhere: for the eager per-column Expr.cast there is
no field object to preserve (the expression builds a new array), so this only really applies
where a backend casts a whole table; happy to open an issue to track it if useful.

Checks: prek, pyright/mypy/pyrefly and the full-coverage suite were run locally before the
push; everything on the PR is green now.