2. Override standard transformations

cleanframes provides standard transformations but you would like to redefine some of them and use with others.

It's possible by defining own type-class Cleaner instance and obeying implicits precedence rules.

Cleaner type-class instance call be implemented using materializer method in: Cleaner.materialize.

Redefine, for example, Integer transformation:

implicit val redefinedInt: Cleaner[Option[Int]] = {
  import org.apache.spark.sql.functions
  Cleaner.materialize { (frame, name, alias) =>
    List(
      functions.when(
        functions.not(
          frame.col(name.get).isNaN
        ),
        frame.col(name.get) * 1000
      ) cast IntegerType as alias.get
    )
  }
}

If needed, add rest transformations:

import cleanframes.instances.all._

Call API:

frame
   .clean[CaseClass]