The Schema currently uses a array of StructFields to represent the schema.
However, that is exactly what a StructType is used for in Spark. Additionally, a StructType provides convenient methods to modify the StructFields that it stores. Using a StructType will also be handy, to interact with Spark Dataframes.
Therefore, I propose to change the array of StructFields to a single StructType.
It is also possible to directly use the StructType instead of creating our own schema class.
The
Schemacurrently uses a array ofStructFields to represent the schema.However, that is exactly what a
StructTypeis used for in Spark. Additionally, aStructTypeprovides convenient methods to modify theStructFields that it stores. Using aStructTypewill also be handy, to interact with Spark Dataframes.Therefore, I propose to change the array of
StructFields to a singleStructType.It is also possible to directly use the
StructTypeinstead of creating our own schema class.