variant_get should follow JSONpath semantics

[scovich]

From @scovich in #8354 (comment):

there's still an open question of whether we intend to follow the jsonpath spec in our path step logic, as e.g. spark does? #8354 (comment)

The jsonpath spec requires foo[100] to return NULL if foo is not an array, and also requires returning NULL if foo has fewer than 101 elements. Similarly, foo.bar should return NULL if foo is not a struct and should also return NULL if foo has no field named bar. Safe casting would only influence actual casting decisions, e.g. a variant_get call that specifically requests a string and the requested path points to a struct.

In contrast, our current struct handling code currently returns an error if safe casting is disabled and:

• a Field path step encounters a "wrong" type (L169)
• an Index path step encounters a "wrong" type (L224)
• an Index path step is out of bounds (L99)

[scovich]

I think spark is just implementing jsonpath semantics:

A syntactically valid segment MUST NOT produce errors when executing the query. This means that some operations that might be considered erroneous, such as using an index lying outside the range of an array, simply result in fewer nodes being selected.

Here, "syntactically valid" is referring to the previous section (2.1):

A JSONPath implementation MUST raise an error for any query that is not well-formed and valid. The well-formedness and the validity of JSONPath queries are independent of the JSON value the query is applied to. No further errors relating to the well-formedness and the validity of a JSONPath query can be raised during application of the query to a value. This clearly separates well-formedness/validity errors in the query from mismatches that may actually stem from flaws in the data.

Note: Integer overflow in an index is well-formed but not valid, so it's allowed to produce an error.

Originally posted by @scovich in #8354 (comment)

[scovich]

Also from @alamb in #8354 (comment):

@alamb -- any opinions about supporting jsonpath semantics or not? Or ideas on who we should seek input from?

In my opinion supporing jsonpath sounds like a good idea in general

The usecase I know of (and have) for variant_get is to mirror spark's variant_get function: https://docs.databricks.com/aws/en/sql/language-manual/functions/variant_get

So having the arrow-rs implementation match sounds good too

[alamb]

label_issue.py automatically added labels {'parquet-variant'} from #9599