chore(arrow-cast): support white space prefixed parse for ints and floats - #10374
Conversation
|
cc @alamb @aryan-212 @Rafferty97 since you had interest in #9538 & #9537 |
|
supporting post-fix white space should be pretty similar as well |
|
@Rich-T-kid Since we are only concerned with ascii whitespaces, you can use |
@aryan-212 thanks for the suggestion, ill update the PR |
|
the screenshotted benchmarks are for string -> f32, but it seems f32/f16 paths are not changed only f64. i think we need to add to f32/f16 and rerun those benchmarks |
whoops 😅. I forgot to add this to the f32/16 branches. @Jefffrey should be good now |
|
run benchmark cast_kernels |
This comment was marked as duplicate.
This comment was marked as duplicate.
|
🤖 Arrow criterion benchmark completed (GKE) | trigger Instance: CPU Details (lscpu)Details
Resource Usagebase (merge-base)
branch
File an issue against this benchmark runner |
nice |
|
run benchmark cast_kernels |
This comment was marked as duplicate.
This comment was marked as duplicate.
|
🤖 Arrow criterion benchmark completed (GKE) | trigger Instance: CPU Details (lscpu)Details
Resource Usagebase (merge-base)
branch
File an issue against this benchmark runner |
|
thanks @Rich-T-kid & @aryan-212 |
|
Nice! |
# Which issue does this PR close? <!-- We generally require a GitHub issue to be filed for all bug fixes and enhancements and this helps us generate change logs for our releases. You can link an issue to this PR using the GitHub syntax. --> - Closes #9538. # Rationale for this change - see #9538 - see #10374 <!-- Why are you proposing this change? If this is already explained clearly in the issue then this section is not needed. Explaining clearly why changes are proposed helps reviewers understand your changes and offer better suggestions for fixes. --> # What changes are included in this PR? this PR adds support for suffix white space when calling parse <!-- There is no need to duplicate the description in the issue here but it is sometimes worth providing a summary of the individual changes in this PR. --> # Are these changes tested? yes <!-- We typically require tests for all PRs in order to: 1. Prevent the code from being accidentally broken by subsequent changes 2. Serve as another way to document the expected behavior of the code If tests are not included in your PR, please explain why (for example, are they covered by existing tests)? If this PR claims a performance improvement, please include evidence such as benchmark results. --> # Are there any user-facing changes? yes, users will be able to parse strings that have white space prefixed/suffixed onto numbers/floats. <!-- If there are user-facing changes then we may require documentation to be updated before approving the PR. If there are any breaking changes to public APIs, please call them out. --> --------- Co-authored-by: rich-T-kid <richardbaah@MacBook-Air-de-Richard.local> Co-authored-by: Jeffrey Vo <jeffrey.vo.australia@gmail.com> Co-authored-by: Andrew Lamb <andrew@nerdnetworks.org>
Which issue does this PR close?
Rationale for this change
currently parsing ints and floats with white space prefiing the number caused the cast to fail
What changes are included in this PR?
this PR adds a check to the string before parsing. it checks the first byte, if its not an ascii digit it will trim the left side of the string.
Performance was a consideration to avoid regressions here — see #9537 (comment). In the worst case where a string doesn't need trimming (no leading whitespace), the cost is just one slice operation to grab the first byte plus a range comparison to check if it's an ASCII digit.
benchmarks show no significant change

Are these changes tested?
yes. I created a test with 12 assertiosn to validate behavoir
Are there any user-facing changes?
yes. arrow-rs/datafusion cast operations will no longer fail when a number is prefixed with white space.