[FIP-40] merge fluss-rust#3401
Open
fresh-borzoni wants to merge 288 commits into
Open
Conversation
…pache#100) --------- Co-authored-by: 赵海源 <zhaohaiyuan@xiaohongshu.com>
--------- Co-authored-by: luoyuxia <luoyuxia@alumni.sjtu.edu.cn>
…efault ZSTD compression is configured (apache#120) --------- Co-authored-by: luoyuxia <luoyuxia@alumni.sjtu.edu.cn>
…pache#116) --------- Co-authored-by: luoyuxia <luoyuxia@alumni.sjtu.edu.cn>
--------- Co-authored-by: 赵海源 <zhaohaiyuan@xiaohongshu.com>
…--release` to complete (apache#129)
…ert implementation (apache#158)
- Add `Datum::Row(Box<GenericRow>)` variant with `as_row()` accessor - Add `get_row()` to `InternalRow` trait with default error impl - Implement `GenericRow::get_row()` and `CompactedRow::get_row()` delegation - Implement `ColumnarRow::get_row()` with Arrow StructArray extraction + OnceLock caching - Add `InnerValueWriter::Row(RowType)` and write path via nested CompactedRowWriter - Add `DataType::Row` arm in `CompactedRowDeserializer` for eager nested decode - Add `InnerFieldGetter::Row` and hook up FieldGetter/ValueWriter pipeline - Handle `Datum::Row` in `resolve_row_types` (C++ bindings) - Add round-trip tests: simple nesting, deep nesting, nullable fields, ROW as primary key Wire format matches Java: varint-length-prefixed blob of a complete CompactedRow.
* Add RecordBatchLogReader for bounded log reading * address comments * update doc * update doc and inline comments * rebase and follow up after rebase of new changes, and fix a corner issue * feedback * run tests in thread to avoid asyncio event loop starvation * address feedback
* add scanner fetch and remote download metrics * add scanner fetch and remote download metrics
…apache#560) * [rust] integration tests for MAP dt + restructure tests around complex types (apache#549) * address comments
* Add scanner poll timing metrics * cargo fmt * improve overlapping polls scenario * address feedback
…ache#565) * add database and table label to scanner metrics * revert dropped comments * revert dropped comments * refactor to avoid clones many times * optimize tests
…e#566) * feat(elixir): Extend NifConfig with writer tuning fields This commit adds eight new fields to NifConfig covering writer tuning, such as: acks, retries, request max size, idempotence, buffer memory / wait timeout, max in flight requests per bucket, and the bucket-assigner enum (:sticky or :round_robin atoms via NifUnitEnum) * feat(elixir): Expose writer tuning options on Fluss.Config This commit adds eight writer-tuning fields to Fluss.Config and exposes the corresponding setters: acks, retries, request max size, idempotence, buffer, memory/wait timeout, max inflight requests per bucket, and bucket-assigner enum.
…ble_ready helper. (apache#564) * [tests] replace fixed tokio::time::sleep workarounds with wait_for_table_ready helper. * [tests] replace fixed tokio::time::sleep workarounds with wait_for_table_ready helper. * [tests] replace fixed tokio::time::sleep workarounds with wait_for_table_ready helper.
Implements a one-shot bounded BatchScanner backed by a single LimitScanRequest RPC (fixes apache#316): - adds TableScan::limit and create_batch_scanner - eager RPC with leader resolution (mirrors Lookuper) - Arrow IPC (log) and KV -> RecordBatch (PK) decoding - projection support Squashed from PR apache#515.
…e#574) * feat(elixir): Extend NifConfig with scanner tuning fields This commit adds 8 more scanner-tuning fields to NifConfig, covering the remote log prefetch, download concurrency, max poll records and fetch size/time limits. * feat(elixir): Expose scanner tuning options on Fluss.Config This commit adds 8 scanner-tuning fields to Fluss.Config with corresponding set_* setters: remote prefetch count, remote log read concurrency, file download thread count, max poll records and fetch size/time bounds.
…pache#576) * fix rustdoc and promote warning to error in ci * address comments
…meout options (apache#577) * feat(elixir): Extend NifConfig with security & connect timeout fields This commit adds 5 fields to NifConfig: connect timeout ms and security_* settings (protocol, sasl mechanism, usernama and password) * feat(elixir): Expose security opts and connect timeout in Fluss.Config This commit adds 5 fields to Fluss.Config with the corresponding setters: connect timeout, security protocol, sasl mechanism, username and password. To ensure that the password is redacted, we add a custom Inspect impl, essentially mirroring the upstream rust debug redaction.
fresh-borzoni
force-pushed
the
fip-40-merge-rust
branch
3 times, most recently
from
4249f9b to
bee6296
Compare
fresh-borzoni
force-pushed
the
fip-40-merge-rust
branch
from
02c8001 to
c232100
Compare
Member
Author
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
20 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
closes #3407
Brings the Rust, Python, C++, Elixir clients (apache/fluss-rust) into this repo under fluss-rust/, per FIP-40 (https://cwiki.apache.org/confluence/pages/viewpage.action?pageId=421957144)