Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
61 commits
Select commit Hold shift + click to select a range
d15338f
create FuseFS, implement basic methods
Ycyken Apr 4, 2025
92ab57f
add Send trait bound for Chunker, Hasher, Scrub
Ycyken Apr 4, 2025
7071341
implement read with exact size, FileHandle::set_offset()
Ycyken Apr 4, 2025
014c678
finalized write and read in fuse_filesystem
Ycyken Apr 4, 2025
9bf46ff
add some docs to fuse_filesystem
Ycyken Apr 4, 2025
5581bdf
add fuse fs test
Ycyken Apr 5, 2025
0397aff
change FileHandle::offset return type from &usize to usize
Ycyken Apr 5, 2025
9b1d094
add ability to change offset of any file handle
Ycyken Apr 5, 2025
bac565a
add example of FS mounting
Ycyken Apr 5, 2025
5b854ac
change FileSpan::{offset, len} return type from &usize to usize
Ycyken Apr 5, 2025
5b73523
add more comments in FileSystem::read method
Ycyken Apr 5, 2025
b1e54ca
change keys type from Vec<&K> to &[&K] in Database::get_multi
Ycyken Apr 5, 2025
e5e068d
refactor: libc flags import, remove usize.clone()
Ycyken Apr 6, 2025
117bf92
some refactor
Ycyken Apr 19, 2025
5f22426
fix FuseFS::write() - was checking permission for read instead of write
Ycyken Apr 20, 2025
bf74f04
add permissions tests for FuseFS
Ycyken Apr 20, 2025
7e77eda
remove metadata change possibility with write access, add ctime change
Ycyken Apr 20, 2025
cb5f6af
add atime and ctime update in read()
Ycyken Apr 20, 2025
db0cfe0
add fuse tests: readdir, metadata time
Ycyken Apr 20, 2025
b60c015
fix release(): was removing by file handle by inode counter instead o…
Ycyken Apr 20, 2025
7719512
fix clippy warnings
Ycyken Apr 21, 2025
dc903c9
add installation of FUSE dependencies in CI
Ycyken Apr 21, 2025
e82352b
fix clippy warnings
Ycyken Apr 21, 2025
54b9b25
add cache to FuseFS
Ycyken Apr 23, 2025
0f1e8d9
refactor FuseFS
Ycyken Apr 23, 2025
d51a4e2
renamed mixed-up fuse tests
Ycyken Apr 23, 2025
e93f785
fix global cache drop in FuseFS: was removing file handle before iter…
Ycyken Apr 24, 2025
69618d9
add tests with different cache scenarios, concurrent file handles
Ycyken Apr 24, 2025
5d92017
clippy warnings
Ycyken Apr 24, 2025
54fea19
add 7.28 FUSE ABI version as fuser feature
Ycyken Apr 24, 2025
db6689c
add AutoUnmount option to tests and example
Ycyken Apr 24, 2025
2c95124
CI: add user_allow_other to /etc/fuse.conf
Ycyken Apr 24, 2025
88fe88f
reduce FuseFS cache: big cache was causing rare, long writes
Ycyken Apr 25, 2025
77a5225
add fuse bench
Ycyken Apr 25, 2025
bb7d22d
add init method in FuseFS
Ycyken Apr 25, 2025
3c4ef62
add flush() method in FuseFS
Ycyken Apr 25, 2025
a429b04
change mount points in fuse tests to mount_dir/X_mount_point
Ycyken Apr 26, 2025
65712c3
fix single chunk read with uneven offset
Ycyken Apr 26, 2025
462a815
fix cache drop: was dropping by underlying fh offset without it's ass…
Ycyken Apr 26, 2025
741e1ab
update cache tests with flush() and new cache sizes
Ycyken Apr 26, 2025
bcdbfa9
add O_DIRECT to all write-read fuse tests: cache can return wrongly c…
Ycyken Apr 26, 2025
542644e
fuse tests: replace SimpleHasher with Sha256Hasher, add FuseFixture::…
Ycyken Apr 26, 2025
c6de056
add fuse tests with boundary chunk reads
Ycyken Apr 26, 2025
a88e74a
add cache drop test
Ycyken Apr 26, 2025
a246e1e
some refactor
Ycyken Apr 26, 2025
933a8e8
ref: typos, impl order, spaces
Ycyken May 1, 2025
96f46fe
test: change error assertion to concrete error type
Ycyken May 1, 2025
f646ab9
fix readdir test
Ycyken May 1, 2025
f055e5c
fix lookup permission check: FUSE was sendind EIO on incorrect daemon…
Ycyken May 1, 2025
fe221ae
add more docs to FuseFS
Ycyken May 1, 2025
e843aa1
fix EIO FUSE error causing by incorrect file generation update on eve…
Ycyken May 2, 2025
72afbaf
move reply in FuseFS::create() after all filesystem updates to avoid …
Ycyken May 2, 2025
9d579f4
add O_DIRECT flag to fuse benches
Ycyken May 2, 2025
f1f0ec5
fix FuseFS: total cache was not increasing in write()
Ycyken May 8, 2025
8cb0939
add dedup ratio and avg chunk size request for FuseFS via ioctl
Ycyken May 8, 2025
de92078
update Readme with native dependencies for FUSE
Ycyken May 8, 2025
6656e49
fix global cache drop with concurrent file handles in FuseFS
Ycyken May 8, 2025
6310a90
change custom Dataset structure to bench::Dataset in fuse_write_read …
Ycyken May 8, 2025
b70e410
fix benches: super and ultra were mix up
Ycyken May 8, 2025
213e1a5
increase FuseFS cache sizes
Ycyken May 8, 2025
c33c321
Merge branch 'main' into fuse
Piletskii-Oleg Jul 10, 2026
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
11 changes: 11 additions & 0 deletions .github/workflows/rust.yml
Original file line number Diff line number Diff line change
Expand Up @@ -12,13 +12,24 @@ jobs:

steps:
- uses: actions/checkout@v3

- name: Install FUSE dependencies
run: |
sudo apt update
sudo apt install -y fuse3 libfuse3-dev
echo "user_allow_other" | sudo tee -a /etc/fuse.conf

- name: Formatting
run: cargo fmt -- --check

- name: Clippy
run: cargo clippy --all-targets --tests -- -D warnings

- name: Build
run: cargo build --all-features --all-targets --verbose

- name: Run tests
run: cargo test --verbose

- name: Run binary
run: cargo build -p chunkfscli --verbose
126 changes: 126 additions & 0 deletions Cargo.lock

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

8 changes: 8 additions & 0 deletions Cargo.toml
Original file line number Diff line number Diff line change
Expand Up @@ -28,6 +28,9 @@ csv = { version = "1", optional = true }
serde = { version = "1", optional = true, features = ["derive"] }
serde_with = { version = "3", optional = true }
chrono = { version = "0.4", optional = true, features = ["serde"] }
libc = "0.2.171"
fuser = { version = "0.15.1", features = ["abi-7-28"] }
nix = { version = "0.30.1", features = ["ioctl"] }

[features]
chunkers = ["cdc-chunkers", "fastcdc"]
Expand All @@ -40,6 +43,7 @@ criterion = "0.5"
sha3 = "0.10"
approx = "0.5"
tempfile = "3.14"
filetime = "0.2.25"

serde_json = "1"

Expand All @@ -50,3 +54,7 @@ harness = false
[[bench]]
name = "chunker_sizes"
harness = false

[[bench]]
name = "fuse_write_read"
harness = false
15 changes: 15 additions & 0 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -82,6 +82,21 @@ To use provided chunkers and hashers, use the corresponding features:
chunkfs = { version = "0.1", features = ["chunkers", "hashers"] }
```

## FUSE native dependepcies

To use FuseFS structure, mountable with FUSE, you need to install the following native dependencies:

```bash
sudo apt update
sudo apt install -y fuse3 libfuse3-dev
```

To use the file system correctly, you need to allow other users to access the mounted file system.

```bash
echo "user_allow_other" | sudo tee -a /etc/fuse.conf
```

## Examples

Examples for chunkfs usage and benching are provided in [examples](examples) folder.
Expand Down
4 changes: 2 additions & 2 deletions benches/chunker_sizes.rs
Original file line number Diff line number Diff line change
Expand Up @@ -54,8 +54,8 @@ fn get_chunker(algorithm: Algorithms, params: SizeParams) -> ChunkerRef {
match algorithm {
Algorithms::Rabin => RabinChunker::new(params).into(),
Algorithms::Leap => LeapChunker::new(params).into(),
Algorithms::Super => UltraChunker::new(params).into(),
Algorithms::Ultra => SuperChunker::new(params).into(),
Algorithms::Super => SuperChunker::new(params).into(),
Algorithms::Ultra => UltraChunker::new(params).into(),
}
}

Expand Down
Loading
Loading