bcachefs: implement online filesystem shrinking

[jullanggit]

Implement online filesystem shrinking through reconcile. Closes #781 once done.
This is hopefully complementary to #1070, which targets offline shrink.

Goal

A robust online shrinking implementation, that automatically resumes after restarts/crashes, as shrinking is a potentially long-running operation, and supports changing the target size mid-shrink.

Current state

• Data movement, including stripes seems to work
• Not all metadata (especially surrounding bucket gens) seems to be correctly handled yet
• Journals are not explicitly handled, but may need to be
• Resuming and changing target size is not yet implemented
• Uses a whole-device reconcile scan, I plan to make it only scan the affected area

Implementation

Reuses large parts of the device remove/evacuate paths

Documentation

Not yet written

Testing

See https://github.com/jullanggit/ktest/tree/shrink for the tests used

[koverstreet]

Nice work — the reconcile-based approach for online shrinking is the right direction. Some feedback:

On-disk format:

Adding target_nbuckets to struct bch_member is a reasonable approach for tracking in-progress shrinks across restarts. But appending to bch_member needs care — make sure you're checking sizeof for the member struct version in the superblock validation path so older kernels reading a newer superblock don't read garbage. (This might already be handled by the existing versioned member struct machinery, but worth verifying.)

The evacuation loop:

The tail_is_empty() / sleep / retry loop in bch2_dev_shrink() is the right structure, but there are some issues:

• The TODO about cached data is a real problem. After reconcile evacuates, the old extents get marked cached, so tail_is_empty() (which checks backpointers) might never see them clear. You'll need to handle this — either by making evacuate fully remove the old extent rather than caching it, or by adding a pass that invalidates cached buckets in the shrink region.

• schedule_timeout_killable(HZ/2) is a reasonable poll interval but you should probably also check c->reconcile progress or wait on an event rather than blind polling.

• The signal handling (signal_pending → -EINTR) is good — long-running operations should be interruptible.

Allocation cutoff:

if (unlikely(ca->mi.target_nbuckets && bucket >= ca->mi.target_nbuckets)) {

This replaces bch2_bucket_nouse() — but buckets_nouse was also used for other things (marking individual buckets bad). You've removed ENOMEM_buckets_nouse and no_resize_with_buckets_nouse from the error codes. Make sure nothing else was relying on that bitmap.

bch2_ptr_bad_or_evacuating_rcu:

This inline function does a division (div_u64(ptr->offset, ca->mi.bucket_size)) in what can be a hot path. Consider whether you can precompute the cutoff sector instead.

Commented-out code:

The __bch2_dev_resize_alloc block is commented out with a TODO. Either figure out what's needed there or remove it — commented-out code shouldn't ship.

Style nits:

• Opening braces on function definitions should be on their own line (kernel style)
• // TODO comments are fine for WIP but should be resolved before merge
• The _typos.toml change doesn't belong in this PR
• Several unrelated typo fixes (sentinal→sentinel, dosen't→doesn't, minumum→minimum, elligible→eligible) — those are welcome but should be a separate commit

Testing:

Good that you have ktest tests for this: https://github.com/jullanggit/ktest/tree/shrink. Consider adding cases for:

• Shrink interrupted by signal, then resumed
• Shrink with concurrent heavy writes to the device being shrunk
• Shrink that needs to move striped/EC data
• Shrink below journal location

Overall this is solid WIP. The hard parts (cached data handling, journal, resume after crash) are acknowledged as TODOs, which is the right approach — get the happy path working first.

— ProofOfConcept

[jullanggit]

Thank you for the review! I'll continue working on this, and will ping you once I feel like another review would help.

[Komzpa]

Small helper PR for this branch:

• bcachefs: shrink: address review cleanups jullanggit/bcachefs#1

It is intentionally narrow: kernel brace style for the new pointer helper, avoids the hot-path divide in bch2_ptr_bad_or_evacuating_rcu(), replaces the stale tail-snapshot TODO with the current journal/superblock handling note, and removes the commented-out __bch2_dev_resize_alloc() block.

Related tools-side bridge:

• device: let online resize delegate shrink to kernel bcachefs-tools#650

That tools PR is clean/green at head 81ee809c6d8922a04c0e5d2cebbb19f7950ad47e and only lets mounted bcachefs device resize delegate smaller target sizes to kernels that implement shrink; it does not claim full shrink support by itself.

This does not try to solve the larger cached-data, progress/wakeup, journal/resume, or ktest follow-ups from Kent’s review.

[Komzpa]

I took another pass on this because it is blocking the high-score shrink cluster (koverstreet/bcachefs#781 / koverstreet/bcachefs-tools#268).

Rebased helper branch:

• https://github.com/Komzpa/bcachefs/tree/komzpa/shrink-rebase-check-20260629
• head: 4c16f1c1e5ca (bcachefs: port shrink discard guard to resize target)
• base used locally: koverstreet/bcachefs@ca944a61e079

What changed beyond the existing shrink stack:

• rebased the branch over current master;
• kept the existing author stack intact;
• added one small compatibility commit for the rebase: the shrink discard guard now uses the current resize target state instead of the deleted buckets_nouse bitmap, so both the normal and fast discard paths still skip the shrinking tail.

Validation I ran:

• git diff --check origin/master..HEAD
• no conflict markers or stale bch2_bucket_nouse / dev_bucket_nouse references under fs/bcachefs
• targeted build:
  make -j32 fs/bcachefs/alloc/buckets.o fs/bcachefs/alloc/discard.o fs/bcachefs/alloc/foreground.o fs/bcachefs/data/reconcile/work.o fs/bcachefs/data/ec/init.o fs/bcachefs/init/dev.o fs/bcachefs/init/fs.o fs/bcachefs/btree/interior.o fs/bcachefs/journal/write.o

I did not force-push jullanggit:shrink; this is just a review/rebase branch people can pull from.