Skip to content
Draft
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
34 commits
Select commit Hold shift + click to select a range
0c0fe3d
block: add mirror module skeleton
Coffeeri Apr 27, 2026
6b005aa
block: add MirroringAsyncIo skeleton
Coffeeri Apr 27, 2026
c74ead5
block: add range lock primitive for mirror
Coffeeri Apr 28, 2026
f9bddf3
block: add CompletionWaiter for single-fd waits
Coffeeri Apr 29, 2026
2c1716e
block: mirror mutating I/O to destination
Coffeeri Apr 28, 2026
dc1910a
block: add background copy worker for mirror
Coffeeri Apr 29, 2026
ce1eaa1
virtio-devices: swap disk_image via queue commands
Coffeeri Apr 29, 2026
7bd8700
virtio-devices: pre-allocate per-queue command slots
Coffeeri Apr 29, 2026
fff5673
block: add BlockMirrorHandle
Coffeeri Apr 29, 2026
38d785e
block: add MirroringAsyncIo::create
Coffeeri Apr 30, 2026
389905f
virtio-devices: add Block::start_mirror
Coffeeri Apr 30, 2026
f3cf938
virtio-devices, block: add mirror status helper
Coffeeri Apr 30, 2026
be2205f
vmm: add device manager block mirror start and status
Coffeeri Apr 30, 2026
8d66b62
vmm: add vm.disk-mirror-start REST endpoint
Coffeeri Apr 30, 2026
fe95d58
vmm: add vm.disk-mirror-status REST endpoint
Coffeeri Apr 30, 2026
8af2fdd
block: implement MirroringAsyncIo::submit_batch_requests
Coffeeri Apr 30, 2026
045d570
block: add AsyncIo::has_inflight_requests
Coffeeri May 3, 2026
6f1aca8
virtio-devices: drain mirror wrapper before disk_image swap
Coffeeri May 3, 2026
b455827
virtio-devices, block: add Block::complete_mirror
Coffeeri May 3, 2026
bdb2d27
vmm: add vm.disk-mirror-complete REST endpoint
Coffeeri May 3, 2026
7d43b83
virtio-devices, block: add Block::cancel_mirror
Coffeeri Jun 10, 2026
b8e4bd0
vmm: deny conflicting ops while a disk mirror runs
Coffeeri Jun 10, 2026
eec7803
vmm: add vm.disk-mirror-cancel REST endpoint
Coffeeri Jun 11, 2026
7e6cf98
block: preserve sparseness in CopyWorker
Coffeeri Jun 17, 2026
66aa370
block: add mirror unit tests
Coffeeri Jun 22, 2026
c1569fb
block: test range guard held across mirror write
Coffeeri Jun 23, 2026
b38f504
virtio-devices, block: reject mirror ops on a paused device
Coffeeri Jun 23, 2026
87e96c2
vmm: reject mirror destination already backing a disk
Coffeeri Jun 24, 2026
1b2b717
vmm: seccomp: allow io_uring and eventfd2 on vcpus
Coffeeri Jun 24, 2026
a23a91c
block: test batched submit on partial failure
Coffeeri Jun 24, 2026
0a812e1
block: test mirror phase transitions and op fan-out
Coffeeri Jun 24, 2026
54bd335
docs: add disk mirroring guide
Coffeeri Jun 25, 2026
4ef3bad
vmm: add configurable mirror destination mode
Coffeeri Jun 29, 2026
f13781a
vmm: dedup block device lookup in DeviceManager
Coffeeri Jul 6, 2026
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
8 changes: 8 additions & 0 deletions block/src/async_io.rs

@arctic-alpaca arctic-alpaca Jul 8, 2026

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm still reviewing, but had some questions:

  • Writes are mirrored through MirroringAsyncIo, so we could track what areas where already written by the mirroring and omit/skip those in the copy worker. This would allow us to avoid coordination through the RangeLockManager. Would that be possible?

  • Why does snapshotting conflict with storage migration?

  • The trade-off is added write latency, which is fine for storage rebalancing.

    Do you have some benchmarks, tests or something else that supports this? I have no clue whether this is reasonable or not.

File unrelated, just so the discussion can be resolved.

Copy link
Copy Markdown
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thank you for taking the time to look at the PR!

  1. Tracking written ranges would work as an optimization to reduce copy work, but I sadly don't see how it could replace the range locking, because the race lives in the submit-to-completion window, not in "was this range written before". Both CopyWorker and virtqueue workers can still interleave.
  2. The Mirror job is a runtime-only state: phase, copy progress and destination handle are not living in the snapshot state. A restored VM would come up with the mirror silently gone and the destination left behind as a half-copied file, without the operator ever seeing a cancel. Moreover snapshotting requires the VM to be paused. A paused vm would have parked the virtqueue worker threads, so we could never complete the mirror as we need to switch all backends to the destination AsyncIO backend. Maybe we can add support for this in the future. I tried to keep the logic/ state "simple" here.
  3. Nope not yet. The cost is per-write latency and lost pipelining: the virtqueue worker waits inline for both completions instead of letting io_uring run ahead, so I/O on that queue stalls behind an in-flight mirrored write.

Original file line number Diff line number Diff line change
Expand Up @@ -109,4 +109,12 @@ pub trait AsyncIo: Send {
fn alignment(&self) -> u64 {
SECTOR_SIZE
}

/// Returns true when this implementation has request pairings in flight
/// that have not yet been acked to the guest. Only the mirroring
/// implementation tracks such pairings, plain backends always return
/// false.
fn has_inflight_requests(&self) -> bool {
false
}
}
20 changes: 20 additions & 0 deletions block/src/error.rs
Original file line number Diff line number Diff line change
Expand Up @@ -42,6 +42,18 @@ pub enum BlockErrorKind {
NotFound,
/// An internal counter or limit was exceeded.
Overflow,
/// The file already exists, when disk creation was requested.
AlreadyExists,
/// A mirror operation was requested but no mirror is active for the device.
MirrorNotActive,
/// A completion was requested but the mirror has not reached the ready phase.
MirrorNotReady,
/// A mirror swap was requested but was unsuccessful.
MirrorSwap,
/// A mirror completion is already in progress.
MirrorCompletionInProgress,
/// A mirror operation was requested while the device is paused.
MirrorDevicePaused,
}

impl Display for BlockErrorKind {
Expand All @@ -54,6 +66,14 @@ impl Display for BlockErrorKind {
Self::OutOfBounds => write!(f, "Out of bounds"),
Self::NotFound => write!(f, "Not found"),
Self::Overflow => write!(f, "Overflow"),
Self::AlreadyExists => write!(f, "Already exists"),
Self::MirrorNotActive => write!(f, "No active mirror for the device"),
Self::MirrorNotReady => write!(f, "Mirror is not yet ready, cannot complete"),
Self::MirrorSwap => write!(f, "Failed to swap AsyncIO in virtqueue worker for mirror"),
Self::MirrorCompletionInProgress => write!(f, "Mirror completion already in progress"),
Self::MirrorDevicePaused => {
write!(f, "Mirror operation rejected: the device is paused")
}
}
}
}
Expand Down
44 changes: 44 additions & 0 deletions block/src/factory.rs
Original file line number Diff line number Diff line change
Expand Up @@ -9,6 +9,7 @@
//! support, and constructs the appropriate backend. Callers receive
//! a trait object that is ready for use by virtio queue workers.

use std::io::ErrorKind;
use std::os::unix::fs::OpenOptionsExt;
use std::path::Path;
use std::sync::OnceLock;
Expand All @@ -21,6 +22,7 @@ use crate::block_io_uring_is_supported;
use crate::disk_file::AsyncFullDiskFile;
use crate::error::{BlockError, BlockErrorKind, BlockResult};
use crate::fixed_vhd_disk::FixedVhdDisk;
use crate::qcow::{QcowFile, RawFile};
use crate::qcow_disk::QcowDisk;
use crate::raw_disk::{RawBackend, RawDisk};
use crate::vhdx_sync::VhdxDiskSync;
Expand Down Expand Up @@ -203,6 +205,48 @@ fn open_qcow2(
))
}

/// Create a new disk image at `options.path` of the given image type
/// and logical `size`. The file must not exist yet.
pub fn create_disk(
options: &DiskOpenOptions<'_>,
image_type: ImageType,
size: u64,
) -> BlockResult<()> {
let file = fs::OpenOptions::new()
.read(true)
.write(true)
.create_new(true)
.open(options.path)
.map_err(|e| {
let kind = match e.kind() {
ErrorKind::AlreadyExists => BlockErrorKind::AlreadyExists,
_ => BlockErrorKind::Io,
};
BlockError::from_kind(kind)
.with_path(options.path)
.with_source(e)
})?;

match image_type {
ImageType::Raw => {
file.set_len(size)
.map_err(|e| BlockError::from(e).with_path(options.path))?;
}
ImageType::Qcow2 => {
let raw_file = RawFile::new(file.try_clone()?, options.direct);
QcowFile::new(raw_file, 3, size, options.sparse)
.map_err(|e| e.with_path(options.path))?;
}
_ => {
return Err(
BlockError::from_kind(BlockErrorKind::UnsupportedFeature).with_path(options.path)
);
}
}

Ok(())
}

#[cfg(test)]
mod unit_tests {
use std::io::Write;
Expand Down
1 change: 1 addition & 0 deletions block/src/lib.rs
Original file line number Diff line number Diff line change
Expand Up @@ -20,6 +20,7 @@ pub mod fixed_vhd;
pub mod fixed_vhd_async;
pub mod fixed_vhd_disk;
pub mod fixed_vhd_sync;
pub mod mirror;
pub mod qcow;
#[cfg(feature = "io_uring")]
pub(crate) mod qcow_async;
Expand Down
Loading
Loading