A storage deduplication tool for MKV files and their source media (DVD ISOs, Blu-ray backups).
mkvdup reduces storage requirements for MKV files by referencing their source media. Since the underlying codec data (video frames, audio packets) is identical between an MKV and its source—just at different offsets with different container framing—we can store only the unique MKV data plus an index mapping MKV offsets to source offsets.
Example: A 3.4GB MKV can be stored as ~50MB by referencing the source ISO.
This tool is intended for personal backup and archival of legally owned media. It does not perform any copy protection circumvention.
- DVD support - Works with ISO files containing VOB (MPEG-PS) content
- Blu-ray support - Works with BDMV directory structures and Blu-ray ISO files
- FUSE filesystem - Mount deduplicated files and access them transparently
- Permission & timestamp customization -
chmod/chown/touchsupport with persistent metadata storage (file mtimes are derived from the dedup file and overridable) - Verification - Byte-for-byte verification of reconstructed files
brew tap stuckj/mkvdup
brew install mkvdupCanary (pre-release): brew install mkvdup-canary — installs alongside stable as mkvdup-canary
# Add the GPG key
curl -fsSL https://stuckj.github.io/mkvdup/gpg-key.asc | sudo gpg --dearmor -o /usr/share/keyrings/mkvdup.gpg
# Add the repository
echo "deb [signed-by=/usr/share/keyrings/mkvdup.gpg arch=amd64,arm64] https://stuckj.github.io/mkvdup/apt stable main" | sudo tee /etc/apt/sources.list.d/mkvdup.list
# Install
sudo apt update
sudo apt install mkvdupThis repository carries the current release only. To install or pin an older version, add the archive repository as well — it indexes every version ever published.
Enabling both is safe. They are two sources for the same package, not two
packages, so apt merges them and still installs exactly one mkvdup — one
binary, one man page, one mount helper. The current release appears in both, with
the same checksum, and apt simply lists two sources for it. (The canary channel
is the different case: it installs under its own name, mkvdup-canary, and is
meant to sit alongside a stable install.) Both are signed with the same key.
echo "deb [signed-by=/usr/share/keyrings/mkvdup.gpg arch=amd64,arm64] https://github.com/stuckj/mkvdup/releases/download/apt-history/ ./" | sudo tee /etc/apt/sources.list.d/mkvdup-history.list
sudo apt update
apt list -a mkvdup # every published version
sudo apt install mkvdup=1.8.0 # pin oneCanary (pre-release)
# Add the GPG key (same as stable)
curl -fsSL https://stuckj.github.io/mkvdup/gpg-key.asc | sudo gpg --dearmor -o /usr/share/keyrings/mkvdup.gpg
# Add the canary repository (current canary only)
echo "deb [signed-by=/usr/share/keyrings/mkvdup.gpg arch=amd64,arm64] https://stuckj.github.io/mkvdup/apt canary main" | sudo tee /etc/apt/sources.list.d/mkvdup-canary.list
# ...or every canary ever published
echo "deb [signed-by=/usr/share/keyrings/mkvdup.gpg arch=amd64,arm64] https://github.com/stuckj/mkvdup/releases/download/apt-history-canary/ ./" | sudo tee /etc/apt/sources.list.d/mkvdup-canary-history.list
# Install
sudo apt update
sudo apt install mkvdup-canaryThis repository indexes every version published, so no separate archive repository is needed.
Requires rpm 4.16 or newer — Fedora, RHEL/Alma/Rocky 9 and 10. The packages are signed with an ed25519 key, and rpm only learned to read EdDSA signatures in 4.16.0. EL8 ships rpm 4.14, which cannot import the key at all (
key 1 import failed), sogpgcheck=1cannot be satisfied there. Debian and Ubuntu are unaffected: apt verifies the signedReleasefile rather than a signature inside each package.
# Add the repository
sudo tee /etc/yum.repos.d/mkvdup.repo << 'EOF'
[mkvdup]
name=mkvdup
baseurl=https://stuckj.github.io/mkvdup/yum
enabled=1
gpgcheck=1
gpgkey=https://stuckj.github.io/mkvdup/yum/gpg-key.asc
EOF
# Install
sudo dnf install mkvdup
# ...or pin an older version
dnf list --showduplicates mkvdup
sudo dnf install mkvdup-1.8.0Canary (pre-release)
Same rpm 4.16 requirement as the stable repository above.
# Add the canary repository
sudo tee /etc/yum.repos.d/mkvdup-canary.repo << 'EOF'
[mkvdup-canary]
name=mkvdup-canary
baseurl=https://stuckj.github.io/mkvdup/yum-canary
enabled=1
gpgcheck=1
gpgkey=https://stuckj.github.io/mkvdup/yum-canary/gpg-key.asc
EOF
# Install
sudo dnf install mkvdup-canaryAvailable from the first release cut after Arch support landed; earlier tags have neither the AUR packages nor the pacman repository.
From the AUR, if you use an AUR helper:
yay -S mkvdupOr add the binary repository, which serves the same package and updates through
pacman -Syu without a rebuild. x86_64 only — Arch publishes no aarch64 container for CI to
build one in, so on Arch Linux ARM use the AUR route above, which builds natively:
# Trust the signing key (public; the same key the APT and YUM repositories use)
curl -fsSL https://stuckj.github.io/mkvdup/gpg-key.asc | sudo pacman-key --add -
sudo pacman-key --lsign-key 3AABF4C834FFE7E08D91A9BACDB7B8F88AFCCBE3
# Add the repository
sudo tee -a /etc/pacman.conf << 'EOF'
[mkvdup]
SigLevel = Required
Server = https://github.com/stuckj/mkvdup/releases/download/pacman-$arch
EOF
# Install
sudo pacman -Syu mkvdupBoth routes install the same mkvdup package, so you can add the repository later
without uninstalling the AUR build — pacman takes over the upgrade from there.
Canary (pre-release)
Installs as mkvdup-canary, providing the mkvdup-canary command alongside stable.
yay -S mkvdup-canaryOr via the canary repository:
# Trust the signing key (public; same as stable)
curl -fsSL https://stuckj.github.io/mkvdup/gpg-key.asc | sudo pacman-key --add -
sudo pacman-key --lsign-key 3AABF4C834FFE7E08D91A9BACDB7B8F88AFCCBE3
# Add the canary repository
sudo tee -a /etc/pacman.conf << 'EOF'
[mkvdup-canary]
SigLevel = Required
Server = https://github.com/stuckj/mkvdup/releases/download/pacman-canary-$arch
EOF
# Install
sudo pacman -Syu mkvdup-canaryOnce mkvdup lands in nixpkgs (submission pending) that becomes the recommended route — pre-built binaries, no flakes needed:
nix profile install nixpkgs#mkvdup # or add mkvdup to environment.systemPackagesUntil then, install from this repo's flake, pointing it at a release tag:
nix profile install github:stuckj/mkvdup/v<version>#mkvdupThe #mkvdup output is available from v1.8.2 onward. Older tags are limited by what the
flake looked like at the time: v1.8.1 offers only #mkvdup-canary, which installs the command as
mkvdup-canary, and tags at or before v1.8.0 have no flake at all.
Declaratively, add this repo as a flake input and put mkvdup.packages.${system}.mkvdup in
environment.systemPackages (or home.packages).
mkvdup mounts FUSE filesystems, so it needs fusermount3 from fuse3 at runtime. On NixOS, set
programs.fuse.userAllowOther = true; if you mount with allow_other.
Canary (pre-release)
Nix needs no separate canary repository — the flake reference is the selector, so you can build
any branch, tag or commit directly. It installs as mkvdup-canary, alongside a stable mkvdup.
# try a development branch without installing anything
nix shell github:stuckj/mkvdup/feat/my-branch#mkvdup-canary
# or install it
nix profile install github:stuckj/mkvdup/feat/my-branch#mkvdup-canary
# or pin an immutable canary tag
nix profile install github:stuckj/mkvdup/v1.8.2-canary.1#mkvdup-canaryCanary tags from v1.8.2-canary.1 onward are installable this way. Earlier ones were tagged
before the hash automation existed and generally fail to build; check out the tag and run
./scripts/update-nix-vendor-hash.sh if you need one of them.
nix shell is usually the right tool for testing a branch: the binary is on PATH for that shell
only, with nothing to uninstall afterwards. Branch names containing / work as written.
Installing from a branch pins the revision Nix resolved at the time, so use nix profile upgrade to
move it forward.
go install github.com/stuckj/mkvdup/cmd/mkvdup@latestCanary: go install github.com/stuckj/mkvdup/cmd/mkvdup@v0.x.x-canary.N (see releases for versions)
mkvdup create video.mkv /path/to/source/dir video.mkvdupmkvdup mount /mnt/videos config.yamlEach .mkvdup file gets a companion YAML config:
# /data/dedup/video1.mkvdup.yaml
name: "Movies/Video 1.mkv"
dedup_file: video1.mkvdup
source_dir: /data/sources/Video1_DVDA top-level config includes them all using glob patterns:
# /etc/mkvdup.conf
includes:
- "/data/dedup/**/*.mkvdup.yaml"Include globs in mount configs aren't re-expanded at runtime, so new files
added to matched directories won't appear until the config is reloaded.
Use expand-config to resolve globs to explicit paths:
# Use an existing mount config with include globs as the source of truth
# e.g., /etc/mkvdup.conf contains:
# includes:
# - "/data/dedup/**/*.mkvdup.yaml"
# Expand to explicit file list
mkvdup expand-config /etc/mkvdup.conf --output expanded.yaml
# Mount using the expanded config
mkvdup mount /mnt/videos expanded.yamlWhen new .mkvdup.yaml files are added, re-run expand-config to regenerate
the explicit config, then reload the running mount (mkvdup reload or SIGHUP).
If the file list hasn't changed, the output file is not rewritten. See
docs/CLI.md for full details.
/etc/mkvdup.conf /mnt/videos fuse.mkvdup nofail 0 0
nofail lets the system boot normally if the mount fails (e.g., source media unavailable). The mount helper automatically enables allow_other so that non-root users can access the filesystem.
For a directory of config files instead of a single file:
/etc/mkvdup.d /mnt/videos fuse.mkvdup config_dir,nofail 0 0
See docs/FUSE.md for full configuration details including source watching, error notifications, and permissions.
mkvdup verify video.mkvdup /path/to/source/dir original.mkvmkvdup info video.mkvdup- Index the source - Parse the DVD/Blu-ray container and build a hash index of codec packets
- Parse the MKV - Extract codec data locations from the MKV file
- Match packets - Find MKV codec data in the source using hash lookups
- Create dedup file - Store the index mapping plus any MKV-only data (headers, chapters, etc.)
- Reconstruct on-demand - FUSE filesystem stitches data from source files and the dedup file
See docs/CLI.md for the full command-line reference.
- DESIGN.md - Architecture overview and technical decisions
- docs/MATCHING.md - Matching algorithms and ES-aware indexing
- docs/FILE_FORMAT.md - Binary specification for .mkvdup files
- docs/FUSE.md - FUSE filesystem configuration
- docs/CLI.md - Command-line interface reference
- CONTRIBUTING.md - Development guidelines
- Performance Benchmarks - Historical performance tracking
MIT