CVMFS Config Package Builder for Subspatial

Build system for creating CVMFS client configuration packages for Subspatial repositories.

Quick Start

# 1. Add your repository's public key
mkdir -p src/etc/cvmfs/keys/cryoet-opendata-poc.subspatial.org
cp /path/to/cryoet-opendata-poc.subspatial.org.pub \
   src/etc/cvmfs/keys/cryoet-opendata-poc.subspatial.org/

# 2. Build packages (requires fpm)
./build.sh 0.1.0

# 3. Packages appear in output/
ls output/

Directory Structure

cvmfs-config-subspatial-builder/
├── README.md                   # This file
├── build.sh                    # Main build script
│
├── src/                        # Source files for packages
│   └── etc/cvmfs/
│       ├── domain.d/
│       │   └── subspatial.org.conf          # Domain-wide settings
│       ├── config.d/
│       │   ├── cryoet-opendata-poc.subspatial.org.conf   # Per-repo config
│       │   └── TEMPLATE.subspatial.org.conf               # Template
│       └── keys/
│           ├── cryoet-opendata-poc.subspatial.org/
│           │   └── cryoet-opendata-poc.subspatial.org.pub
│           └── <other-repo>.subspatial.org/
│               └── <other-repo>.subspatial.org.pub
│
├── build/                      # Temporary build files (generated)
└── output/                     # Final packages (generated)
    ├── cvmfs-config-subspatial-VERSION.tar.gz
    ├── cvmfs-config-subspatial_VERSION-1_all.deb
    └── cvmfs-config-subspatial-VERSION-1.noarch.rpm

Adding a New Repository

Step 1: Create Repository Configuration

Create src/etc/cvmfs/config.d/<repo-name>.subspatial.org.conf:

cat > src/etc/cvmfs/config.d/spatial-omics.subspatial.org.conf << 'EOF'
# Spatial Omics Data Repository
# Repository: spatial-omics.subspatial.org

# S3 backend for CVMFS metadata
CVMFS_SERVER_URL="https://spatial-omics-metadata.s3.eu-west-1.amazonaws.com/spatial-omics.subspatial.org"

# S3 bucket for external data
CVMFS_EXTERNAL_URL="https://spatial-omics-data.s3.eu-west-1.amazonaws.com/spatial-omics.subspatial.org"

# Public key location
CVMFS_KEYS_DIR="/etc/cvmfs/keys/spatial-omics.subspatial.org"
EOF

Step 2: Add Public Key

# Create key directory
mkdir -p src/etc/cvmfs/keys/spatial-omics.subspatial.org

# Copy public key from publisher
scp user@publisher:/etc/cvmfs/keys/spatial-omics.subspatial.org.pub \
    src/etc/cvmfs/keys/spatial-omics.subspatial.org/

Step 3: Build Packages

./build.sh 0.1.1

Repository Configuration Details

Domain Config (domain.d/subspatial.org.conf)

• Shared settings for ALL *.subspatial.org repositories
• Default proxy settings
• Cache quotas
• Timeouts

Repository Config (config.d/<repo>.subspatial.org.conf)

• Repository-specific URLs
• Each repo has its own:
  • Metadata S3 bucket (for CVMFS catalogs)
  • Data S3 bucket (for large external files)
  • Public key directory

Key Structure

• Each repository has its own key directory
• Keys are included IN the packages
• Format: /etc/cvmfs/keys/<repo-name>.subspatial.org/<repo-name>.subspatial.org.pub

Building Packages

Prerequisite: install fpm (https://github.com/jordansissel/fpm), e.g. gem install fpm. On macOS, also install gnu-tar (for DEB) and rpm (for RPM) via Homebrew.

./build.sh [VERSION]

# Examples:
./build.sh 0.1.0    # First release
./build.sh 0.1.1    # Added new repository
./build.sh 0.2.0    # Major update

Package Contents

All packages include:

1. Domain configuration (shared settings)
2. Per-repository configurations
3. Public keys for all configured repositories
4. Documentation and templates

Installation (For End Users)

Red Hat/Rocky/Alma/Fedora:

sudo yum install cvmfs
sudo rpm -ivh cvmfs-config-subspatial-0.1.0-1.noarch.rpm
echo 'CVMFS_REPOSITORIES="cryoet-opendata-poc.subspatial.org"' | sudo tee -a /etc/cvmfs/default.local
sudo cvmfs_config setup
sudo cvmfs_config probe

Debian/Ubuntu:

sudo apt install cvmfs
sudo dpkg -i cvmfs-config-subspatial_0.1.0-1_all.deb
echo 'CVMFS_REPOSITORIES="cryoet-opendata-poc.subspatial.org"' | sudo tee -a /etc/cvmfs/default.local
sudo cvmfs_config setup
sudo cvmfs_config probe

Manual (Tarball):

sudo tar -xzf cvmfs-config-subspatial-0.1.0.tar.gz -C /
echo 'CVMFS_REPOSITORIES="cryoet-opendata-poc.subspatial.org"' | sudo tee -a /etc/cvmfs/default.local
sudo cvmfs_config setup
sudo cvmfs_config probe

Design Philosophy

This follows the EESSI model:

• Domain config: Shared defaults for the domain
• Repository configs: Specific URLs and keys per repo
• Separate S3 buckets: Each repo can use different buckets/regions
• Keys included: No manual key distribution needed

Maintenance

Updating a Repository

1. Update config file in src/etc/cvmfs/config.d/
2. Update public key if needed in src/etc/cvmfs/keys/
3. Rebuild with new version number
4. Distribute updated packages

Adding S3 Mirrors

Edit repository config to add mirrors:

# In src/etc/cvmfs/config.d/<repo>.subspatial.org.conf
CVMFS_SERVER_URL="https://bucket1.s3.region1.amazonaws.com/@fqrn@;https://bucket2.s3.region2.amazonaws.com/@fqrn@"

License

GPLv3

Repository maintained by: subspatial
Last updated: 2026-01-22
CVMFS version tested: 2.13.3
Status: Proof-of-concept (production-ready)