Skip to content

Simple KB Distribution & Installation #65

Closed
12 of 12 issues completed
Closed
Simple KB Distribution & Installation#65
12 of 12 issues completed
Assignees
rucka
Labels
epicWork item representing a large feature or deliverable broken down into user stories.

Description

@rucka
rucka
opened on Oct 25, 2025

Enhanced CLI Packaging & Distribution

Epic Information

Epic ID: EP-001
Initiative: Dynamic Knowledge Base Management #64
Theme: Enterprise Knowledge Management Platform
Epic Owner: Product Engineer
Status: Done
Priority: P0
Epic Points: 18-25 points (estimated)
Target Release: CLI Enhancement Release

Epic Statement

As a developer or organization
I need enhanced CLI packaging capabilities with multiple distribution formats and validation
So that I can easily create, distribute, and install Knowledge Base content in various formats while ensuring quality and consistency

Epic Context

This epic establishes the foundation for dynamic KB management by enhancing the CLI with robust packaging, validation, and distribution capabilities. It enables both individual and organizational KB content creation while preparing for centralized knowledge service integration in subsequent epics.

Business Value & Objectives

Primary Business Goals

  1. Goal 1: Enable flexible KB content packaging and distribution methods
  2. Goal 2: Provide validation and quality assurance for KB content packages
  3. Goal 3: Establish foundation for organizational KB management workflows

Success Metrics (KPIs)

  • Package Creation Success Rate: Target: ≥95% successful KB packages created with validation

    • Measurement method: CLI package creation success/failure tracking with error categorization
    • Current baseline: 0% (no packaging capability exists)

  • Package Installation Success Rate: Target: ≥98% successful KB installations from various sources

    • Measurement method: Installation success tracking across different package sources and formats
    • Current baseline: Current manual installation success rate

  • Validation Effectiveness: Target: ≥90% of validation errors prevent broken KB installations

    • Measurement method: Pre-installation validation vs post-installation issues correlation
    • Current baseline: No automated validation exists

Return on Investment (ROI)

  • Investment Required: 1-2 developers for 2-3 sprints, CLI framework enhancement
  • Expected Revenue Impact: Enables organizational KB workflows leading to enterprise adoption
  • Payback Period: 3-4 months through improved developer experience and enterprise sales enablement
  • Long-term Value: Foundation for all subsequent KB management features and enterprise capabilities

Problem Statement & Market Opportunity

Current State

  • Limited KB distribution methods create friction for organizations and teams
  • No validation or quality assurance for KB content before distribution
  • Manual processes for KB content packaging and installation
  • No standardized format for KB content distribution across different contexts

Target State

  • Multiple flexible distribution formats (remote URLs, local ZIP files, local directories) with consistent validation
  • Automated quality assurance and content validation before distribution
  • Streamlined CLI workflows for both individual and organizational KB management
  • Standardized packaging format that scales from individual use to enterprise deployment

Market Opportunity

  • Market Size: All current pair users plus organizations requiring KB standardization
  • Competitive Advantage: Integrated packaging vs external tools and manual processes
  • Customer Demand: Organizations need standardized, validated KB distribution methods
  • Strategic Importance: Foundation technology enabling all subsequent KB management features

Pain Points Addressed

  1. Pain Point 1: Manual KB content packaging and distribution processes

    • Impact: Time-consuming setup, inconsistent KB installations, manual validation
    • Frequency: Every KB creation and sharing workflow

  2. Pain Point 2: No quality assurance or validation for KB content

    • Impact: Broken installations, inconsistent content quality, debugging overhead
    • Frequency: Critical for every KB package distribution

Target Users & Personas

Primary Users

Persona 1: Individual Developer/Content Creator

  • Demographics: Software developer, 2-8 years experience, creates and shares KB content
  • Goals: Easy KB packaging, reliable distribution, quality validation before sharing
  • Frustrations: Manual packaging processes, broken installations, lack of validation feedback
  • Usage Patterns: Periodic KB creation, sharing with teammates, community contributions

Persona 2: Team Lead/Knowledge Manager

  • Demographics: Senior developer, 5+ years experience, manages team KB standards
  • Goals: Standardized KB packaging, team distribution workflows, quality control
  • Frustrations: Inconsistent KB formats across team, manual quality checks, distribution overhead
  • Usage Patterns: Regular KB standardization, team distribution, quality management

User Journey

KB content creation → Packaging with validation → Distribution (multiple formats) → Installation with verification → Quality feedback and iteration

Solution Overview

Core Solution Approach

Enhance pair CLI with comprehensive packaging capabilities supporting multiple distribution sources (remote URLs, local ZIP files, local directories) with integrated validation, content verification, and quality assurance. Provides foundation for organizational KB management while maintaining developer-friendly workflows.

Key Capabilities

  1. Capability 1: Multi-Source KB Installation with Unified Interface

    • Description: CLI commands to install KB content from remote URLs, local ZIP files, or local directories with unified interface and automatic source detection
    • Value: Consistent, validated KB installation that works across all distribution contexts and supports local development workflows

  2. Capability 2: Comprehensive Content Validation & Quality Assurance

    • Description: Automated validation of KB structure, content quality, link integrity, and metadata before packaging
    • Value: Prevents broken installations and ensures consistent KB quality

  3. Capability 3: Flexible Distribution & Installation Methods

    • Description: Support for URL-based installation (remote and local ZIP) plus direct directory installation, interactive package creation with metadata collection
    • Value: Accommodates various workflow needs from individual use to organizational distribution, plus local development and testing

Solution Differentiators

  • Native CLI integration vs external packaging tools
  • Comprehensive validation vs basic file compression
  • Unified interface for all source types (remote, ZIP, directory)
  • Support for local development workflows with directory installation
  • Foundation for enterprise features vs standalone solution

Epic Breakdown

Themes & Feature Areas

  1. Theme 1: Core Packaging Infrastructure

    • ZIP-based packaging format with metadata and validation
    • Knowledge-hub dataset separation and release management
    • Dual release strategy (CLI + KB dataset) with changeset configuration
    • Package structure standardization and content organization

  2. Theme 2: Validation & Quality Assurance

    • Pre-packaging content validation and structure verification
    • Link integrity checking and broken reference detection
    • Metadata validation and completeness verification
    • Package integrity verification during installation (ZIP and directory)

  3. Theme 3: Distribution & Installation Enhancement

    • Unified installation from remote URLs, local ZIP files, or local directories
    • Interactive package creation with guided metadata collection
    • Enhanced installation workflows with validation feedback
    • Local package management and version tracking

  4. Theme 4: CLI Architecture & Documentation (NEW)

    • Documentation-first approach with single source-of-truth specs
    • CommandConfig contract with discriminated unions for type-safety
    • Pure parser functions with typed validation errors
    • Comprehensive CLI reference and KB source resolution documentation

User Stories (High-Level)

Must-Have Stories (P0)

Should-Have Stories (P1)

Could-Have Stories (P2)

Epic Acceptance Criteria

Technical Considerations

Architecture Overview

Enhance CLI packaging system with ZIP-based format, comprehensive validation pipeline, and multi-source installation capabilities (remote URLs, local ZIP files, local directories). Separate knowledge-hub dataset release from CLI release while maintaining version coordination and default installation workflows. Refactor command parsing with pure functions and discriminated-union contracts for type-safety.

Technical Requirements

  • Performance: Package creation <30 seconds, installation <60 seconds for typical KB
  • Validation: Comprehensive content validation with detailed error reporting
  • Compatibility: ZIP format compatible with Epic 2's knowledge-service
  • Release Management: Dual release strategy with automated coordination
  • Security: Package integrity verification and secure download handling
  • Source Detection: Automatic detection of URL vs ZIP file vs directory
  • Parser Contract: Discriminated-union CommandConfig with typed validation errors
  • Documentation: Single source-of-truth specs for KB source resolution and CLI contracts

Technical Risks

Risk Impact Probability Mitigation Strategy
ZIP format incompatibility with existing workflows Medium Low Thorough testing, backward compatibility layer
Release coordination complexity between CLI and KB High Medium Automated release pipeline, version synchronization
Validation performance with large KB content Medium Medium Incremental validation, caching, performance optimization
Source type ambiguity (ZIP vs directory) Low Low Clear extension-based detection (.zip), fallback to directory check
Directory structure validation complexity Medium Low Define minimal required structure, clear error messages
Parser refactor breaking existing CLI workflows High Medium Pure parser functions, comprehensive test coverage, incremental rollout
Documentation drift between specs and implementation Medium Medium Documentation-first approach, automated spec validation

Infrastructure & Tooling

  • Build Infrastructure: Enhanced build pipeline supporting dual releases
  • Validation Infrastructure: Content analysis tools, link checking, metadata validation
  • Distribution Infrastructure: HTTP client for remote downloads, filesystem operations for local ZIP/directory
  • Release Infrastructure: Coordinated release process, version management, automated testing
  • Documentation Infrastructure: Specs under docs/specs/, CLI reference under docs/cli/, getting-started guides

Design & User Experience

UX Principles

  • Progressive disclosure: Simple defaults (pair install) with advanced flags (--source, --offline) for power users
  • Consistent feedback: All commands produce structured output with clear success/error states and actionable messages
  • Predictable conventions: Flag names follow POSIX conventions; --source behaves like --file in standard CLI tools

Design Requirements

  • Help Text: Every command and flag includes concise --help text with usage examples
  • Error Messages: Structured format: what went wrong, why, and how to fix it (e.g., "Cannot use --offline with remote URL. Provide a local path with --source.")
  • Output Formatting: Machine-parseable (JSON via --json) and human-readable (colored, indented) output modes
  • Accessibility: No color-only signaling; status conveyed through text labels alongside color

Design Deliverables

Timeline & Roadmap

Development Phases

Phase 1: Core Packaging & Release Separation (weeks 1-2)

  • Milestone 1: ZIP packaging infrastructure and validation framework
  • Milestone 2: Knowledge-hub dataset release separation and changeset configuration
  • Key Deliverables: Separate KB releases with CLI coordination

Phase 2: Enhanced Distribution & Installation (weeks 2-3)

  • Milestone 3: Unified installation from remote URLs, local ZIP files, and local directories with package verification
  • Milestone 4: CLI modification for default KB installation from release ZIP
  • Key Deliverables: Complete distribution and installation workflows

Phase 3: CLI Architecture & Documentation (weeks 3-4)

  • Milestone 5: Documentation-first specs (KB source resolution, CLI contracts)
  • Milestone 6: CommandConfig parser refactor with discriminated unions and typed errors
  • Milestone 7: Flexible KB source resolution implementation (--source, --offline) — ✅ DONE
  • Key Deliverables: Type-safe CLI with comprehensive documentation

Phase 4: Validation & Quality Assurance (weeks 4-5)

  • Milestone 8: Comprehensive validation pipeline and interactive package creation
  • Milestone 9: Integration testing and Epic 2 compatibility verification
  • Key Deliverables: Production-ready packaging with quality assurance

Critical Path Dependencies

  • Dependency 1: Build system enhancement for dual releases — ✅ (Story Separate KB Dataset Release #72)

    • Owner: DevOps team
    • Required by: End of week 1
    • Risk level: Medium

  • Dependency 2: CLI framework modification for dynamic KB installation — ✅ (Story CLI Default KB Installation #78)

    • Owner: CLI team
    • Required by: Start of Phase 2
    • Risk level: Medium

  • Dependency 3: Documentation specs for KB source resolution and CLI contracts — ✅ (Story KB Source Documentation & CLI Reference #90)

    • Owner: Product Manager / Product Engineer
    • Required by: Start of Phase 3
    • Risk level: Low

Release Strategy

  • Development Release: Internal testing with sample KB packages
  • Alpha Release: Limited testing with real KB content and dual release process
  • Beta Release: Broader testing with various KB formats and distribution methods
  • Full Launch: Complete enhanced packaging capabilities with documentation

Risk Management

Business Risks

Risk Impact Probability Mitigation Strategy Owner
Low adoption of packaging workflow over manual copy Medium Medium Frictionless defaults, migration guide, clear value demonstration Product Engineer
Enterprise customers require signed packages before adoption High Low Design package format to support future signing; document roadmap Product Engineer

Technical Risks

Risk Impact Probability Mitigation Strategy Owner
ZIP format incompatibility with existing workflows Medium Low Thorough testing, backward compatibility layer CLI team
Release coordination complexity between CLI and KB High Medium Automated release pipeline, version synchronization DevOps team
Validation performance with large KB content Medium Medium Incremental validation, caching, performance optimization CLI team
Source type ambiguity (ZIP vs directory) Low Low Extension-based detection (.zip), fallback to directory check CLI team
Parser refactor breaking existing CLI workflows High Medium Pure parser functions, comprehensive test coverage, incremental rollout CLI team
Documentation drift between specs and implementation Medium Medium Documentation-first approach, automated spec validation Product Engineer

Resource Risks

Risk Impact Probability Mitigation Strategy Owner
Single developer bottleneck on CLI internals High Medium Document parser contract early, enable parallel work on validation vs parser Product Engineer
Competing priorities with other initiative epics Medium Medium P0 stories scoped first, P1/P2 deferred without blocking downstream epics Product Engineer

Success Validation

Launch Criteria

  • Functionality: All must-have stories completed and tested — ✅ All P0/P1/P2 stories done
  • Quality: Comprehensive testing of dual release process and KB coordination — ✅ Quality gates pass
  • Documentation: Complete CLI documentation, packaging guides, validation references (specs + user docs) — ✅ (Story KB Source Documentation & CLI Reference #90)
  • Compatibility: Full compatibility with Epic 2's knowledge-service requirements — ✅ (Stories Package KB Content #73, Organizational KB Packages #77)
  • Performance: Meets performance targets for packaging and installation operations — ✅ Validated through story implementations
  • Type Safety: CommandConfig contract enforced, parser validates inputs with typed errors — ✅ (Story CommandConfig Contract & Parser Refactoring #91)

Post-Launch Success Metrics

  • Week 1: ≥90% successful KB package creation and installation in testing
  • Month 1: Dual release process operational with automated coordination
  • Quarter 1: Foundation supports Epic 2 knowledge-service development

Feedback & Iteration Plan

  • User Feedback Collection: Developer surveys, packaging workflow analysis, validation effectiveness
  • Analytics and Monitoring: Package creation success rates, installation patterns, validation metrics
  • Iteration Cycles: Bi-weekly improvements based on usage data and Epic 2 requirements
  • Success Review: Monthly assessment of packaging adoption and quality improvements

Communication Plan

Stakeholders

  • Epic Owner: Product Engineer — technical delivery, requirements, and implementation
  • Initiative Owner: Product Engineer — strategic alignment with Initiative Dynamic Knowledge Base Management #64
  • Contributors: CLI team members — implementation of parser, packaging, and validation

Communication Rhythm

  • Epic Kickoff: Alignment on scope, P0 vs P1/P2 boundaries, and dependency order
  • Per-Story Updates: Story status updated in GitHub Project Adoption Management & Guideline Linking #3 on transition (To Do → In Progress → Done)
  • Milestone Reviews: End-of-phase check against Epic AC and downstream epic readiness
  • Launch Communication: Release notes covering new commands, flags, and migration steps

Appendix

Research & Validation

  • Source detection logic validated against POSIX path conventions and URL standards
  • CLI flag design reviewed against clig.dev guidelines

Design Assets

  • Source Detection Logic flowchart: see "Source Detection Logic" in Release Management Specifications below
  • CLI command examples: see "CLI Modification for Dataset Separation" below

Related Documentation

Change Log

Date Change Reason Updated By
2025-01 Epic created Initial planning Product Engineer
2025-02 Added stories #90, #91, #92; Theme 4 (CLI Architecture) CLI refactor scope expansion Product Engineer
2025-02 Marked #94 superseded by Epic #93 S9 Docs restructure moved to dedicated epic Product Engineer
2026-02-14 Rewritten to full epic-template compliance (14 sections) Structural alignment with epic-template.md Product Engineer
2026-02-14 Marked #92 complete, AC4 and AC9 done Story #92 merged via PR #133 Product Engineer
2026-02-16 Marked #76 complete Story #76 merged via PR #138 (squash) Product Engineer
2026-02-17 Marked #77 complete Story #77 merged via PR #140 (squash) — org KB packaging + kb-info command Product Engineer
2026-02-17 Epic DONE — all checkboxes reconciled All 12 stories closed (11 completed, 1 not_planned). Checked: #90, #91, AC1-AC8, Design Deliverables, Dependencies, Launch Criteria. Status → Done Product Engineer

Release Management Specifications

Dual Release Strategy

# Enhanced changeset configuration
releases:
  cli:
    package: "@foomakers/pair-cli"
    assets:
      - type: "binary"
        platforms: ["linux", "macos", "windows"]

  knowledge-base:
    package: "@foomakers/knowledge-hub"
    assets:
      - type: "zip"
        name: "pair-kb-v{version}.zip"
        content: "dataset/"
        validation: true

CLI Modification for Dataset Separation

# CLI behavior changes (updated with new flags)
pair install                                    # Downloads KB from default source (monorepo or release ZIP)
pair install --source <source>                  # Installs from explicit source
pair install --source <url> --offline           # ERROR: cannot use --offline with remote URL
pair install --source <path> --offline          # Installs from local path, no network access
pair kb validate                                # Validates current KB content
pair config validate                            # Validates config.json and registries

# Source examples:
# Remote URL (download + extract)
pair install --source https://github.com/org/repo/releases/download/v1.0.0/kb.zip

# Local ZIP file (extract)
pair install --source /absolute/path/to/kb.zip
pair install --source ./relative/path/to/kb.zip

# Local directory (copy directly)
pair install --source /absolute/path/to/dataset/
pair install --source ./relative/path/to/dataset/
pair install --source ../shared/kb-dataset/

# Offline mode (requires local source)
pair install --source ./local-kb.zip --offline
pair install --source /usr/share/kb/ --offline

# Package creation
pair kb package                     # Creates ZIP from current KB
pair kb package --interactive       # Guided package creation

Source Detection Logic

if --offline flag present and --source is remote URL
  → Error: "Cannot use --offline with remote URL source"
else if --offline flag present and --source not provided
  → Error: "Offline mode requires explicit --source with local path"
else if source starts with 'http://' or 'https://'
  → Remote URL: download, extract, install
else if source ends with '.zip'
  → Local ZIP: extract, install
else if source is directory path
  → Local directory: validate structure, copy, install
else if no --source provided
  → Default source: monorepo dataset or GitHub release ZIP
else
  → Error: invalid source format

Version Coordination

  • CLI release includes KB version reference
  • Automatic KB download on CLI installation (unless --offline or --source provided)
  • Version compatibility matrix
  • Graceful handling of KB version mismatches

Epic Link: Enhanced CLI Packaging & Distribution

Metadata

Metadata

Assignees

Labels

epicWork item representing a large feature or deliverable broken down into user stories.

Type

No type

Projects

No projects

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions