Skip to content

Add USB Mass Storage host support infrastructure for Hekate - #1

Draft
nafields with Copilot wants to merge 8 commits into
masterfrom
copilot/add-usb-mass-storage-support
Draft

Add USB Mass Storage host support infrastructure for Hekate#1
nafields with Copilot wants to merge 8 commits into
masterfrom
copilot/add-usb-mass-storage-support

Conversation

Copilot AI commented Oct 29, 2025

Copy link
Copy Markdown

Implements foundational USB host mode to enable reading from USB mass storage devices (flash drives, SSDs) connected via USB-C. Core transfer implementations remain incomplete pending hardware testing.

Infrastructure

  • xHCI host controller (bdk/usb/xhci.c/h) - Controller init, ring management, slot allocation. Transfer functions stubbed for future completion.
  • MSC Bulk-Only Transport (bdk/usb/usb_msc_host.c/h) - SCSI command layer (TEST_UNIT_READY, INQUIRY, READ_CAPACITY, READ/WRITE). Relies on xHCI bulk transfers.
  • Storage integration (nyx/nyx_gui/emummc_storage_usb.c/h) - Block device interface with 64-sector cache, read-only default mode.
  • Clock support - Added clock_enable_xusb_{host,ss,fs}() for XUSB controller clocks.

Safety & Configuration

Default read-only with explicit usb_write_enable flag required for writes. Cache gracefully degrades on allocation failure. All magic numbers replaced with named constants (USB_DEFAULT_PORT, SECTOR_SIZE_BYTES, etc.).

Architecture

USB host available through Nyx only (not raw bootloader). Single device support, no hub enumeration. Polling mode for simplicity.

Remaining Work

Transfer implementation requires:

  • Control/bulk TRB queueing and event ring processing (~200 LOC)
  • USB descriptor parsing for endpoint extraction (~100 LOC)
  • PHY initialization (UTMI/SuperSpeed) (~150 LOC)
  • Hardware validation and device quirks

Documentation

  • README_USB_STORAGE.md - Usage, safety warnings, troubleshooting
  • IMPLEMENTATION_STATUS.md - Technical status and TODO tracking
  • DEVELOPMENT_SUMMARY.md - Architecture decisions and metrics
  • CODE_REVIEW_SUGGESTIONS.md - Comprehensive code review with 20 prioritized items
  • TESTING_FRAMEWORK.md - Complete testing strategy and procedures
  • MISSING_ITEMS.md - Detailed checklist of remaining work with estimates

Testing & CI/CD

GitHub Actions Workflows:

  • .github/workflows/build-check.yml - Automated build verification with devkitARM, validates USB object files are compiled
  • .github/workflows/static-analysis.yml - Code quality checks (cppcheck, documentation validation, header guards)

Testing Materials:

  • Device compatibility matrix for tracking tested devices
  • Performance benchmarking templates
  • Manual testing checklists for basic functionality
  • Build verification and code quality scripts

Code Review Summary

Comprehensive review identified 20 items organized by priority:

  • P0 (Critical): PHY init, transfer implementation, event processing (~650 LOC to MVP)
  • P1 (High): Descriptor parsing, firmware loading (~250 LOC)
  • P2 (Medium): Timeout handling, memory alignment, error codes, logging (~320 LOC)
  • P3 (Low): Device quirks, hot-plug, performance optimizations (~840 LOC)

Path to Minimum Viable Product requires completing P0 and P1 items (~900 LOC total).

Original prompt

Implement USB mass storage support for Hekate using the following steps:

  1. Tegra XUSB Host Bring-Up:

    • Initialize the USB host controller in a minimal bare-metal form.
    • Configure clocks and the PHY.
    • Load the xusb.bin firmware if required.
    • Set the controller into host mode.
  2. xHCI/HCD Subset:

    • Implement the xHCI/HCD subset to enumerate a single Mass Storage Class (MSC) device. Avoid hub support initially.
  3. MSC BOT Transport and SCSI:

    • Implement MSC Bulk-Only Transport (BOT).
    • Add a minimal SCSI layer to support the following commands:
      • TEST UNIT READY
      • INQUIRY
      • READ CAPACITY (10/16)
      • READ/WRITE (10/12)
      • REQUEST SENSE
  4. Block Device Veneer:

    • Provide a block device interface with the following functions:
      • Read: read(lba, count)
      • Write: write(lba, count)
    • Integrate this into Hekate as emummc_storage_usb.c.
  5. Bootloader Policy:

    • Require connecting through a powered USB hub.
    • Start in read-only mode for safety.
    • Allow write operations only after extended testing and enabling an explicit flag.
  6. Performance Enhancements:

    • Add a small sector cache and implement read-ahead for boot partitions to reduce latency.
  7. Testing Matrix:

    • Test with various USB devices, including:
      • USB thumb drives
      • Powered SSD enclosures
    • Test scenarios such as:
      • Device replug during early boot
      • Read/write stress tests
      • Handling of unaligned 4K sectors
  8. Compatibility Improvements:

    • Map behavior from U-Boot/Linux and add quirks for popular USB controllers.

This pull request was created as a result of the following prompt from Copilot chat.

Implement USB mass storage support for Hekate using the following steps:

  1. Tegra XUSB Host Bring-Up:

    • Initialize the USB host controller in a minimal bare-metal form.
    • Configure clocks and the PHY.
    • Load the xusb.bin firmware if required.
    • Set the controller into host mode.
  2. xHCI/HCD Subset:

    • Implement the xHCI/HCD subset to enumerate a single Mass Storage Class (MSC) device. Avoid hub support initially.
  3. MSC BOT Transport and SCSI:

    • Implement MSC Bulk-Only Transport (BOT).
    • Add a minimal SCSI layer to support the following commands:
      • TEST UNIT READY
      • INQUIRY
      • READ CAPACITY (10/16)
      • READ/WRITE (10/12)
      • REQUEST SENSE
  4. Block Device Veneer:

    • Provide a block device interface with the following functions:
      • Read: read(lba, count)
      • Write: write(lba, count)
    • Integrate this into Hekate as emummc_storage_usb.c.
  5. Bootloader Policy:

    • Require connecting through a powered USB hub.
    • Start in read-only mode for safety.
    • Allow write operations only after extended testing and enabling an explicit flag.
  6. Performance Enhancements:

    • Add a small sector cache and implement read-ahead for boot partitions to reduce latency.
  7. Testing Matrix:

    • Test with various USB devices, including:
      • USB thumb drives
      • Powered SSD enclosures
    • Test scenarios such as:
      • Device replug during early boot
      • Read/write stress tests
      • Handling of unaligned 4K sectors
  8. Compatibility Improvements:

    • Map behavior from U-Boot/Linux and add quirks for popular USB controllers.

✨ Let Copilot coding agent set things up for you — coding agent works faster and does higher quality work when set up for your repo.

Copilot AI and others added 6 commits October 29, 2025 22:12
Co-authored-by: nafields <2613057+nafields@users.noreply.github.com>
Co-authored-by: nafields <2613057+nafields@users.noreply.github.com>
Co-authored-by: nafields <2613057+nafields@users.noreply.github.com>
Co-authored-by: nafields <2613057+nafields@users.noreply.github.com>
Co-authored-by: nafields <2613057+nafields@users.noreply.github.com>
Co-authored-by: nafields <2613057+nafields@users.noreply.github.com>
Copilot AI changed the title [WIP] Implement USB mass storage support for Hekate Add USB Mass Storage host support infrastructure for Hekate Oct 29, 2025
Copilot AI requested a review from nafields October 29, 2025 22:28
@nafields

Copy link
Copy Markdown
Owner

@copilot review this. Suggests edits. Thing we may have missed. And a GitHub testing framework. Based on documentation and actions you may be aware in other repos

Co-authored-by: nafields <2613057+nafields@users.noreply.github.com>

Copilot AI commented Oct 29, 2025

Copy link
Copy Markdown
Author

@copilot review this. Suggests edits. Thing we may have missed. And a GitHub testing framework. Based on documentation and actions you may be aware in other repos

Comprehensive review complete! I've created detailed documentation covering all your requests:

📋 Code Review (CODE_REVIEW_SUGGESTIONS.md)

20 items identified with priority levels (P0-P3):

  • P0 Critical: PHY init, transfer implementation, event processing (~650 LOC to MVP)
  • P1 High: Descriptor parsing, firmware loading (~250 LOC)
  • P2 Medium: Timeout handling, memory alignment, error codes, logging (~320 LOC)
  • P3 Low: Device quirks, hot-plug, performance opts (~840 LOC)

Each item includes code examples and implementation guidance.

🧪 Testing Framework (TESTING_FRAMEWORK.md)

GitHub Actions workflows created:

  1. .github/workflows/build-check.yml - Automated build with devkitARM, verifies USB objects
  2. .github/workflows/static-analysis.yml - cppcheck, documentation checks, header guards

Manual testing materials:

  • Device compatibility matrix (flash drives, SSDs, HDDs)
  • Performance benchmarking templates
  • Basic functionality checklists
  • Test scripts for build verification

🔍 Missing Items (MISSING_ITEMS.md)

Complete checklist of 20 remaining items with:

  • Estimated line counts per item
  • Priority assignments
  • Implementation references
  • Success criteria

Path to MVP: ~900 LOC needed for P0+P1 items (PHY, transfers, events, descriptors)

✅ What's Working

  • Clean foundation architecture
  • Safety features (read-only default)
  • Build system integration
  • Comprehensive documentation (6 guides, 1500+ lines)

All materials committed in 8f5aeb4. The workflows will run automatically on future pushes.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants