Implement parallel workers as default for HTML validation with progress tracking

[Copilot]

Addresses issue #143 by making parallel processing the default behavior for HTML validation with visual progress tracking.

🚀 Key Features

Default Parallel Processing

• Always On: Parallel processing with worker threads is now the default behavior (no flags or environment variables needed)
• 4 Workers: Uses 4 parallel workers by default, configurable via HTML_VALIDATE_WORKERS environment variable
• Automatic Load Balancing: Work is distributed across available workers for optimal performance

Enhanced Progress Visualization

• MultiBar Progress: Individual progress bars for each worker showing current file being processed
• Overall Progress: Combined progress bar tracking total completion across all workers
• Real-time Updates: Live status updates showing which files each worker is processing

Performance Benefits

Multi-threading provides performance improvements for:

• Large numbers of HTML files
• I/O-bound operations (external link checking)
• CPU-intensive validation rules
• Scalable processing on multi-core systems

🛠️ Implementation

The system automatically uses parallel processing when validating HTML files:

# Regular test now uses parallel processing by default
yarn test

# Manual validation also runs in parallel
yarn node test/build-html-validate.mjs

Visual Demo

When running with multiple files, you'll see parallel processing in action:

🧪 Validating 8 files with 4 parallel workers...
Worker #1 [████████████████████████████████████████] 100% | 2/2 | Complete
Worker #2 [████████████████████████████████████████] 100% | 2/2 | Complete  
Worker #3 [████████████████████████████████████████] 100% | 2/2 | Complete
Worker #4 [████████████████████████████████████████] 100% | 2/2 | Complete
Worker Overall [████████████████████████████████████████] 100% | 8/8 | 8/8 completed

📊 Results Summary:
✅ 8 files passed validation
✨ All tests passed!

Workers include a temporary 1-second delay for testing purposes to visualize the parallel processing. This will be removed after testing.

The implementation significantly improves the testing experience and handles larger sites efficiently with perfect load balancing across workers.

---

💬 Share your feedback on Copilot coding agent for the chance to win a $200 gift card! Click here to start the survey.

[fulldecent]

@copilot good progress, but the purpose of the multi parallel mode is to be default.

Remove the feature where you are creating 100 copies of the file. Remove the single threaded runner. Remove the multi threaded file that you spammed into the project directory.

In the worker, add a one second delay before returning the results as a message. This will allow me to manually play with it a little bit and then we will remove that when I merge

[Copilot]

@copilot good progress, but the purpose of the multi parallel mode is to be default.

Remove the feature where you are creating 100 copies of the file. Remove the single threaded runner. Remove the ...

Implemented all requested changes in commit 90b9735:

✅ Made parallel processing the default - Removed environment variables and flags, parallel mode now runs automatically
✅ Removed 100-file generation feature - Deleted setup-multithread-test.mjs and valid-test-page.html
✅ Removed single-threaded runner - Deleted the sequential validation fallback, only parallel mode remains
✅ Removed separate multi-threaded file - Deleted build-html-validate-parallel.mjs from project directory
✅ Added 1-second delay in worker - Workers now pause 1 second before returning results for testing visualization
✅ Cleaned up package.json - Removed all demo scripts, keeping only the main test script

The HTML validation now runs with 4 parallel workers by default, showing individual progress bars for each worker with the 1-second delay allowing you to see the parallel processing in action.

Warning

Firewall rules blocked me from connecting to one or more addresses

I tried to connect to the following addresses, but was blocked by firewall rules:

• api.PacificMedicalTraining.com
  • Triggering command: curl --head --silent --max-time 5 --max-redirs 0 --write-out %{http_code} --dump-header - --output /dev/null REDACTED (dns block)
• data.jsdelivr.com
  • Triggering command: curl --silent --fail --location REDACTED (dns block)
• en.wikipedia.org
  • Triggering command: curl --head --silent --fail --max-time 10 --location REDACTED (dns block)

If you need me to access, download, or install something from one of these locations, you can either:

• Configure Actions setup steps to set up my environment, which run before the firewall is enabled
• Add the appropriate URLs or hosts to the custom allowlist in this repository's Copilot coding agent settings (admins only)