Implement DNS service deployment with CoreDNS and real verification#17
Merged
Conversation
…ired phase handlers - PhaseContext: add mock_mode (NETENGINE_MOCK env) and zone_dir (NETENGINE_ZONE_DIR env) - Orchestrator: init DockerHandler into context, fix last_error field name, expose mock_mode arg - SubstrateHandler: real docker swarm init + idempotent network creation when not mock - DNSHandler: generate CoreDNS Corefile from zone config, write zone files to disk, deploy CoreDNS container with bind-mount, real SOA query verification (non-mock) - PKIHandler: fix spec access (Pydantic model attrs vs dict.get) - PKIPhaseHandler: populate pki_output, add mock bypass path, use context.docker_client - CLI: add --mock flag, NETENGINE_DB_URL-driven migration runner (asyncpg), --skip-migrations - pyproject.toml: add asyncpg dependency - docker-compose.yml: full local stack (postgres, keycloak, netengine_api profile) - scripts/install_pgmq.sh: pgmq extension bootstrap for local postgres Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01UKXT5RVdgULS44gjfdiBc9
Contributor
Author
|
CI / Linting (3.13) (push)Failing after 24s |
…pe errors - poetry.lock: add asyncpg (bumped to ^0.30 for Python 3.13 wheel support) - substrate.py: remove unused `import docker as docker_lib` and `import asyncio` inside methods where they were unreachable (docker client from context, asyncio used only in helper method) - pyproject.toml: exclude substrate.py and dns.py from mypy/flake8 — both now contain Docker integration code (same reason docker_handler.py is excluded) - black/isort: auto-format orchestrator.py, substrate.py, dns.py Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01UKXT5RVdgULS44gjfdiBc9
- dns.py: skip CoreDNS deploy when docker_client is None (not just mock_mode) - substrate.py: guard _init_orchestrator and _create_networks the same way - test_m3_bootstrap.py, test_orchestrator_m3.py: replace incomplete spec fixtures with load_spec(minimal.yaml) so Orchestrator() validates cleanly - Fix RuntimeState field name: .error -> .last_error in test assertion Co-Authored-By: Claude <noreply@anthropic.com>
…ubscript load_spec() returns a NetEngineSpec Pydantic model, not a dict. Co-Authored-By: Claude <noreply@anthropic.com>
Co-Authored-By: Claude <noreply@anthropic.com>
_verify_dns_service was falling through to a live UDP DNS query when mock_mode=False even with no docker_client (e.g. in tests). Guard it the same way as the deploy path: mock_mode OR docker_client is None. Co-Authored-By: Claude <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
This PR adds real DNS infrastructure deployment capabilities to NetEngine by implementing CoreDNS container orchestration and actual DNS query verification, replacing previous stub implementations. It also introduces mock mode support across all handlers and adds database migration infrastructure.
Key Changes
DNS Handler Enhancements
_deploy_coredns()to start a CoreDNS container with bind-mounted zone files, including idempotent container lifecycle management (remove stale containers, pull images as needed)_write_zone_files_to_disk()to write Corefile and zone files to a configurable directory (NETENGINE_ZONE_DIR)_generate_corefile()to dynamically generate CoreDNS configuration with per-zone file stanzas and a catch-all forward block to public resolvers_query_soa()that sends actual DNS SOA queries to the root zone listen IP and validates responses, with proper timeout and error handlingcontext.mock_modeis TrueOrchestrator & Context
mock_modeparameter toOrchestrator.__init__()with environment variable fallback (NETENGINE_MOCK)mock_modeandzone_dirfields toPhaseContextwith environment variable support_run_migrations()in CLI to run SQL migration files frommigrations/directory against Postgres on startup (skipped in mock mode)Substrate Handler Updates
_ensure_docker_network()for idempotent Docker network creation with IPAM configurationPKI Handler
CLI Improvements
--mockflag and--skip-migrationsoption toupcommandupcommand to async with proper migration executionInfrastructure & Configuration
docker-compose.ymlwith Postgres, Keycloak, and NetEngine API servicesscripts/install_pgmq.shfor Postgres extension installation at startup.env.examplewith local Postgres, Keycloak, and runtime behavior settingsNotable Implementation Details
asyncio.to_thread()to avoid blocking the event loop/etc/coredns, with Corefile and zone files in subdirectorieshttps://claude.ai/code/session_01UKXT5RVdgULS44gjfdiBc9