Skip to content

Hermetic DNS/HTTP tests + resolve4/6 wire-protocol migration (#495)#869

Merged
nickna merged 2 commits into
mainfrom
wrk/issue-495-live-network-hermeticity
Jun 21, 2026
Merged

Hermetic DNS/HTTP tests + resolve4/6 wire-protocol migration (#495)#869
nickna merged 2 commits into
mainfrom
wrk/issue-495-live-network-hermeticity

Conversation

@nickna

@nickna nickna commented Jun 21, 2026

Copy link
Copy Markdown
Owner

Closes #495.

Problem

A few tests made real outbound DNS/HTTP calls. On a flaky CI runner the in-flight request keeps the event loop alive (the #320 mechanism) until the 30s test timeout → false-red; and the matrix's default fail-fast then cancels the other two platforms (1 flake = 3 red jobs). A prior Skip.If(output.Length==0) band-aid masked genuine no-output regressions and didn't even catch the real hang mode.

Goal: every test is either fully hermetic or explicitly tagged LiveNetwork and excluded from CI — enforced by a guardrail meta-test — and the Skip band-aid is gone.

Two commits: A (test-infra, no runtime change) and B (a Node/c-ares-accurate runtime semantics change).

Commit A — hermetic test infrastructure

  • FetchReturnsPromise → local MockHttpServer instead of example.com.
  • Deleted the two redundant DnsRecordTypeTests.*_InvalidDomain_* (already covered deterministically by DnsFakeServerModuleTests NXDOMAIN tests).
  • Tagged the genuinely-live tests (google.com LiveSmoke + invalid-hostname lookup) [Trait("Category","LiveNetwork")]; removed the Skip.If(output.Length==0) guards.
  • New Infrastructure/LiveNetworkHosts.cs — the single sanctioned home for external host literals.
  • New CompilerTests/LiveNetworkHermeticityTests.cs guardrail (modeled on StandaloneDllTests): fails the build if an inline external-network literal (a non-loopback fetch('http…'), google.com, or a reserved *.example name) appears outside that file, and that any file using a LiveNetworkHosts constant is LiveNetwork-tagged. Deliberately keyed on the network-call pattern, not bare example.com (which has ~120 legit hermetic uses: URL parsing, Request/Response, Agent.getName, loopback DNS tests).
  • CI: strategy.fail-fast: false + dotnet test --filter "Category!=LiveNetwork".

Commit B — resolve4/resolve6 → DNS wire protocol

resolve4/resolve6 (and resolve() with the default/A/AAAA rrtype) used the OS resolver (Dns.GetHostEntry/getaddrinfo), which reads the hosts file and has no SHARPTS_DNS_SERVER redirect seam — making them non-hermetic and divergent from Node (where resolve* uses c-ares). They now route through the existing emitted DNS wire protocol (the A/AAAA parse already existed in both modes).

  • Interpreter (DnsRecordResolver, DnsModuleInterpreter.ResolveAddresses) + the three compiled emit sites in RuntimeEmitter.Dns.cs; removed the now-dead EmitDnsResolveAddresses. No new SharpTS.dll late-binding (the wire-protocol path is already pure emitted IL).
  • Widened the resolve4/resolve6 callback catch from SocketException to Exception (the wire protocol throws Exception("Runtime Error: dns.x ECODE host")).
  • reverse/lookup/lookupService intentionally stay on getaddrinfo (Node parity).
  • Tests for resolve4/resolve6 now run against a loopback FakeDnsServer (SHARPTS_DNS_SERVER / setServers) with exact-value assertions, both runtimes.

Behavior change (documented in docs/node-modules-api.md + STATUS-NODE.md): resolve4('localhost') now returns ENOTFOUND rather than 127.0.0.1 — use dns.lookup for hosts-file resolution. Matches Node.

Verification

  • Guardrail plant-tested: failed on an injected fetch('https://google.com'), green on revert; confirmed it does not flag the existing legit example.com lines.
  • Green (Debug + Release, 0 warnings): 132 DNS tests (both modes, incl. the nested-callback test in compiled mode), 399 Release DNS/HTTP/Fetch/guard with the CI filter, StandaloneDll (29), ILVerification + EventLoop + RuntimeTypeSync + EmitterSync (102).
  • Confirmed exactly 7 tests are tagged LiveNetwork and excluded by --filter "Category!=LiveNetwork".

Follow-up (out of scope, tracked separately)

A fail-fast timeout on the remaining getaddrinfo path (lookup/lookupService/reverse) — after this PR there's no getaddrinfo hang surface left in CI, so it's deferred as a product-robustness hardening.

nickna added 2 commits June 20, 2026 16:30
…ind a trait

A handful of tests made real outbound DNS/HTTP calls. On a flaky CI runner the
in-flight request keeps the event loop alive to the 30s test timeout (false-red),
and the matrix's default fail-fast then cancels the other platforms. A prior
Skip.If(output.Length==0) band-aid masked genuine no-output regressions.

- FetchReturnsPromise: hit the local MockHttpServer instead of example.com.
- DnsRecordTypeTests: delete the two *_InvalidDomain_* tests (already covered
  deterministically by DnsFakeServerModuleTests NXDOMAIN); tag the google.com
  LiveSmoke tests [Trait Category=LiveNetwork]; drop the Skip-on-empty guard.
- DnsModuleTests: tag the two invalid-hostname lookup tests LiveNetwork
  (dns.lookup uses getaddrinfo - no fake-server seam).
- LiveNetworkHosts.cs (new): the single sanctioned home for external host literals.
- LiveNetworkHermeticityTests.cs (new): guardrail meta-test - fails the build if an
  inline external literal appears outside that file, or a file using it isn't tagged.
- CI: strategy.fail-fast: false; Test step --filter "Category!=LiveNetwork".
- DnsFakeServerModuleTests: answer A/AAAA from the fake server (used by the resolve4
  migration in the next commit).

Refs #495
…semantics)

resolve4/resolve6 (and resolve() with the default/A/AAAA rrtype) previously used
the OS resolver (Dns.GetHostEntry / getaddrinfo), which reads the hosts file and
has no SHARPTS_DNS_SERVER redirect seam. That made them non-hermetic in tests and
diverged from Node, where resolve* uses c-ares (querying the configured DNS server,
not the hosts file). They now go through the existing emitted DNS wire protocol.

Behavior change: resolve4('localhost') no longer returns 127.0.0.1 from the hosts
file — it queries the DNS server and typically returns ENOTFOUND, matching Node.
Use dns.lookup for hosts-file resolution (unchanged — still getaddrinfo). reverse,
lookup, and lookupService also stay on getaddrinfo (Node parity).

- Interpreter: DnsRecordResolver.ResolveA/ResolveAaaa and the custom-server A/AAAA
  cases route to DnsWireProtocol.Query; DnsModuleInterpreter.ResolveAddresses too.
  Widened the resolve4/resolve6 callback catch from SocketException to Exception
  (the wire protocol throws Exception("Runtime Error: dns.x ECODE host");
  ExtractErrorCode already parses both shapes).
- Compiled: the three A/AAAA emit sites in RuntimeEmitter.Dns.cs (DnsResolveRecord,
  DnsWrapper_resolve4/6, DnsWrapper_resolve) route through the emitted wire protocol
  (DnsDoQuery); removed the now-dead EmitDnsResolveAddresses helper. No new
  SharpTS.dll late-binding — DnsResolveRecord is already pure emitted IL.
- Tests: DnsAsyncTests + DnsResolverTests resolve4/resolve tests now run against a
  loopback FakeDnsServer (SHARPTS_DNS_SERVER / setServers) with exact-value
  assertions, in both runtimes. dns.lookup/reverse tests stay on loopback.
- Docs: note the resolve* vs lookup distinction (node-modules-api.md, STATUS-NODE.md).

Refs #495
@nickna nickna merged commit 5bdd70a into main Jun 21, 2026
3 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Test infra: live-network DNS/HTTP tests cause false-red CI (hang to 30s timeout when runner has no network)

1 participant