Skip to content

Harden proxy integration tests: remove dependency on external httpbin.org / api.ipify.org (flaky) #178

Description

@iliyan-velichkov

Problem

Two integration tests depend on external third-party services and intermittently fail when those services are slow, rate-limited, or temporarily down:

  • AirflowPerspectiveIT.testPerspective → proxies to http://httpbin.org and asserts an <h2> containing "httpbin.org" renders in the iframe.
  • ProxyIT.textProxyPath → proxies to https://api.ipify.org.

This bit us in #177 (parent bump 13.1.0 → 13.2.0): the pull-request / integration-tests check went red, but the failure had nothing to do with the change. The CI screenshot showed the proxied iframe rendering 503 Service Temporarily Unavailable — httpbin.org was transiently down. The proxy worked correctly; it just forwarded the upstream 503. A plain re-run went green once httpbin.org recovered.

Because these failures are indistinguishable at a glance from real regressions, every dependabot/parent-bump PR carries a recurring "is this real or just httpbin flaking again?" investigation cost.

Proposed hardening

Remove the dependency on external hosts from these tests. Options (in rough order of preference):

  1. Local stub server — point AppConfig.AIRFLOW_URL at a lightweight in-test HTTP stub (e.g. WireMock / an embedded handler) that serves a known HTML page (<h2>httpbin.org</h2>) and the asset/redirect responses the proxy rewriters care about. This makes the test fully hermetic and also lets us assert the rewriting behavior deterministically rather than relying on httpbin's Swagger UI.
  2. Retry-on-transient-error — if we want to keep exercising a real external site, wrap the navigation/assertion so a transient upstream 5xx triggers a reload (bounded by a timeout) instead of a hard fail. Weaker, still externally dependent.

Option 1 is preferred: it removes flakiness entirely and strengthens what the proxy tests actually verify (TextResponseBodyRewriter / RelativeLocationHeaderRewriter path rewriting).

Acceptance criteria

  • AirflowPerspectiveIT and ProxyIT pass without reaching any external network host.
  • The proxy's path/URL rewriting is asserted against controlled stub responses.

Context

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type
    No fields configured for issues without a type.

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions