Skip to content

Port from Lwt to Eio#59

Open
mtelvers wants to merge 3 commits into
mirage:masterfrom
mtelvers:eio
Open

Port from Lwt to Eio#59
mtelvers wants to merge 3 commits into
mirage:masterfrom
mtelvers:eio

Conversation

@mtelvers
Copy link
Copy Markdown

Replaces the Lwt-based runtime with direct-style Eio. The bulk of the diff is removing Lwt.t from every signature, but a few things are worth noting.

API changes:

  • CollectorRegistry.collect now returns a snapshot directly rather than a snapshot Lwt.t. Collectors and pre-collect hooks may perform IO via effects, so the parallel _lwt-suffixed API is gone: register_lwt, register_pre_collect_lwt and the internal metrics_lwt / pre_collect_lwt fields are all removed.
  • Gauge.track_inprogress, Gauge.time, Summary.time and Histogram.time take a plain (unit -> 'a) and return 'a. Internally they use Fun.protect so the gauge/observation is still updated if the inner function raises. This is covered in a new regression test.
  • Prometheus_app.Cohttp functor is replaced by a single top-level Prometheus_app.callback for cohttp-eio (no transport parameterisation needed).
  • Prometheus_unix.serve returns a (unit -> unit) list of fiber bodies. The caller composes them with Eio.Fiber.all / Fiber.fork rather than passing a switch. Each fiber manages its own listening socket via an internal switch and shuts down cleanly on cancellation. New optional ?backlog and ?addr parameters let callers tune the listen queue or bind to IPv6.
  • Prometheus_unix.Logging.init now requires ~clock:_ Eio.Time.clock. The Logs reporter sources timestamps from Eio.Time.now instead of Unix.gettimeofday, so it must be called from inside Eio_main.run.
  • New top-level Prometheus_unix.init ~clock () replaces the implicit module-init registration of process_start_time_seconds. The metric's value is captured from the supplied clock when init is called. Callers must invoke this once, inside Eio_main.run.

Package deps and constraints:

  • prometheus: ocaml >= 4.08 (was 4.01.0) needed for Fun.protect.
  • prometheus-app: ocaml >= 5.0, depends on cohttp-eio and eio instead of cohttp-lwt / cohttp-lwt-unix / lwt; alcotest-lwt dropped from test deps.

Tests:

  • The former "Lwt collectors" test is renamed and now uses Eio.Fiber.yield () in place of Lwt.pause () to exercise a collector that suspends.
  • New "Gauge track_inprogress on raise" test confirms the finaliser semantics of the new Fun.protect-based implementation.

Example:

  • examples/example.ml uses a while-loop counter, calls Eio.Fiber.all over Prometheus_unix.serve.

The interpretation of "thread-safe" in the docs is qualified: safe under OCaml 5 effect-based schedulers running on a single domain, NOT under multi-domain parallelism.

Replaces the Lwt dependency throughout with direct-style Eio. As flagged in
the v1.2 CHANGES entry, the Lwt collectors were a temporary shim for pre-5.0
OCaml; with OCaml 5 and effects they can go away entirely.

- CollectorRegistry.collect is now synchronous. Collectors may still perform
  IO via effects (e.g. Eio operations).
- Drop register_lwt / register_pre_collect_lwt and the separate metrics_lwt /
  pre_collect_lwt state; a single register / register_pre_collect pair serves
  both sync and effectful collectors.
- Gauge.track_inprogress and the *.time helpers take (unit -> 'a) -> 'a and
  use Fun.protect, so the gauge/observation is updated even when the inner
  function raises.
- Prometheus_app: the Cohttp(Server) functor becomes a top-level callback
  suitable for Cohttp_eio.Server.make.
- Prometheus_unix.serve takes ~sw and ~net explicitly and forks the server
  onto the caller's switch.
- Bump ocaml lower bound to 5.0. Drop lwt, cohttp-lwt, cohttp-lwt-unix,
  alcotest-lwt. Add eio, cohttp-eio, eio_main (tests/example).
- Prometheus_unix.serve now returns a (unit -> unit) list and creates
  its own switch internally; callers compose with Eio.Fiber.all/fork
  rather than passing ~sw.
- Expose ?backlog and ?addr on serve so users can bind to IPv6/dual
  stack or tune the listen queue.
- Example uses while-loop counter, idiomatic Term.(const $ ...)
  cmdliner spelling, and Eio.Fiber.all to express concurrency at the
  call site.
- Fix cohttp-eio log source name: 'cohttp.eio' -> 'cohttp.eio.io'.
- Relax prometheus.opam ocaml floor from >= 5.0 to >= 4.08 (Fun.protect
  is the only stdlib-version-sensitive call; the core no longer needs
  Eio).
Eliminate the two remaining Unix.gettimeofday calls in
Prometheus_unix:

  - process_start_time_seconds was captured at module-init time. It
    now comes from Eio.Time.now clock, captured the first time
    Prometheus_unix.init ~clock () is called. The metric is no longer
    auto-registered at module load.
  - The Logs reporter installed by Logging.init used to call
    Unix.gettimeofday for every log line; it now closes over a clock
    and calls Eio.Time.now.

Both Logging.init and the new top-level init require ~clock and must
be called from inside Eio_main.run, where a clock capability is
available. This matches the Eio idiom (capabilities passed
explicitly, no global resources) and matches feedback talex5 gave on
the obuilder Eio port: 'plumb a clock through; useful for tests
anyway.'

Migration for downstream callers is mechanical: move the toplevel
Prometheus_unix.Logging.init () into the body of Eio_main.run, add
~clock, and follow with Prometheus_unix.init ~clock ().
@dinosaure
Copy link
Copy Markdown
Member

I'm not sure that replacing lwt with eio is the right approach. You should certainly take a look at the work we’ve done regarding Let's Encrypt, which involves abstracting away the scheduler in order to continue supporting lwt but also to provide the option of using another scheduler such as Miou (as, to date, the advantage of using eio over other schedulers is not clear-cut).

As for the library, this would therefore involve abstracting the monad so as to retain support for lwt and be able to define the monad as type 'a t = 'a for eio and Miou.

Finally, as for the application, I’m not sure it's fundamentally necessary to use eio; is there a real practical advantage over lwt?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants