-
Notifications
You must be signed in to change notification settings - Fork 1
Logging
As of June 2014 (0.1.0.0), the IO monad is haphazardly threaded throughout Strappy, leaving users uncertain which portions of the code are actually pure and which actually produce side-effects. We want to produce a better logging system which facilitates the following:
- logging for debugging
- logging progress over the course of long simulations
- logging grammars, generated programs, best programs to hit a task, and other data in a machine-parseable format for use in later data visualization and analysis
We could thread a Writer monad through Strappy, collecting a list of log strings over the course of a simulation and writing this (potentially post-processed) log to a file at the end of the simulation. However, Writers have greater memory requirements than other solutions, and they don't provide incremental progress reports. Both of these effects are caused by the fact that the result of a Writer isn't available until the underlying computation is complete.
As I understand it, the streaming data paradigm of Pipes would allow for the threading of IO-based logging throughout the system while maintaining a strict separation between the IO required for logging and any other operation. So, the codebase can remain pure save for logging, which is sprinkled liberally throughout. Moreover, Pipes allow logs to be written as computations occur, unlike Writer monads. See the package author's reasoning for more information. pipes-aeson is a helper-package for Pipes, converting pipe data to and from JSON. This package may be useful for writing log data in a machine-readable format.
Conduit is part of the Yesod project and is geared toward providing streaming data abilities like Pipes. It shares many of the same ideas as Pipes. It differs from Pipes in that it appears to me to be more flexible (and thus more complicated) in areas that I'm not sure are relevant for use with Strappy. It has a helper-package for reading and writing to XML, but it appears the helper-package for JSON uses a JSON server rather than merely reading or writing JSON files.
My conclusion, then, is that we should use the Pipes library in conjunction with the pipes-aeson package to log results and progress to JSON.