diff --git a/docs/docs/executable/config-files.md b/docs/docs/executable/config-files.md new file mode 100644 index 00000000..fe30745c --- /dev/null +++ b/docs/docs/executable/config-files.md @@ -0,0 +1,69 @@ +# Config Files + +Oxygen gives you two complementary ways to work with JSON / YAML config files. They share the same +core loading and merging logic (single file, or a directory merged with `reduceLeft(_ ++ _)`), so +their behavior never diverges — they differ only in *when* and *how* you reach for them. + +| | `@envConfig` | `ConfigFileService` | +|---|---|---| +| Phase | **Startup** — resolved once while the CLI parses its inputs | **Runtime** — call it any time while the app runs | +| Shape | A CLI-param annotation; the framework injects the decoded value | An injectable `ZLayer`-provided ZIO service | +| Direction | Read-only (load + decode) | Read **and** write (load / merge / list / save) | +| Source | One env var → a file *or* a directory | Any `Path` you hand it | + +Both honor `FileSystem.current`, so both are testable by pointing at a temp directory (or a test +file-system). + +## `@envConfig` — startup + +Use it when a config is an *input* to the app: resolved once, up front, and injected already decoded. + +```scala +final case class ServeCmd( + @envConfig("APP_CONFIG") cfg: AppConfig, // env var holds a file path or a directory +) extends CliApp[Any, Any] derives CompiledCliApp.DeriveRootApp +``` + +If `APP_CONFIG` points at a **file** it is decoded (dispatch on `.json` / `.yaml` / `.yml`); if it +points at a **directory**, every supported file inside is merged (later files, sorted by path, win). + +## `ConfigFileService` — runtime + +Use it when the app needs to read or **write** config files while running — e.g. a CLI that manages +`./.my-cli/local.json` and `~/.my-cli/global.json`. + +```scala +import oxygen.executable.config.* +import oxygen.zio.system.Path + +for { + local <- Path.of("./.my-cli/local.json") + cfg <- ConfigFileService.load[AppConfig](local) + updated = cfg.copy(port = 8080) + _ <- ConfigFileService.save(local, updated) // atomic: temp-file + move +} yield () +``` + +Provide the service with `ConfigFileService.live` (aliased as `.default`; `.test` is the same layer — +testability comes from `FileSystem.current`). + +### Operations + +- `load[A: JsonDecoder](file)` / `loadJson(file)` — read + decode a single file. +- `save[A: JsonEncoder](file, value)` — **atomic** write (serialize by extension → temp sibling → + `moveTo`), creating parent directories as needed. +- `mergeDirectory[A: JsonDecoder](dir)` / `mergeDirectoryJson(dir)` — merge every supported file in + a directory, `reduceLeft(_ ++ _)` (later files, sorted by path, override earlier ones). +- `loadResolved[A: JsonDecoder](path)` — the `@envConfig` semantics: file → load, directory → merge. +- `list(dir)` / `exists(file)`. + +### Errors + +All operations fail into the typed `ConfigFileError` ADT — `FileSystem` (wrapping `FileSystemError`), +`UnsupportedExtension`, `JsonDecodeFailure`, `YamlDecodeFailure`, `EmptyDirectory`, +`NotFileOrDirectory`, `PathDoesNotExist`. + +### Supported extensions + +`.json` (via `oxygen-json`), `.yaml` / `.yml` (via `oxygen-yaml`). Any other extension yields +`UnsupportedExtension`. diff --git a/docs/mkdocs.yml b/docs/mkdocs.yml index b37d3ad2..fba0bd0c 100644 --- a/docs/mkdocs.yml +++ b/docs/mkdocs.yml @@ -29,6 +29,7 @@ nav: - 'Oxygen Executable': - 'Executable Overview': 'executable/index.md' - 'CLI Annotations': 'executable/cli.md' + - 'Config Files': 'executable/config-files.md' - 'Migrating from v1': 'executable/migration-from-v1.md' - 'Tab Completion': 'executable/completion.md' - 'Future Plans': 'future-plans.md' diff --git a/modules/general/executable/src/main/scala/oxygen/executable/config/ConfigFileError.scala b/modules/general/executable/src/main/scala/oxygen/executable/config/ConfigFileError.scala new file mode 100644 index 00000000..28169b51 --- /dev/null +++ b/modules/general/executable/src/main/scala/oxygen/executable/config/ConfigFileError.scala @@ -0,0 +1,50 @@ +package oxygen.executable.config + +import oxygen.predef.core.* +import oxygen.zio.error.FileSystemError +import oxygen.zio.system.Path + +/** + * Typed errors raised by [[ConfigFileService]]. + * + * Wraps lower-level [[FileSystemError]]s together with JSON/YAML decode failures, so callers get a + * single, exhaustive error channel for the whole config-file lifecycle. Style mirrors + * `TestContainerError` / `MigrationError`. + */ +sealed trait ConfigFileError extends Throwable { + + val path: Path.PathName + + override final def getMessage: String = this match + case ConfigFileError.FileSystem(path, cause) => + s"File-system error for config file ($path): ${cause.safeGetMessage}" + case ConfigFileError.UnsupportedExtension(path, extension) => + s"Unsupported config file extension${extension.fold("")(e => s" ($e)")} for ($path). Supported: ${ConfigFileError.supportedExtensions.mkString(", ")}" + case ConfigFileError.JsonDecodeFailure(path, message) => + s"Unable to decode JSON config file ($path): $message" + case ConfigFileError.YamlDecodeFailure(path, message) => + s"Unable to decode YAML config file ($path): $message" + case ConfigFileError.EmptyDirectory(path) => + s"Config directory ($path) contains no valid config files (${ConfigFileError.supportedExtensions.mkString(", ")})" + case ConfigFileError.NotFileOrDirectory(path) => + s"Config path ($path) is neither a file nor a directory" + case ConfigFileError.PathDoesNotExist(path) => + s"Config path does not exist ($path)" + +} +object ConfigFileError { + + val supportedExtensions: Set[String] = Set("json", "yaml", "yml") + + final case class FileSystem(path: Path.PathName, cause: FileSystemError) extends ConfigFileError + final case class UnsupportedExtension(path: Path.PathName, extension: Option[String]) extends ConfigFileError + final case class JsonDecodeFailure(path: Path.PathName, message: String) extends ConfigFileError + final case class YamlDecodeFailure(path: Path.PathName, message: String) extends ConfigFileError + final case class EmptyDirectory(path: Path.PathName) extends ConfigFileError + final case class NotFileOrDirectory(path: Path.PathName) extends ConfigFileError + final case class PathDoesNotExist(path: Path.PathName) extends ConfigFileError + + /** Lift a [[FileSystemError]] into the [[ConfigFileError]] channel. */ + def fromFileSystem(error: FileSystemError): ConfigFileError = ConfigFileError.FileSystem(error.path, error) + +} diff --git a/modules/general/executable/src/main/scala/oxygen/executable/config/ConfigFileService.scala b/modules/general/executable/src/main/scala/oxygen/executable/config/ConfigFileService.scala new file mode 100644 index 00000000..6f74be99 --- /dev/null +++ b/modules/general/executable/src/main/scala/oxygen/executable/config/ConfigFileService.scala @@ -0,0 +1,179 @@ +package oxygen.executable.config + +import oxygen.core.PlatformCompat +import oxygen.json.{Json, JsonDecoder, JsonEncoder} +import oxygen.predef.core.* +import oxygen.yaml.{YamlParser, YamlWriter} +import oxygen.zio.error.FileSystemError +import oxygen.zio.system.Path +import zio.* + +/** + * Reusable, injectable service that centralizes the config-file lifecycle: load / merge / validate + * / persist JSON + YAML config files (e.g. `./.my-cli/local.json`, `~/.my-cli/global.json`). + * + * All path operations go through [[Path]] / `FileSystem.current`, so the service is testable by + * swapping the current file-system or pointing at a temp directory. + * + * Supported extensions: `.json`, `.yaml`, `.yml`. + * + * Contrast with `@envConfig` (see `ConfigLoader`): `@envConfig` resolves a single env var to a + * file/directory ONCE at application startup, whereas this service is the runtime API for reading, + * writing, listing and merging config files while the app is running. Both share the same core + * loading/merging logic so their behavior can never diverge. + */ +trait ConfigFileService { + + /** True iff `file` exists (any type). */ + def exists(file: Path): IO[ConfigFileError, Boolean] + + /** List the supported config files (`.json`/`.yaml`/`.yml`) directly inside `dir`, sorted by path. */ + def list(dir: Path): IO[ConfigFileError, Chunk[Path]] + + /** Read + parse a single config `file` (dispatch on extension) into a raw [[Json]]. */ + def loadJson(file: Path): IO[ConfigFileError, Json] + + /** Read + parse + decode a single config `file` (dispatch on extension) into `A`. */ + def load[A: JsonDecoder](file: Path): IO[ConfigFileError, A] + + /** Merge every supported config file in `dir` into a single [[Json]] via `reduceLeft(_ ++ _)`. */ + def mergeDirectoryJson(dir: Path): IO[ConfigFileError, Json] + + /** Merge every supported config file in `dir` and decode the result into `A`. */ + def mergeDirectory[A: JsonDecoder](dir: Path): IO[ConfigFileError, A] + + /** Resolve `path`: if a file -> [[loadJson]]; if a directory -> [[mergeDirectoryJson]]. */ + def loadResolvedJson(path: Path): IO[ConfigFileError, Json] + + /** Resolve `path` (file or directory) and decode the result into `A`. This is the `@envConfig` semantics. */ + def loadResolved[A: JsonDecoder](path: Path): IO[ConfigFileError, A] + + /** + * Atomically persist `value` to `file`, serialized by its extension (JSON pretty / YAML). + * Writes to a sibling temp file then moves it into place, so readers never observe a partial file. + * Parent directories are created if missing. + */ + def save[A: JsonEncoder](file: Path, value: A): IO[ConfigFileError, Unit] + +} +object ConfigFileService { + + ////////////////////////////////////////////////////////////////////////////////////////////////////// + // Layers + ////////////////////////////////////////////////////////////////////////////////////////////////////// + + /** The real implementation. Stateless: behavior is entirely a function of `FileSystem.current`. */ + val live: ULayer[ConfigFileService] = ZLayer.succeed(Live) + + /** Alias for [[live]] — the default wiring. */ + val default: ULayer[ConfigFileService] = live + + /** + * Alias for [[live]] — there is nothing to stub. Point `FileSystem.current` at a temp directory + * (or a test file-system) to exercise the service in tests. + */ + val test: ULayer[ConfigFileService] = live + + ////////////////////////////////////////////////////////////////////////////////////////////////////// + // Accessors + ////////////////////////////////////////////////////////////////////////////////////////////////////// + + def exists(file: Path): ZIO[ConfigFileService, ConfigFileError, Boolean] = ZIO.serviceWithZIO(_.exists(file)) + def list(dir: Path): ZIO[ConfigFileService, ConfigFileError, Chunk[Path]] = ZIO.serviceWithZIO(_.list(dir)) + def loadJson(file: Path): ZIO[ConfigFileService, ConfigFileError, Json] = ZIO.serviceWithZIO(_.loadJson(file)) + def load[A: JsonDecoder](file: Path): ZIO[ConfigFileService, ConfigFileError, A] = ZIO.serviceWithZIO(_.load[A](file)) + def mergeDirectoryJson(dir: Path): ZIO[ConfigFileService, ConfigFileError, Json] = ZIO.serviceWithZIO(_.mergeDirectoryJson(dir)) + def mergeDirectory[A: JsonDecoder](dir: Path): ZIO[ConfigFileService, ConfigFileError, A] = ZIO.serviceWithZIO(_.mergeDirectory[A](dir)) + def loadResolvedJson(path: Path): ZIO[ConfigFileService, ConfigFileError, Json] = ZIO.serviceWithZIO(_.loadResolvedJson(path)) + def loadResolved[A: JsonDecoder](path: Path): ZIO[ConfigFileService, ConfigFileError, A] = ZIO.serviceWithZIO(_.loadResolved[A](path)) + def save[A: JsonEncoder](file: Path, value: A): ZIO[ConfigFileService, ConfigFileError, Unit] = ZIO.serviceWithZIO(_.save[A](file, value)) + + ////////////////////////////////////////////////////////////////////////////////////////////////////// + // Live + ////////////////////////////////////////////////////////////////////////////////////////////////////// + + case object Live extends ConfigFileService { + + private def wrapFs[A](effect: IO[FileSystemError, A]): IO[ConfigFileError, A] = + effect.mapError(ConfigFileError.fromFileSystem) + + override def exists(file: Path): IO[ConfigFileError, Boolean] = + wrapFs(file.exists) + + override def list(dir: Path): IO[ConfigFileError, Chunk[Path]] = + wrapFs(dir.children).map { children => + children + .filter(_.fileName.hasExtension(ConfigFileError.supportedExtensions)) + .sortBy(_.pathName.unwrap) + } + + override def loadJson(file: Path): IO[ConfigFileError, Json] = + file.fileName.extension match + case Some("json") => + wrapFs(file.read).flatMap { str => + ZIO.fromEither(JsonDecoder.json.decodeJsonString(str)).mapError { e => ConfigFileError.JsonDecodeFailure(file.pathName, e.safeGetMessage) } + } + case Some("yaml" | "yml") => + wrapFs(file.read).flatMap { str => + ZIO.fromEither(YamlParser.parseJson(str)).mapError { e => ConfigFileError.YamlDecodeFailure(file.pathName, e) } + } + case ext => + ZIO.fail(ConfigFileError.UnsupportedExtension(file.pathName, ext)) + + override def load[A: JsonDecoder as decoder](file: Path): IO[ConfigFileError, A] = + loadJson(file).flatMap(decodeAst[A](file.pathName, _)) + + override def mergeDirectoryJson(dir: Path): IO[ConfigFileError, Json] = + list(dir).flatMap { files => + ZIO.foreach(files)(loadJson).flatMap { + case c if c.isEmpty => ZIO.fail(ConfigFileError.EmptyDirectory(dir.pathName)) + case c => ZIO.succeed(c.reduceLeft(_ ++ _)) + } + } + + override def mergeDirectory[A: JsonDecoder](dir: Path): IO[ConfigFileError, A] = + mergeDirectoryJson(dir).flatMap(decodeAst[A](dir.pathName, _)) + + override def loadResolvedJson(path: Path): IO[ConfigFileError, Json] = + wrapFs(path.status).flatMap { + case Path.Type.File => loadJson(path) + case Path.Type.Directory => mergeDirectoryJson(path) + case Path.Status.DoesNotExist => ZIO.fail(ConfigFileError.PathDoesNotExist(path.pathName)) + case Path.Type.Other => ZIO.fail(ConfigFileError.NotFileOrDirectory(path.pathName)) + } + + override def loadResolved[A: JsonDecoder](path: Path): IO[ConfigFileError, A] = + loadResolvedJson(path).flatMap(decodeAst[A](path.pathName, _)) + + override def save[A: JsonEncoder as encoder](file: Path, value: A): IO[ConfigFileError, Unit] = + encode[A](file, value) match + case Left(error) => ZIO.fail(error) + case Right(contents) => + val tmp: Path = tempSiblingOf(file) + val write: IO[FileSystemError, Unit] = + ZIO.foreachDiscard(file.parentOption)(_.createDirectories) *> + tmp.write(contents) *> + tmp.moveTo(file, replaceExisting = true) + wrapFs(write).onError { _ => tmp.deleteIfExists.ignore } + + ////////////////////////////////////////////////////////////////////////////////////////////////////// + // Helpers + ////////////////////////////////////////////////////////////////////////////////////////////////////// + + private def decodeAst[A](pathName: Path.PathName, json: Json)(using decoder: JsonDecoder[A]): IO[ConfigFileError, A] = + ZIO.fromEither(decoder.decodeJsonAST(json)).mapError { e => ConfigFileError.JsonDecodeFailure(pathName, e.safeGetMessage) } + + private def encode[A](file: Path, value: A)(using encoder: JsonEncoder[A]): Either[ConfigFileError, String] = + file.fileName.extension match + case Some("json") => encoder.encodeJsonStringPretty(value).asRight + case Some("yaml" | "yml") => YamlWriter.writeJsonOf(value).asRight + case ext => ConfigFileError.UnsupportedExtension(file.pathName, ext).asLeft + + private def tempSiblingOf(file: Path): Path = { + val name: String = s".${file.fileName.name}.${PlatformCompat.randomUUID()}.tmp" + file.parentOption.fold(file.resolve(name))(_.resolve(name)) + } + + } + +} diff --git a/modules/general/executable/src/main/scala/oxygen/executable/generic/ConfigLoader.scala b/modules/general/executable/src/main/scala/oxygen/executable/generic/ConfigLoader.scala index 85983029..40532bf9 100644 --- a/modules/general/executable/src/main/scala/oxygen/executable/generic/ConfigLoader.scala +++ b/modules/general/executable/src/main/scala/oxygen/executable/generic/ConfigLoader.scala @@ -1,45 +1,24 @@ package oxygen.executable.generic -import oxygen.json.{Json, JsonDecoder} +import oxygen.executable.config.ConfigFileService +import oxygen.json.JsonDecoder import oxygen.predef.core.* import oxygen.predef.zio.* -import oxygen.yaml.YamlParser import zio.* +/** + * Startup-time loader backing `@envConfig`: resolves an env var's value to a file/directory and + * decodes it into `T`. + * + * Delegates to [[ConfigFileService.Live]] so the `@envConfig` startup path and the runtime + * [[ConfigFileService]] share a single implementation and can never diverge (same file/dir + * resolution, same JSON/YAML handling, same directory-merge order). + */ private[executable] object ConfigLoader { - def loadDecoded[T: JsonDecoder as decoder](varName: String, raw: String): IO[String, T] = - resolveJson(raw) - .flatMap { json => ZIO.fromEither { decoder.decodeJsonAST(json).leftMap(_.getMessage) } } + def loadDecoded[T: JsonDecoder](varName: String, raw: String): IO[String, T] = + Path.normalizedAbsolute(raw).mapError(_.safeGetMessage) + .flatMap { path => ConfigFileService.Live.loadResolved[T](path).mapError(_.safeGetMessage) } .mapError { error => s"Error extracting environment variable config [$varName]: $error" } - private def resolveJson(raw: String): IO[String, Json] = - Path.normalizedAbsolute(raw).mapError(_.safeGetMessage).flatMap { path => - path.status.mapError(_.safeGetMessage).flatMap { - case Path.Type.File => readFileJson(path) - case Path.Type.Directory => mergeDirectoryJson(path) - case Path.Status.DoesNotExist => ZIO.fail(s"Path does not exist (${path.pathName})") - case _ => ZIO.fail(s"Path is not a file or directory (${path.pathName})") - } - } - - // TODO (KR) : add support for yaml? - private def readOptFileJson(file: Path): IO[String, Option[Json]] = - file.fileName.extension match { - case Some("json") => file.readDecodeJson[Json].mapError(_.safeGetMessage).asSome - case Some("yaml" | "yml") => - file.read.mapError(_.safeGetMessage) - .flatMap { str => ZIO.fromEither { YamlParser.parseJson(str) }.mapError { e => s"Invalid yaml at (${file.pathName}): $e" } }.asSome - case _ => ZIO.none - } - - private def readFileJson(file: Path): IO[String, Json] = - readOptFileJson(file).someOrFail(s"Invalid file extension (${file.pathName})") - - private def mergeDirectoryJson(dir: Path): IO[String, Json] = - dir.childStream.mapError(_.safeGetMessage).mapZIO(readOptFileJson).collectSome.runCollect.flatMap { - case children if children.isEmpty => ZIO.fail(s"Directory contains no valid config files (${dir.pathName})") - case children => ZIO.succeed { children.reduceLeft(_ ++ _) } - } - } diff --git a/modules/general/executable/src/test/scala/oxygen/executable/config/ConfigFileServiceSpec.scala b/modules/general/executable/src/test/scala/oxygen/executable/config/ConfigFileServiceSpec.scala new file mode 100644 index 00000000..c046bece --- /dev/null +++ b/modules/general/executable/src/test/scala/oxygen/executable/config/ConfigFileServiceSpec.scala @@ -0,0 +1,153 @@ +package oxygen.executable.config + +import java.nio.file.Files +import oxygen.json.JsonCodec +import oxygen.predef.test.* +import oxygen.zio.system.Path + +object ConfigFileServiceSpec extends OxygenSpecDefault { + + final case class AppConfig(name: String, port: Int) derives JsonCodec + + /** A fresh, real temp directory (via `FileSystem.current`) for each test. */ + private val tempDir: UIO[Path] = + ZIO.attempt(Files.createTempDirectory("oxygen-config-file-spec")).orDie + .flatMap(jpath => Path.of(jpath.toString).orDie) + + override def testSpec: TestSpec = + suite("ConfigFileServiceSpec")( + suite("round-trip")( + test("json save -> load returns the original value") { + val cfg = AppConfig("svc", 8080) + for { + dir <- tempDir + file = dir.resolve("local.json") + _ <- ConfigFileService.save(file, cfg) + loaded <- ConfigFileService.load[AppConfig](file) + } yield assertTrue(loaded == cfg) + }, + test("yaml save -> load returns the original value") { + val cfg = AppConfig("svc", 9090) + for { + dir <- tempDir + file = dir.resolve("global.yaml") + _ <- ConfigFileService.save(file, cfg) + loaded <- ConfigFileService.load[AppConfig](file) + } yield assertTrue(loaded == cfg) + }, + test("save overwrites an existing file (second write wins)") { + val first = AppConfig("first", 1) + val second = AppConfig("second", 2) + for { + dir <- tempDir + file = dir.resolve("local.json") + _ <- ConfigFileService.save(file, first) + _ <- ConfigFileService.save(file, second) + loaded <- ConfigFileService.load[AppConfig](file) + } yield assertTrue(loaded == second) + }, + test("save creates missing parent directories") { + val cfg = AppConfig("nested", 1) + for { + dir <- tempDir + file = dir.resolve("a").resolve("b").resolve("c.json") + _ <- ConfigFileService.save(file, cfg) + loaded <- ConfigFileService.load[AppConfig](file) + } yield assertTrue(loaded == cfg) + }, + ), + suite("mergeDirectory")( + test("merges every supported file, later (sorted) files win on key conflicts") { + for { + dir <- tempDir + _ <- dir.resolve("01-base.json").write("""{"name":"A","port":1}""").orDie + _ <- dir.resolve("02-override.json").write("""{"name":"B"}""").orDie + merged <- ConfigFileService.mergeDirectory[AppConfig](dir) + } yield assertTrue(merged == AppConfig("B", 1)) + }, + test("merges across json + yaml files") { + for { + dir <- tempDir + _ <- dir.resolve("01.json").write("""{"name":"json"}""").orDie + _ <- dir.resolve("02.yaml").write("port: 42\n").orDie + merged <- ConfigFileService.mergeDirectory[AppConfig](dir) + } yield assertTrue(merged == AppConfig("json", 42)) + }, + test("empty directory fails with EmptyDirectory") { + for { + dir <- tempDir + res <- ConfigFileService.mergeDirectory[AppConfig](dir).either + } yield assertTrue(res.left.toOption.exists(_.isInstanceOf[ConfigFileError.EmptyDirectory])) + }, + ), + suite("loadResolved (the @envConfig semantics)")( + test("resolves a single file") { + val cfg = AppConfig("file", 7) + for { + dir <- tempDir + file = dir.resolve("cfg.json") + _ <- ConfigFileService.save(file, cfg) + loaded <- ConfigFileService.loadResolved[AppConfig](file) + } yield assertTrue(loaded == cfg) + }, + test("resolves a directory by merging") { + for { + dir <- tempDir + _ <- dir.resolve("a.json").write("""{"name":"A","port":1}""").orDie + loaded <- ConfigFileService.loadResolved[AppConfig](dir) + } yield assertTrue(loaded == AppConfig("A", 1)) + }, + test("missing path fails with PathDoesNotExist") { + for { + dir <- tempDir + missing = dir.resolve("does-not-exist.json") + res <- ConfigFileService.loadResolved[AppConfig](missing).either + } yield assertTrue(res.left.toOption.exists(_.isInstanceOf[ConfigFileError.PathDoesNotExist])) + }, + ), + suite("list / exists")( + test("list returns only supported files, sorted by path") { + for { + dir <- tempDir + _ <- ConfigFileService.save(dir.resolve("b.yaml"), AppConfig("b", 2)) + _ <- ConfigFileService.save(dir.resolve("a.json"), AppConfig("a", 1)) + _ <- dir.resolve("notes.txt").write("ignored").orDie + files <- ConfigFileService.list(dir) + } yield assertTrue(files.map(_.fileName.name) == Chunk("a.json", "b.yaml")) + }, + test("exists reflects presence") { + for { + dir <- tempDir + file = dir.resolve("c.json") + before <- ConfigFileService.exists(file) + _ <- ConfigFileService.save(file, AppConfig("c", 3)) + after <- ConfigFileService.exists(file) + } yield assertTrue(!before, after) + }, + ), + suite("errors")( + test("save to an unsupported extension fails with UnsupportedExtension") { + for { + dir <- tempDir + res <- ConfigFileService.save(dir.resolve("config.txt"), AppConfig("x", 1)).either + } yield assertTrue(res.left.toOption.exists(_.isInstanceOf[ConfigFileError.UnsupportedExtension])) + }, + test("load of an unsupported extension fails with UnsupportedExtension") { + for { + dir <- tempDir + _ <- dir.resolve("config.txt").write("whatever").orDie + res <- ConfigFileService.load[AppConfig](dir.resolve("config.txt")).either + } yield assertTrue(res.left.toOption.exists(_.isInstanceOf[ConfigFileError.UnsupportedExtension])) + }, + test("load of malformed json fails with JsonDecodeFailure") { + for { + dir <- tempDir + file = dir.resolve("bad.json") + _ <- file.write("""{"name":"x"}""").orDie // missing required `port` + res <- ConfigFileService.load[AppConfig](file).either + } yield assertTrue(res.left.toOption.exists(_.isInstanceOf[ConfigFileError.JsonDecodeFailure])) + }, + ), + ).provide(ConfigFileService.test) + +} diff --git a/modules/general/zio/src/main/scala/oxygen/zio/system/JavaPath.scala b/modules/general/zio/src/main/scala/oxygen/zio/system/JavaPath.scala index 43e12b57..ae16917b 100644 --- a/modules/general/zio/src/main/scala/oxygen/zio/system/JavaPath.scala +++ b/modules/general/zio/src/main/scala/oxygen/zio/system/JavaPath.scala @@ -125,9 +125,10 @@ final case class JavaPath(javaPath: J.Path) extends Path { FileSystem.attempt(pathName, s"copy to $destination") { J.Files.copy(this.javaPath, destination.javaPath) } } - override def moveTo(destination: Path): IO[FileSystemError, Unit] = + override def moveTo(destination: Path, replaceExisting: Boolean): IO[FileSystemError, Unit] = JavaPath.safeJava(destination).flatMap { destination => - FileSystem.attempt(pathName, s"move to $destination") { J.Files.move(this.javaPath, destination.javaPath) } + val options: Seq[J.CopyOption] = if replaceExisting then Seq(J.StandardCopyOption.REPLACE_EXISTING) else Seq.empty + FileSystem.attempt(pathName, s"move to $destination") { J.Files.move(this.javaPath, destination.javaPath, options*) } } /////// Delete /////////////////////////////////////////////////////////////// diff --git a/modules/general/zio/src/main/scala/oxygen/zio/system/Path.scala b/modules/general/zio/src/main/scala/oxygen/zio/system/Path.scala index 56f01ffc..becbe646 100644 --- a/modules/general/zio/src/main/scala/oxygen/zio/system/Path.scala +++ b/modules/general/zio/src/main/scala/oxygen/zio/system/Path.scala @@ -71,7 +71,7 @@ trait Path { def createDirectories: IO[FileSystemError, Unit] def copyTo(destination: Path): IO[FileSystemError, Unit] - def moveTo(destination: Path): IO[FileSystemError, Unit] + def moveTo(destination: Path, replaceExisting: Boolean = false): IO[FileSystemError, Unit] /////// Delete /////////////////////////////////////////////////////////////// diff --git a/report/OXY-143.md b/report/OXY-143.md new file mode 100644 index 00000000..ff016783 --- /dev/null +++ b/report/OXY-143.md @@ -0,0 +1,84 @@ +# OXY-143 — Create a service for managing config files + +## Goal +Reusable, injectable ZIO service (`ConfigFileService`) centralizing config-file lifecycle: +load / merge / validate / persist JSON+YAML config files. Refactor private +`ConfigLoader` (`@envConfig`) to share logic with the runtime service. + +## Key decisions / assumptions +- **Module placement: `oxygen-executable`** (the ticket's documented alternative), NOT `oxygen-zio`. + Reason: the service *requires* `YamlParser` (`.yaml`/`.yml` handling), which lives in + `oxygen-yaml`. `oxygen-zio` does not depend on `oxygen-yaml` (would force a new dep on the + low-level module). `oxygen-executable` already depends on BOTH `oxygen-zio` (Path/FileSystem via + `oxygen-cli`) and `oxygen-yaml`, and it is where `ConfigLoader` lives — so the shared-logic + refactor stays in-module. Path ops still honor `FileSystem.current` for testability. +- **Package: `oxygen.executable.config`** (new public package). `ConfigLoader` stays + `private[executable]` in `oxygen.executable.generic` and delegates to the shared core. +- **Error ADT `ConfigFileError`**: sealed trait extends `Throwable` with `getMessage`, mirroring + `TestContainerError`/`MigrationError` style. Wraps `FileSystemError` + JSON/YAML decode failures. +- **Format dispatch by extension**: `.json` -> oxygen-json (`encodeJsonStringPretty` / + `readDecodeJson`); `.yaml`/`.yml` -> `YamlParser`/`YamlWriter`; anything else -> + `UnsupportedExtension`. +- **Atomic save**: write to a sibling temp file in the same directory (`..tmp-`), + then `moveTo` destination (atomic on same filesystem). `createDirectories` on parent first. +- **`mergeDirectory`**: reuses ConfigLoader semantics — read each supported file in the directory, + `reduceLeft(_ ++ _)` (later files win on key conflicts, preserving existing order), decode the + merged JSON. Empty dir -> `EmptyDirectory` error. +- **`loadResolved`**: file-or-directory dispatch (the `@envConfig` behavior). ConfigLoader now + delegates here, so startup and runtime paths cannot diverge. +- **Layers**: `live`/`default`/`test` all provide the same `Live` impl (stateless; behavior is + entirely a function of `FileSystem.current`). `default` and `test` are aliases documented as such. + Testability comes from swapping `FileSystem.current` or using a real temp dir (no in-memory FS + exists in the repo; `JavaFileSystem` is the only concrete impl besides `UnimplementedFileSystem`). +- **`load` accepts `Path`** (String convenience via existing `Path.of`). Keeps surface small and FS-aware. + +## Building blocks used +- `oxygen.zio.system.Path` / `FileSystem` (+ `FileSystem.current`) +- `oxygen.yaml.{YamlParser, YamlWriter}` +- `oxygen.json.{Json, JsonDecoder, JsonEncoder}` (`Json.++` = merge; `encodeJsonStringPretty`) +- `oxygen.zio.error.FileSystemError` + +## Progress +- [x] Explored codebase / module graph / building blocks +- [x] Write `ConfigFileError` +- [x] Write `ConfigFileService` (+ shared core `ConfigFiles`) +- [x] Refactor `ConfigLoader` to delegate +- [x] Tests (round-trip, directory-merge, missing-path) +- [x] Doc: `@envConfig` (startup) vs service (runtime) +- [x] Build + test verification + +## Files +- NEW `modules/general/executable/.../config/ConfigFileError.scala` — typed error ADT +- NEW `modules/general/executable/.../config/ConfigFileService.scala` — trait + `live`/`default`/`test` layers + accessors + `Live` +- NEW `modules/general/executable/src/test/.../config/ConfigFileServiceSpec.scala` — 14 tests +- MOD `modules/general/executable/.../generic/ConfigLoader.scala` — now delegates to `ConfigFileService.Live.loadResolved` +- NEW `docs/docs/executable/config-files.md` + nav entry in `docs/mkdocs.yml` + +## Verification +- `oxygen-executableJVM/test` -> 43 passed / 0 failed (14 new + 29 existing, incl. envConfig path). +- `oxygen-executableJS/compile` + `oxygen-executableNative/compile` -> success (cross-Pure OK). + +## Build environment note (not part of the PR) +- sbt-git's JGit `hasUncommittedChanges` throws `NoWorkTreeException` inside a *linked git worktree*, + which blocks `sbt` from loading. Worked around locally with a TEMPORARY untracked + `zzz-worktree-git-workaround.sbt` pinning `git.git*` keys to constants; **deleted before commit**. + Nothing in the committed diff touches the build. A normal (non-worktree) checkout is unaffected. + +## Intentional behavior changes / deviations +- Directory merge now **sorts files by path** before `reduceLeft(_ ++ _)`. The old `ConfigLoader` + relied on unsorted `childStream` order (nondeterministic). Sorting makes merge precedence + deterministic ("later sorted file wins") for BOTH `@envConfig` and the service — strictly safer. +- `@envConfig` error message prefixes are slightly reworded (now sourced from `ConfigFileError`). + No test asserted on them; behavior/exit semantics unchanged. +- Placed in `oxygen-executable` (not `oxygen-zio`) — see decisions above; `oxygen-zio` lacks the + `oxygen-yaml` dependency required for YAML support. + +## CONFIDENCE SCORE: 8.5 / 10 +- High: compiles on all 3 platforms; new + existing tests green; ConfigLoader/`@envConfig` unified + with the runtime service (core ticket goal met); atomic save, typed errors, JSON+YAML, list/exists, + merge, and docs all delivered. +- Risk / unknowns: (1) module-placement judgement call (executable vs zio) — defensible and the + ticket named it as the alternative, but a reviewer may prefer zio + a new yaml dep; (2) the sort + change to merge order is a deliberate but real behavior tweak; (3) atomic-save temp-file uses a + sibling `.tmp` file — assumes same-filesystem move (standard); (4) no in-memory FS exists so the + `test` layer is an alias of `live` (testability via temp dirs / `FileSystem.current`).