From c67abf7aafd7010eb733b65a08408f27fa21e20a Mon Sep 17 00:00:00 2001 From: Matthias Wahl Date: Fri, 13 Mar 2026 21:39:46 +0100 Subject: [PATCH 1/5] Add RFC for Json.print() api in stdlib json package --- text/0000-json-encode-jsonvalue.md | 75 ++++++++++++++++++++++++++++++ 1 file changed, 75 insertions(+) create mode 100644 text/0000-json-encode-jsonvalue.md diff --git a/text/0000-json-encode-jsonvalue.md b/text/0000-json-encode-jsonvalue.md new file mode 100644 index 00000000..3302e80b --- /dev/null +++ b/text/0000-json-encode-jsonvalue.md @@ -0,0 +1,75 @@ +- Feature Name: json-encode-jsonvalue +- Start Date: 2026-03-12 +- RFC PR: (leave this empty) +- Pony Issue: (leave this empty) + +# Summary + +Adapt the stdlib [`json` package][json-package] in order to add means to encode arbitrary [`JsonValue`][JsonValue]s into `String` or `Array[U8]`. And while we are at it, provide a matching API for decode `String` or `Array[U8]` into [`JsonValue`] for completeness. + +# Motivation + +The basic functionality of a json library should be to provide a representation for structured json values (which is [JsonValue][JsonValue]) and methods to decode a string or bytes as JSON into this [JsonValue][JsonValue] and vice versa, to encode all possible representations of [JsonValue][JsonValue] into valid JSON bytes (or string). The current [`json` package][json-package] is lacking the means of encoding all [JsonValue][JsonValue] instances into valid JSON. + +`JsonObject` and `JsonArray` provide methods for encoding them as valid JSON with the `.string()` and `.pretty_string()` methods, but for all other possible JSON values (string, number, bool, null) there is no such method. The `.string()` method on `None` is producing `"None"` (without the quotes), which is invalid JSON. Another example: the String `"\""` is copying itself when `.string()` is called on it, without any escaping, which is also invalid JSON. The method `.pretty_string()` is missing from all those other [JsonValue][JsonValue] instances. + +This package needs a consistent and convenient way to encode and decode json bytes from and to all possible [JsonValue][JsonValue] instances. + +# Detailed design + +This PR suggests adding a new primitive called `Json`: + +```pony +primitive Json + fun print(value: JsonValue, pretty: Bool = false): String => + if pretty then + _JsonPrint.pretty(value) + else + _JsonPrint.compact(value) + end + + fun parse(source: String): (JsonValue | JsonParseError) => + JsonParser.parse(source) +``` + +All the bits and pieces for encoding arbitrary values are there already, they just aren't exposed. `_JsonPrint` is private. + +The naming of the functions is not settled yet. The dual of `encode` and `decode` would be ideal, but the `JsonParser` already returns `JsonParseError`, hence `parse` makes more sense for the decoding step, but it lacks a proper dual name for the process of decoding. For further discussion of the naming, see [Alternatives](#alternatives). + +Proper docstrings are missing, obviously. They should explain what can be expected, under which conditions errors are raised etc. and also list usage examples. They were left out for brevity. + +We might want to remove the `JsonObject.string()` and `JsonArray.string()` (and `.string_pretty()`) functions. As all [JsonValue][JsonValue] members implement `ToString` and thus all have a `.string()` method, but don't all produce valid JSON, this might be a source of confusion. This would also shrink the amount of different ways to achieve one thing to one: `Json.print()`. + +The `JsonParser` primitive should also be removed and the contents of its `parse()` method should be moved to the `Json.parse()` method, to have only one way of decoding a string as JSON. + +# How We Teach This + +The package documentation of the `json` package should show usage examples of these functions most prominently and then show all other components of the crate, like `JsonValue`, `JsonPath` or `JsonNav`. + +The intend of this primitive is to expose an easy-to-use interface for turning strings into structures JSON values and vice versa. This should answer the question of how to get from bytes or strings from the network or from files to structured JSON that can be analyzed (Pass the string to `Json.parse()`), and how to serialize Pony objects and data-structures as JSON (build a `JsonValue`, then pass it those `Json.print()`). + + +# How We Test This + +Additional property-based tests will be added for encoding arbitrary [JsonValue][JsonValue]s via `Json.encode(...)` and parsing them again via `Json.parse(...)` and ensuring equivalence of the initial values and the ones produces by the roundtrip. + +# Drawbacks + +While exposing a way of printing/encoding/serializing [JsonValue][JsonValue]s is necessary, removing `JsonParser` and `JsonObject.string` etc. might not be. These changes are breaking. + +# Alternatives + +Function naming alternatives: + +- `parse` / `print` - this would follow the existing naming in the package the closest, hence considered as the best candidate. +- `encode` / `decode` - this pair would make sense if the decoding would not expose `JsonParseError`. It feels inconsistent to create a `ParseError` from a `decode` function call. +- `serialize` / `deserialize` - these terms are not used anywhere yet in pony, so didn't favor them. +- `to_string` / `from_string` - The most humble variant of them all, getting along without any fancy terminology. + +# Unresolved questions + +- Should a partial version of parsing be also exposed? +- Should `Json.print()` expose finer controls than just the `pretty` flag? + +[json-package]: https://github.com/ponylang/ponyc/tree/main/packages/json +[JsonValue]: https://github.com/ponylang/ponyc/blob/main/packages/json/json.pony#L243 From 393484e24f3adac66e75abee82416226c54dec9d Mon Sep 17 00:00:00 2001 From: Matthias Wahl Date: Sat, 4 Apr 2026 21:22:13 +0200 Subject: [PATCH 2/5] Rephrase the PR to add JsonPrinter primitive and rename string methods --- text/0000-json-encode-jsonvalue.md | 46 +++++++++++++----------------- 1 file changed, 20 insertions(+), 26 deletions(-) diff --git a/text/0000-json-encode-jsonvalue.md b/text/0000-json-encode-jsonvalue.md index 3302e80b..1e681378 100644 --- a/text/0000-json-encode-jsonvalue.md +++ b/text/0000-json-encode-jsonvalue.md @@ -5,7 +5,7 @@ # Summary -Adapt the stdlib [`json` package][json-package] in order to add means to encode arbitrary [`JsonValue`][JsonValue]s into `String` or `Array[U8]`. And while we are at it, provide a matching API for decode `String` or `Array[U8]` into [`JsonValue`] for completeness. +Adapt the stdlib [`json` package][json-package] in order to add a `JsonPrinter` primitive exposing functions to encode arbitrary [`JsonValue`][JsonValue]s into `String iso^`, in a similar manner as the existing `JsonParser` for parsing [`JsonValue`][JsonValue] from String. # Motivation @@ -17,49 +17,44 @@ This package needs a consistent and convenient way to encode and decode json byt # Detailed design -This PR suggests adding a new primitive called `Json`: +This PR suggests adding a new primitive called `JsonPrinter`: ```pony -primitive Json - fun print(value: JsonValue, pretty: Bool = false): String => - if pretty then - _JsonPrint.pretty(value) - else - _JsonPrint.compact(value) - end - - fun parse(source: String): (JsonValue | JsonParseError) => - JsonParser.parse(source) +primitive JsonPrinter + """ + Serialize any `JsonValue` to a JSON string. + """ + fun print(value: JsonValue): String iso^ => + """Compact JSON serialization of any `JsonValue`.""" + _JsonPrint.compact(value) + + fun pretty(value: JsonValue, indent: String = " "): String iso^ => + """Pretty-printed JSON serialization of any `JsonValue`.""" + _JsonPrint.pretty(value, indent) ``` All the bits and pieces for encoding arbitrary values are there already, they just aren't exposed. `_JsonPrint` is private. -The naming of the functions is not settled yet. The dual of `encode` and `decode` would be ideal, but the `JsonParser` already returns `JsonParseError`, hence `parse` makes more sense for the decoding step, but it lacks a proper dual name for the process of decoding. For further discussion of the naming, see [Alternatives](#alternatives). - -Proper docstrings are missing, obviously. They should explain what can be expected, under which conditions errors are raised etc. and also list usage examples. They were left out for brevity. - -We might want to remove the `JsonObject.string()` and `JsonArray.string()` (and `.string_pretty()`) functions. As all [JsonValue][JsonValue] members implement `ToString` and thus all have a `.string()` method, but don't all produce valid JSON, this might be a source of confusion. This would also shrink the amount of different ways to achieve one thing to one: `Json.print()`. - -The `JsonParser` primitive should also be removed and the contents of its `parse()` method should be moved to the `Json.parse()` method, to have only one way of decoding a string as JSON. +To avoid confusion and misuse, the methods `JsonObject.string()` and `JsonArray.string()` (and `.string_pretty()`) are being renamed to `JsonObject.print()`, `JsonArray.print()` and `JsonObject.pretty_print()`, `JsonArray.pretty_print()`. This has the side-effect of both `JsonObject` and `JsonArray` and thus `JsonValue` not implementing `Stringable` anymore. # How We Teach This -The package documentation of the `json` package should show usage examples of these functions most prominently and then show all other components of the crate, like `JsonValue`, `JsonPath` or `JsonNav`. +The package documentation of the `json` package should show usage examples of the new `JsonPrinter` functions as prominently as `JsonParser` usage and then show all other components of the crate, like `JsonValue`, `JsonPath` or `JsonNav`. -The intend of this primitive is to expose an easy-to-use interface for turning strings into structures JSON values and vice versa. This should answer the question of how to get from bytes or strings from the network or from files to structured JSON that can be analyzed (Pass the string to `Json.parse()`), and how to serialize Pony objects and data-structures as JSON (build a `JsonValue`, then pass it those `Json.print()`). +The intend of this primitive is to expose an easy-to-use interface for turning JSON values into strings as a dual to `JsonParser`. This should also answer the question of how to serialize Pony objects and data-structures as JSON (build a `JsonValue`, then pass it to `JsonPrinter.print(value)`). # How We Test This -Additional property-based tests will be added for encoding arbitrary [JsonValue][JsonValue]s via `Json.encode(...)` and parsing them again via `Json.parse(...)` and ensuring equivalence of the initial values and the ones produces by the roundtrip. +Additional property-based tests will be added for encoding arbitrary [JsonValue][JsonValue]s via `JsonPrinter.print(...)` and parsing them again via `JsonParser.parse(...)` and ensuring equivalence of the initial values and the ones produces by the roundtrip. # Drawbacks -While exposing a way of printing/encoding/serializing [JsonValue][JsonValue]s is necessary, removing `JsonParser` and `JsonObject.string` etc. might not be. These changes are breaking. +Renaming `.string()` and `.string_pretty()` on `JsonObject` and `JsonArray` to `.print()` and `.pretty_print()` is a breaking change. It will cause libraries and applications using the `json` package to be changed in order to be compiled with the ponyc version containing this change. # Alternatives -Function naming alternatives: +Function naming alternatives for parsing/printing: - `parse` / `print` - this would follow the existing naming in the package the closest, hence considered as the best candidate. - `encode` / `decode` - this pair would make sense if the decoding would not expose `JsonParseError`. It feels inconsistent to create a `ParseError` from a `decode` function call. @@ -68,8 +63,7 @@ Function naming alternatives: # Unresolved questions -- Should a partial version of parsing be also exposed? -- Should `Json.print()` expose finer controls than just the `pretty` flag? +- Should the options, like pretty-printing or not, be exposed as optional parameters to `JsonPrinter.print()` instead of exposing `.print()` and `pretty()`? [json-package]: https://github.com/ponylang/ponyc/tree/main/packages/json [JsonValue]: https://github.com/ponylang/ponyc/blob/main/packages/json/json.pony#L243 From be2554c32df9105b3b29feadd99a9747f180ea79 Mon Sep 17 00:00:00 2001 From: Matthias Wahl Date: Sun, 3 May 2026 22:42:23 +0200 Subject: [PATCH 3/5] Update text/0000-json-encode-jsonvalue.md Co-authored-by: Joe Eli McIlvain --- text/0000-json-encode-jsonvalue.md | 2 ++ 1 file changed, 2 insertions(+) diff --git a/text/0000-json-encode-jsonvalue.md b/text/0000-json-encode-jsonvalue.md index 1e681378..ad2647ed 100644 --- a/text/0000-json-encode-jsonvalue.md +++ b/text/0000-json-encode-jsonvalue.md @@ -61,6 +61,8 @@ Function naming alternatives for parsing/printing: - `serialize` / `deserialize` - these terms are not used anywhere yet in pony, so didn't favor them. - `to_string` / `from_string` - The most humble variant of them all, getting along without any fancy terminology. +Note that this RFC only includes `print` in scope, so renaming the `parse` would be out of scope unless we decided to pursue one of the above alternatives for a broader change. + # Unresolved questions - Should the options, like pretty-printing or not, be exposed as optional parameters to `JsonPrinter.print()` instead of exposing `.print()` and `pretty()`? From 4fafb6db3d14bb389696fb63209a737d7d3b222e Mon Sep 17 00:00:00 2001 From: Matthias Wahl Date: Mon, 4 May 2026 07:48:26 +0200 Subject: [PATCH 4/5] Clarify what tests are required. --- text/0000-json-encode-jsonvalue.md | 12 ++++++++---- 1 file changed, 8 insertions(+), 4 deletions(-) diff --git a/text/0000-json-encode-jsonvalue.md b/text/0000-json-encode-jsonvalue.md index ad2647ed..3590475e 100644 --- a/text/0000-json-encode-jsonvalue.md +++ b/text/0000-json-encode-jsonvalue.md @@ -26,11 +26,15 @@ primitive JsonPrinter """ fun print(value: JsonValue): String iso^ => """Compact JSON serialization of any `JsonValue`.""" - _JsonPrint.compact(value) + JsonPrinter.print_with_options(value where indent = "", add_newlines = false, add_spaces = false) fun pretty(value: JsonValue, indent: String = " "): String iso^ => """Pretty-printed JSON serialization of any `JsonValue`.""" - _JsonPrint.pretty(value, indent) + JsonPrinter.print_with_options(value, indent where add_newlines = true, add_spaces = true) + + fun print_with_options(value: JsonValue, indent: String = "", add_newlines: Bool = false, add_spaces: Bool = false): String iso^ => + """JSON serialization with all possible formatting options exposed. Defaults to compact formatting.""" + // TBD ``` All the bits and pieces for encoding arbitrary values are there already, they just aren't exposed. `_JsonPrint` is private. @@ -46,7 +50,7 @@ The intend of this primitive is to expose an easy-to-use interface for turning J # How We Test This -Additional property-based tests will be added for encoding arbitrary [JsonValue][JsonValue]s via `JsonPrinter.print(...)` and parsing them again via `JsonParser.parse(...)` and ensuring equivalence of the initial values and the ones produces by the roundtrip. +The json package already has print-parse roundtrip tests with arbitrary json values. Those need to be adapted to use the public API. Furthermore several "example"-tests are needed, verifying that printing a concrete `JsonValue` yields an expected valid JSON string (i.e. bools, ints, nulls, floats, strings with values needing escape sequences etc.). Those are there to proof that we don't have a common bug in printing and parsing which makes roundtripping work, but produces invalid JSON. # Drawbacks @@ -65,7 +69,7 @@ Note that this RFC only includes `print` in scope, so renaming the `parse` would # Unresolved questions -- Should the options, like pretty-printing or not, be exposed as optional parameters to `JsonPrinter.print()` instead of exposing `.print()` and `pretty()`? +- Should the options, like pretty-printing or not, be exposed as optional parameters to `JsonPrinter.print()` instead of exposing `.print()` and `.pretty()`? [json-package]: https://github.com/ponylang/ponyc/tree/main/packages/json [JsonValue]: https://github.com/ponylang/ponyc/blob/main/packages/json/json.pony#L243 From b530eefc0874a4cdf4f9e1378a582975e2f83398 Mon Sep 17 00:00:00 2001 From: Matthias Wahl Date: Mon, 4 May 2026 08:30:25 +0200 Subject: [PATCH 5/5] Flesh out some considered alternatives --- text/0000-json-encode-jsonvalue.md | 32 ++++++++++++++++++++---------- 1 file changed, 22 insertions(+), 10 deletions(-) diff --git a/text/0000-json-encode-jsonvalue.md b/text/0000-json-encode-jsonvalue.md index 3590475e..35152c2a 100644 --- a/text/0000-json-encode-jsonvalue.md +++ b/text/0000-json-encode-jsonvalue.md @@ -26,18 +26,14 @@ primitive JsonPrinter """ fun print(value: JsonValue): String iso^ => """Compact JSON serialization of any `JsonValue`.""" - JsonPrinter.print_with_options(value where indent = "", add_newlines = false, add_spaces = false) + _JsonPrint.compact(value) fun pretty(value: JsonValue, indent: String = " "): String iso^ => """Pretty-printed JSON serialization of any `JsonValue`.""" - JsonPrinter.print_with_options(value, indent where add_newlines = true, add_spaces = true) - - fun print_with_options(value: JsonValue, indent: String = "", add_newlines: Bool = false, add_spaces: Bool = false): String iso^ => - """JSON serialization with all possible formatting options exposed. Defaults to compact formatting.""" - // TBD + _JsonPrint.pretty(value, indent) ``` -All the bits and pieces for encoding arbitrary values are there already, they just aren't exposed. `_JsonPrint` is private. +All the bits and pieces for encoding arbitrary values are there already, they just aren't exposed: `_JsonPrint` is private. To avoid confusion and misuse, the methods `JsonObject.string()` and `JsonArray.string()` (and `.string_pretty()`) are being renamed to `JsonObject.print()`, `JsonArray.print()` and `JsonObject.pretty_print()`, `JsonArray.pretty_print()`. This has the side-effect of both `JsonObject` and `JsonArray` and thus `JsonValue` not implementing `Stringable` anymore. @@ -50,7 +46,7 @@ The intend of this primitive is to expose an easy-to-use interface for turning J # How We Test This -The json package already has print-parse roundtrip tests with arbitrary json values. Those need to be adapted to use the public API. Furthermore several "example"-tests are needed, verifying that printing a concrete `JsonValue` yields an expected valid JSON string (i.e. bools, ints, nulls, floats, strings with values needing escape sequences etc.). Those are there to proof that we don't have a common bug in printing and parsing which makes roundtripping work, but produces invalid JSON. +The [`json` package][json-package] already has print-parse roundtrip tests with arbitrary json values. Those need to be adapted to use the public API. Furthermore several "example"-tests are needed, verifying that printing a concrete `JsonValue` yields an expected valid JSON string (i.e. bools, ints, nulls, floats, strings with values needing escape sequences etc.). Those are there to proof that we don't have a common bug in printing and parsing which makes roundtripping work, but produces invalid JSON. # Drawbacks @@ -58,7 +54,7 @@ Renaming `.string()` and `.string_pretty()` on `JsonObject` and `JsonArray` to ` # Alternatives -Function naming alternatives for parsing/printing: +## Function naming alternatives - `parse` / `print` - this would follow the existing naming in the package the closest, hence considered as the best candidate. - `encode` / `decode` - this pair would make sense if the decoding would not expose `JsonParseError`. It feels inconsistent to create a `ParseError` from a `decode` function call. @@ -67,9 +63,25 @@ Function naming alternatives for parsing/printing: Note that this RFC only includes `print` in scope, so renaming the `parse` would be out of scope unless we decided to pursue one of the above alternatives for a broader change. +## Removing pretty + +It has also been considered to condense both `pretty` and `print` into only one exposed function `JsonPrinter.print()`, similar to `JsonPrinter.print_with_options()` below. But this makes it cumbersome to enable pretty-printing: All parameters need to be examined and understood to configure a printing mode which is equivalent to what `.pretty()` would do. + +```pony +primitive JsonPrinter + fun print_with_options(value: JsonValue, indent: String = "", add_newlines: Bool = false, add_spaces: Bool = false): String iso^ => + """JSON serialization with all possible formatting options exposed. Defaults to compact formatting.""" + // TBD +``` + +## Exposing verbose `print_with_options` as third method of JsonPrinter + +Another alternative is to expose `print_with_options` above as third function and define `.print()` and `.pretty()` as aliases with arguments provided that are equivalent to their current formatting. +This would allow for greater control for users, if both `.print()` and `.pretty()` formatting do not fit their needs. This was not suggested as the use-case for further control was considered way too exotic. + # Unresolved questions -- Should the options, like pretty-printing or not, be exposed as optional parameters to `JsonPrinter.print()` instead of exposing `.print()` and `.pretty()`? +None. [json-package]: https://github.com/ponylang/ponyc/tree/main/packages/json [JsonValue]: https://github.com/ponylang/ponyc/blob/main/packages/json/json.pony#L243