CI: custom json formatter

[mgovers]

In light of recent events, we are aiming to reduce our amount of dependencies - in particular the npm ones. Unfortunately, that means we need to revert some of the changes in #1256 in favor of small custom scripts.

Relates to #1499

[mgovers]

we need this, or there will be some ugly error-prone repeated code between the code quality workflow and this repo

[figueroa1395]

I'd argue that if we add this custom formatter, let's also include the following three

re.compile(rf"{REPO_PATH}/tests/data/.*/input\.json"),
re.compile(rf"{REPO_PATH}/tests/data/.*/.*_output.*\.json"),
re.compile(rf"{REPO_PATH}/tests/data/.*/update_batch\.json"),

I don't see what's the problem. We would get a BIG PR but it's just formatting.

[mgovers]

currently, we do a lot of compact formatting in our input/output/update. that would no longer be possible and also significantly increase test data size with literally only whitespaces due to indentation. this was also discussed when introducing biome

[kornerc]

@mgovers I like to idea to get rid of npm.

One alternative came into my mind to a custom JSON formatter: Biome has standalone binaries (it’s written in Rust).
However, I think it’s unfortunately rather difficult to integrate this nicely into pre-commit and to manage the versions properly.

Unfortunately, this PR to publish Biome as Python package is currently not progressing biomejs/biome#8818

In summary I guess a custom JSON formatter - like your drafted one - might be the best solution (until there is a Python package for Biome).

[mgovers]

@mgovers I like to idea to get rid of npm.

One alternative came into my mind to a custom JSON formatter: Biome has standalone binaries (it’s written in Rust). However, I think it’s unfortunately rather difficult to integrate this nicely into pre-commit and to manage the versions properly.

Unfortunately, this PR to publish Biome as Python package is currently not progressing biomejs/biome#8818

In summary I guess a custom JSON formatter - like your drafted one - might be the best solution (until there is a Python package for Biome).

Hi @kornerc,

Thank you for the input. We really appreciate it, especially after the great in-depth investigation you have done so far. 🤞 fingers crossed that that PR for biome will be published soon...

[figueroa1395]

This is generally fine by me, but let's discuss with the rest of the team before proceeding.

[figueroa1395]

I'd argue that if we add this custom formatter, let's also include the following three

re.compile(rf"{REPO_PATH}/tests/data/.*/input\.json"),
re.compile(rf"{REPO_PATH}/tests/data/.*/.*_output.*\.json"),
re.compile(rf"{REPO_PATH}/tests/data/.*/update_batch\.json"),

I don't see what's the problem. We would get a BIG PR but it's just formatting.

[kornerc]

*.jsonc with comments or trailing commas cannot be loaded by json.loads().

Comment:

import json

print(json.loads(
"""
{
  // comment
  "a": 1
}
"""
))

Trailing comma:

import json

print(json.loads(
"""
{
  "a": 1,
}
"""
))

Even if json.loads() would be able to loads .jsonc files with comments, the comments would get lost.

Therefore, it might be a good decision to not format .*jsonc files

[mgovers]

good catch

[sonarqubecloud]

Quality Gate passed

Issues
0 New issues
0 Accepted issues

Measures
0 Security Hotspots
0.0% Coverage on New Code
0.0% Duplication on New Code

See analysis details on SonarQube Cloud