Plugin Reload: Migrate to System.Text.Json

[Jorteck]

With the addition of the NUI API, an extra dependency was added to Newtonsoft.Json due to its simplicity for serializing polymorphic objects.

Unfortunately it has a few issues from sometimes holding hard dependencies to plugins, preventing AssemblyLoadContexts from being unloading and causing hot reloads to fail.

[Jorteck]

Should not start working on this until we have appropriate test coverage: #463

[Samuel-IH]

Just saw this!
While this is a long ways away, I'd like to mention that System.Text.Json may not solve the issue. I've used it in one of my plugins to help with hot reloading with no success. I certainly could be missing something in my setup, but this issue was pretty convincing to me: dotnet/runtime#65323

I ended up having to use Jil instead: https://github.com/kevin-montrose/Jil

[zunath]

I had a hard time working with JIL - the serialization it uses is pretty finicky with what I was doing. If you're doing simple (de)serialization it's probably fine but it wasn't covering my use case.

I did find a hack that works to clear the cache during a Dispose call in a service. It ain't pretty but even Microsoft acknowledges it: https://github.com/dotnet/runtime/blob/dae890906431049d32e24d498a1d707a441a64a8/src/libraries/System.Text.Json/tests/System.Text.Json.Tests/Serialization/CacheTests.cs#L213

private void ClearJsonCache()
{
    Assembly stjAssembly = typeof(JsonSerializer).Assembly;

    Type? updateHandlerType = stjAssembly.GetType("System.Text.Json.JsonSerializerOptionsUpdateHandler", throwOnError: false);
    if (updateHandlerType == null)
    {
        throw new InvalidOperationException("Could not find System.Text.Json.JsonSerializerOptionsUpdateHandler in the assembly.");
    }

    MethodInfo? clearCacheMethod = updateHandlerType.GetMethod("ClearCache",
        BindingFlags.Public | BindingFlags.Static);

    if (clearCacheMethod == null)
    {
        throw new InvalidOperationException("Could not find ClearCache method on JsonSerializerOptionsUpdateHandler.");
    }

    clearCacheMethod.Invoke(null, [null]);
}

So far so good but I need to do further testing.