more lenient json parsing

[david-andrew]

During a run, the agent initially made a mistake with one of its inputs. When it went back to redo the action, it included a comment next to the part that was adjusted. The new input was correct, but since json normally doesn't support comments, parsing failed, forcing the agent to try again.

Consider allowing the input json to contain comments. This would probably require either pre-filtering any comments before parsing:

def remove_comments(json_string):
    # This regex matches C-style comments (// or /* */)
    json_without_comments = re.sub(r'(/\*.*?\*/|//[^\r\n]*$)', '', json_string, flags=re.MULTILINE|re.DOTALL)
    return json_without_comments

or would require using a library that supports comments in json, such as json5

Here's the relevant portions of the instance I noticed this:

thought: I need to test tool9, which requires specific input parameters.
tool: tool9
tool_input: {'item': {'a': {'a': 5, 'b': []}, ... }}}

error: error running tool "tool9": 1 validation error for M3
a.b
  Input should be a valid integer [type=int_type, input_value=[], input_type=list]
    For further information visit https://errors.pydantic.dev/2.7/v/int_type
⠇ thinking...[2024-08-22 13:44:02,408] {_client.py:1026} INFO - HTTP Request: POST https://api.openai.com/v1/chat/completions "HTTP/1.1 200 OK"
error: failed to parse action. Action must be a single valid json dictionary {"thought": ..., 
"tool": ..., "tool_input": ...}. There may not be any text or comments outside of the json 
object. Your input was: {
  "thought": "The input for tool9 is invalid because the 'b' field in 'a' should be an integer 
instead of an empty list. I will correct this and try again.",
  "tool": "tool9",
  "tool_input": {
    "item": {
      "a": {
        "a": 5,
        "b": [1]  // Providing a valid integer in the list
      },
     ...
    }
  }
}