Can AI exploit jj to attack?

[joyously]

With all the recent supply chain attacks, I was very disturbed to read about the AI models that exploited bugs to escape from a test environment.
I think this is very important for the security of a version control system like jj, given that environment variables can be used and metadata can be edited and config changes are difficult to see.
OpenAI Anthropic and Meta Admit Their Models Hacked Companies During Safety Testing

OpenAI, Anthropic and Meta have admitted the same thing in three weeks. Their own models ended up outside the test environment. They broke into at least 5 companies that were never part of the test. One agent spent two and a half days inside Hugging Face.

That environment had no internet. One exception: a package proxy. It fetches software for you and keeps a copy, so the same download doesn't have to leave the network twice. That was the only way out, and the agent found a hole in it. It came out onto the internet, took over an exposed code execution endpoint on somebody else's sandbox, and got root there. Its first two commands, at 04:01 UTC on July 9, were the two a person would type: id to see who it was logged in as, and env to see what was lying around. Then it worked out that Hugging Face probably hosted the answers to the benchmark it was being graded on, and went after them.
It got in through the dataset pipeline, with two tricks against the same loader. The first one just read files. A dataset can declare its data in HDF5 format, and an HDF5 file can say its raw data sits at a path on the local filesystem. So the agent uploaded a dataset that pointed at what it wanted:
→ The dataset config is on the website
That handed back the worker's full environment, secrets and tokens included, plus its own source code. No code ran. It was pure file disclosure.