hook to write transformed data as JSON string

[bob-carpenter]

It would be nice to be able to recover the transformed data generated with RNGs and even without RNGs. There's no way to do that now.

To be usable, it will need a hook through the services in the stan-dev/stan repo, for which there is an existing issue:

stan-dev/stan#3349

[WardBrian]

This would be reasonable. Do we think it should also include the data, or just the transformed data?

[bob-carpenter]

I think just the transformed data, because the user will already have the data and copying it will use space.

[WardBrian]

I think we will have some technical troubles doing this, since the best method we currently have is the structured_writer class, which the generated code could use, but because users can write anything in the transformed data we would need a lot more overloads than are currently available (we would in fact need it to be fully templated, which I'm not sure we can do, since it is used as a virtual base class)

[spinkney]

I sometimes have the need to recover transformed data and I just pass it again to generated data. Couldn't you just pass all the transformed data into the generated data output in the backend?

[WardBrian]

If we didn't mind flattening it, re-using the machinery we do for write_array is definitely possible. It would be nicer as JSON, though

[bob-carpenter]

@spinkey: By "generated data" do you mean the result of the generated quantities block? The issue there is that it outputs every iteration and the transformed data might be huge. So I don't think we'd want to tag it into write_array() but rather have a separate way to recover it.

[spinkney]

@spinkey: By "generated data" do you mean the result of the generated quantities block? The issue there is that it outputs every iteration and the transformed data might be huge. So I don't think we'd want to tag it into write_array() but rather have a separate way to recover it.

Ah yes, I wasn't thinking about the per sampling iteration. Anyway, I'd love to have this as I often want to check if what I'm doing is correct in the transformed data block.