A new repo format is probably not strictly needed for large file support

[manuel]

In https://github.com/radarroark/xit/blob/master/docs/compat.md you write:

In particular, I think better merging and better large file support can't be attained without moving to a new repo format.

I think better large file support could be had without a new repo format: split large files into content-defined chunks, and store those chunks as individual blobs. I think bup does this.

[xeubie]

In reality, you are still making a new format when you do that, no? Git would not read those chunks correctly...it would have no way of piecing them together back into a file. You also presumably would not want to zlib-compress the blobs like git normally does.

[manuel]

Then I guess I don't understand how xit works. Are you storing large files as-is on the Git server?

[xeubie]

The chunking is only how the objects are stored locally. When xit pushes objects to a server, it puts them in the format the server expects (in a pack file, zlib compressed, etc). So I guess you were suggesting storing the chunks as separate blobs in order to get the server to store them that way? Hmm, that's an interesting idea...

[manuel]

So I guess you were suggesting storing the chunks as separate blobs in order to get the server to store them that way?

Yes, that's what I meant.