workspace add/forget: bridge jj workspaces with git worktrees

[calebdw]

jj workspace add now supports --colocate/--no-colocate flags to control whether a corresponding Git worktree is created alongside the jj workspace. By default, a worktree is created when the current workspace is colocated and the git.colocate config is true.

If Git HEAD does not yet point to a commit (freshly initialized repo with no commits), workspace add errors rather than silently falling back to a non-colocated workspace. This is a Git limitation: git worktree add requires a valid HEAD reference.

jj workspace forget removes the corresponding Git worktree when one exists by deleting the .git gitlink file and pruning the worktree metadata, preserving the workspace directory contents.

The git worktree is created before the jj workspace is initialized, so the directory structure contains both .git (gitlink to the worktree) and .jj/ (pointing back to the shared repo store). Git worktrees are created with relative paths to match jj's convention for portable repositories.

For non-colocated repos, behavior is unchanged.

Checklist

If applicable:

• I have updated CHANGELOG.md
• I have updated the documentation (README.md, docs/, demos/)
• I have updated the config schema (cli/src/config-schema.json)
• I have added/updated tests to cover my changes
• I fully understand the code that I am submitting (what it does,
  how it works, how it's organized), including any code drafted by an LLM.
• For any prose generated by an LLM, I have proof-read and copy-edited with
  an eye towards deleting anything that is irrelevant, clarifying anything
  that is confusing, and adding details that are relevant. This includes,
  for example, commit descriptions, PR descriptions, and code comments.

[PhilipMetzger]

I still think this should be opt-in (with the default being shared with git.colocate = true) and are you aware that this is the third attempt at this which now suddenly has maintainer attention (#8667, #8834 where the previous discussions were with some better abstractions).

[calebdw]

I still think this should be opt-in (with the default being shared with git.colocate = true)

Not sure I see the why worktrees should not be colocated if the main repo is---I considered adding options to disable it, but figured that's just pointless. By virtue of being in a colocated repo there's no real reason why the worktrees should not also be colocated with the workspaces---of course in a non-colocated repo this doesn't matter (and it goes without saying that you should not be able to colocate a workspace if the main workspace is also not colocated)

are you aware that this is the third attempt at this which now suddenly has maintainer attention (#8667, #8834 where the previous discussions were with some better abstractions).

I didn't check before I started working on this (I suppose I should have), I can close out the rest of my PRs if there's better minds than mine already working on this.

My first PR was just merged and lays the groundwork for the worktree colocation work by tracking HEAD per worktree (there was an existing TODO in the code to do this anyway)

cc: @sjawhar

[PhilipMetzger]

I still think this should be opt-in (with the default being shared with git.colocate = true)

Not sure I see the why worktrees should not be colocated if the main repo is---I considered adding options to disable it, but figured that's just pointless. By virtue of being in a colocated repo there's no real reason why the worktrees should not also be colocated with the workspaces---of course in a non-colocated repo this doesn't matter (and it goes without saying that you should not be able to colocate a workspace if the main workspace is also not colocated)

I mean have git.colocate set to false but I'd still like to opt-in to colocated workspaces when necessary, so I don't think I agree with "it goes without saying [...]" just because you don't expect someone to not have it on by default shouldn't mean that they cannot opt-in. Unlike the the relative path PR which you did send earlier providing an option here has value.

I didn't check before I started working on this (I suppose I should have), I can close out the rest of my PRs if there's better minds than mine already working on this.

I would keep it open because it'll hopefully force the maintainers to finally reconcile on the situation.

[calebdw]

I mean have git.colocate set to false but I'd still like to opt-in to colocated workspaces when necessary,

If the main jj repository is not colocated with a git repository, then how do you expect to colocate a child workspace? My point is the "opt-in" is whether or not the repository is colocated---there shouldn't be a separate option to colocate worktrees/workspaces but not the main repository (which technically are a worktree/workspace in and of themselves)

[PhilipMetzger]

If the main jj repository is not colocated with a git repository, then how do you expect to colocate a child workspace?

By working with the bare repo within .jj/.

My point is the "opt-in" is whether or not the repository is colocated---there shouldn't be a separate option to colocate worktrees/workspaces but not the main repository (which technically are a worktree/workspace in and of themselves)

I disagree, since you should be able to opt-in, and its also something we already discussed on Discord (I suggest you to look at the surrounding discussion).

[yuja]

I still think this should be opt-in (with the default being shared with git.colocate = true)

+1

(#8667, #8834 where the previous discussions were with some better abstractions).

I'm open to adding a simpler version of these PRs. IIRC, my main concern with #4644/#4588 was that it was too big and included unnecessary changes. I'm not sure if its successor, #8867, addressed these issues. I'm also not sure if the author of #8867 understands its design and implementation.

[PhilipMetzger]

(#8667, #8834 where the previous discussions were with some better abstractions).

I'm open to adding a simpler version of these PRs.

In what terms do you mean simpler? #8667 now only contains the MaybeColocatedRepo abstraction without any warnings and the minor parts from #4644. The noisy warning from the initial PR also has been removed since I don't think we want that at the moment. So it'd be nice if you finally could take a look at it, since @steadmon and I spent much time on it (so it hopefully takes less of your time) and its also a thing many people are waiting on.

I'm also not sure if the author of #8667 understands its design and implementation.

While this is a fair opinion to have, I don't think it should block work on which so many people are waiting on. Otherwise its something you should discuss on our AI policy issue (#9219) since that just relates to the author and his tool usage.

[yuja]

(#8667, #8834 where the previous discussions were with some better abstractions).

I'm open to adding a simpler version of these PRs.

In what terms do you mean simpler? #8667 now only contains the MaybeColocatedRepo abstraction without any warnings and the minor parts from #4644.

For instance, it would be nice if we could get rid of "workspace: Make backend factories aware of the workspace root". This PR doesn't seem to have that one. (I'm not sure if we can avoid it for the foreseeable future, but it's a controversial change.)

I'm also not sure if the author of #8667 understands its design and implementation.

While this is a fair opinion to have, I don't think it should block work on which so many people are waiting on.

I personally don't want to spend my time repeatedly reviewing patches revived without an understanding of what the original problem was. I understand that this is an important feature, so I'm reviewing this PR stack.

[calebdw]

@yuja, @PhilipMetzger, I've added --colocate/--no-colocate flags to the jj workspace add command to allow overriding the default

[PhilipMetzger]

minor comments

[PhilipMetzger]

(#8667, #8834 where the previous discussions were with some better abstractions).

I'm open to adding a simpler version of these PRs.

In what terms do you mean simpler? #8667 now only contains the MaybeColocatedRepo abstraction without any warnings and the minor parts from #4644.

For instance, it would be nice if we could get rid of "workspace: Make backend factories aware of the workspace root". This PR doesn't seem to have that one. (I'm not sure if we can avoid it for the foreseeable future, but it's a controversial change.)

OK, this is a fair point although you probably did find it OK two years ago? This set of PRs is vastly simpler exactly because it doesn't try to wire itself to all the StoreFactories mechanisms.

I personally don't want to spend my time repeatedly reviewing patches revived without an understanding of what the original problem was.

OK, thanks for clarifying this and since that is a personal opinion it would've been nice to have on the PR since its clearly better than saying nothing for 1,5 years.