Is JJ really distributed? Can history modification be described as adding commits?

[noamraph]

Please forgive me - I just watched this introduction to JJ and read a bit of documentation. So there's a good chance that I am completely wrong.

I tried to understand the underlying model that JJ uses. I understand that it's very similar to git, with the addition of recording conflicts in a commit (which is nice!)

From the introduction that I watched, it seems to me that the main benefit of JJ, besides having a saner UI than git, is that it makes it much easier to modify history. However, all this has to be local - once you share your work, you shouldn't modify the history. Am I correct?

If I'm correct, I'm a bit disappointed. I personally don't like modifying history. Here's a specific example:

Where I work, the standard way to update a pull request to incorporate changes to master is to rebase it. There's even a button in GitLab to do it. But I don't like it. If I press the "rebase" button in GitLab, and run git status in my terminal, it says that my history and the branch have diverged. If instead, I decide to rebase on my machine, I have to use git push -f, and if someone else committed something to the branch (or even I did it from a different directory and forgot about it), the change is going to be deleted. Another problem with this rebase is that it's very hard to go back to the state before the rebase. That's one reason why I much prefer adding a merge commit from master to the branch.

If I'm correct about JJ, it's not going to fix this issue. It may even make it worse, since it'll be easier to rebase!

I think I have an idea for a solution. Can the model be append-only, with new commits having metadata that describe which old commits they deprecate? A rebase won't remove any commits by default, but instead just add new commits with some metadata. When I view the repository, it will look like a normal rebase, but I would be able to see the state before the rebase. If I rebase from one place (say a web UI) and pull from another place (say the terminal), the system will understand what happened, and if I have added changes, the system could automatically rebase them on top of the updated branch.

In other words: it seems to me that the main new thing that JJ brings to the table isn't currently distributed. The actual history is only kept in the local repository, and if it gets deleted then the history is lost, and if you share the state with others (even with a web UI) then many advantages goes away. Perhaps with some extra metadata, JJ could become really distributed, and also really preserve history?

What do you think?

Cheers,
Noam

[arxanas]

Mercurial's commit evolution supports distributed rewriting: https://www.mercurial-scm.org/doc/evolution/

jj used to have commit evolution as a feature, but I believe it was removed as it didn't add much beyond what the underlying operation log supports. I don't think it was ever distributed, either.

The operation log could conceivably be distributed, though. I think it's less obvious how to do so since it's a DAG (?) rather than a grow-only set of edges.

[noamraph]

Thanks! I didn't know that Mercurial has this feature, it's good to know that there's prior art!

(And another reason that makes me sad that Mercurial lost to git.)

jj used to have commit evolution as a feature, but I believe it was removed as it didn't add much beyond what the underlying operation log supports. I don't think it was ever distributed, either.

Perhaps the commit evolution feature didn't add much because it wasn't distributed.

The operation log could conceivably be distributed, though. I think it's less obvious how to do so since it's a DAG (?) rather than a grow-only set of edges.

Distributing the operation log doesn't sound to me like a good idea. The model isn't "Google Docs", where everyone edits the same document simultaneously, but rather everyone working on their version and updating the main database when they see fit. So the operation log is really for my local work. But if I want to modify the history after I shared it outside of my computer, I need commit evolution.

[arxanas]

I don't see why it's a bad idea to distribute the operation log? It would also let us distribute rewrites in the same way as commit evolution. As I understand it, you can implement strictly more functionality by distributing the operation log vs taking the Mercurial approach.

[noamraph]

I guess you can have the server keep the operation log of all users. But you must have a separate operation log for each user. I think it's a significantly more complicated data model than just having a shared set of commits, with some commits marking other commits as obsolete.

[tim-janik]

Another problem with this rebase is that it's very hard to go back to the state before the rebase

JJ makes it very easy to go back to the state before a rebase. You can use jj op log to see the recent operations on a repository.
Using an operation ID, you can look at the history jj log --at-operation <ID> or diffs jj diff --at-operation <ID> of a previous operation. With restore, undo etc, you can also revive old repository state. Here is a screencast that shows a rebase, the op log and jj undo:
https://asciinema.org/a/687049

[noamraph]

Thanks for showing this. I realize that locally it's easy to revert a rebase. But my point is that it remains local. You still can't have a "rebase" button in a web UI and have a good user experience. And if you made a mistake, and you went off to vacation, your coworker can't get to the original commit and find what went wrong.

[tim-janik]

Thanks for showing this. I realize that locally it's easy to revert a rebase. But my point is that it remains local. You still can't have a "rebase" button in a web UI and have a good user experience. And if you made a mistake, and you went off to vacation, your coworker can't get to the original commit and find what went wrong.

If the remote is a JJ repo, or if a cowoker pulls a rebased branch into his own JJ repo, those JJ repos will have their own operation log. And that can just as well be used to undo/revert a rebase that was pulled from a remote instead of rebased locally.

[noamraph]

I agree that it would be possible, but it would require significant effort to dig in logs of remote servers. If instead the model is a set of commits which are shared between everyone, with some commits marking other commits as obsolete, then you have this history available to you like the regular commit history.

[tim-janik]

I agree that it would be possible, but it would require significant effort to dig in logs of remote servers.

You don't need to do that.

• If I have a local JJ repo, I can undo a rebase easily, with jj undo or using the op log.
• If I push into a JJ repo, e.g. from Desktop to Laptop, I can undo the push or parts of it, using the op log on the laptop.
• If I push a rebase onto Github and you pull that into your local JJ repo, you can peek at the op log and revive old repository state from before you pulled.

There is no need to peek into remote logs, your local operation log is all you need.

[noamraph]

Let's say you're working on a pull request, meaning a branch that you publish to a web UI such as GitHub (or a hypothetical web UI which is JJ-based). During the process you rebased your branch on top of the new changes in the master branch. Then you went on a vacation, and I, a co-worker, got the job of merging your pull request. I notice a problem, and I suspect that it is the result of the rebase, so I want to check the code before the rebase. What can I do?

Currently, I need to check the log of the server to find the state of the branch before the rebase. GitHub and GitLab let you do this, but you need to use the web UI to see the history of the pull request, find the commit before the rebase, and fetch it. (In GitLab you have to tag it using the web UI if you want to fetch it to your local machine, I don't know about GitHub). All this is external to the VCS.

With something similar to Mercurial's evolve, the actual history of the branch is available as an integral part of the VCS, with commits having the metadata that shows the original commits from which they were rebased. I think that this will be very valuable to jj as well.