I've been waiting for the server-side of things to be open-sourced ever since this announcement.
In the meantime, I've been enjoying jj/jujutsu (https://github.com/martinvonz/jj), which started as a 20% project and has been out (and developed) in the open ever since.
"Jujutsu brings to the table a few key concepts — none of which are themselves novel, but the combination of which is really nice to use in practice:
Changes are distinct from revisions: an idea borrowed from Mercurial, but quite different from Git’s model.
Conflicts are first-class items: an idea borrowed from Pijul and Darcs.
The user interface is not only reasonable but actually really good: an idea borrowed from… literally every VCS other than Git."
The server-side IS open source -- it's basically been in the source tree since day 1, but "inert" due to facebook-only dependencies that made the code unusable, but most of the stuff was actually there if you were willing to dig deep. And actually, they just recently said it became "usable for unsupported experimentation"[1], so it's been on my TODO list to give Mononoke a run and see how it operates.
Unfortunately the builds for Mononoke are not actually uploaded on GitHub due to a bug in the GHA setup[2] that I reported :')
In practice, for more "modern" projects, it works fine: they tend to have good hygiene anyway. For Rust projects, for example, you .gitignore the target directory, and you're good for 99% of projects (and cargo new already generates said ignore.)
For older projects that leave temporary files everywhere, or for workflows that may generate intermediate files (like coredumps are often generated in the cwd and not somewhere easily ignored), it can be a pain.
However, in the next release, a feature related to this has landed: you can add a configuration, `snapshot.auto-track="none()"`, to turn off automatically tracking files, and then `jj track` individual paths (with globs, of course), if you prefer that workflow.
It basically works fine, but there are some annoying edge cases that even I, a jj developer, have run into. Probably the most "common" one (and it's not that common, just moreso than the rest) when you switch branches with different `.gitignore` files, you may stop ignoring some junk from the other branch when you switch. Kind of annoying.
But it's overall pretty marginal in projects with good ignore hygiene, and there are solutions to that problem too (to varying degrees of ease.)
I have less experience working in large (heavily tracked active, e.g. 1k+ committers) repos, however we try to keep conflicts to a minimum - of course you can't get rid of them completely, normally merge/rebase I have close to zero conflicts on the regular, or those that I have are automatically (non-ai) merged (correctly).
I suppose I find the idea of conflicts as first class entities as somewhat intriguing, do they function as a sort of super-feature? If e.g. you have 10 features all touching the same area of code, then do you get maybe 3 combined conflict(ing) features?
At any rate, I think while it might be a useful metric, there should be an aim to keep conflicts to an absolute minimum....
One thing that "conflicts as first class items" implies is that commits can exist in a conflicted state. This implies that you don't have to deal with conflicts right away. Let me explain.
jj's rebase works just like git's rebase conceptually: take these commits and move them on top of some different commits. But unlike git, `jj rebase` will aways succeed, and succeed immediately. The resulting rebased commits/changes will just be in a conflicted state if a conflict happens. This decouples the "do the rebase" from "fix the conflicts" in time, meaning I can choose if I want to handle them right away or later.
This ends up being very powerful in a few ways: for one, because it will always succeed, and succeed quickly, we can start automatically rebasing. Let's say you have a branch with three changes on it. You go back and edit the first change, because you forgot to do something. jj will then automatically rebase the two children, and you immediately see if you've introduced a conflict. You can then fix it right now, or you can continue working on change #1, until you're ready to address the conflict. Let's say you decide to do that, and the conflict originates in change #2: when you fix that conflict, #3 gets rebased again, and in this hypothetical story, the conflict goes away.
It's also nice if you have a bunch of outstanding work, and you want to rebase it all on top of your latest trunk/main/master branch: run the command to do so, and it'll all get rebased right away, and you can immediately see what has a conflict and what doesn't, and fix them when you feel like it, rather than doing it right now just so that the command completes.
There are secondary advantages, like the fact that this means rebases can happen in-memory, rather than through materializing files. This means it's super fast, and doesn't interrupt what you're doing. But I think those kinds of things are harder to grok than the workflow improvements.
It doesn't work like that. I should probably rewrite that "first class" part in the docs and use a real example (I wrote it originally), but basically it comes down to this:
When a commit is in a conflicted state, the fact it is conflicted is recorded inside the commit object.
Let's say I have a base B which is the main branch, three commits X Y Z that do 3 different things, and my set of changes that aren't committed, called "@".
B (main) ---> X ---> Y ---> Z ---> @
Now someone pushes to main, so the full graph now actually looks like this with the new main called B'
B ---> B' (main)
\
\---> X ---> Y ---> Z ---> @
Let's say that B' has a change that will cause a conflict in Y. What does that mean? It means that if we were to change the parent of X to B', then the state of Y would be invalid and thus in conflict, because the changes are not compatible with the state of the system.
`jj rebase -d main -s X` will give you a graph just like this, with such a conflict:
B ---> B' ---> X ---> Y ---> Z ---> @
C C C
The marker 'C' means "This commit is conflicted." Note that all descendants of a conflicted commit are conflicted too, unless they solve the conflict. (This sentence is phrased very carefully because I'm about to show you a magic trick.)
Okay... So now in my filesystem, I can go see the conflict and I resolve it. Maybe Y renamed a variable that B' got rid in file foo.c, or something.
So now I can solve this conflict. But how do I resolve it? This is too much of a topic to discuss in general but here is the magic trick:
- Solve the conflict in your working copy
- Move the conflict resolution into the first conflicted commit, Y
- The conflict resolution will be propagated to all descendants, just the same way the conflict itself was propagated.
Step 1: solve the conflict in your working copy. Now my history looks like this.
B ---> B' ---> X ---> Y ---> Z ---> @
C C
Note: @ is no longer conflicted! We solved the conflict there, so it is OK. Now how do we resolve the conflict in Y and Z?
Step 2: `jj squash --from @ --into Y --interactive` will move any changes you select in a diff editor and then move that diff into the other commit.
Now the graph looks like this:
B ---> B' ---> X ---> Y ---> Z ---> @
I moved the resolution of the conflict into Y. And so the resolution of the conflict is propagated to Z.
Step 3: There is no step 3. You are done.
So the secret is that, Jujutsu tracks conflicts and the relationships between conflicts in the commit graph, just like commits. This is why they are "first class." Git basically doesn't do any of this. A commit with a conflict and a commit without one are indistinguishable in Git, unless you look at the actual diff and see rejected hunk markers. A conflicted hunk in a modified file in Git is no different than any other hunk.
This is already too long but as an addendum what I used to do is describe the conflict support as "git rebase --update-refs, combined with git rerere, on 1000x steroids." Except that's actually a shit way to describe this functionality, because only like 5 people on Planet Earth know about --update-refs or rerere. So you really need to experience it yourself or see a step-by-step, I'm afraid.
The readme did not make it clear: does this have a TUI workflow for moving line chunks around?
That is my one reluctance towards jj is that my (bad practices?) frequently have me touch a few different unrelated bits simultaneously (eg editing foo, but fix a typo in bar) which I use stage to iteratively commit. From the jj stuff I have read thus far, my workflow would be a bit clunky to adapt.
To be fair, the website has changed very little since OSS launch (when this blog post first came out) and almost all of the main points of the blog are still the same, and the selling points are the same, too. So, I think it's fine, but definitely the website is the next place to look, yes.
Probably the biggest major changes in Sapling I can personally think of are the introduction of the experimental "Dotgit" mode (.git exists next to .sl) and that the server-side stuff has recently become more usable. But none of it is stable yet, so for users not much has changed (yet).
My biggest issue with Juijitsu and Sapling (especially) is compatibility with a repo where nobody else uses these tools. Sapling's problem is pretty obvious: I would have to get others to use FBs merge stack tool and get that past security approval.
JJ? I spent a day trying to rebase/merge from trunk into my PR branch and truly fucked it up in a way that I have never managed with plain old Git - Google results were pretty scarce and unhelpful. For Git, I `switch foo; rebase main; push --force-with-lease`, for JJ (apparently) I `rebase -b main -d foo` - great! How do I access the results of the rebase and update my PR branch? How do I force push that branch? It feels like the documentation is in the same place that Git was in during the early days - I assumes you are deeply familiar with the idiomatic workflow. The effort put into the migration guide is minimal[1].
> How do I access the results of the rebase and update my PR branch?
So, one small difference between jj and git is that jj's branches don't move automatically if you add new changes on top of them. In the next jj release, "branches" will be renamed to "bookmarks" to kinda emphasize this difference, even though they're used for git interop. Additionally, there's some discussion about what making branches/bookmarks move like git branches move could mean, with an experimental setting to give that a try.
With all of that said, when you rebased, it shouldn't have changed where the head of your branch was pointed to, so you shouldn't have needed to actually move anything here. This is a side effect of the change/commit distinction: the branch points to a change, so when you rebase your stuff, the commits will change, but the change will not. That means all you needed to do was...
> How do I force push that branch?
`jj git push` will force push all of your changes.
> It feels like the documentation is in the same place that Git was in during the early days - I assumes you are deeply familiar with the idiomatic workflow.
I agree that the documentation isn't in a super awesome place yet, but I think the cause is kind of the opposite: there isn't really one idiomatic workflow, but instead a few common ones, and a hesitation to demand that a specific workflow is idiomatic.
That being said, I have an in-progress tutorial that I have been waiting for two recent changes to land (one being the branch -> bookmark rename, the other being a different rename (obslog -> evolog)), that is intended to land upstream once I'm done with it: https://steveklabnik.github.io/jujutsu-tutorial/
After that, if I can keep finding the energy, I'd love to keep improving the documentation, but I have a lot less FOSS energy than I used to, so we'll see.
Thanks for taking the time to address my concerns. I am still very much open to trying it again in the future, but just like you I have time constraints and have to find a time box for it.
What I would really like to see an extremely simple walkthrough that covers some really basic scenarios without the ceremony of setting up a new JJ repo, for busy people who want to pilot something new - that covers all of:
* Central repository using branches (single remote) OR GitHub-style forked repository (multiple remotes)
* Merge-based workflow OR rebase-based workflow, both without conflicts
Using github.com/martinvonz/jj as an example repo - show us how to draw the entire owl
Take me from opening a terminal in order to make my changes, to creating a commit with those changes, merging/rebasing the trunk to my branch, to pushing it to my remote branch. Remember: I (and the rest of the 80%) have a job to do, so we want the cliff notes to reach success, and then we can dive deeper into things as we encounter them naturally.
Once you have the trivial workflow out of the way you can progressively disclose other things. For example, for conflict resolution/merging, you could start with an example that's already in a conflicted state and walk through that - i.e. an example that solely focuses on a specific issue. Remember, your new users want to achieve things first - think from their perspective "oh shit, this rebase has resulted in a conflict, what do I do?" or "my team uses gitflow, how do I interact with that?" Tiny little tutorials that cover common scenarios go a lot further than a monolithic detailed tutorial.
oh hey- that's your guide? It's awesome, highly recommend to anyone else wanting to give jujutsu a try! ... I read through it when I didn't know what jujutsu was. It not only convinced me to try jujutsu out, but laid the groundwork for me to get started really quickly and easily.
If you come into jj and try to just use it without going through the tutorial[1] you’ll just gert frustrated and disappointed. Jj is not git and the workflow is very much different, arguably far better than git. I went through the tutorial first and had no issues at all and I actually understood the how it works under the hood as a VCS. With git I’ve never understood how it works and just used clone, rebase, commit, push.
Seems like you had a bad day. Luckily the day I tried it it occurred me to first RTFM.
1: Steve Klabnik’s tutorial is arguably the best one atm.
> Sapling's problem is pretty obvious: I would have to get others to use FBs merge stack tool and get that past security approval.
You don't have to use Sapling's integration at all, FWIW. I completely avoided PR functionality when I first started using it, and would just do a branch for every commit (which is what it would do for you, anyway.) I think almost every tool imaginable can do a diff between branches?
> for JJ (apparently) I `rebase -b main -d foo`
These arguments are backwards. What you want is to flip the -d and -b switches which mean "destination" and "branch" respectively:
jj rebase -d main -b whatever-branch-you-want
jj git push -b whatever-branch-you-want
You don't have to switch to any branches or do anything else to make this work. These two commands will basically work no matter what the state of the tree is.
> It feels like the documentation is in the same place that Git was in during the early days
Yeah. We've talked about this. It's not great but we have so many things going on right now, nobody has totally taken over documentation. Steve Klabnik's Jujutsu tutorial is very popular and we've even considered asking if we could rewrite our documentation using it, but again, so little time.
I used it for the last year or two at a company where all of our source control was with github. It has bindings to interact with github (not sure about what else) that allowed me to use it locally without anyone else having to change their workflows/install anything. Granted, I was mostly using it for stacked pr management among a few other things and not really fully taking advantage of all of it.
Yes, it uses Git as the default backend, so it's more or less just a different interface to a Git repository. Everyone today uses it this way.
The server-side components have been open source, but not fully usable due to fb-only code. That's changing and you can in theory build a working OSS server now that works on mysql/s3, but it isn't supported yet.
You can do that with Sapling, in fact it's both very easy and conceptually more robust than Git because with Sapling you are just manipulating commits. Sapling also includes `sl web` which makes interactive commits, splitting, and rebasing a breeze. In Sapling, commits are easy to manipulate, so there's nearly no need for the staging area, unlike Git where the staging area is easy to manipulate but commits are unreasonably hard. You can stage temporary changes to your hearts desire with a slight mindset shift.
Readit News
In the meantime, I've been enjoying jj/jujutsu (https://github.com/martinvonz/jj), which started as a 20% project and has been out (and developed) in the open ever since.
An introduction by Chris Krycho for those familiar with some form of vcs: https://v5.chriskrycho.com/essays/jj-init/
"Jujutsu brings to the table a few key concepts — none of which are themselves novel, but the combination of which is really nice to use in practice:
Changes are distinct from revisions: an idea borrowed from Mercurial, but quite different from Git’s model. Conflicts are first-class items: an idea borrowed from Pijul and Darcs. The user interface is not only reasonable but actually really good: an idea borrowed from… literally every VCS other than Git."
Unfortunately the builds for Mononoke are not actually uploaded on GitHub due to a bug in the GHA setup[2] that I reported :')
[1] https://github.com/facebook/sapling?tab=readme-ov-file#monon...
[2] https://github.com/facebook/sapling/issues/922
It feels like you'd end up with a bunch of junk in your repo.
For older projects that leave temporary files everywhere, or for workflows that may generate intermediate files (like coredumps are often generated in the cwd and not somewhere easily ignored), it can be a pain.
However, in the next release, a feature related to this has landed: you can add a configuration, `snapshot.auto-track="none()"`, to turn off automatically tracking files, and then `jj track` individual paths (with globs, of course), if you prefer that workflow.
But it's overall pretty marginal in projects with good ignore hygiene, and there are solutions to that problem too (to varying degrees of ease.)
I have less experience working in large (heavily tracked active, e.g. 1k+ committers) repos, however we try to keep conflicts to a minimum - of course you can't get rid of them completely, normally merge/rebase I have close to zero conflicts on the regular, or those that I have are automatically (non-ai) merged (correctly).
I suppose I find the idea of conflicts as first class entities as somewhat intriguing, do they function as a sort of super-feature? If e.g. you have 10 features all touching the same area of code, then do you get maybe 3 combined conflict(ing) features?
At any rate, I think while it might be a useful metric, there should be an aim to keep conflicts to an absolute minimum....
jj's rebase works just like git's rebase conceptually: take these commits and move them on top of some different commits. But unlike git, `jj rebase` will aways succeed, and succeed immediately. The resulting rebased commits/changes will just be in a conflicted state if a conflict happens. This decouples the "do the rebase" from "fix the conflicts" in time, meaning I can choose if I want to handle them right away or later.
This ends up being very powerful in a few ways: for one, because it will always succeed, and succeed quickly, we can start automatically rebasing. Let's say you have a branch with three changes on it. You go back and edit the first change, because you forgot to do something. jj will then automatically rebase the two children, and you immediately see if you've introduced a conflict. You can then fix it right now, or you can continue working on change #1, until you're ready to address the conflict. Let's say you decide to do that, and the conflict originates in change #2: when you fix that conflict, #3 gets rebased again, and in this hypothetical story, the conflict goes away.
It's also nice if you have a bunch of outstanding work, and you want to rebase it all on top of your latest trunk/main/master branch: run the command to do so, and it'll all get rebased right away, and you can immediately see what has a conflict and what doesn't, and fix them when you feel like it, rather than doing it right now just so that the command completes.
There are secondary advantages, like the fact that this means rebases can happen in-memory, rather than through materializing files. This means it's super fast, and doesn't interrupt what you're doing. But I think those kinds of things are harder to grok than the workflow improvements.
When a commit is in a conflicted state, the fact it is conflicted is recorded inside the commit object.
Let's say I have a base B which is the main branch, three commits X Y Z that do 3 different things, and my set of changes that aren't committed, called "@".
Now someone pushes to main, so the full graph now actually looks like this with the new main called B' Let's say that B' has a change that will cause a conflict in Y. What does that mean? It means that if we were to change the parent of X to B', then the state of Y would be invalid and thus in conflict, because the changes are not compatible with the state of the system.`jj rebase -d main -s X` will give you a graph just like this, with such a conflict:
The marker 'C' means "This commit is conflicted." Note that all descendants of a conflicted commit are conflicted too, unless they solve the conflict. (This sentence is phrased very carefully because I'm about to show you a magic trick.)Okay... So now in my filesystem, I can go see the conflict and I resolve it. Maybe Y renamed a variable that B' got rid in file foo.c, or something.
So now I can solve this conflict. But how do I resolve it? This is too much of a topic to discuss in general but here is the magic trick:
- Solve the conflict in your working copy
- Move the conflict resolution into the first conflicted commit, Y
- The conflict resolution will be propagated to all descendants, just the same way the conflict itself was propagated.
Step 1: solve the conflict in your working copy. Now my history looks like this.
Note: @ is no longer conflicted! We solved the conflict there, so it is OK. Now how do we resolve the conflict in Y and Z?Step 2: `jj squash --from @ --into Y --interactive` will move any changes you select in a diff editor and then move that diff into the other commit.
Now the graph looks like this:
I moved the resolution of the conflict into Y. And so the resolution of the conflict is propagated to Z.Step 3: There is no step 3. You are done.
So the secret is that, Jujutsu tracks conflicts and the relationships between conflicts in the commit graph, just like commits. This is why they are "first class." Git basically doesn't do any of this. A commit with a conflict and a commit without one are indistinguishable in Git, unless you look at the actual diff and see rejected hunk markers. A conflicted hunk in a modified file in Git is no different than any other hunk.
This is already too long but as an addendum what I used to do is describe the conflict support as "git rebase --update-refs, combined with git rerere, on 1000x steroids." Except that's actually a shit way to describe this functionality, because only like 5 people on Planet Earth know about --update-refs or rerere. So you really need to experience it yourself or see a step-by-step, I'm afraid.
That is my one reluctance towards jj is that my (bad practices?) frequently have me touch a few different unrelated bits simultaneously (eg editing foo, but fix a typo in bar) which I use stage to iteratively commit. From the jj stuff I have read thus far, my workflow would be a bit clunky to adapt.
Will try it for sure.
Probably the biggest major changes in Sapling I can personally think of are the introduction of the experimental "Dotgit" mode (.git exists next to .sl) and that the server-side stuff has recently become more usable. But none of it is stable yet, so for users not much has changed (yet).
JJ? I spent a day trying to rebase/merge from trunk into my PR branch and truly fucked it up in a way that I have never managed with plain old Git - Google results were pretty scarce and unhelpful. For Git, I `switch foo; rebase main; push --force-with-lease`, for JJ (apparently) I `rebase -b main -d foo` - great! How do I access the results of the rebase and update my PR branch? How do I force push that branch? It feels like the documentation is in the same place that Git was in during the early days - I assumes you are deeply familiar with the idiomatic workflow. The effort put into the migration guide is minimal[1].
/rant
[1]: https://martinvonz.github.io/jj/latest/git-comparison/
> Google results were pretty scarce and unhelpful.
This is true, the Discord is very active and helpful though: https://discord.gg/bKWAbtpH Discord isn't for everyone, but you can get help quickly. You can also ask questions as GitHub Discussions: https://github.com/martinvonz/jj/discussions
> How do I access the results of the rebase and update my PR branch?
So, one small difference between jj and git is that jj's branches don't move automatically if you add new changes on top of them. In the next jj release, "branches" will be renamed to "bookmarks" to kinda emphasize this difference, even though they're used for git interop. Additionally, there's some discussion about what making branches/bookmarks move like git branches move could mean, with an experimental setting to give that a try.
With all of that said, when you rebased, it shouldn't have changed where the head of your branch was pointed to, so you shouldn't have needed to actually move anything here. This is a side effect of the change/commit distinction: the branch points to a change, so when you rebase your stuff, the commits will change, but the change will not. That means all you needed to do was...
> How do I force push that branch?
`jj git push` will force push all of your changes.
> It feels like the documentation is in the same place that Git was in during the early days - I assumes you are deeply familiar with the idiomatic workflow.
I agree that the documentation isn't in a super awesome place yet, but I think the cause is kind of the opposite: there isn't really one idiomatic workflow, but instead a few common ones, and a hesitation to demand that a specific workflow is idiomatic.
That being said, I have an in-progress tutorial that I have been waiting for two recent changes to land (one being the branch -> bookmark rename, the other being a different rename (obslog -> evolog)), that is intended to land upstream once I'm done with it: https://steveklabnik.github.io/jujutsu-tutorial/
After that, if I can keep finding the energy, I'd love to keep improving the documentation, but I have a lot less FOSS energy than I used to, so we'll see.
What I would really like to see an extremely simple walkthrough that covers some really basic scenarios without the ceremony of setting up a new JJ repo, for busy people who want to pilot something new - that covers all of:
* Central repository using branches (single remote) OR GitHub-style forked repository (multiple remotes)
* Merge-based workflow OR rebase-based workflow, both without conflicts
Using github.com/martinvonz/jj as an example repo - show us how to draw the entire owl
Take me from opening a terminal in order to make my changes, to creating a commit with those changes, merging/rebasing the trunk to my branch, to pushing it to my remote branch. Remember: I (and the rest of the 80%) have a job to do, so we want the cliff notes to reach success, and then we can dive deeper into things as we encounter them naturally.
Once you have the trivial workflow out of the way you can progressively disclose other things. For example, for conflict resolution/merging, you could start with an example that's already in a conflicted state and walk through that - i.e. an example that solely focuses on a specific issue. Remember, your new users want to achieve things first - think from their perspective "oh shit, this rebase has resulted in a conflict, what do I do?" or "my team uses gitflow, how do I interact with that?" Tiny little tutorials that cover common scenarios go a lot further than a monolithic detailed tutorial.
Seems like you had a bad day. Luckily the day I tried it it occurred me to first RTFM.
1: Steve Klabnik’s tutorial is arguably the best one atm.
You don't have to use Sapling's integration at all, FWIW. I completely avoided PR functionality when I first started using it, and would just do a branch for every commit (which is what it would do for you, anyway.) I think almost every tool imaginable can do a diff between branches?
> for JJ (apparently) I `rebase -b main -d foo`
These arguments are backwards. What you want is to flip the -d and -b switches which mean "destination" and "branch" respectively:
You don't have to switch to any branches or do anything else to make this work. These two commands will basically work no matter what the state of the tree is.> It feels like the documentation is in the same place that Git was in during the early days
Yeah. We've talked about this. It's not great but we have so many things going on right now, nobody has totally taken over documentation. Steve Klabnik's Jujutsu tutorial is very popular and we've even considered asking if we could rewrite our documentation using it, but again, so little time.
https://steveklabnik.github.io/jujutsu-tutorial/introduction...
(I'm one of the Jujutsu developers, in case it wasn't clear from the last paragraph.)
What's new since? Where are we seeing some adoption? Any progress on:
> When used with our Sapling-compatible server and virtual file system (we hope to open-source these in the future),
Is there significant git/sapling adoption at Meta, or is mercurial/eden still the main thing folks use?
It's a neat idea. The smartlog command being a top level entrance point to a subdirectory in a monorepo seems smart as heck, a fine start.
The server-side components have been open source, but not fully usable due to fb-only code. That's changing and you can in theory build a working OSS server now that works on mysql/s3, but it isn't supported yet.
Besides that, there are powerful "split" feature to split a commit into smaller commits or move changes between commits.