docs/design: Topics

Continues the conversation started in Discord and hopefully concludes it with a final
design.

docs/design/topics.md

@@ -0,0 +1,187 @@
+# Topics
+
+Authors: [Philip Metzger](mailto:[email protected]), [Noah Mayr](mailto:[email protected])
+ [Anton Bulakh](mailto:[email protected])
+
+## Summary
+
+Introduce Topics as a truly Jujutsu native way for topological branches, which
+also replace the current bookmark concept for Git interop. As they have been
+documented to be confusing users coming from Git. They also supersede the
+`[experimental-advance-branches]` config for those who currently use it, as
+such a behavior will be built-in for Topics.
+
+Topics have been discussed heavily since their appearance in
+[this Discussion][gh-discuss]. As Noah, Anton and I had a long
+[Discord discussion][dc-thread] about them, which then also poured into the
+[Topic issue][issue].
+
+## Prior work
+
+Currently there only is Mercurial which has a implementation of
+[Topics][hg-topics]. There also is the [Topic feature][gerrit-topics] in Gerrit,
+which groups commits with a single identifier.
+
+
+## Goals and non-goals
+
+### Goals
+
+The goals for this Project are small, see below.
+
+* Introduce the concept of native topological branches for Jujutsu.
+* Simplify Git interop by reducing the burden on Bookmarks.
+* Remove the awkward `bookmark` to Git `branch` mapping.
+* Add Change metadata as a storage concept.
+
+### Non-Goals
+
+* Making bookmarks unnecessary.
+
+## Overview
+
+Until now, Jujutsu had no native set of topological branches, just
+[Bookmarks][bm] which interact poorly with Git's expectation of branches.
+Topics on the otherhand are can be made to represent Git branches as users
+expect them, see [Julia Evans poll][jvns-poll]. They also allow us to
+seamlessly take over the [tracking-branches][tb] concept.
+
+Other use-cases they're useful for are representing a set of
+[archived commits][archived] or even a [checkout history][checkout].
+
+
+#### Behavior
+
+A Topic is a set of Changes marked with a name, which infectiously moves
+forward with each descendant you create. All Changes without a named topic are
+implicitly in a anonymous topic, which gets separately tracked as soon as you
+send it in for review (so materialize it as a Git branch). A Topic may have
+multiple heads. A single Change may belong to multiple Topics, which means
+that they also can get exported as overlapping Git branches.
+
+
+TODO: Example here
+
+#### Major Use-Cases
+
+
+##### Obsoletion of `[experimental.auto-advance-bookmarks]`
+
+Since Topics are infectious by nature, they can perfectly map to a Git branch.
+This alleviates the need for the auto-advancing bookmarks. It also makes the
+use-case of the solo developer working on Git `main` easier, as you just mark
+all descendants of `trunk()` belonging to the `main` topic, which then gets
+translated to the `main` branch on `jj git push`.
+
+
+##### Change Archival
+
+For the archival use-case the infectious property of a Topic isn't as
+important. So having a
+`jj topic create -r 'trunk()..@ & description("archive:")' should mark them all
+revisions with a `archive:` description as archived. For this use-case the
+non-contiguous property of Topics really shine.
+
+##### Checkout history
+
+For the checkout history use-case, we can each Change after either a `jj edit`
+or `jj new` onto the `checkout` topic, which is is non-contiguous by default.
+
+### Detailed Design
+
+#### Git Interop
+
+For all contiguous Topics we can just simply export them as Git Branches. For
+Topics which contain non-contiguous parts, we should allow exporting them
+one-by-one or with a generated name, such as `<topic-name>-1` for each part.
+A single Topic which is representable as a Git branch is defined with this
+revset ``.
+
+
+##### Nested branches
+
+Since Git allows a branch `foo` to be a child of another branch `bar`, each
+pointing to a different Git ref. Topics should be able to reflect these
+longliving branches, as we still can use the respective bookmarks pointing
+at the common ancestor.
+
+TODO: how again, Discord Thread?
+
+#### New Revsets
+
+#### Storage
+
+We should store `Topics` as metadata on the serialized proto, without
+considering the resulting Gencode. To avoid rewriting a Commit on each metadata
+change, we need to teach the "ContentHash" macro to ignore fields.As Topics
+will be stored on _each_ Commit struct.
+
+
+```protobuf
+// A simple Key-Value pair.
+message StringPair {
+  string key = 1;
+  string value = 2;
+  // Could be extended by a protobuf.Any see the future possibilities section.
+}
+
+message Commit {
+  //...
+  repeated StringPair metadata = N;
+}
+```
+
+while the actual code should look like this:
+
+```rust
+#[derive(ContentHash, ...)]
+struct Commit {
+  //...
+  //
+  // This avoids rewriting the Change-ID, but must be implemented.
+  #[ContentHash(ignore)]
+  metadata: HashMap<String, String>
+}
+```
+
+#### Backend implications
+
+If Topics were stored as commit metadata, it would allow backends to drop
+them if necessary. This property can be useful to mark tests as passing
+on a specific client or avoiding a field entirely in database backed backends.
+
+For the Git backend, we could either embed them in the message, like Arcanist
+or Gerrit do or store them as Git Notes, if necessary.
+
+## Alternatives considered
+
+### Storing Topics out-of-band
+
+See [Noah's prototype][prototype] for the variant of keeping them out of band.
+While this works it falls short of having the metadata synced by multiple
+clients, which is something desirable. The prototype thus also avoids rewriting
+the Change-ID which is a good thing, but makes them only locally available.
+
+
+### Single Head Topics
+
+While these are conceptually simpler, they wouldn't help with Git interop where
+it is useful to map a single underlying Topic to multiple Git branches. This
+also worsens the `jj`-`Git` interop story.
+
+## Future Possibilities
+
+In the future we could attach a `google.protobuf.Any` to the Change metadata,
+which would allow specific clients, such as testrunners to directly attach test
+results to a Change which could be neat.
+
+[archived]: https://github.com/martinvonz/jj/discussions/4180
+[bm]:  ../bookmarks.md
+[checkout]: https://github.com/martinvonz/jj/issues/3713
+[dc-thread]: https://discord.com/channels/968932220549103686/1224085912464527502
+[gerrit-topics]: https://gerrit-review.googlesource.com/Documentation/cross-repository-changes.html
+[gh-discuss]: https://github.com/martinvonz/jj/discussions/2425#discussioncomment-7376935
+[hg-topics]: https://www.mercurial-scm.org/doc/evolution/tutorials/topic-tutorial.html#topic-basics
+[issue]: https://github.com/martinvonz/jj/discussions/2425#discussioncomment-7376935
+[jvns-poll]: https://social.jvns.ca/@b0rk/111709458396281239
+[prototype]: https://github.com/martinvonz/jj/pull/3613

mkdocs.yml

@@ -140,5 +140,3 @@ nav:
     - 'Sparse Patterns v2': 'design/sparse-v2.md'
     - 'Tracking branches': 'design/tracking-branches.md'
     - 'Copy tracking and tracing': 'design/copy-tracking.md'
-
-- 'Development Roadmap': "roadmap.md"