FR: Make special forms not look like functions

[arxanas]

Is your feature request related to a problem? Please describe.

Most functions in the revset language are pure and referentially-transparent. You can evaluate them in any order and produce the same result, and substitute any expression for its resolved value without changing the result of the entire query.

Currently, there's one exception: at_operation(op, rev) is not referentially-transparent, since the result of evaluating rev depends on the result of evaluating op. You can't evaluate rev separately, then substitute its literal contents as the operand to at_operation and be guaranteed to get the same results.

Describe the solution you'd like

There should be a syntactic distinction between "special forms", which indicate non-referentially-transparent computations, to avoid potential user surprise. It might also make script authors more likely to generate robust revset expressions as parts of queries, since it'll be more obvious that you can't just splice expressions textually in all cases.

Possible future special forms:

• Objects on a remote could be potentially be expressed with a special form syntax.
  • Essentially, they are today via the @ operator (which, incidentally, doesn't seem to be in the revset docs?). Since it's an operator, it's probably less surprising that it has unusual evaluation semantics.
  • It could also be used to expose colocated Git interop without overloading the remote mechanism. As I understand it, we currently rely on a special remote named git.
• Something to do with traversing evolution history, as discussed in FR: revset functions for traversing the obslog #4129
• A literal syntax for revset expressions, similar to Bazel's set() syntax form.
  • This would address the case of interpolating revset literals from other contexts, particularly shell scripting.
  • This has implications for parsing rather than evaluation, since the special form is on the syntax level instead of just the semantic level.
• I proposed a hidden: modifier here, which would be similar to at_operation if it were implemented (although it doesn't seem likely): Rebasing (and some other operations) is confusing for hidden commits #4544 (comment)

Describe alternatives you've considered

Some ideas:

• Do nothing, and keep the function syntax.
  • It might be that this is a non-issue in practice and nobody will be confused or rely on referential transparency in a problematic way.
  • It might be that we never want to add any other special form, so generalizing it for at_operation only would be too much overhead.
  • This might only be problem for an overly-academic programming language design standpoint, given that people are not (yet?) writing large programs in the revset syntax, and that it's optimized for interactive use.
• Use different syntax.
  • Example: Reuse and generalize the modifier syntax to work in arbitrary expressions, allowing you to write foo(bar(at_op(baz): qux())).
  • Example: Add a sigil to indicate that a function is a special form. Rust uses !; we could write at_operation!(<op>, <rev>) instead.
  • Example: Use an operator instead of a function, like mentioned in FR: revset syntax to resolve an expression at an old operation #1283 (comment), or somehow combine the design with the existing @ operator.
• Lift all special forms to command-line flags or other global/out-of-band mechanisms.
  • This seems undesirable and limiting. (jj already had --at-op when the at_operation function was requested.)

Additional context

• at_operation was originally created to address FR: revset syntax to resolve an expression at an old operation #1283 and implemented in revset: add at_operation(op, expression) #4614.

[emilazy]

(I suspect this also applies to if() in revsets? Though maybe not, if the language is total, or consistent with non‐strict semantics.)

[arxanas]

(I suspect this also applies to if() in revsets? Though maybe not, if the language is total, or consistent with non‐strict semantics.)

Interestingly, if is only documented in the templating language at present: https://martinvonz.github.io/jj/v0.23.0/templates/.

• Perhaps if doesn't exist in the revset language right now?
• While I imagine users can infer special evaluation by convention from the name if, it might not be obvious for functions like coalesce (assuming that coalesce is similar).
• Regardless, the same issues as per the original issue probably apply. (I suspect that the core revset language is total, but stuff like string manipulation might produce fatal errors as side-effects, and that users use if to avoid them.)

I took a look at the fileset language just now as well, but it seems to be simple enough that there's no unusual functions for now.

[emilazy]

Sorry, I meant to say “in templates”. We don’t have if() in revsets, though we do have coalesce() (which I believe has pure semantics).

Fatal errors aren’t necessarily a semantic problem as long as the language semantics are consistent with non‐strict evaluation (at most, perhaps if is lazier than every other function). at_operation() cannot be explained in terms of non‐strict evaluation, so the problem is more pressing there.

[scott2000]

Here's another weird quirk which seems related: you can have a revset alias like this which uses an argument as both a commit ID and a number:

[revset-aliases]
"test(cond, n)" = "coalesce(cond & n, ancestors(@, n))"

If 9 is a valid short commit ID prefix, then test(all(), 9) is equivalent to all() & 9 so it evaluates to the commit with unique prefix 9 (e.g. 9e09d307), and test(none(), 9) is equivalent to ancestors(@, 9) so it returns 9 ancestors of @.

[martinvonz]

FYI, another oddity is that referring to a hidden commit makes it temporarily visible in some ways. For example, if X is a hidden commit, then jj log -r 'X & all()' will show that commit even though a plain all() doesn't include it.