[RFC] Definitions and References

[deathaxe]

Motivation

The primary motivation of this RFC is to define a common guideline of how syntax definitions should scope entities in order to work well with Sublime Text's Goto Definition and Goto Reference features using a minimal set of rules and to provide a starting point to extend the feature syntax specifically.

Sublime Text's Goto Definition and Goto Reference requires scoping tokens of definitions or declarations and pairing them with those looking like a reference of the entity.

Pairing works by adding those tokens to Sublime Text's index database.

The following code blocks show the most basic pair, which is defined by Sublime Text out of the box.

Definitions

<dict>
    <key>scope</key>
    <string>
        meta.function entity.name.function
    </string>
    <key>settings</key>
    <dict>
        <key>showInIndexedSymbolList</key>
        <integer>1</integer>
    </dict>
</dict>
</plist>

References

<dict>
    <key>scope</key>
    <string>
        meta.function-call variable.function,
        meta.function-call support.function
    </string>
    <key>settings</key>
    <dict>
        <key>showInIndexedReferenceList</key>
        <integer>1</integer>
    </dict>
</dict>
</plist>

Note: The disussion was initiated at #1841 (comment)

Issues

The function identifier can be one of the following atomic or complex expresions:

type	scope
identifier	variable.function
builtin function	support.function
macro	constant.other.macro
builtin macro	support.constant.macro
expression	...

Examples with macros

C

#define FUNC_NAME function

//// FUNCTION DEFINITION //////////////////////////////////

void FUNC_NAME(int arg1, int *arg2) { }

// equal to

void function(int arg1, int *arg2) { }

//// FUNCTION CALL ////////////////////////////////////////

int main(int argc, int *argv) {
    
    return FUNC_NAME(argc, argv)
}

Erlang

-define(FUNC_NAME, function).

%%%% FUNCTION DEFINITION %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%

?FUNC_NAME(arg1, arg2) -> .

% equal to

function(arg1, arg2) -> .

%%%% FUNCTION CALL %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%

main(argc, argv) ->

    ?FUNC_NAME(argc, argv).

The FUNC_NAME is defined in line 1. Its reference is used as identifier in the function definition.

The FUNC_NAME in line 5 is now both - a reference of a previously defined macro and the identifier of the function definition.

The FUNC_NAME is also used as identifier in the function call in the last line.

Note: Macros and ordinary identifiers may not be distinguished in syntaxes like C/C++ for sure, while they can in others like Erlang.

Questions

1. How to scope FUNC_NAME in the function definition?

• constant.other.macro
• entity.name.function
• entity.name.function.macro
• entity.name.function constant.other.macro

2. How to scope FUNC_NAME in the function call?

• constant.other.macro
• variable.function
• variable.function.macro
• variable.function constant.other.macro

Answer

1. A macro should be scoped as such in all situations, which removes [entity|variable].function.macro from the list of candidates.
2. The constant.other.macro only is not a proper candidate as it might be used in the function body with different meaning.
3. Stacking two scopes which are used for highlighting directly seems weird as well.

Proposal

This PFC proposes to use (existing) meta scopes to solve the issue and introduce detailed subscopes for the different parts of a complex definition/reference statement based on #884

Note: Even though this proposal uses the function definition and function call as an example, it may be applied to any kind of definition/reference expression.

Simple definitions and references

Simple definitions and references, which are uniquely identified by one word, don't need extra meta scoping.

definition	reference
entity.name.<definition-type>	variable.other.<definition-type>
entity.name.<definition-type>	constant.other.<definition-type>

Example

A custom object definition could use entity.name.object while its references are scoped variable.other.object.

Complex definitions and references

If a syntax supports sophisticated names or expressions to be used as identifier in a (function-) definition the whole definition statement should ...

1. be wrapped into a meta.<definition-type> scope,
2. use entity.name.<definition-type> for the identifier,
3. use sub meta scopes like meta.<definition-type>.name to wrap complex identifier expressions.

If a syntax supports sophisticated names or expressions to be used as identifier in a (function-) reference the whole reference statement should ...

1. be wrapped into a meta.<definition-type>-call scope,
2. use one of the existing top-level scopes like variable.other.<definition-type> or constant.other.<definition-type> for the identifier,
3. use sub meta scopes like meta.<definition-type>-call.name to wrap complex identifier expressions.

Notes:

• An entity. which is used as identifier in a definition should not be used to scope the reference of the same object. In short: entity is dedicated to definitions.

• Scopes being directly used for highlighting should not be stacked together in order to create specialized tokens. Don't use variable.function constant.other or something like that.

Example

   % quoted identifier

   'my-complex\'type' (Arg1, 'arg-2') -> ok.
   ^^^^^^^^^^^^^^^^^^^ meta.function.name
                      ^^^^^^^^^^^^^^^ meta.function.parameters
                                     ^^^ meta.function
                                        ^^^^ meta.function.body
   ^^^^^^^^^^^^^^^^^^ entity.name.function
   
   % macro identifier and recursive usage
   
   ?FUNC_NAME (Arg1, 'arg-2') -> ?FUNC_NAME(Arg1, 'arg-2') when is_list(Arg1).
   ^^^^^^^^^^^ meta.function.name
              ^^^^^^^^^^^^^^^ meta.function.parameters
                             ^^^ meta.function
                                ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ meta.function.body
                                 ^^^^^^^^^^ meta.function-call.name
                                           ^^^^^^^^^^^^^^^ meta.function-call.arguments
   ^^^^^^^^^^ constant.other.macro
                                 ^^^^^^^^^^ constant.other.macro   

Benefit

All sorts of definitions or references - even if they use macros as identifier - can be added to the index or be addressed by color schemes.

function definition: meta.function.name & (entity.name.function | constant.other.macro)

function call: meta.function-call.name & (variable.function | support.function | support.constant.macro | constant.other.macro)

[Thom1729]

I missed this when it was added. I have three concerns.

The first is that in (say) JavaScript, not all functions (scoped with meta.function) are declarations. Anonymous function may optionally have names, but they do not declare those names for surrounding scopes. The syntax definition necessarily makes this distinction, and it we're standardizing scopes, we should make sure that we can distinguish them.

The second is that we can't rely on the scope at time of reference. For example:

function f() {}
//       ^ entity.name.function
console.log(f);
//          ^ variable.other

The third is that I'm not sure how we would use this to implement GOTO features. If the appropriate meta scopes were all present, then we could implement a fairly reliable GOTO system in Python, but it's not trivial. I've done this for JavaScript, and it requires a fair bit of language-specific code. We could define a new kind of JSON-based specification format and write an engine that consumes it, but I'm not sure that this is within the scope of the proposal.

[deathaxe]

This issue was not raised with the goal to force everybody to scope the most obvious impossible declaration-reference pairs as illustrated in the previous post.

It was rather created with the question in mind: "If a syntax allows us to scope definitions and references, how to name those in order to make use of the goto feature?"

It is obvious that this question can't be answered for all syntaxes the same way we know ST's syntax engine can't handle all syntaxes on earth with 100% accuracy.

[ismell]

This is probably not the right place to ask, but I'll do it anyway.

What about variable declarations and variable references? I would like to have goto references know when a function has been used as a variable. i.e. specifying a callback. So adding variable.other to the references index. This would require that variable declarations have a different scope so they don't get included. So maybe entity.name.variable?

I'm thinking of the following example:

char buffer[1024];
//   ^ entity.name.variable
void my_callback(void *data) { ... }
//   ^ meta.function entity.name.function

struct driver my_driver = {
    .data = buffer,
//          ^ variable.other
    .callback = &my_callback
//               ^ variable.other
};

[deathaxe]

This is the correct place to ask and this is basically the scope of the issue. Your question is partly discussed in #1842. Was not able to follow any detail though.

Explicit variable declarations as in C/C++ might be matched as entity.name.variable, I think, while l-values of assignments or implicit declarations in languages like pyhton are not easy or worth to be handled in special manner.