[Haskell] The syntax highlighting changes are too dramatic to bear.

[absop]

Problem description

I am using the One Light color scheme, and I just updated my default packages with the master branch (under the help of this script), noticed that syntax highlighting in Haskell has changed dramatically as shown in the two screenshots below.

In my opinion, the previous syntax highlighting file was rough and lacked some features, such as the inability to jump to type definitions, which the new syntax highlighting file fixes. However, the new syntax highlighting caused the whole interface to look a bit exaggerated. Of course, this had something to do with the color scheme, but I still felt it was a problem.

Before the update

After the update

[jwortmann]

The One color schemes use a red color for the variable scope, and it looks like the updated syntax introduced variable.other as the scope for regular variables (which I think is a good thing).
Maybe try adding the following rule as a color scheme customization, and it should look a lot less exaggerated:

                {
			"scope": "variable.other",
			"foreground": "var(--foreground)",
		},

[michaelblyons]

Can you copy/paste the code in question so it needn't be transcribed by each person attempting to reproduce?

[absop]

In the screenshot LetFun should be a constructor function, not a type storage. And I don't think we should assign the variable scope to lexical elements in Haskell.

[absop]

data LetExpr =  LetApp LetExpr LetExpr | LetDef [([Int], LetExpr)] LetExpr |
                LetFun Int | LetVar Int | LetNum Int
                deriving (Show, Eq)

parseLet :: String -> Maybe LetExpr
parseLet s = case parse letExpr s of
              [(e, "")] -> Just e
              _         -> Nothing

letExpr = letApp <|> letExprWithoutApp

-- 用于在letApp Parser中消除左递归
letExprWithoutApp = letFun
                <|> letVar
                <|> letNum
                <|> letDef
                <|> wrapped letExpr

letFun = LetFun <$> fun
letVar = LetVar <$> var
letNum = LetNum <$> nat
letDef = LetDef <$ symbol "let" <*> eqnList <* symbol "in" <*> letExpr
letApp = do
  es <- letExprWithoutApp `sepBy` oneOrMoreSpaces
  case es of
    f:xxs@(_:_) -> return $ foldl LetApp f xxs
    _           -> empty

eqnList :: Parser [([Int], LetExpr)]
eqnList = eqn `sepBy` (tok ';')

varList :: Parser [Int]
varList = var `sepBy` oneOrMoreSpaces

eqn :: Parser ([Int], LetExpr)
eqn = (,) <$> left <* tok '=' <*> right
  where left = (:) <$> fun <* oneOrMoreSpaces <*> varList
        right = letExpr

fun = char 'f' >> nat
var = char 'x' >> nat
tok = token . char

wrapped :: Parser a -> Parser a
wrapped p = id <$ tok '(' <*> p <* tok ')'

sepBy :: Parser a -> Parser b -> Parser [a]
sepBy p q = (:) <$> p <*> many (q >> p)

-- 一个或多个 space 的 parser
oneOrMoreSpaces :: Parser ()
oneOrMoreSpaces = some (sat isSpace) >> return ()

[absop]

The One color schemes use a red color for the variable scope, and it looks like the updated syntax introduced variable.other as the scope for regular variables (which I think is a good thing). Maybe try adding the following rule as a color scheme customization, and it should look a lot less exaggerated:

                {
			"scope": "variable.other",
			"foreground": "var(--foreground)",
		},

Thanks, it looks better!

[absop]

data LetExpr =  LetApp LetExpr LetExpr | LetDef [([Int], LetExpr)] LetExpr |
                LetFun Int | LetVar Int | LetNum Int
                deriving (Show, Eq)

[deathaxe]

And I don't think we should assign the variable scope to lexical elements in Haskell.

Any token of any meaning should receive a valuable scope to enable color schemes to address it. Otherwise they are forced to be highlighted as plain text under all circumstances.

One may argue about the used scope name though. But ideally it should be one of the more common basic scopes used by most (and default) color schemes to ensure consistent highlighting accross syntaxes.

In the screenshot LetFun should be a constructor function, not a type storage.

VSCode scopes them constant.other which seems wrong, too.

The idea behind storage.type (or support.type) is illustrated in your most recent post. A constructor function is defined by data LetExpr ... which (even though being a function) can be read as a data type declaration - very much like a class definition - maybe derived from more basic data types.

A class also calls a constructor to create new objects of certain type. Nothing else a Haskell constructor function does. We could also scope it variable.function.constructor but using entity.name.function.constructor for LetExpr in data LetExpr = feels odd.

[absop]

For the type definition shown below, we can compare syntax highlighting on Github, VSC and ST. Obviously, Github and VSC do a better job.

Github

data LetExpr =  LetApp LetExpr LetExpr | LetDef [([Int], LetExpr)] LetExpr |
                LetFun Int | LetVar Int | LetNum Int
                deriving (Show, Eq)

VSC

ST

[absop]

I don't pay much attention to scope names, but I think they should be consistent and differentiated.

For the code shown below, all locations of LetExpr should have the same scope name, which might be storage.type. And for LetApp, LetDef, LetFun, LetVar, LetNum and LetFun, they may have an entity.name.function.constructor or other, but they must different from LetExpr

data LetExpr =  LetApp LetExpr LetExpr | LetDef [([Int], LetExpr)] LetExpr |
                LetFun Int | LetVar Int | LetNum Int
                deriving (Show, Eq)

[deathaxe]

Choosing colors is the task of a color-scheme while syntax definitions only define valuable and semantically correct names for certain tokens. It's easy to achieve tha same highlighting with ST.

While many older syntax definitions tried to achieve certain highlighting by abusing scope names we have decided to move on defining semantically correct scope names as a stable base for everything long ago because the former approach only works well for certain color schemes and languages.

Sure, this path may result in some maintenance work for color scheme authors. But it should allow color schemes of reduced complexity in the long term by reducing the need to write language specific rules.

... all location of LetExpr should have the same scope name ...

ST traditionally scopes all constructs entity.name when being defined - if possible. Hence data LetExpr uses entity.name.type. That's the way how Goto Definition is powered. All references to the constructor are scoped storage.type to be able to drive Goto Reference. This strategy can be found in many syntaxes.

They can be highlighted the same way if the color scheme decides to do so.

It doesn't look that tramatical with the default setup of ST4126 using Mariana.

[absop]

ST traditionally scopes all constructs entity.name when being defined - if possible. Hence data LetExpr uses entity.name.type. That's the way how Goto Definition is powered. All references to the constructor are scoped storage.type to be able to drive Goto Reference. This strategy can be found in many syntaxes.

Yes, data LetExpr should have an entity.name.type, but it can also have a storage.type which the other LetExprs have.

[absop]

The LetApp in LetApp LetExpr LetExpr should be considered as a bug.

[deathaxe]

Why? With the scheme of all constructor references using storage.type LetApp uses the same scope as all the other constructors in tha line.

[FichteFoll]

Only LetApp is a constructor in data LetExpr = LetApp LetExpr LetExpr. The LetExpr instances are type references (which makes them even stronger contenders for using storage.type).

In general, it comes down to the decision of what scope a constructor should receive, in which case I'm always happy to reference my favorite (unresolved) RFC discussion at #1842. TL;DR: it's complicated (?)

Edit: added data … = for clarification

[deathaxe]

But as the syntax chooses to use storage.type for all constructors as well, the scope is the same. Otherwise we'd end up with 3 scopes being used for the same token depending on position of use - if even possible in a consistent manner. If we'd want to scope constructors variable.function.constructor (which is the existing one used in C++ etc.) we'd scope LetApp

1. entity.name.type in data LetApp = ...,
2. variable.function.constructor in ... = LetApp ...
3. storage.type in any other position.

Even if we'd use entity.name.function.constructor and variable.function.constructor, we'd end up with the same situation as we have now.

I may be wrong, but I read ... = LetApp as something like ... = new LetApp() in Java or C++.

[absop]

Why? With the scheme of all constructor references using storage.type LetApp uses the same scope as all the other constructors in tha line.

For goto definition, LetApp should have a entity.name.constructor (or other better scope), and it should be distinguished from the following LetExprs whom have a storage.type, so LetApp should not have storage.type.

[deathaxe]

Isn't LetApp defined in a data LetApp = ... expression as well? So this would be the location entity.name would be valid, no? Not that I am opposed to tweak that part of the syntax, but I am still on my way to understand the relational. The first token after the assignment is always something like a function call, while the following tokens are just some kind of arguments for it?

[absop]

In my opinion, The following is what it should be.

data LetExpr =  LetApp LetExpr LetExpr | LetDef [([Int], LetExpr)] LetExpr |
--   ^^^^^^^  entity.name.type storage.type
--              ^^^^^^ entity.name.function.constructor constant.other
--                     ^^^^^^^ ^^^^^^^ storage.type

[absop]

The first token after the assignment is always something like a function call, while the following tokens are just some kind of arguments for it?

Following the assignment, Each branch is a definition, define a constuctor, here are LetApp, tokens after LetApp are type arguments.

[deathaxe]

So finally all the constructors LetApp, LetDef etc. can be used to create an "object" of type LetExpr?

[absop]

So finally all the constructors LetApp, LetDef etc. can be used to create an "object" of type LetExpr?

Yes.

[absop]

The first token after the assignment is always something like a function call, while the following tokens are just some kind of arguments for it?

Following the assignment, Each branch is a definition, define a constuctor, here are LetApp, tokens after LetApp are type arguments.

tokens after LetApp are type arguments

Represents the type of the corresponding parameter, which can be either a certain type (beginning with an uppercase letter) or a type variable (beginning with a lowercase letter).

[pennychase]

I agree with @absop's highlighting of the example. LetApp is not defined in a data LetApp = ... Its definition is on the RHS of the data expression, which says that LetExpr is a new type with value constructors LetApp and LetDef. The tokens that appear after LetApp and LetDef are always types in this context. BTW, in this expression LetExpr is defined recursively, and as you can see in @absop's example it should be highlighted the same way throughout, since it is always a type. But LetApp and LetDef actually are like function calls. In a type definition the "arguments" are types, and when they are used in other expressions in the code, the arguments will be values

[pennychase]

Another issue is how Just x and Nothing are handled. Nothing is support.constant.prelude.haskell and Just is support.type.prelude.haskell, but again both are the value constructors of the Maybe type, so in my opinion, they should be highlighted the same way (even though one has an argument and the other doesn't). See how they're handled in the ParseLet definition in the above comment. I think GitHub gets this right, by having all value constructors highlighted the same way, both in the type definition and when used to construct values in expressions. Well, with the exception of Left and Right, the value constructors of Either, which GitHub treats as a built-in. So I think Just and Nothing could be handled like ordinary value constructors, or could be treated as special (from Prelude), but should be of the same scope.

[absop]

But LetApp and LetDef actually are like function calls. In a type definition the "arguments" are types, and when they are used in other expressions in the code, the arguments will be values

I think LetApp and LetDef are more like function definitions or function prototype declarations (the functions themselves are generated by the compiler) than function calls. Only when they are applied to create instances of the LetExpr type, which says

letDef = LetDef <$ ...
letApp = ... LetApp f xxs ...

, they are called.

[absop]

Another issue is how Just x and Nothing are handled. Nothing is support.constant.prelude.haskell and Just is support.type.prelude.haskell, but again both are the value constructors of the Maybe type, so in my opinion, they should be highlighted the same way (even though one has an argument and the other doesn't).

I agree with @pennychase's opinion. Nothing is the same as Just, they are both constructors of the Maybe type. But we can consider all constructors in Haskell are constants.

[pennychase]

In a comment to PR 2697 I gave an example of highlighting qualified imports that uses different scopes in the qualified name. @deathaxe explained that the scope naming scheme for all syntaxes differentiate between qualifiers and the entity that is imported. But in a qualified import you're not importing anything, rather you're giving a short name for the namespace. So in the example I gave HTTP.Types isn't an imported entity, but a shortened name that will be used instead of Network.HTTP.Types to provide a namespace to disambiguate entities referred to in code, e.g., a qualified import lets you use HTTP.Types.Status instead of Network.HTTP.Types.Status.

[deathaxe]

I think I got the point about types vs. constructors. But please note ST's old Haskell syntax didn't distinguish them as well. All types and constructors were scoped constant.other only. What's new is them all being scoped storage.type now.

data LetExpr =  LetApp LetExpr LetExpr | LetDef [([Int], LetExpr)] LetExpr |
--   ^^^^^^^ constant.other
--              ^^^^^^ constant.other
--                     ^^^^^^^ ^^^^^^^ constant.other

I am not currently sure if constructors and types can be distinghished and thus highlighted correctly in all the different expressions.

[absop]

I am not currently sure if constructors and types can be distinghished and thus highlighted correctly in all the different expressions.

Both types and constructors start with a upper case letter, but constructors are applicative, types are not applicative, the will not be called, they are basically only referenced in type definitions.

In something like a function call, if the function (means the first token in f x y and Just x) starts with a capital letter, it's pretty much a constructor.

[jrappen]

@absop I assume your issue has been resolved? Maybe close this?

[jrappen]

/cc @AmjadHD FYI, as this was your color scheme being talked about in the opening comment.

[absop]

@absop I assume your issue has been resolved? Maybe close this?

OK, thanks

[AmjadHD]

@absop Are you satisfied with the look of the color scheme after deathaxe's commit ?

[deathaxe]

Partly answered by #3321