[D] Syntax Improvements

[BenjaminSchaaf]

This is a complete re-implementation of the D syntax highlighting to support all current features and mark entities correctly. I've validated correctness by going through some portion of the D standard library and vibed.

Also includes improvements to the build system, with error highlighting and dub support.

[BenjaminSchaaf]

@wbond Thanks for the review, I think I've responded/fixed most of it. In terms of the ; consistency I'm not sure what should be prioritized. In D ; is used both as a terminator and a separator:

auto b = [1, 2, 3];
foreach (int a; b) { … }

int fn()
out(i; i > 0)
body { return 1; }

[wbond]

There were a few comments that I didn't see a response to, or a change related to. I believe everything else that you replied to I either resolved or left my thoughts on what I think should happen.

This round I added some comments with specific meta. scopes we should probably add. Additionally pairs of parens and braces should get meta.parens and meta.braces (or meta.block) for the symbols and everything between them. The official docs about meta. scopes include a few others. I may have missed some situations where they would be appropriate: https://www.sublimetext.com/docs/3/scope_naming.html#meta.

[wbond]

Here are a few constructs from the dub codebase that seem not to be handled properly, although the issue for some may just be my understanding. 🙂

---

https://github.com/dlang/dub/blob/master/source/dub/internal/vibecompat/core/log.d#L65

appender!string()

The ! should probably be something like punctuation.section.generic. The parens and everything between have meta.function-call doubled up.

---

https://github.com/dlang/dub/blob/master/source/dub/internal/vibecompat/core/log.d#L69

auto threadid = () @trusted {

The @ is marked as punctuation for an annotation, but there is no meta.annotation scope and trusted is not variable.annotation, but rather storage.attribute.

---

https://github.com/dlang/dub/blob/master/source/dub/commandline.d#L243

dub = new Dub(NativePath(getcwd()));

Right now Dub is storage.type, so it won't show up in Goto References. I think we'll need to add a .tmPreferences to make sure meta.function-call storage.type is indexed as a reference. See Default/Indexed Reference List.tmPreferences for an example. However, that will probably catch type casts in function calls. We may need to change storage.type (in this case) to something like variable.function or variable.class (with an appropriate tmPreferences file).

---

https://github.com/dlang/dub/blob/master/source/dub/description.d#L49

Exception("Package '"~name~"' not found in dependency tree.")

Right now the ~ is scoped as an arithmetic operator. I am guessing that is a concatenation operator?

---

https://github.com/dlang/dub/blob/master/source/dub/dub.d#L128

PackageSupplier[] m_packageSuppliers;

PackageSupplier is not recognized as a type, presumably because of the [].

---

https://github.com/dlang/dub/blob/master/source/dub/dub.d#L764

final switch (list_type) with (ListBuildSettingsFormat) {

The with clause is marked as invalid.

---

https://github.com/dlang/dub/blob/master/source/dub/dub.d#L958

(a, b) => a.version_ < b.version_

b is scoped variable.parameter, but a is variable.other.

---

https://github.com/dlang/dub/blob/master/source/dub/init.d#L124-L141

What exactly is this here? It seems to be D code, but inside of an unquoted string.

---

https://github.com/dlang/dub/blob/master/source/dub/init.d#L181-L196

Are the % placeholders a built-in string thing?

[BenjaminSchaaf]

@wbond

@trusted is a built-in attribute just like pure, static, auto, etc. It just has a @ in front for some reason.

---

~ is indeed concatenation

---

PackageSupplier[] m_packageSuppliers;

Can't be reliably parsed as [] is a valid operation:

PackageSupplier[].map!(foo => foo.bar);

---

(a, b) => ...

Is another parsing ambiguity, though one that we could probably special-case. The following is a worst-case:

(foo!(123, [2, 3, 4])[] a)
{}

---

q{} is a token string, all contents must be valid D tokens. This is useful when writing code for mixin or in this case when writing default code to src/app.d when initializing a dub project.

---

format does printf-style formatting. D has uniform function call syntax so:

void f(string a);
f("foo");
"foo".f(); // equivalent