Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[Python] Add syntax highlighting for type comments #1925

Closed
wants to merge 8 commits into from
Closed

[Python] Add syntax highlighting for type comments #1925

Show file tree
Hide file tree
Changes from 2 commits
Commits
Show all changes
8 commits
Select commit Hold shift + click to select a range
137308c
[Python] Add syntax highlighting for type comments
TeamSpen210 Mar 24, 2019
ed61bb4
[Python] Add function type hint comment syntax
TeamSpen210 Mar 26, 2019
56ff18e
[Python] Small fixes/improvements to type comments
TeamSpen210 Oct 17, 2019
482f0ec
Merge branch 'master' into type-comment
wbond Jul 21, 2020
14bf602
[Python] Clean up type comment scopes
wbond Jul 21, 2020
2a57368
Merge branch 'master' into type-comment
wbond Jul 22, 2020
e3b57a9
[Python] Fixed various type comment tests to use valid code
wbond Jul 22, 2020
97ae638
[Python] Ensure type comments are exited at newline
wbond Jul 22, 2020
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
90 changes: 90 additions & 0 deletions Python/Python.sublime-syntax
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -402,6 +402,96 @@ contexts:
pop: true

comments:
# Type-ignore must be the only hint allowed!
- match: '(#)\s*(type)(:)\s*(ignore)'
TeamSpen210 marked this conversation as resolved.
Show resolved Hide resolved
captures:
1: punctuation.definition.comment.python comment.line.number-sign.python
2: keyword.other.annotation.type-comment
TeamSpen210 marked this conversation as resolved.
Show resolved Hide resolved
3: punctuation.separator.annotation.type-comment
TeamSpen210 marked this conversation as resolved.
Show resolved Hide resolved
4: keyword.other.annotation.type-comment.ignore
TeamSpen210 marked this conversation as resolved.
Show resolved Hide resolved
set:
- meta_scope: comment.line.type-hint
TeamSpen210 marked this conversation as resolved.
Show resolved Hide resolved
- include: comments_noannotation
- match: \n
pop: true
- match: \S
scope: invalid.illegal
TeamSpen210 marked this conversation as resolved.
Show resolved Hide resolved
- match: '(#)\s*(type)(:)'
scope: comment.line.type-hint
captures:
1: punctuation.definition.comment.python comment.line.number-sign.python
2: keyword.other.annotation.type-comment
3: punctuation.separator.annotation.type-comment
embed: type-comment
escape: '\n'
embed_scope: comment.line.type-hint

wbond marked this conversation as resolved.
Show resolved Hide resolved
- include: comments_noannotation

type-comment:
- include: comments_noannotation

wbond marked this conversation as resolved.
Show resolved Hide resolved
# Don't use expressions here, only a few types are reasonable
# as annotations.
- include: constants
- include: builtin-exceptions
- include: builtin-types
- include: builtin-functions
- match: '{{identifier}}'
scope: support.class.python

wbond marked this conversation as resolved.
Show resolved Hide resolved
- match: '\s*(\[)'
captures:
1: meta.item-access.python punctuation.section.brackets.begin.python
push:
- meta_content_scope: meta.item-access.arguments.python
- match: \]
scope: meta.item-access.python punctuation.section.brackets.end.python
pop: true
- match: ':'
scope: punctuation.separator.slice.python
- include: type-comment

wbond marked this conversation as resolved.
Show resolved Hide resolved
- match: '\('
scope: punctuation.section.parens.begin.python
push:
- meta_content_scope: meta.function.parameters.python
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This should probably be meta_scope so it applies to the ( and ), right?

- match: \)
scope: punctuation.section.parens.end.python
pop: true
- match: \(
scope: invalid.illegal.stray.brace.round.python

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Style: remove this blank line

- match: \*{3,}
scope: invalid.illegal.syntax.python
- match: \*\*
scope: keyword.operator.unpacking.mapping.python
- match: \*
scope: keyword.operator.unpacking.sequence.python

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Style: remove this blank line

- match: '->'
scope: invalid.illegal.stray.return.arrow.python
- include: type-comment

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Style: remove this blank line

- match: '->'
scope: punctuation.separator.sequence.python meta.function.return-type.python
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The meta scope should come before the punctuation scope

set:
- match: '->'
scope: invalid.illegal.stray.return.arrow.python
- include: type-comment
- meta_scope: meta.function.return-type.python
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This should be moved up 3 lines so it is the first entry in the anonymous context

- match: ','
scope: punctuation.separator.sequence.python
- match: \)
scope: invalid.illegal.stray.brace.round.python
- match: \]
scope: invalid.illegal.stray.brace.square.python
- match: \}
scope: invalid.illegal.stray.brace.curly.python
- match: \S
scope: invalid.illegal

comments_noannotation:
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Use a - here instead of a _ for stylistic consistency

- match: "#"
scope: punctuation.definition.comment.python
push:
Expand Down
46 changes: 46 additions & 0 deletions Python/syntax_test_python.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -1350,6 +1350,52 @@ class Starship:
# ^ punctuation.separator.annotation.variable.python
# ^ keyword.operator.assignment

# Type comments - type: ignore must be by itself.

primes = 5 # type: ignore # type: not-a-type-comment
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Just for clarification

primes = 5  # type: ignore # type: str   // technically ok but weird
primes = 5  # type: str  # type: ignore  // not tested here but typical
primes = 5  # type: str  # type: str     // not tested here and invalid

(I don't know how fine-grained these tests are here.)

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The impression I had gotten from the previous implementation was that line comments added into a type comment effectively ended the processing of mypy. Is that true?

From your comment above, it almost sounds like mypy will process nested comments that start with type: and the only logical combinations are type: ignore and a normal type: comment, right?

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Maybe mypy stops after the first type: ignore comment. 🤷 After that everything is "egal". The only thing you see in practice is # type: str # type: ignore to brutally force a type. And typical # type: ignore[foo-rule] # mypy bug https:\\issue-nr because you really get hit by a mypy bug every other day.

# ^ comment.line.type-hint punctuation.definition.comment.python comment.line.number-sign.python
wbond marked this conversation as resolved.
Show resolved Hide resolved
# ^^^^ comment.line.type-hint keyword.other.annotation.type-comment
# ^ comment.line.type-hint punctuation.separator.annotation.type-comment
# ^^^^^^ comment.line.type-hint keyword.other.annotation.type-comment.ignore
# ^ comment.line.type-hint comment.line.number-sign.python punctuation.definition.comment.python
# ^^^^^^^^^^^^^^^^^^^^^^^^ comment.line.type-hint comment.line.number-sign.python
deathaxe marked this conversation as resolved.
Show resolved Hide resolved

primes = 5 # type: List[Dict[*property:str, ...]], bool # comment
wbond marked this conversation as resolved.
Show resolved Hide resolved
deathaxe marked this conversation as resolved.
Show resolved Hide resolved
# ^ comment.line.type-hint punctuation.definition.comment.python comment.line.number-sign.python
# ^^^^ comment.line.type-hint keyword.other.annotation.type-comment
# ^ comment.line.type-hint punctuation.separator.annotation.type-comment
# ^^^^ comment.line.type-hint support.class.python
# ^ comment.line.type-hint meta.item-access.python punctuation.section.brackets.begin.python
# ^^^^ comment.line.type-hint meta.item-access.arguments.python support.class.python
# ^ comment.line.type-hint meta.item-access.arguments.python meta.item-access.python punctuation.section.brackets.begin.python
# ^ comment.line.type-hint meta.item-access.arguments.python meta.item-access.arguments.python invalid.illegal
# ^^^^^^^^ comment.line.type-hint meta.item-access.arguments.python meta.item-access.arguments.python support.function.builtin.python
# ^ comment.line.type-hint meta.item-access.arguments.python meta.item-access.arguments.python punctuation.separator.slice.python
# ^^^ comment.line.type-hint meta.item-access.arguments.python meta.item-access.arguments.python support.type.python
# ^ comment.line.type-hint meta.item-access.arguments.python meta.item-access.arguments.python punctuation.separator.sequence.python
# ^^^ comment.line.type-hint meta.item-access.arguments.python meta.item-access.arguments.python constant.language.python
# ^ comment.line.type-hint meta.item-access.arguments.python meta.item-access.python punctuation.section.brackets.end.python
# ^ comment.line.type-hint meta.item-access.python punctuation.section.brackets.end.python
# ^ comment.line.type-hint punctuation.separator.sequence.python
# ^^^^ comment.line.type-hint support.type.python
# ^ comment.line.type-hint comment.line.number-sign.python punctuation.definition.comment.python

# Python 2.7 function annotations.
def function(a, b, *c, **d):
# type: (int, str, *List[str], **bool) -> Dict[str, str] # type: noncomment
wbond marked this conversation as resolved.
Show resolved Hide resolved
# ^ comment.line.type-hint punctuation.section.parens.begin.python
# ^^^ comment.line.type-hint meta.function.parameters.python support.type.python
# ^ comment.line.type-hint meta.function.parameters.python punctuation.separator.sequence.python
# ^ comment.line.type-hint meta.function.parameters.python keyword.operator.unpacking.sequence.python
# ^^^^ comment.line.type-hint meta.function.parameters.python support.class.python
# ^^ comment.line.type-hint meta.function.parameters.python keyword.operator.unpacking.mapping.python
# ^^^^ comment.line.type-hint meta.function.parameters.python support.type.python
# ^ comment.line.type-hint punctuation.section.parens.end.python
# ^^ comment.line.type-hint meta.function.return-type.python punctuation.separator.sequence.python meta.function.return-type.python
# ^^^^ meta.function.return-type.python support.class.python
# ^ meta.function.return-type.python comment.line.number-sign.python punctuation.definition.comment.python
# ^^^^^^^^^^^^^^^^ meta.function.return-type.python comment.line.number-sign.python
deathaxe marked this conversation as resolved.
Show resolved Hide resolved


# <- - meta
# ensure we're not leaking a context