Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[Python] Add syntax highlighting for type comments #1925

Closed
wants to merge 8 commits into from
Closed

[Python] Add syntax highlighting for type comments #1925

Show file tree
Hide file tree
Changes from 1 commit
Commits
Show all changes
8 commits
Select commit Hold shift + click to select a range
137308c
[Python] Add syntax highlighting for type comments
TeamSpen210 Mar 24, 2019
ed61bb4
[Python] Add function type hint comment syntax
TeamSpen210 Mar 26, 2019
56ff18e
[Python] Small fixes/improvements to type comments
TeamSpen210 Oct 17, 2019
482f0ec
Merge branch 'master' into type-comment
wbond Jul 21, 2020
14bf602
[Python] Clean up type comment scopes
wbond Jul 21, 2020
2a57368
Merge branch 'master' into type-comment
wbond Jul 22, 2020
e3b57a9
[Python] Fixed various type comment tests to use valid code
wbond Jul 22, 2020
97ae638
[Python] Ensure type comments are exited at newline
wbond Jul 22, 2020
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
70 changes: 35 additions & 35 deletions Python/Python.sublime-syntax
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -415,55 +415,67 @@ contexts:
- match: '(#)\s*(type)(:)\s*(ignore)\b'
scope: comment.line.type-hint.python
captures:
1: punctuation.definition.comment.python comment.line.number-sign.python
1: punctuation.definition.comment.python
2: keyword.other.annotation.type-comment.python
3: punctuation.separator.annotation.type-comment.python
4: keyword.other.annotation.type-comment.ignore.python
set: comments_afterignore
push: comments-afterignore
- match: '(#)\s*(type)(:)'
scope: comment.line.type-hint.python
captures:
1: punctuation.definition.comment.python comment.line.number-sign.python
1: punctuation.definition.comment.python
2: keyword.other.annotation.type-comment.python
3: punctuation.separator.annotation.type-comment.python
embed: type-comment
escape: '\n'
embed_scope: comment.line.type-hint.python
push: type-comment
- match: "#"
scope: punctuation.definition.comment.python
push: comments-basic-remainder

comments-basic:
- match: "#"
scope: punctuation.definition.comment.python
set: comments-basic-remainder

wbond marked this conversation as resolved.
Show resolved Hide resolved
- include: comments_noannotation
comments-basic-remainder:
- meta_scope: comment.line.number-sign.python
- match: \n
pop: true

comments_afterignore:
- meta_scope: comment.line.type-hint.python
- include: comments_noannotation
comments-afterignore:
- meta_content_scope: comment.line.type-hint.python
- match: \n
pop: true
- match: '(?=#)'
set: comments-basic
- match: \[
# Mypy's # type: ignore[error-code1, error-code2]
scope: punctuation.section.brackets.begin.python
set:
- meta_scope: comment.line.type-hint.python
- meta_content_scope: comment.line.type-hint.python
- match: \[
scope: invalid.illegal.python
- match: '[a-zA-Z_-][a-zA-Z_0-9-]+'
scope: comment.line.type-hint.python.mypy-errorcode
- match: '[0-9]'
scope: constant.other.error-code.python
- match: '[0-9]\w*'
scope: invalid.illegal.python
- match: ','
scope: punctuation.separator.sequence.python.mypy-errorcode
scope: punctuation.separator.sequence.python
- match: \]
scope: punctuation.section.brackets.end.python
set:
# Don't allow a second block.
- match: \[
# Don't allow a second block.
- meta_content_scope: comment.line.type-hint.python
- match: '(?=#)'
set: comments-basic
- match: \S
scope: invalid.illegal.python
pop: true
- include: comments_afterignore
- match: \S
scope: invalid.illegal.python

type-comment:
- include: comments_noannotation

- meta_content_scope: comment.line.type-hint.python
- match: '(?=#)'
set: comments-basic
# Don't use expressions here, only a few types are reasonable
# as annotations.
- include: constants
Expand All @@ -472,7 +484,6 @@ contexts:
- include: builtin-functions
- match: '{{identifier}}'
scope: support.class.python

- match: '\s*(\[)'
captures:
1: meta.item-access.python punctuation.section.brackets.begin.python
Expand All @@ -483,8 +494,8 @@ contexts:
pop: true
- match: ':'
scope: punctuation.separator.slice.python
- match: '(?=#)'
- include: type-comment

- match: '\('
scope: punctuation.section.parens.begin.python
push:
Expand All @@ -494,25 +505,22 @@ contexts:
pop: true
- match: \(
scope: invalid.illegal.stray.brace.round.python

- match: \*{3,}
scope: invalid.illegal.syntax.python
- match: \*\*
scope: keyword.operator.unpacking.mapping.python
- match: \*
scope: keyword.operator.unpacking.sequence.python

- match: '->'
scope: invalid.illegal.stray.return.arrow.python
- include: type-comment

- match: '->'
scope: punctuation.separator.sequence.python meta.function.return-type.python
scope: meta.function.return-type.python punctuation.separator.sequence.python
set:
- meta_content_scope: comment.line.type-hint.python meta.function.return-type.python
- match: '->'
scope: invalid.illegal.stray.return.arrow.python
- include: type-comment
- meta_scope: meta.function.return-type.python
- match: ','
scope: punctuation.separator.sequence.python
- match: \)
Expand All @@ -524,14 +532,6 @@ contexts:
- match: \S
scope: invalid.illegal.python

comments_noannotation:
- match: "#"
scope: punctuation.definition.comment.python
push:
- meta_scope: comment.line.number-sign.python
- match: \n
pop: true

constants:
- match: \b(None|True|False|Ellipsis|NotImplemented|__debug__)\b
scope: constant.language.python
Expand Down
108 changes: 59 additions & 49 deletions Python/syntax_test_python.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -1490,70 +1490,80 @@ class Starship:
# Type comments - type: ignore must be by itself.

primes = 5 # type: ignore # type: not-a-type-comment
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Just for clarification

primes = 5  # type: ignore # type: str   // technically ok but weird
primes = 5  # type: str  # type: ignore  // not tested here but typical
primes = 5  # type: str  # type: str     // not tested here and invalid

(I don't know how fine-grained these tests are here.)

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The impression I had gotten from the previous implementation was that line comments added into a type comment effectively ended the processing of mypy. Is that true?

From your comment above, it almost sounds like mypy will process nested comments that start with type: and the only logical combinations are type: ignore and a normal type: comment, right?

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Maybe mypy stops after the first type: ignore comment. 🤷 After that everything is "egal". The only thing you see in practice is # type: str # type: ignore to brutally force a type. And typical # type: ignore[foo-rule] # mypy bug https:\\issue-nr because you really get hit by a mypy bug every other day.

# ^ comment.line.type-hint punctuation.definition.comment.python comment.line.number-sign.python
# ^^^^ comment.line.type-hint keyword.other.annotation.type-comment
# ^ comment.line.type-hint punctuation.separator.annotation.type-comment
# ^^^^^^ comment.line.type-hint keyword.other.annotation.type-comment.ignore
# ^ comment.line.type-hint comment.line.number-sign.python punctuation.definition.comment.python
# ^^^^^^^^^^^^^^^^^^^^^^^^ comment.line.type-hint comment.line.number-sign.python
# ^^^^^^^^^^^^^^^ comment.line.type-hint - comment.line.number-sign
# ^ punctuation.definition.comment
# ^^^^ keyword.other.annotation.type-comment
# ^ punctuation.separator.annotation.type-comment
# ^^^^^^ keyword.other.annotation.type-comment.ignore
# ^^^^^^^^^^^^^^^^^^^^^^^^^^ comment.line.number-sign - comment.line.type-hint
# ^ punctuation.definition.comment

# This is not a type-ignore comment.
value = 3 # type: ignore_class_starting_with_ignore
# ^ comment.line.type-hint.python punctuation.definition.comment.python comment.line.number-sign.python
# ^^^^ comment.line.type-hint.python keyword.other.annotation.type-comment.python
# ^ comment.line.type-hint.python punctuation.separator.annotation.type-comment.python
# ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ comment.line.type-hint.python support.class.python
# ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ comment.line.type-hint - comment.line.number-sign
# ^ punctuation.definition.comment
# ^^^^ keyword.other.annotation.type-comment
# ^ punctuation.separator.annotation.type-comment
# ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ support.class

# Mypy ignore extension
value = 3 # type: ignore[error-code-1, 4noint][error]
# ^^^^^^ comment.line.type-hint.python keyword.other.annotation.type-comment.ignore
# ^ comment.line.type-hint.python comment.line.type-hint.python punctuation.section.brackets.begin.python
# ^^^^^^^^^^^^ comment.line.type-hint.python comment.line.type-hint.python.mypy-errorcode
# ^ comment.line.type-hint.python punctuation.separator.sequence.python.mypy-errorcode
# ^ invalid.illegal.python
# ^ comment.line.type-hint.python punctuation.section.brackets.end.python
# ^ invalid.illegal.python
value = 3 # type: ignore[error-code-1, 4noint][error] # foo
# ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ comment.line.type-hint - comment.line.number-sign
# ^^^^^^ keyword.other.annotation.type-comment.ignore
# ^ punctuation.section.brackets.begin
# ^^^^^^^^^^^^ constant.other.error-code
# ^ punctuation.separator.sequence
# ^^^^^^ invalid.illegal
# ^ punctuation.section.brackets.end
# ^^^^^^^ invalid.illegal
# ^^^^^ comment.line.number-sign - comment.line.type-hint

# With/for allow commas
for a, b in lst: # type: str, int
# ^ comment.line.type-hint.python punctuation.separator.sequence.python
# ^ comment.line.type-hint punctuation.separator.sequence


primes = 5 # type: List[Dict[*property:str, ...]], bool # comment
wbond marked this conversation as resolved.
Show resolved Hide resolved
deathaxe marked this conversation as resolved.
Show resolved Hide resolved
# ^ comment.line.type-hint.python punctuation.definition.comment.python comment.line.number-sign.python
# ^^^^ comment.line.type-hint.python keyword.other.annotation.type-comment.python
# ^ comment.line.type-hint.python punctuation.separator.annotation.type-comment.python
# ^^^^ comment.line.type-hint.python support.class.python
# ^ comment.line.type-hint.python meta.item-access.python punctuation.section.brackets.begin.python
# ^^^^ comment.line.type-hint.python meta.item-access.arguments.python support.class.python
# ^ comment.line.type-hint.python meta.item-access.arguments.python meta.item-access.python punctuation.section.brackets.begin.python
# ^ comment.line.type-hint.python meta.item-access.arguments.python meta.item-access.arguments.python invalid.illegal.python
# ^^^^^^^^ comment.line.type-hint.python meta.item-access.arguments.python meta.item-access.arguments.python support.function.builtin.python
# ^ comment.line.type-hint.python meta.item-access.arguments.python meta.item-access.arguments.python punctuation.separator.slice.python
# ^^^ comment.line.type-hint.python meta.item-access.arguments.python meta.item-access.arguments.python support.type.python
# ^ comment.line.type-hint.python meta.item-access.arguments.python meta.item-access.arguments.python punctuation.separator.sequence.python
# ^^^ comment.line.type-hint.python meta.item-access.arguments.python meta.item-access.arguments.python constant.language.python
# ^ comment.line.type-hint.python meta.item-access.arguments.python meta.item-access.python punctuation.section.brackets.end.python
# ^ comment.line.type-hint.python meta.item-access.python punctuation.section.brackets.end.python
# ^ comment.line.type-hint.python punctuation.separator.sequence.python
# ^^^^ comment.line.type-hint.python support.type.python
# ^ comment.line.type-hint.python comment.line.number-sign.python punctuation.definition.comment.python
# ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ comment.line.type-hint - comment.line.number-sign
# ^ punctuation.definition.comment
# ^^^^ keyword.other.annotation.type-comment
# ^ punctuation.separator.annotation.type-comment
# ^^^^ support.class
# ^^^^^^^^^^^^^^^^^^^^^^^^^^ meta.item-access
# ^punctuation.section.brackets.begin
# ^^^^^^^^^^^^^^^^^^^^^^^^ meta.item-access.arguments
# ^^^^ support.class
# ^ meta.item-access punctuation.section.brackets.begin
# ^ invalid.illegal
# ^^^^^^^^ support.function.builtin
# ^ punctuation.separator.slice
# ^^^ support.type
# ^ punctuation.separator.sequence
# ^^^ constant.language
# ^ punctuation.section.brackets.end
# ^ punctuation.section.brackets.end
# ^ punctuation.separator.sequence
# ^^^^ support.type
# ^^^^^^^^^^ comment.line.number-sign
# ^ punctuation.definition.comment

# Python 2.7 function annotations.
def function(a, b, *c, **d):
# type: (int, str, *List[str], **bool) -> Dict[str, str] # type: noncomment
wbond marked this conversation as resolved.
Show resolved Hide resolved
# ^ comment.line.type-hint.python punctuation.section.parens.begin.python
# ^^^ comment.line.type-hint.python meta.function.parameters.python support.type.python
# ^ comment.line.type-hint.python meta.function.parameters.python punctuation.separator.sequence.python
# ^ comment.line.type-hint.python meta.function.parameters.python keyword.operator.unpacking.sequence.python
# ^^^^ comment.line.type-hint.python meta.function.parameters.python support.class.python
# ^^ comment.line.type-hint.python meta.function.parameters.python keyword.operator.unpacking.mapping.python
# ^^^^ comment.line.type-hint.python meta.function.parameters.python support.type.python
# ^ comment.line.type-hint.python punctuation.section.parens.end.python
# ^^ comment.line.type-hint.python meta.function.return-type.python punctuation.separator.sequence.python meta.function.return-type.python
# ^^^^ meta.function.return-type.python support.class.python
# ^ meta.function.return-type.python comment.line.number-sign.python punctuation.definition.comment.python
# ^^^^^^^^^^^^^^^^ meta.function.return-type.python comment.line.number-sign.python
# ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ comment.line.type-hint
# ^ punctuation.section.parens.begin
# ^^^ meta.function.parameters support.type
# ^ meta.function.parameters punctuation.separator.sequence
# ^ meta.function.parameters keyword.operator.unpacking.sequence
# ^^^^ meta.function.parameters support.class
# ^^ meta.function.parameters keyword.operator.unpacking.mapping
# ^^^^ meta.function.parameters support.type
# ^ punctuation.section.parens.end
# ^^^^^^^^^^^^^^^^^^ meta.function.return-type
# ^^ punctuation.separator.sequence
# ^^^^ support.class
# ^^^^^^^^^^^^^^^^^^ comment.line.number-sign - comment.line.type-hint
# ^ punctuation.definition.comment


##################
Expand Down