Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Syntax highlighting breaks when single quoted string starts with sql keyword and ends with N #8

Open
MoonE opened this issue Aug 12, 2023 · 1 comment
Labels
bug Something isn't working

Comments

@MoonE
Copy link

MoonE commented Aug 12, 2023

Description

Syntax highlighting breaks when string starts with AND , 'SELECT , ALTER , etc. and the string ends with a capital N, followed by another string on the same line

Steps to Reproduce

  1. Paste example code into file with PHP syntax highlighting:
<?php
['AND N' => 1, '']; 
['AND DAMN' => 1, '']; 
[
    'AND
    N' => 1, '',
]; 
['AND M' => 1, '']; 
["AND N" => 1, ""]; 
  1. See wrong highlighting:
    image

Expected behavior:
This part: => 1, should not be highlighted as part of a string

Actual behavior:
The line is highlighted as string from first single quote to last single quote.

It works correctly with double quoted strings or if the last character is anything else than N.

Reproduces how often: 100 %

Versions

VSCode 1.81.1

@MoonE
Copy link
Author

MoonE commented Aug 30, 2023

N' apparently is the start of a unicode literal in mssql.
See: microsoft/vscode-mssql#1057
The offending rule is https://github.com/microsoft/vscode/blob/41c7fc34e084dad4087760597516ffbeaba0b155/extensions/sql/syntaxes/sql.tmLanguage.json#L523

But if it is inside a single-quoted string it should only match this if the single quote is escaped, e.g.

$sql = 'SELECT N\'asdf\'';

The generated tokens comparison:

  it 'should tokenize N correctly',
    {tokens} = grammar.tokenizeLine "['AND N' => 1, ''];"

    expect(tokens[0]).toEqual value: '[', scopes: ["source.php", "punctuation.section.array.begin.php"]
    expect(tokens[1]).toEqual value: '\'', scopes: ['source.php', 'string.quoted.single.sql.php', 'punctuation.definition.string.begin.php']
    expect(tokens[2]).toEqual value: 'AND', scopes: ['source.php', 'string.quoted.single.sql.php', 'source.sql.embedded.php', 'keyword.other.DML.sql']
    expect(tokens[3]).toEqual value: ' ', scopes: ['source.php', 'string.quoted.single.sql.php', 'source.sql.embedded.php']
    expect(tokens[4]).toEqual value: 'N', scopes: ['source.php', 'string.quoted.single.sql.php', 'source.sql.embedded.php', 'string.quoted.single.sql']
    expect(tokens[5]).toEqual value: '\'', scopes: ['source.php', 'string.quoted.single.sql.php', 'source.sql.embedded.php', 'string.quoted.single.sql', 'punctuation.definition.string.begin.sql']
    expect(tokens[6]).toEqual value: ' => 1, ', scopes: ['source.php', 'string.quoted.single.sql.php', 'source.sql.embedded.php', 'string.quoted.single.sql']
    expect(tokens[7]).toEqual value: '\'', scopes: ['source.php', 'string.quoted.single.sql.php', 'source.sql.embedded.php', 'string.quoted.single.sql', 'punctuation.definition.string.end.sql']
    expect(tokens[8]).toEqual value: '\'', scopes: ['source.php', 'string.quoted.single.sql.php', 'punctuation.definition.string.end.php']
    expect(tokens[9]).toEqual value: ']', scopes: ['source.php', 'punctuation.section.array.end.php']
    expect(tokens[10]).toEqual value: ';', scopes: ['source.php', 'punctuation.terminator.expression.php']

  it 'should tokenize M correctly', ->
    {tokens} = grammar.tokenizeLine "['AND M' => 1, ''];"

    expect(tokens[0]).toEqual value: '[', scopes: ["source.php", "punctuation.section.array.begin.php"]
    expect(tokens[1]).toEqual value: '\'', scopes: ['source.php', 'string.quoted.single.sql.php', 'punctuation.definition.string.begin.php']
    expect(tokens[2]).toEqual value: 'AND', scopes: ['source.php', 'string.quoted.single.sql.php', 'source.sql.embedded.php', 'keyword.other.DML.sql']
    expect(tokens[3]).toEqual value: ' M', scopes: ['source.php', 'string.quoted.single.sql.php', 'source.sql.embedded.php']
    expect(tokens[4]).toEqual value: '\'', scopes: ['source.php', 'string.quoted.single.sql.php', 'punctuation.definition.string.end.php']
    expect(tokens[5]).toEqual value: ' ', scopes: ['source.php']
    expect(tokens[6]).toEqual value: '=>', scopes: ['source.php', 'keyword.operator.key.php']
    expect(tokens[7]).toEqual value: ' ', scopes: ['source.php']
    expect(tokens[8]).toEqual value: '1', scopes: ['source.php', 'constant.numeric.decimal.php']
    expect(tokens[9]).toEqual value: ',', scopes: ['source.php', 'punctuation.separator.delimiter.php']
    expect(tokens[10]).toEqual value: ' ', scopes: ['source.php']
    expect(tokens[11]).toEqual value: '\'', scopes: ['source.php', 'string.quoted.single.php', 'punctuation.definition.string.begin.php']
    expect(tokens[12]).toEqual value: '\'', scopes: ['source.php', 'string.quoted.single.php', 'punctuation.definition.string.end.php']
    expect(tokens[13]).toEqual value: ']', scopes: ['source.php', 'punctuation.section.array.end.php']
    expect(tokens[14]).toEqual value: ';', scopes: ['source.php', 'punctuation.terminator.expression.php']

@KapitanOczywisty KapitanOczywisty added the bug Something isn't working label Oct 30, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Something isn't working
Projects
None yet
Development

No branches or pull requests

2 participants