[ and ] should have a modifier of typeHint when used in type hints

[NorthIsUp]

Environment data

Name: Pylance
Id: ms-python.vscode-pylance
Description: A performant, feature-rich language server for Python in VS Code
Version: 2021.10.2-pre.1
Publisher: Microsoft
VS Marketplace Link: https://marketplace.visualstudio.com/items?itemName=ms-python.vscode-pylance

• Language Server version: ^^
• OS and version: ubuntu 18
• Python version (& distribution if applicable, e.g. Anaconda): 3.7.6

Expected behaviour

Semantic coloring should be consistent in the type hint, but the item access [ and ] don't get the typeHint modifier

Actual behaviour

The item accessors [ and ] don't get the typeHint modifier in a type hint

Code Snippet / Additional information

from typing import List

# I expect the "[" to have a modifier of "typeHint"
def foo(some_stuff: List[str]) -> None:
    return ['hi']

in this screenshot i'd expect the [ and ] to be red like the rest of the type hits.

[jakebailey]

We don't emit semantic tokens for any tokens like this (only names, builtin constants None/True/False/__debug__, and match/case). I'm not certain it's by design (similarly reported in #459 with other bugs), but our parse tree doesn't always have the information to make it straightforward to color these types of tokens, and we've relied on VS Code's TextMate profile to do the rest.

Reading the LSP spec for semantic tokens, I'm not entirely certain there's a token that we can use for these, though: https://microsoft.github.io/language-server-protocol/specification#textDocument_semanticTokens

We'd have to compare to something like TS with its <> or similar.

[yvvt0379]

From my perspective, I think this can be an option(not as default), because when a type hint is complicated(like dict[int, Union[str, Iterable[str]]]), highlighting square brackets the same color as types just makes the code harder to read & understand.

[NorthIsUp]

From my perspective, I think this can be an option(not as default), because when a type hint is complicated(like dict[int, Union[str, Iterable[str]]]), highlighting square brackets the same color as types just makes the code harder to read & understand.

agreed, ideally they could be colored as their own entity like typehint.item-access or something.

[heejaechang]

implementation wise, we will need to go through tokens to support this since for now, we only process parse tree and name node.

[judej]

Moving this issue to discussion as an enhancement request for comments and upvotes.