Autocompletion bug with multiple hyphens in options

[alexrosenfeld10]

First Check

• I added a very descriptive title here.
• I used the GitHub search to find a similar question and didn't find it.
• I searched the Typer documentation, with the integrated search.
• I already searched in Google "How to X in Typer" and didn't find any information.
• I already read and followed all the tutorials in the docs and didn't find an answer.
• I already checked if it is not related to Typer but to Click.

Commit to Help

• I commit to help with one of those options 👆

Example Code

import typer


app = typer.Typer(help="Testing 1234", no_args_is_help=True)

def autocomplete_test(incomplete: str):
    vals = ["foo-test-1234", "foo-anothertest-1234", "bar-test-1234"]
    return [val for val in vals if incomplete in val]


@app.command()
def test(
    name: str = typer.Option(None, "--name", "-n", help="Filter by name", autocompletion=autocomplete_test),
):
    typer.echo(f"foo {name}")

Description

When the above code is run in a CLI and the input my_cli test --name foo[TAB] is entered into the shell (where [TAB] indicates the user pressing tab to autocomplete the --name argument), the following is the result of the autocompletion:

❯ my_cli test --name foo--1234

I have observed similar behaviors in slightly different scenarios, for example, if the autocomplete function looks like this instead:

def autocomplete_test(incomplete: str):
    vals = ["foo-test-1234", "bar-test-1234"]
    return [val for val in vals if incomplete in val]

then a similar behavior happens when running my_cli test --name [TAB]:

❯ my_cli test --name -test-1234

Note - if the autocompletion options only have one typo, instead of two or more, it works fine. As in, this has no issue:

def autocomplete_test(incomplete: str):
    vals = [
        "foo-test",
        "foo-anothertest",
        "bar-test",
    ]
    return [val for val in vals if incomplete in val]

Operating System

macOS

Operating System Details

No response

Typer Version

0.12.5

Python Version

3.12.3

Additional Context

No response

[alexrosenfeld10]

@svlandeg @tiangolo this seems like a pretty clear bug, I took a look at the source code but didn't find any unit tests that trigger tab completion. If you can point me in the right direction I'll take a stab at fixing it

[svlandeg]

Hi @alexrosenfeld10, thanks for the report!

You can find some autocompletion tests in tests/test_completion, perhaps you have a look whether you can contribute some unit tests that fail on master? And then we can start looking into fixing the bug 🙏

[alexrosenfeld10]

Hmm. I tried this and wasn't able to reproduce through the unit tests. I'll find some time to make a minimal reproduction project and share it here

[alexrosenfeld10]

@svlandeg this doesn't seem to reproduce easily via unit tests... however, I've reproduced it for you here: https://github.com/alexrosenfeld10/typer-autocomplete-bug

Thanks in advance for taking a look. I'll be responsive here or in the issue page in my repo if you need any help setting up or reproducing the problem.

[alexrosenfeld10]

@svlandeg let me know if there's anything else I can to do help debug this

[alexrosenfeld10]

Hi @svlandeg , did you get a chance to try out the reproduction project? If you can point me in the right direction I'll take a stab at fixing the issue.

[alexrosenfeld10]

@tiangolo possible this could get looked at? It's a pretty common case to have hyphenated autocomplete suggestions, and this totally breaks that

[alexrosenfeld10]

@tiangolo would be great if we could get some eyes on this, having autocomplete is one of the main things typer writes about in the homepage. Again I'm happy to help if you can point me in the right direction

[svlandeg]

Hi @alexrosenfeld10, I've finally had some time to look at this in detail. Thanks for the detailed report and repo to reproduce.

I've tried to write some unit tests to reproduce this issue, but I couldn't - they are all green on master: #1095

I've looked at the details of the examples you've provided. For instance:

vals = ["foo-test-1234", "foo-anothertest-1234", "bar-test-1234"]

and autocompleting with

--name foo[TAB] 

leads to

--name foo--1234

Which seems wrong, as "foo--1234" is not a correct string from the options. But, importantly, as I can see from your video from the repo you created to reproduce, the cursor in this case would still be after "foo-" and before "-1234", right? Because in that sense the autocompletion is "correct": when starting with "foo" there are two options, both of which starting with "foo-" and ending in "-1234". But you still need to fill in the correct middle part.

In this sense, I wonder whether maybe your title and focus on "two hyphens" is a red herring: I think what really matters here is that the beginning and end string are the same for two options. The same is true on your reproduction repo, where the options are

"start-test-exmpl",
"start1234-other-exmpl",
"start1234-test-exmpl",

and the auto completion gives "start--exmpl" because that's the part of the string that overlaps between all three options 🤔

So, one question I have is this: what happens when you remove the hyphens from the strings, and have e.g. options

vals = ["footest1234", "fooanothertest1234", "bartest1234"]

and autocomplete on

--name foo[TAB] 

does it lead to

--name foo|1234

with | the meta mark for the cursor? Because if this is the case, then the behaviour you're seeing is not caused by the multiple hyphens, but by the string overlaps.

Then one other question: what happens if you hit TAB twice and not just once? Do you see the full options as strings?

Because on my system this does in fact work:

> typer disc_965.py run --name foo- [TAB TAB]

gives me as output:

foo-anothertest-1234  foo-test-1234

i.e. the two correct options are given in full. This is also what we see in the succeeding unit tests of #1095.

[alexrosenfeld10]

@svlandeg just checking in on this, thanks in advance. if there's anything I can do to help let me know

[svlandeg]

Still looking into this. Quick update: I can now reproduce this locally (it seems to only happen with the zsh shell) and found that a single hyphen can already trigger this, e.g. with options

vals = ["fooQtest-1234", "fooQanothertest-1234", "barQtest-1234"]

and completing at fooQ gives fooQ|-1234. Note that it doesn't give fooQ|test-1234 which is interesting. Somehow somewhere someone is spawning that end bit following the hyphen. I'll keep digging. Still can't reproduce with a unit test BTW.

[alexrosenfeld10]

Yeah, I concur it seems to be a zsh-specific problem.

[svlandeg]

So to come back to this observation I made earlier:

vals = ["foo-test-1234", "foo-anothertest-1234", "bar-test-1234"]

and autocompleting with

--name foo[TAB] 

leads to

--name foo--1234

Which seems wrong, as "foo--1234" is not a correct string from the options. But, importantly, as I can see from your video from the repo you created to reproduce, the cursor in this case would still be after "foo-" and before "-1234", right? Because in that sense the autocompletion is "correct": when starting with "foo" there are two options, both of which starting with "foo-" and ending in "-1234". But you still need to fill in the correct middle part.

I am not that familiar with zsh, but the behaviour I describe above might actually be intentional. For instance, this post lists it as correct behaviour in its first part:

So basically I'm starting to wonder whether this is really just a zsh setting, and nothing to do with Typer really.

More reading: https://zsh.sourceforge.io/FAQ/zshfaq04.html#l50 & https://superuser.com/a/603017

[svlandeg]

For instance, when I put this in my .zshc profile:

zstyle ':completion:*' completer _list _expand _complete _approximate _match

then the value isn't being completed with the cursor in the middle, instead it just immediately shows me the two correct options:

typer disc_965.py run --name f
foo-anothertest-1234  foo-test-1234

[alexrosenfeld10]

@svlandeg that works for me as well but has another downside - when there's only one option, it no longer auto-fills, instead it prints the "list" of options as if there were multiple, even though there is only one

[alexrosenfeld10]

think you're right though, generally this doesn't seem like a typer bug, more of a zsh / omz issue

[alexrosenfeld10]

This: ohmyzsh/ohmyzsh#1398 (comment) seems to be the silver bullet for me