unsetopt CASE_GLOB breaks performance

[JustWhateverIOnlyWantBetterCodeView]

By default "completion" module does unsetopt CASE_GLOB. This completely breaks performance, see my comment on this issue: nix-community/home-manager#2255 (comment).

This option is also harmful and could result in privilege escalation.
Assume I add path /a/b to fpath and some module/script sources scripts from this fpath. If unprivileged user has write access to /a, but not to /a/b, and a sticky bit is set on directory /a, then it could create directory /a/B, and any glob that uses fpath, would also match paths in attacker-controlled directory /a/B. Though this scenario is a bit theoretical as such setup of privileges seems unlikely on any real system.

[turboMaCk]

I can confirm that the performance is an issue and that setting:

zstyle ':prezto:*:*' case-sensitive 'yes'

leads to significant improvement indeed.

Anyway I started to have performance issues only like for last 2 years (and kept pushing looking into them for about that long). I was always using case insensitive autocomplete but it never lead to performance issues before. Based on that I think it's likely also performance regression though I can't say anything to how it happened.

[JustWhateverIOnlyWantBetterCodeView]

Some further explanation why this breaks performance.

If you glob a path like /a/b/c* then with case-sensitive globbing zsh will open directory /a/b, go though the list of all files inside of it and returns directories matching glob c.

With case-insensitive globbing, zsh will enter root directory, go through the list of all files inside of it and search for directories matching glob {a,A}, then, once it finds directory /a/ (and possible also /A/ as it also matches...), it will open directory /a and, again go though the list of all files inside of it and search for directories matching glob {b,B}. Only after that it will open directory /a/b and list all directories inside of it matching glob {c,C}.

The drop in performance depends on the size of all the directories along the path /a/b.

For a concrete example, let's take this particular line, where I encountered the problem the first time: https://github.com/zsh-users/zsh/blob/28410bd5bc71fda6343b13c2b6abad06bd2eaaee/Functions/Prompts/promptinit#L23.
This grep is supposed to return all files named prompt_*_setup inside of fpath. For simplicity let's say there's only one path in fpath. With case-sensitive globbing zsh will simply open that directory, and for each file inside of it it will try to match it with glob prompt_*_setup. With case-insensitive globbing however, it will open each directory along the fpath and go though the list of all files inside them.

The drop in performance is probably negligible if there's only hundreds or so of files. But on nix /nix/store directory has hundreds of thousands of files and going through the list of all files takes few hundreds of milliseconds (which is further multiplied by the amount of times /nix/store is present in fpath).

[turboMaCk]

@JustWhateverIOnlyWantBetterCodeView Thanks for additional clarification. This is what I understood from your comment in home-manager (linked above as well).

This behaviour makes a lot of sense of course. Since if you're on FS with case-sensitive file names, the only (sane) way to do the case-insensitivity on top is to do the listing of all content. It's also clear that number of directories zsh has to enter to collect this has a great (at least o(n)) impact on this so globbing /nix/store creates a pathological case here.

That's being said I think what might have changed is the amount of globing that is being done when profile gets loaded.

For instance in my case the primary source of slowness is prompt zstyle ':prezto:module:prompt' ... When I remove this setting I don't experience any performance problems

[turboMaCk]

I think that:

1. case insensitive globing is indeed source of the slowness
2. the thing that triggers the gobbing to happen is call to vcs_info 'prompt' which many folks do in prompt.

[turboMaCk]

I already posted this to nix-community/home-manager#2255 (comment)

but for greater visibility this is the work around I endup with for now.

file: .zshrc:

# Triger `vcs_info 'prompt'` before loading Prezto.
# We want the first call to this happen before prezto will
# `unsetopt CASE_GLOB` which will have negative effect on the performance.
# Calling this before case sensitive globing makes initial start much faster.
# see: https://github.com/nix-community/home-manager/issues/2255
autoload -Uz vcs_info
vcs_info 'prompt'

# Source Prezto.
if [[ -s "${ZDOTDIR:-$HOME}/.zprezto/init.zsh" ]]; then
  source "${ZDOTDIR:-$HOME}/.zprezto/init.zsh"
fi