bash-completion as part of GNU Emacs

[bhavin192]

There was a mention of making emacs-bash-completion a part of GNU Emacs: https://debbugs.gnu.org/26661#24
@szermatt what do you think about it?

AFAIK this will also involve getting the copyright assignment papers signed by contributors who have contributed significant amount of changes to this repository.

[szermatt]

Hi! Thank you for letting me know.

I'm ok with transferring copyright assignment and dealing with the paperwork, if there's enough interest.

I have to say that I personally find bash-completion a bit hackish and fragile (by nature) to be something that'd come as part of standard Emacs, but that's something for Emacs maintainers to decide.

[manuel-uberti]

Hi,

I don't know what the status of this is but what about ELPA or NonGNU ELPA?

[szermatt]

Nothing has happened so far. I haven't had much time to spend on emacs-bash-completion.

Adding Emacs to ELPA or NonGNU ELPA are reasonable alternatives.

[manuel-uberti]

Can I suggest then to contact @monnier to see how this can be done? :)

[peniblec]

(Apologies in advance for the ramblings @szermatt; I felt compelled to blurt all that out for some reason. It's certainly not a call to action, there's no emergency here as far as I am concerned; please don't bother with this unless you feel like you can spare the time)

Hi folks,

FWIW, in the long run, I really think it would make sense for Emacs to use Bash's native completion system, out-of-the-box. To rehash the thoughts I laid down in bug#26661 last year, and to add a couple more things I've ruminated since then:

• I'm intrigued by the idea that bash-completion (assuming you are referring to the Elisp package; or are you referring to Bash's completion API?) is hackish by nature, and I wonder how much of a problem it is in practice:

  • I don't feel like this issue tracker is overflowing with bugs;
  • python.el's comint support has native completion enabled by default; in what regards is its implementation more robust?
  • even if we take that fragility for granted, the feature can always be made opt-in, so that users will not be worse off by default.

• I find that there are too many ways for stock Emacs's M-x shell to lose track of what's going on in Bash sessions, and the completion candidates quickly get irrelevant; to name a few hitches:

  • autocd breaks shell-dirtrack-mode: ./ TAB will keep suggesting files in the previous folder,
  • changes in $PATH are not taken into account: TAB will not suggest programs from added entries,
  • changes of environment in general (e.g. starting a subshell in a Docker container) are not taken into account…

(Granted, those are all solvable bugs; it's just more code that has to be piled into shell.el, vs. adding support for native completion and letting Bash do the heavy lifting)

• @skangas brought up pcomplete; I think that's an orthogonal topic. pcomplete certainly could use more love, and I'm sure it can provide powerful and more ad-hoc completions that what native Bash completions offer, but while waiting for someone to invent these smarter completions, bash-completion lets Emacs leverage

  1. all the logic Bash has for its builtins,
  2. all the code upstream authors have already written to provide completions for their CLI applications.

• To close off on a somewhat weaker, less technical argument: Bash and Emacs are both parts of the GNU project; it feels like they ought to play off each other's strenghts IMO.

Again, apologies for the ramblings. Even though I think it would make eminent sense for Emacs to use native Bash completions by default, it's fine if the package ends on (Non)GNU ELPA; that doesn't preclude an "out-of-the-box" inclusion later on AFAIK.

Do let us know if there is anything we can do to help, though!

[monnier]

Again, apologies for the ramblings. Even though I think it would make eminent sense for Emacs to use native Bash completions by default, it's fine if the package ends on (Non)GNU ELPA; that doesn't preclude an "out-of-the-box" inclusion later on AFAIK.

I for one would welcome support in shell.el (or in a GNU ELPA package) to delegate completion to bash (or to query bash for the completion candidates, and then do the completion itself in Emacs).

[szermatt]

Just to be sure it's clear: I'm not opposed to making this package part of Emacs. I just haven't gotten around to it. At this point, I don't have much time to change or maintain this package, but the situation should improve next year.

I'm intrigued by the idea that bash-completion (assuming you are referring to the Elisp package; or are
you referring to Bash's completion API?) is hackish by nature

Completion in Bash is understandably tightly connected with readline, which handles the communication with the user. The bash-completion.el package has to simulate the way readline normally triggers completion in an attempt to make this work. There are corner-cases where the simulation breaks down that I haven't been able to fix because it's just not what it was meant for - this is understandable as, after all, bash knows it has no terminal and that readline is not available so why would completion be necessary? The end-result is that bash-completion.el doesn't always work as it should.

To improve the situation, as you mentioned, would mean work on the bash side to make external (non-readline) completion a supported feature, fix any corner-cases, document it and make it an officially supported interface. I haven't contacted anyone from the bash project about that, but would expect it to be doable. It's just a matter of someone putting the required time and effort into this.

I don't feel like this issue tracker is overflowing with bugs;

I have to say that it's hard for me to tell whether this means that the package is working and useful or whether people just give up on it without bothering with filing bugs.

I find that there are too many ways for stock Emacs's M-x shell to lose track of what's going on in Bash sessions

Yes, exactly. Even just tracking the current directory is a problem much more difficult than it should really be, because the only channel of communication there is between the shell and Emacs is a terminal that's meant for human being to interact with, not machines. Again, this is solvable - and many solutions have been provided already - but it remains disappointingly fragile. bash-completion.el faces the same problem - but worse since it needs two-way communication between the bash and the Emacs process.

all the code upstream authors have already written to provide completions for their CLI applications.

I agree that this is the strength of this elisp package. This is why I wrote bash-completion.el to begin with. Without this motivation, it would be better to provide the completion fully on the Emacs side.

[peniblec]

Just to be sure it's clear:

(Crystal clear as far as I'm concerned. I hope my message didn't come across as pushy; I just wanted to write all those arguments down for the record. I'm confident the inexorable March Of Progress™ will resume in due course; there's no urgency as far as I'm concerned)

Even just tracking the current directory is a problem much more difficult than it should really be, because the only channel of communication there is between the shell and Emacs is a terminal that's meant for human being to interact with, not machines.

(Related: I've just remembered that Emacs 28 will add (opt-in) support the OSC 7 escape sequence, so that particular problem will be sort-of solved… as long as one configures their shell(s) to emit the sequence (cf. VTE's implementation), as suggested in the docstring of comint-osc-directory-tracker)

[skangas]

For the record, I agree with that this would be a welcome feature, in GNU ELPA or maybe even Emacs itself. Thanks to everyone who is working on it.

[phikal]

ping?

[szermatt]

Stefan Monnier has just added bash-completion NonGNU ELPA 😄 !

On my side, there's been no progress on the full integration into GNU Emacs.

[monnier]

On my side, there's been no progress on the full integration into GNU Emacs.

On an unrelated note, I wonder if it can be generalized to other shells and/or other programs offering completion via readline.

[szermatt]

On an unrelated note, I wonder if it can be generalized to other shells and/or other programs offering completion via readline.

I wish!

The approach currently taken in bash-completion integrates at the bash level. It needs to know how Bash works and split lines into commands and arguments in exactly the right way.

An approach that would integrate at the readline level would be much more efficient and robust - and it would generalize to other readline-based tools. That would be exactly the right thing to do! I don't know whether readline is capable of doing that, but it would be a nice extension to readline which other terminal emulators (others than Emacs) could make good use of.