Bootstrapping the Emacs test suite

[CeleritasCelery]

Emacs ships with over 7000 elisp tests. Bootstrapping these tests would make a good milestone and help ensure correctness. These tests can be run with make check in GNU Emacs.

[gtrak]

I want to look into this one as an intro to rust, and I have a little elisp experience. Do you have any starting thoughts for it?

I was able to get this repo to compile. How would we run a single test against it?

For example, would we integrate with a rust test library or do some scripting to run the elisp against a compiled binary?

Fundamentally, we need a way to signal success or failure and report it.

[CeleritasCelery]

I am glad you are interested!

To run the tests we first need to bootstrap ert. This may have other dependencies that we will need to enable along the way. I copy the lisp files that have been bootstrapped into a matching directory here and then make sure they get loaded in the bootstrap. This will get loaded with cargo run --release.

Once that is working we can enable some of the test in the src directory that test many of the built-in functionality.

[bhankas]

I added ert.el to rune/emacs-lisp/, ran cargo run --release and it finished successfully.

I do not see ert being loaded in the list though. Am I missing something?

[bhankas]

Adding seq.el did result in an error!

@CeleritasCelery I'm just playing around, been following from sidelines for a while, and now seems like a good time to try to help to my meager abilities and learn some Rust along the way.

If you think I can help out, please let me know the stupidest way to start.

[gtrak]

Forgot to update this, I added a few things and ran into a missing native function:

        lisp/emacs-lisp/backtrace.el
        lisp/emacs-lisp/debug.el
        lisp/emacs-lisp/ert.el
        lisp/emacs-lisp/ewoc.el
        lisp/emacs-lisp/find-func.el
        lisp/emacs-lisp/map.el
        lisp/emacs-lisp/pp.el

diff --git a/lisp/bootstrap.el b/lisp/bootstrap.el
index 8fdcf4a..b5adcdb 100644
--- a/lisp/bootstrap.el
+++ b/lisp/bootstrap.el
@@ -4,6 +4,8 @@
 (load "cconv")
 (load "warnings") ;; should be autoloaded
 (load "bytecomp")
+(defvar source-directory nil)
+(load "ert")

 ;; check bytecodes

any ideas?

MESSAGE: byte-compile: byte-compile-dynamic-variable-op
Loading bytecomp Done
Loading ert...
Loading debug...
Loading backtrace...
Loading find-func...
-----LOAD ERROR START-----


(defvar find-function-C-source-directory
  (let ((dir (expand-file-name "src" source-directory)))
    (if (file-accessible-directory-p dir) dir))
  "Directory where the C source files of Emacs can be found.
If nil, do not try to find the source code of functions and variables
defined in C.")
-----LOAD ERROR END-----
-----LOAD ERROR START-----
         ; if-let
(require 'find-func)
-----LOAD ERROR END-----
-----LOAD ERROR START-----

(require 'backtrace)
-----LOAD ERROR END-----
-----LOAD ERROR START-----

(require 'debug)
-----LOAD ERROR END-----
-----LOAD ERROR START-----

(load "ert")
-----LOAD ERROR END-----
Error: Invalid function: file-accessible-directory-p

BEGIN_BACKTRACE
0: require [find-func ]
1: require [backtrace ]
2: require [debug ]
3: load ["ert" ]
END_BACKTRACE
Error: ()

[gtrak]

I got past that and hit the next issue, I'll spin up a branch

[gtrak]

#87

[gtrak]

A couple style questions,

1. Are you annotating which native defuns are untested vs which ones are considered to be complete?
   I expect once this test suite runs, there will be a lot of breakage and we need to revisit the native implementations.
   Are you concerned at all about semantic correctness in what gets merged now before we get that suite running?

2. I copy/pasted the docstring from the upstream C impl into the rust for reference. I noticed you also use gplv3 so license might not be an issue. Do you have a preference either way?

[CeleritasCelery]

Are you annotating which native defuns are untested vs which ones are considered to be complete?

I am not, but this would be a good thing to track. Things that have obvious missing functionality should have a TODO in the body. But i am assuming now that the native functions have just enough functionality to get through the bootstrap process.

I expect once this test suite runs, there will be a lot of breakage and we need to revisit the native implementations.

I suspect this will be the case, which is part of the reason why this would be so valuable.

Are you concerned at all about semantic correctness in what gets merged now before we get that suite running?

Obviously the more correct the better. but correctness is judged by “what does Emacs do”. So things like ERT will help put us on firmer footing.

I copy/pasted the docstring from the upstream C impl into the rust for reference. I noticed you also use gplv3 so license might not be an issue. Do you have a preference either way?

Adding the docstring is fine. I have not be doing that because I have been developing this in Emacs and I have the docstrings at hand.

[CeleritasCelery]

Also keep in mind that we are pulling lisp from the 29.1 branch.

[CeleritasCelery]

@bhankas to load a new file either add it to bootstrap.el or pass it as an argument via --load `.