Interlisp VM spec, corresponding releases, Xerox archive

[rochus-keller]

There is a 1979 version of "The Interlisp Virtual Machine Specification" here: https://www.cs.utexas.edu/~moore/publications/interlisp-vm.pdf (the original version is from 1976 and I didn't find a later than 1979 version yet).

Here is my DJVU version of it (with OCR): The Interlisp Virtual Machine Specification (1979).djvu.zip

I thought that maybe it would be fun and feasible to implement a VM based on this spec which is able to run the old versions like Chorus, Fugue, Carol or even the later releases. For this purpose I had a look at the https://github.com/Interlisp/history/tree/master/1980s files here, but wasn't sure whether they are complete to build and run a system.

Since I already spent a lot of time with the CHM Xerox archives when implementing my Smalltalk and Cedar tools, I went back and searched for Interlisp directories. And there are indeed a lot of corresponding directories and files. I assume you are aware of e.g.

https://xeroxparcarchive.computerhistory.org/eris/lisp/.index.html

where there seem to be all releases from Chorus to Koto. There are actually quite a lot of Interlisp related directories here:

https://xeroxparcarchive.computerhistory.org/eris/.index.html

and I even found a sysout of the Medley 1.0 release from 1988 here: https://xeroxparcarchive.computerhistory.org/phylum/medley/1.0/.index.html (unfortunately it doesn't seem to work with Maiko; neither the earlier sysouts seem to work; the error message of the VM is "sysout_loader:IFPAGE says sysout size is 0, But, sysout size from UNIX is 7499").

Which of these releases/directories looks most complete and is a good starting point for implementing a compatible VM?

[nbriggs]

Yeah, pretty familiar with the dumps. See, for example, {phylum}<lisp>lyric>library> -- I am "Briggs.PA".

[rochus-keller]

Cool. Did you therefore start with the Lyric release? Interestingly, I didn't find the sources of Medley 1.0 nor Maiko anywhere on this server. Was there already a Maiko VM for Lyric?

[rochus-keller]

After going over all the releases and available documentation, I tend to zero in on the Koto release. First of all it appeared only two years later than the Smalltalk-80 version I used for my project, and there is a whole set of excellent documentation for this specific release available. Also the files available for the Koto release in the Xerox archive seem pretty complete for such an adventure (in contrast to e.g. Chorus¹), though it would be great if someone of the experts could check that. Comments are welcome.

Footnotes

1. actually Chorus, which was released in February 1983 (or the first Fugue version), would be closer to Smalltalk-80 v2, but there is no release specific documentation (appart from the release notes) and the files provided at CHM are much less complete. ↩

[nbriggs]

I started working on applications built in the Harmony release, 1984.

Note that the J Strother Moore VM spec is for Interlisp, as it existed then, not Interlisp-D -- which was the evolution onto the D-machines with bitmapped displays. For example, you won't find a spec for the Pilot BITBLT opcode.

Koto predates the introduction of Common Lisp features into the system, which required a new bytecode compiler (that can compile both Interlisp and Common code). I don't remember the exact compatibility between Maiko and Lisp versions - it's encoded in a few fields in the sysout image (in the interface page, lversion, minrversion, minbversion, rversion, bversion) and the microcode checks at startup for mutual compatibility. It was possible to bring a D-machine generated sysout (Medley, maybe Lyric?) over to an (early?) Maiko-based system and run it but it was a one-way trip - details hazy, I was less involved in Lisp projects by then.

Maiko's first incarnation was on MC68020 based machines (why you'll find "68k" for host pointer variable names), then subsequently on Sun SPARC, and then x86. The files were not stored on the IFSs, or even on PARC servers, after responsibility for the system was transferred to small successor companies. I looked at the files available in our "history" repo - and I think that a number of the .tar.Z files are corrupted - as they also are in the original form we received them. Probably moved via FTP without binary mode (long predating our involvement).

What is out there is Maiko 3.5.1 (as it existed in 2002 and subsequently updated by us), but the preprocessor directives allow for compilation of versions back to 1.15 (based on the RELEASE=xxx and feature setting in version.h) - but there's no guarantee that later changes were tested to ensure they didn't subtly affect older versions (by us or the original responsible people).

That said, there has been some interest in compiling versions of Maiko to be able to run older (Medley) sysouts, and I've had some success in at least getting a 2.0 sysout to load, bring up the display, and start to run - but then it hung. I've not had time to go back and look at that.

There is also Darkstar, the Dandelion hardware emulator, which has successfully loaded and run the Interlisp-D microcode and booted old (pre-Medley) sysouts - so if you were to try implementing a Maiko equivalent for older Lisp releases you'd have something to check the operation against.

[rochus-keller]

Thanks for the information. Where can I download the Interlisp-D microcode?

I meanwhile found something which looks like an Interlips-D VM implemented in BCPL: https://xeroxparcarchive.computerhistory.org/eris/lisp/fugue.6/bcpl/LispBCPLSources.dm!1_/.index.html
Do you have any information or memories about this code?

[nbriggs]

That BCPL code predates my time. It appears to be the sources for the program that you ran at the Alto Exec to start Lisp, and provides subrs that Lisp will call for various operations, but doesn't include the microcode that interprets the Lisp bytecode.

[nbriggs]

See https://xeroxparcarchive.computerhistory.org/eris/lisp/fugue.6/ucode/.index.html

[rochus-keller]

Note that the J Strother Moore VM spec is for Interlisp, as it existed then, not Interlisp-D

Wouldn't it be fair to assume that the spec describes a decently large subset of an Interlisp-D VM, or is it completely different? Is there a specification of the Interlisp-D bytecode and sysout file format somewhere?

What is out there is Maiko 3.5.1 (as it existed in 2002)

Do you have a link for me so I could have a look at it? The earliest release in the Maiko repository is from 2010; I also found this repository: https://github.com/Interlisp/DOS/tree/main/src. Is this really Maiko 2.0?

Related: #489

[nbriggs]

I'm not the person to answer re the VM spec - perhaps Larry Masinter or Ron Kaplan would have something to say. I never wrote or looked at any of the microcode.

The specification of the sysout format is the (Lisp) code that reads it -- it's just the collection of pages of the Lisp virtual memory image dumped to disk when the system shut down. There's a map, FPTOVP, which informs the loader where in Lisp virtual address space to put a particular file page when it is reloaded. There are various parameter values (such as virtual memory addresses for fixed objects like the start of the stack space and the base of the display memory), and structure definitions (for, e.g., the Dandelion IO coprocessor communications page, and the "system" Interface page) defined in LLPARAMS.

The list of opcodes is in medley/sources/LLCODE - the variable \OPCODES (best to just look at it in a running system) - but not the definition of what they must do.

[nbriggs]

The initial checkin of the code in the Maiko repository is the 2002 version, which was checked in in 2015. Nothing changed between 2002 and 2015.

Yes, the DOS version is Medley 2.0.

[rochus-keller]

The specification of the sysout format is the (Lisp) code that reads it

If I have all Lisp source files of an Interlisp-D release, is it possible to run the system if I have a compiler and a VM (the latter two possibly written in C++)? Or is it like in Smalltalk-80, where a bootstrap of the system is no longer possible because there are circular dependencies and objects in a Smalltalk image which are not reflected by source code but just have to be there for the bootstrap?

[masinter]

The development of Interlisp-D and the VM and Maiko differ significantly from the development of Smalltalk-76 -80 and the Mesa PrincOPS and subsequently Cedar. Interlisp-D was built incrementally with a strong requirement for backward compatibility (of source code) with the Interlisp-10 implementation. The Strother-Moore 'VM' document was written before any byte codes were assigned or the overall architecture described, as far as I can recall.

https://github.com/orgs/Interlisp/discussions/1849 had been an Issue and I recently transferred it to be a "Discussion" -- it would be great to have documentation for all of the opcodes and what they do, but I haven't seen such a thing.

There is work in progress to catelog the documentation we do have in the Zotero bibliography.
It would be great if we could coordinate.

I think in the interest of coordination I'm going to transfer this to a Interlisp/medley Discussion and remove the "issue" component of the Interlisp/History repository, even though interlisp/history sounds more appropriate but isn't.

[rochus-keller]

Is this also spposed to be an answer to my quesion (https://github.com/orgs/Interlisp/discussions/1852#discussioncomment-10931379), i.e. do you confirm that an Interlisp system can be bootstrapped from the Lisp source files only, given that there is a pre-existing compiler and VM (but not a pre-existing Lisp image)?

Also note that this question is unrelated to the opcode documentation (your reference), since the mentioned compiler and VM are completeley different implementations using a different technology.