Deforest within the core language

[countvajhula]

Summary of Changes

This introduces the #%deforestable core form and starts to make some of the changes we talked about.

Planned work:

• initial proof of concept just introducing the #%deforestable core form
• hardcoded support for core list operations like map, filter, foldl, foldr, and range
• discern a usable generic core form syntax from these initial forms
• change the core form syntax to the generic version discerned and reimplement the list forms, still hardcoded in the compiler
• write Qi macros for each list operation and expand to the new core form
• make it more generic, using compile time datatype, etc.
• TBD

I hope to give us a head start with this PR so that we can pick up on it in next week's meeting.

Public Domain Dedication

• In contributing, I relinquish any copyright claims on my contribution and freely release it into the public domain in the simple hope that it will provide value.

(Why: The freely released, copyright-free work in this repository represents an investment in a better way of doing things called attribution-based economics. Attribution-based economics is based on the simple idea that we gain more by giving more, not by holding on to things that, truly, we could only create because we, in our turn, received from others. As it turns out, an economic system based on attribution -- where those who give more are more empowered -- is significantly more efficient than capitalism while also being stable and fair (unlike capitalism, on both counts), giving it transformative power to elevate the human condition and address the problems that face us today along with a host of others that have been intractable since the beginning. You can help make this a reality by releasing your work in the same way -- freely into the public domain in the simple hope of providing value. Learn more about attribution-based economics at drym.org, tell your friends, do your part.)

[countvajhula]

FYI @dzoep @michaelballantyne @benknoble

This is WIP. For next steps, I'd like to:

• make the forms use Qi syntax in the f or op positions (e.g. (map f) and (foldl op init)). I think this would involve modifying the codegen step but also invoking codegen on demand while applying the deforestation rewrite rules
• move the matching (which is very clean and modular thanks to your "rockin' refactor" @dzoep ! Has been so easy to modify) up to the expander

Other things to talk about:

• the syntax of forms like range when they are macros rather than functions. Range accepts 3 arguments, and all of them are numbers. Should we support template syntax even though, as a macro, it isn't a partial application of a function anymore? What are the alternatives?
• what should the generic syntax of #%deforestable be, that would allow it to be extensible (instead of explicitly encoding qi/list into the code generation step as it currently does in this WIP)?

Some of these are longer term discussions and we don't need to have answers at this very moment. The main immediate goal is to structure qi/list so that it (1) exhibits forms with Qi syntax and (2) does not do any host language transformations. Then deforestation of remaining forms can proceed in a manner decoupled from these other design considerations, even if they remain closely coupled in qi-lib in the immediate future.

[benknoble]

Presumably once Dominik's branch settles down, you'll rebase this onto there? I was able to try it myself with something like (my remotes are named dominik and sid):

git log --oneline main..sid/deforestable-core-form --author=sid | tail
# copy the hash of the first commit you made on the branch
git rebase --onto=dominik/racket-list-deforestation <hash>^ sid/deforestable-core-form

(The ^ after the pasted <hash> is important; otherwise that commit disappears.) Right now the <hash> is for feeb211 (Introduce a #%deforestable core form, 2024-06-28), and its parent is ce0bc49 (Add attach-form-property to both places needed by current deforestation (CPS) implementation., 2024-06-28).

My reflog also contains enough fetches of Dominik's force-pushes that this is all equivalent to git rebase --fork-point dominik/racket-list-deforestation sid/deforestable-core-form, which also worked for me locally. To see if you can use the short form, try git merge-base --fork-point dominik/racket-list-deforestation sid/deforestable-core-form and confirm it's the same as git rev-parse <hash>^ (i.e., that it gives you the "parent" commit you'd specify with --onto).

[countvajhula]

@benknoble That is some next level Git fu! 🥋 I will try it out when it's time to rebase.

[benknoble]

Is the base branch wrong? Should this PR be trying to merge into drym-org/qi:deforest-all-the-things?

[benknoble]

Ah, drym-org#180 suggests yes.