Release v0.9.0

lunatic is a VM and rust library that brings Erlang-like concurrency and fault tolerance to Rust.

This is one of the most anticipated lunatic releases and it's finally here! 🎉

It's the result of almost 2 years of research on how to fit Erlang/Elixir abstractions into the Rust type system. There is a lot of awesome stuff here, so let's get started.

Testing WebAssembly

Rust's built-in testing is designed around stack unwinding and panic catching. In WebAssembly panic=abort and because of this it wasn't possible to have a great testing experience (cargo test) when running inside of lunatic. Each failed test would just abruptly end the whole execution.

lunatic 0.9 adds a whole cargo compatible testing framework. The only required change is the usage of the #[lunatic::test] annotation instead of #[test]. Everything else should stay the same. Lunatic will be able to also catch specific panics with #[should_panic(expected = "...")] and will understand most cargo flags (cargo test <filter> --test <specific_test> -- --nocapture should work without issues). The #[lunatic::test] macro also detects if it's being compiled for the Wasm target, if not it will fall back to the regular test macro. This way it can be used in libraries that are cross compiled to other targets than Wasm.

There is also a feature branch of criterion that works with Wasm and can be used to benchmark lunatic apps:

[dev-dependencies]
criterion = { git = "https://github.com/bheisler/criterion.rs", branch = "version-0.4", default-features = false  }

Protocols

Protocols are session types for lunatic. They guarantee that messages sent and received between processes are in the expected order and of the expected type:

type AddProtocol = Recv<i32, Recv<i32, Send<i32, End>>>;

let child = spawn_link!(|protocol: AddProtocol|{
    let (protocol, a) = protocol.receive();
    let (protocol, b) = protocol.receive();
    let _ = protocol.send(a + b);
});

let child = child.send(2);
let child = child.send(3);
let (_, result) = child.receive();
assert_eq!(result, 5);

Every time a message is sent or received the protocol is consumed and a new one is returned. If you changed the order of send and receive invocations the program wouldn't even compile. Rust's type inference is powerful enough to automatically generate the "dual" type for the child handle that expects the opposite order of messages/types.

If the protocol is dropped before the End state is reached the process will panic. Together with this property we also only allow spawning linked processes when protocols are used. This means that there is not going to be "hanging processes" waiting for a message from a dead one. Like in Erlang, if a linked process fails the other one will also be immediately terminated.

We have also added helper macros for common use cases, like a @task (a protocol that just runs a function inside another process and returns the result of it):

// Values 2 & 3 are sent as a message to the task. Lunatic's processes don't
// share memory and all data between processes needs to be exchanged through messages.
let task = spawn_link!(@task |a = 2, b = 3| a + b);
assert_eq!(task.result(), 5);

spawn! & spawn_link! macros

The Process::spawn & Process::spawn_link functions can be quite verbose for simple processes. We are introducing two new macros that just analyse the input and fill in the boilerplate. E.g. instead of creating a process that doesn't capture any environment with:

Process::spawn((), |_, mailbox: Mailbox<()>| {
// code
});

you now can write:

spawn!(|mailbox: Mailbox<()>| {
// code
});

It handles just a few basic cases, as we didn't want to introduce too much magic here, but still can save some typing.

AbstractProcess

The main idea behind processes is state management. Usually you move some state into a process and then keep updating or querying it through messages. To provide more structure around this and not force everyone to build their own state machine from a function we are introducing the AbstractProcess trait. Here is an example of simple counter process that can be incremented with messages:

struct Counter(u32);

impl AbstractProcess for Counter {
    type Arg = u32;
    type State = Self;

    fn init(_: ProcessRef<Self>, start: u32) -> Self {
        Self(start)
    }
}

#[derive(serde::Serialize, serde::Deserialize)]
struct Inc;
impl ProcessMessage<Inc> for Counter {
    fn handle(state: &mut Self::State, _: Inc) {
        state.0 += 1;
    }
}

#[derive(serde::Serialize, serde::Deserialize)]
struct Count;
impl ProcessRequest<Count> for Counter {
    type Response = u32;

    fn handle(state: &mut Self::State, _: Count) -> u32 {
        state.0
    }
}


let counter = Counter::start(5, Some("counter"));
counter.send(Inc);
assert_eq!(counter.request(Count), 6);
counter.shutdown();

If you used the actix actor framework this should feel familiar. You simply define messages that a process can receive and then handlers for the messages. A big difference here between lunatic and actix is that lunatic's processes don't share the heap and all messages need to be serialized when crossing process boundaries.

Each struct that implements AbstractProcess can be spawned as a process with the start & start_link functions. The start functions can also take a name as second argument. A named process could be looked up with ProcessRef::<Counter>::lookup("counter").

Supervisor

A concept that is really important to structuring applications in Erlang/Elixir is the supervisor. And finally rust is also getting support for supervisors. A supervisor is just a process that is linked to its children and can restart them if they fail.

The idea behind it is really simple. Long running processes will eventually get into a weird state that the developer couldn't predict and the best course of action is to restart them. The fresh state is always more likely to be well tested, because it's the one the developer keeps implicitly testing all the time while developing. Looking at a long running process while developing is not that fun or even practical.

struct Sup;
impl Supervisor for Sup {
    type Arg = ();
    // Start 3 `Counters` and monitor them for failures.
    type Children = (Counter, Counter, Counter);

    fn init(config: &mut SupervisorConfig<Self>, _: ()) {
        // If a child fails, just restart it.
        config.set_strategy(SupervisorStrategy::OneForOne);
        // Start each `Counter` with a state of `0` & name last child "hello".
        config.children_args((0, None),(0, None),(0, "hello".to_owned()));
    }
}

let sup = Sup::start((), None);
let children = sup.children();
let count1 = children.2.request(Count);

// Get reference by named.
let hello = ProcessRef::<Counter>::lookup("hello").unwrap();
let count2 = hello.request(Count);
assert_eq!(count1, count2);

This example also demonstrates named processes. The start and start_link methods can now also take a process name. The type is also encoded internally in the name, so it's not possible to look up a process with the wrong type.

Supervisors automatically implement the AbstractProcess trait and can be used as children in other supervisors. That way you can build supervision trees, another important design construct taken from Erlang.

What is next?

We had a long way until we arrived here, but it still feels like the beginning. This changes are a strong foundation to build on top of and I'm really excited to see what interesting architectures developers will come up with. I'm always looking for feedback and way to improve lunatic's developer experience. If you are interested in lunatic or would like to help with the development, drop by our discord and say hi!

Now, I'm going to be more focused on building a layer above (db & web frameworks for lunatic), but still will be extending the VM while yak shaving myself out of issues. @withtypes is rebuilding the distributed story of lunatic.

Lunatic is also a YC founded company. We don't have an ad out yet, but soon will be extending our team and hiring. If this is something that excites you and you would like to build lunatic full time, apply at [email protected].