[Federated Execution] Draft Proposal: Revamping the code generation process for federations

[Soroosh129]

@lhstrh and I have been discussing a potentially simpler way of dealing with federated programs that also enables heterogeneous federations (where federates in the federation could be written in different targets).

For more information about federated execution, see here.

For the sake of simplicity, I will use the template FederatedHelloWorld.lf program throughout the proposal:

target C;
reactor MessageGenerator(prefix:string("")) {
    output message:char*;
    state count:int(1);
    timer t(1 sec, 1 sec);
    reaction(t) -> message {=
        // ...
    =}
}

reactor PrintMessage {
    input message:char*;
    reaction(message) {=
        // ...
    =}
}

federated reactor {
    source = new MessageGenerator(prefix = "Hello World");
    print = new PrintMessage();
    source.message -> print.message;
}

In this example, the top-level reactor instances source and print will become their own independent processes that can subsequently be deployed to remote machines. This proposal deals with the implementation details of this process.

Status Quo

For a more in-depth explanation of the current implementation, see #608. However, the short version of it is that we perform an AST transformation which replaces connections across federates with reactions that act as a proxy between them. After the AST transformation, the FederatedHelloWorld.lf program will look as follows:

Note that this is a simplified version of the AST transformation with control reactions not shown.

This design is complicated because these changes happen in an intermediate representation (the AST) that is subsequently hard to reason about and verify. Moreover, the current design highly complicates the logic in the CGenerator because it has to be aware of the fact that not all components of the AST belong to all federates, so it has to figure out what belongs to which federate as it is generating code. Finally, the current design makes it very complicated to enable support for heterogeneous federations because the AST transformation happens inside the code generators for each target.

Proposed New Design

I propose that we implement a new FederatedGenerator that is in charge of taking a federated .lf program and turning it into individual, non-federated .lf programs. These non-federated .lf programs can then be fed into our existing (and simplified) code generators.

Under the new proposed approach, the FederatedGenerator will take FederatedHelloWorld.lf as an input and produce the following structure:

├── fed-gen
│   └── FederatedHelloWorld
│       └── src
│           ├── print.lf
│           └── source.lf
└── src
    └── FederatedHelloWorld.lf

The fed-gen/FederatedHelloWorld/src/print/Main.lf program looks as follows:

The fed-gen/FederatedHelloWorld/src/source/Main.lf program looks as follows:

Each standalone .lf program will have a target that matches the target of the main federate. Subsequently, the print/Main.lf and source/Main.lf programs can individually be fed to the appropriate code generators (depending on their target).

This design decouples the code generators from the complexities that arise in federated execution and makes the process arguably simpler and easier to understand. Since lfc is going to be used to generate code for each federate, it could potentially make deployment also easier, as suggested by @cmnrd here. Finally, it allows for a clear design for the implementation of heterogeneous federations.

Implementation Details

Contents of the "proxy" reactions can be generated by federated extensions for each target. We already have these extensions in a separate org.lflang.federated package, so this should be relatively straightforward.

Currently, the CGenerator code generates certain required initialization code for the federation. This must be done inside preambles in the new proposed design.

The task of producing the launch and distribute scripts are delegated to the FederatedGenerator, which would again make use of our existing extensions to figure out the proper way of compiling and running federates for each target.

Potential Improvements

As advocated by @lhstrh, instead of producing individual top-level components such as network receiver reactions, a proxy reactor could be generated. For example, print/Main.lf will look as follows:

Also advocated by @lhstrh is that the new FederatedGenerator should not manipulate the AST at all. Instead, whenever it encounters a federate that is remote, it would generate a proxy code for that federate instead of actual code.

[edwardalee]

This sounds like a great idea to me.

This would also fix the deadlock bug with the federated pipeline we have that is caused by the fact that reactions can specify a single bank member as a trigger or effect (I can't find the issue to refer to it).

[oowekyala]

A sketch from today

Reactor.lf

Unfederated

Federated program for G, where F is replaced with a proxy

• F is just an empty shell (an interface), the connection to its input port is never triggered
• The sender sends to the network, the output gets received by the receiver
• the causality interface of F might be represented by reactions binding inputs to outputs in F (we should investigate whether this is appropriate)

[edwardalee]

This looks promising! (Just getting back and signing off... See you wall tomorrow) Edward ——————— Edward A. Lee Professor of the Graduate School EECS, UC Berkeley http://eecs.berkeley.edu/~eal

…

On Jun 6, 2022, at 9:17 PM, Clément Fournier ***@***.***> wrote: A sketch from today Reactor.lf Unfederated Federated program for G, where F is replaced with a proxy • F is just an empty shell (an interface), the connection to its input port is never triggered • The sender sends to the network, the output gets received by the receiver • the causality interface of F might be represented by reactions binding inputs to outputs in F (we should investigate whether this is appropriate) — Reply to this email directly, view it on GitHub, or unsubscribe. You are receiving this because you commented.

[lhstrh]

One could think of FProxy as a Proxy<F> (or Federate<F>). Perhaps it would sense to specify it as such in the generated LF code as well. It should be relatively straightforward to generate the appropriate code and diagram for it.

(If the transformation is that simple, however, it begs the question whether it even makes sense to generate the intermediate LF code for each federate.)

[edwardalee]

I think it's still useful to think of it in terms of generating the intermediate .LF code. It gives a simple way to write by hand something that, when run through lfc, gives what the code generator should produce.