Single Process Main

This page discusses the structure of Druid's main routine and how it must change to support a Single Process Druid.

Main

Druid code resides in a single Java jar file. Command line options choose which service to run. The entry point is org.apache.druid.cli.Main.

CLI

Main is essentially a wrapper around the io.airlift.airline.Cli abstraction (marked as deprecated by the Airlift project). The work of each available option occurs in the "command" (runnable) selected by the CLI.

The CLI configuration offers, as an option, "Run one of the Druid server types." So, our first challenge is to alter the CLI to run multiple server types.

Today, we launch a service using server historical, say. The CLI works by building a mapping from commands to Runnables. Here we focus on the list of servers:

    builder.withGroup("server")
           .withDescription("Run one of the Druid server types.")
           .withDefaultCommand(Help.class)
           .withCommands(serverCommands);

    List<Class<? extends Runnable>> toolCommands = Arrays.asList(
        DruidJsonValidator.class,
        PullDependencies.class,
        CreateTables.class,
        DumpSegment.class,
        ResetCluster.class,
        ValidateSegments.class,
        ExportMetadata.class

The first part of the command, server appears above as the name of the group. We include a list of classes. The CLI gets the second name, historical from an annotation:

@Command(
    name = "historical",
    description = "Runs a Historical node, see https://druid.apache.org/docs/latest/Historical.html for a description"
)
public class CliHistorical extends ServerRunnable

Note that we list the command classes above: the CLI mechanism creates an instance for us of the selected command. This pattern conflicts with Guice, which wants to be the one to create instances and thus do dependency injection. This conflict is discussed elsewhere.

Multiple Services

We want to specify multiple services, instead of just one. The current service command is:

<druid exec> service historical

The desired new behavior is to allow multiple services:

<druid exec> service broker historical

However, it seems that the library Druid uses can't quite be made to work as above. An alternative is to introduce a new command:

<druid exec> bundle broker historical

Where the command is bundle and the services are Boolean options defined in a new "bundle command class." The "bundle" command would handle the shared-server setup as distinct from the single-service setup. The detail of that setup is discussed elsewhere.

Launcher

(Details needed. How the class path is set.)

The launcher must be modified to recognize multiple services, and to set only the _common class path in this case, and to invoke the bundle command rather than service. (See below for details.)