Single Process Lifecycle

This page discusses the Druid Lifecycle abstraction and how the behavior must change to support a Single Process Druid.

Lifecycle

The Lifecycle class manages the set of objects for a service.

The GuiceRunnable.initLifecycle() method, implemented on each service CLI object, assumes it is managing the only service in a JVM. For example:

• It logs information about memory and CPU resources.
• It lists all the configuration properties.
• It configures the (global) logger.

To run multiple services, this concept must be split to have one process manager (which does the above) along with a separate service manager (which handles many other tasks.) This is more than just a "wiring" task.

The lifecycle itself is created through Guice, which maintains a singleton instance:

      final Lifecycle lifecycle = injector.getInstance(Lifecycle.class);

The call happens early enough that all services must use the same class, which is good. Unfortunately, if multiple services were run in the same process, they would also use the same Lifecycle instance, which is not so good. Thus, perhaps we'd want to register a "lifecycle factory" to be able to manage multiple instances.

Further, the LifecycleModule class provides static methods that register classes with both the lifecycle and Guice. This class assumes that Guice holds our one and only Lifecycle instance.

Per-Server and Per-Service Configuration

The Lifecycle explained above appears to make assumptions that it a single instance exists per Java process, and that it manages a single service. We must split this concept into two:

• ServerLifecycle which manages a Druid process, and
• ServiceLifecycle which manages a Druid service.

(It is not clear if we need two classes, or just different uses of the current class.)

Further, the ServerRunnable, despite its name, is really a Service Runnable. This code is at the server level:

  public void run()
  {
    ...
    final Lifecycle lifecycle = initLifecycle(injector);
    lifecycle.join();

A service command should have two parts. The first just runs the service. The second runs a server with the service. When launching a single service, the same command handles both tasks. When launching multiple services, a new "composite" command handles the server tasks, while the service commands handle just the service tasks.

LifecycleModule

"A Module to add lifecycle management to the injector." Adds items to both a lifecycle and an injector. Uses Guice scopes to map resources to Lifecycle stages. When the lifecycle starts, it pulls from the Guice injector a single key that has all the resources to be managed by the lifecycle. As a result, the lifecycle can be created only after all service-specific modules add their resources to Guice. That is:

• Add LifecycleModule to the Guice binder.
• Add lifecycle-aware resources (as?).
• Request the lifecycle from the injector (in GuiceRunnable.initLifecycle(). The lifecycle pulls its required resources from the lifecycle key.
• Run the lifecycle (in ServerRunnable.run()).

When used in a split-injector prototype, this module immediately failed since it wants to rebind the "eager binder" key lifecycle to create a "multi-binding" (list), which is then modified per-service. Since Guice does not allow a child injector to replace a key defined in the parent, this mechanism cause the whole split-injector idea to fail.

The eager binder "Registers a key pull out of the injector with an injector.getInstance() call when the lifecycle is created." That is, the lifecycle module builds up a list of classes (not actually keys) that Guice will create (eagerly) so that the lifecycle, when created, can get all classes for a give stage from Guice using the single lifecycle key.

Solution

The first idea to allow multiple services is to allow multiple lifecycles which form a hierarchy:

• Jetty lifecycle which starts/stops the server (and, perhaps the ZK interface and other shared resources.)
• Child lifecycles for each service which manages the web apps within the server.

It might be possible to use a single "uber" lifecycle which represents the web server, however.