Tutorial

Summary for the impatient

1. Read about context and praise it.
2. See examples (CLI, HTTP server) and copy them.
3. Wrap *http.Server in well.HTTPServer.
4. Wrap *http.Client in well.HTTPClient.
   Use Do. Never use Get,Head,Post,PostForm.
5. Use well.CommandContext instead of exec.Command and exec.CommandContext.
6. Use well.FieldsFromContext to prepare log fields.
   Doing so will add request tracking information to your logs.

The rest of this document is structured as follows:

• Overview
• Logging
• Context and Environment
• HTTP server
• HTTP client
• Generic server
• Command execution
• Tracking activities
• Further reading

Overview

github.com/cybozu-go/well is a context-based framework to build well-managed programs.

Our definition of well-managed is:

• Important activities are logged.
• Goroutines can be stopped gracefully.
• Signals are handled properly.

Let's see how we can do this.

Logging

We use github.com/cybozu-go/log for structured logging. Among other features, cybozu-go/log can be configured to output logs in JSON Lines format.

The only thing you must do is to call LogConfig.Apply.

A minimal example looks like:

import (
    "github.com/cybozu-go/log"
    "github.com/cybozu-go/well"
)

func main() {
    flag.Parse()
    well.LogConfig{}.Apply()
    log.Info("test", map[string]interface{}{
        "field1": 123,
        "field2": []string{"a", "b", "c"},
    })
}

More concrete example is here.

Context and Environment

context is a new package introduced in Go 1.7. Problems that context solves well include:

• Stop a goroutine on a signal or a deadline whichever comes first.
• Cancel goroutines immediately when one of them returns an error.
• Carry request-scoped values between function calls.

Environment is built on top of context to provide a kind of barrier synchronization of goroutines. A notable difference from barriers is that Environment owns a base context which is inherited by goroutines started by its Go method.

import (
    "github.com/cybozu-go/log"
    "github.com/cybozu-go/well"
)

func main() {

    // You can pass an arbitrary context to `well.NewEnvironment`.
    // It allows you to stop a goroutine with a signal or a deadline.
    ctx := ...
    env := well.NewEnvironment(ctx)

    // If a goroutine started by Go returns non-nil error,
    // the framework calls env.Cancel(err) to signal other
    // goroutines to stop soon.
    env.Go(func(ctx context.Context) error {
        // do something
    })
    env.Go(...)

    // Stop declares no more Go is called.
    // This is optional if env.Cancel will be called
    // at some point (or by a signal).
    env.Stop()

    // Wait returns when all goroutines return.
    err := env.Wait()

    // err is an error passed to env.Cancel, or nil
    // if no one called Cancel.
    if err != nil {
        log.ErrorExit(err)
    }
}

Calling Cancel for an environment cancels the base context and will effectively signals all goroutines started by its Go to return quickly. If one of the goroutines started by Go returns a non-nil error, the framework calls Cancel immediately.

The global environment

The framework initializes an Environment as the global environment. The global environment can be referenced from everywhere via package-level functions such as Go, and Cancel.

The framework also installs a signal handler that calls Cancel for the global environment when the program receives SIGINT or SIGTERM.

You can create a new environment that inherits the global environment like the following code:

import (
	"context"

	"github.com/cybozu-go/well"
)

func main() {
	well.Go(func(ctx context.Context) error {
		env := well.NewEnvironment(ctx)
		env.Go(func(ctx context.Context) error {
			// do something
		})

		env.Stop()
		env.Wait()
		return nil
	})

	well.Stop()
	well.Wait()
}

HTTP server

HTTPServer is a wrapper for *http.Server. It provides access logs and graceful stop function.

Unlike *http.Server, HTTPServer's ListenAndServe returns immediately. The server itself runs in a goroutine started by Go in an environment.

Example of its usage is here.

HTTP client

HTTPClient is a wrapper for *http.Client. It records HTTP request logs by overriding Do.

Go 1.7 adds context to *http.Request to support context-based cancellation. Users of this framework should set a context to it by http.Request.WithContext, then pass it to Do. For this reason, the framework prohibits use of Get, Head, Post, and PostForm methods. If called, they cause panic.

Example:

import (
    "flag"
    "http"

    "github.com/cybozu-go/log"
    "github.com/cybozu-go/well"
)

func main() {
    flag.Parse()
    well.LogConfig{}.Apply()

    client := &well.HTTPClient{
        Client: &http.Client{},
        Severity: log.LvDebug,    // record successful requests as debug level logs.
    }

    well.Go(func(ctx context.Context() error) {
        req, _ := http.NewRequest("GET", "http://...", nil)
        req = req.WithContext(ctx)
        resp, err := client.Do(req)
        if err != nil {
            return err
        }
        // use resp
        return nil
    })

    well.Stop()
    err := well.Wait()
    if err != nil {
        log.ErrorExit(err)
    }
}

Generic server

Server provides a skeleton implementation of generic network servers. It provides graceful stop function.

A minimal example can be found here.

Command execution

LogCmd is a wrapper for *exec.Cmd. It records command execution logs by overriding *exec.Cmd methods such as Run, Output, and Wait.

Use CommandContext to initialize LogCmd struct.

Tracking activities

This part of tutorial is fairly long and therefore separated in Activity tracking.

Further reading

• github.com/cybozu-go/log
• Graceful restart