Getting Started

Installation

Then, run the following commands:

go install capnproto.org/go/capnp/v3/capnpc-go@latest  # install go compiler plugin
GO111MODULE=off go get -u capnproto.org/go/capnp/v3/  # install go-capnproto to $GOPATH

Lastly, ensure $GOPATH/bin has been added to your shell's $PATH variable. If you're unsure how to do this, you can ask us for help.

Compiling Schema Files

Cap'n Proto works by generating code: your schema can be converted to code for any language that Cap'n Proto supports.

Example Schema File

Consider the following schema, stored in foo/books.capnp:

using Go = import "/go.capnp";
@0x85d3acc39d94e0f8;
$Go.package("books");
$Go.import("foo/books");

struct Book {
    title @0 :Text;
    # Title of the book.

    pageCount @1 :Int32;
    # Number of pages in the book.
}

capnpc-go requires that two annotations be present in your schema:

$Go.package("books"): tells the compiler to place package books at the top of the generated Go files.
$Go.import("foo/books"): declares the full import path within your project. The compiler uses this to generate the import statement in the auto-generated code, when one of your schemas imports a type from another.

Compilation will fail unless these annotations are present.

Compiling the Schema

❗Before you begin❗

Ensure your shell's $PATH variable includes $GOPATH/bin. Ask us for help if you're stuck.

To compile this schema into Go code, run the following command. Note that the source path /foo/books.capnp must correspond to the import path declared in your annotations.

capnp compile -I$GOPATH/src/capnproto.org/go/capnp/std -ogo foo/books.capnp

Tip 👉 For more compilation options, see capnp compile --help.

This will output the foo/books.capnp.go file, containing Go structs that can be imported into your programs. These are ordinary Go types that represent the schema you declared in books.capnp. Each has accessor methods corresponding to the fields declared in the schema. For example, the Book struct will have the methods Title() (string, error) and SetTitle(string) error.

In the next section, we will show how you can write these structs to a file or transmit them over the network.

Marshaling and Unmarshaling Data

Technically speaking, there is no (un)marshal step when transmitting Cap'n Proto structs over the wire. The Go types you compiled in the previous step are wrappers around a []byte buffer, whose getters and setters operate on specific offsets. To send these types over the wire, you need only send the underlying buffer.

In this sense, operations like Marshal and Unmarshal are misnomers. Nevertheless, we use this terminology for three reasons:

it is familiar to most Go developers,
it conveys the pattern for effective use (unmarshal bytes into structs; marshal structs into bytes)
it provides an obvious place for hooking in Cap'n Proto's packed encoding.

In the present section, we will show you how to obtain a []byte buffer from your Cap'n Proto types that can be written to a file or network socket, and we will show you how to perform the reverse operation: transforming a []byte buffer into one of your Cap'n Proto types. Lastly, we will show you how you can use the Encoder and Decoder types to stream capnp messages over any io.Writer or io.Reader, respectively.

Marshalling and Unmarshalling

// TODO

Streaming

package main

import (
    "os"

    "foo/books"
    "capnproto.org/go/capnp/v3"
)

func main() {
    // Make a brand new empty message.  A Message allocates Cap'n Proto structs.
    msg, seg, err := capnp.NewMessage(capnp.SingleSegment(nil))
    if err != nil {
        panic(err)
    }

    // Create a new Book struct.  Every message must have a root struct.
    book, err := books.NewRootBook(seg)
    if err != nil {
        panic(err)
    }
    book.SetTitle("War and Peace")
    book.SetPageCount(1440)

    // Write the message to stdout.
    err = capnp.NewEncoder(os.Stdout).Encode(msg)
    if err != nil {
        panic(err)
    }
}

These datatypes can also be read from byte-streams, as follows:

package main

import (
    "fmt"
    "os"

    "foo/books"
    "capnproto.org/go/capnp/v3"
)

func main() {
    // Read the message from stdin.
    msg, err := capnp.NewDecoder(os.Stdin).Decode()
    if err != nil {
        panic(err)
    }

    // Extract the root struct from the message.
    book, err := books.ReadRootBook(msg)
    if err != nil {
        panic(err)
    }

    // Access fields from the struct.
    title, err := book.Title()
    if err != nil {
        panic(err)
    }
    pageCount := book.PageCount()
    fmt.Printf("%q has %d pages\n", title, pageCount)
}

In addition, each type has a .Message() method that returns a capnp.Message, which can be directly marshaled into []bytes using Message.Marshal, and unmarshaled with a corresponding call to Message.Unmarshal.

Lastly, packed encodings are supported via the following methods:

Remote calls using interfaces

Serializing data structures is useful, but the real power of Cap'n Proto is the RPC Protocol, based on Object Capabilities. Although you do not need to know anything about Object Capabilities to use Cap'n Proto RPC, it is an immensely powerful tool for writing secure systems, and the previous link provides a good conceptual introduction.

But for now, let's skip over the theory and proceed by example.

Schema

Capnp RPC operates on interface types in your schema. We will begin by reproducing the Arith RPC server from the Go Standard Library RPC package documentation.

Open a new file called arith.capnp, and copy/paste the following schema:

using Go = import "/go.capnp";

@0xf454c62f08bc504b;

$Go.package("arith");
$Go.import("arith");

interface Arith {
	multiply @0 (a :Int64, b :Int64) -> (:product :Int64);
	divide   @1 (num :Int64, denom :Int64) -> (quo :Int64, rem :Int64);
}

Now, compile the schema as before:

capnp compile -I$GOPATH/src/capnproto.org/go/capnp/std -ogo arith.capnp

Server Implementation

You should take a moment to inspect the generated types in arith.capnp.go. For interface declarations in your schema, the capnp compiler generates several types:

A server interface: Arith_Server
A client struct: Arith
Param, Result and Future types for each method of Arith

Notice that the only missing piece is the Arith_Server implementation. We'll define that below.

☝️ NOTE: the ability to provide multiple implementations of an RPC server is an extremely powerful pattern. It allows you to create such things as mock implementations for testing, and restricted/revokable interface providers for capability-based security.

Let's define our Arith server. In the same directory, create an arith.go file and paste in the following code:

package arith

import capnp "capnproto.org/go/capnp/v3"

type ArithServer struct{}

func (Arith) Multiply(ctx context.Context, call Arith_multiply) error {
	res, err := call.AllocResults()  // allocate the results struct
	if err != nil {
		return err
	}

	res.SetProduct(call.Args().A() * call.Args().B())
	return nil
}

func (Arith) Divide(ctx context.Context, call Arith_divide) error {
	if call.Args().B() == 0 {
		return errors.New("divide by zero")
	}

	res, err := call.AllocResults()
	if err != nil {
		return err
	}

	res.SetQuo(call.Args().A() / call.Args().B())
	res.SetRem(call.Args().A() % call.Args().B())
	return nil
}

Making RPC calls

We now have a working RPC server implementation for our schema interface. Let's begin by starting a server and listening for incoming RPC calls.

The following snippet instantiates an arith.Arith server, and exports it over a bidirectional stream.

// Create a new locally implemented arith server.
server := arith.Arith_ServerToClient(arith.ArithServer{})

// Listen for calls, using the server as the bootstrap interface.
// The 'rwc' parameter can be any io.ReadWriteCloser, usually net.Conn.
conn := rpc.NewConn(rpc.NewStreamTransport(rwc), &rpc.Options{
	// The BootstrapClient is the RPC interface that will be made available
	// to the remote endpoint by default.  In this case, Arith.
	BootstrapClient: capnp.Client(server),
})
defer conn.Close()

// Block until the connection terminates.
select {
case <-conn.Done():
	return nil
case <-ctx.Done():
	return conn.Close()
}

And here's the corresponding client setup and RPC call:

// As before, rwc can be any io.ReadWriteCloser, and will typically be
// a net.Conn.  The rpc.Options can be nil, if you don't want to override
// the defaults.
conn := rpc.NewConn(rpc.NewStreamTransport(rwc), nil)
defer conn.Close()

// Now we wait until we receive the bootstrap interface from the ArithServer.
// The context can be used to time-out or otherwise abort the bootstrap call.
// It is safe to cancel the context after `Bootstrap` returns.
a := Arith(conn.Bootstrap(ctx))

// Okay! Let's make an RPC call!
//
// There are a few things to notice here:
//  1. We pass a callback function to set parameters on the RPC call.  If the
//     call takes no arguments, you MAY pass nil.
//  2. We return a Future type, representing the in-flight RPC call.  We also
//     return a release function, which MUST be called when you're done with
//     the RPC call and its results.
f, release := a.Multiply(ctx, func(ps arith.Arith_multiply_Params) error {
	ps.SetA(2)
	ps.SetB(42)
	return nil
})
defer release()

// You can do other things while the RPC call is in-flight, but we're going to
// block until the call completes.
res, err := f.Struct()
if err != nil {
	return err
}

log.Println(res.Product())  // prints 84

And that's it! Let's reiterate the key points about calling RPC methods:

For the sake of simplicity, this example uses an in-memory pipe, but you can use TCP connections, Unix pipes, or any other type that implements io.ReadWriteCloser.
The return type for a client call is a promise, not an immediate value. It isn't until the Struct() method is called on a method that the client function blocks on the remote side.

A few additional words on the Future type are in order. If your RPC method returns another interface type, you can use the Future to immediately make calls against that as-of-yet-unreturned interface. This relies on a feature of the Cap'n Proto RPC protocol called promise pipelining, the advantage of which is that Cap'n Proto can often optimize away the additional network round-trips when such method calls are chained. This is one of Cap'n Proto's key advantages.

Next Steps

At this point, it would be a good idea to get familiar with Go Cap'n Proto's reference-counting model. It will allow you to understand and reason about the lifetime of interface types, or capabilities, in your code.

Provide feedback

Saved searches

Use saved searches to filter your results more quickly