Skip to content

Commit

Permalink
Reorganize codelab (#352)
Browse files Browse the repository at this point in the history
* edit codelab link

* posix

* typo

* mysql

* link to mysql

* better wording

* s/the/a

* better phrasing

* single list

* Update cmd/conformance/README.md

* address comments

* a missing one

* use  everywhere

* add mysql database instructions
  • Loading branch information
phbnf authored Dec 2, 2024
1 parent 2db6e06 commit ef536ca
Show file tree
Hide file tree
Showing 4 changed files with 62 additions and 24 deletions.
2 changes: 1 addition & 1 deletion README.md
Original file line number Diff line number Diff line change
Expand Up @@ -93,7 +93,7 @@ Once this process has been completed, a new entry will:

### Getting Started

The best place to start is the codelab provided in the [conformance](./cmd/conformance/) directory.
The best place to start is the [codelab](./cmd/conformance#codelab).
This will walk you through setting up your first log, writing some entries to it via HTTP, and inspecting the contents.

Take a look at the example personalities in the `/cmd/` directory:
Expand Down
9 changes: 8 additions & 1 deletion cmd/conformance/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -6,7 +6,7 @@ This directory contains personalities that serve a dual purpose:

Each subdirectory contains an implementation of the same personality built on top of Tessera.
Implementations are provided that use:
- [A local filesystem](./posix/)
- [A local POSIX-compliant filesystem](./posix/)
- [MySQL](./mysql/)
- [GCP](./gcp/)
- [AWS](deployment/live/aws/codelab/)
Expand All @@ -16,6 +16,13 @@ The contents of any request body will be appended to the log, and the decimal in

## Codelab

This codelab will help you add a few entries to a log, and inspect its contents.
First, you need to bring up personality (a server built with Tessera which manages the log) on the infrastructure of your choice:
- [A local POSIX-compliant filesystem](./posix#bring-up-a-log)
- [MySQL](./mysql#bring-up-a-log)
- [GCP](./gcp)
- [AWS](./aws)

Choose one of the implementations above and deploy it.
In the shell you are going to run this codelab in, define the following environment variables (check the logging output from the implementation you deployed, as these may have been output):
- The write URL: `${WRITE_URL}`
Expand Down
44 changes: 31 additions & 13 deletions cmd/conformance/mysql/README.md
Original file line number Diff line number Diff line change
@@ -1,54 +1,72 @@
# How to Run a Tessera Log (MySQL)
# Conformance MySQL log

This binary runs an HTTP web server that accepts POST HTTP requests to an `/add` endpoint.
This endpoint takes arbitrary data and adds it to a MySQL based Tessera log.

> [!WARNING]
> - This is an example and is not fit for production use, but demonstrates a way of using the Tessera Log with MySQL storage backend.
> - This example is built on the [tlog tiles API](https://c2sp.org/tlog-tiles) for read endpoints and exposes a /add endpoint that allows any POSTed data to be added to the log.
The tessera log with the MySQL storage implementation can be started with either Docker Compose or manual `go run`.
## Bring up a log

This will help you bring up a MySQL database to store a Tessera log, and start a personality
binary that can add entries to it.

You can run this personality using Docker Compose or manually with `go run`.

Note that all the commands are executed at the root directory of this repository.

## Docker Compose
### Docker Compose

### Prerequisites
#### Prerequisites

- [Docker Compose](https://docs.docker.com/compose/install/)
Install [Docker Compose](https://docs.docker.com/compose/install/).

### Starting
#### Start the log

```sh
docker compose -f ./cmd/conformance/mysql/docker/compose.yaml up
```

Add `-d` if you want to run the log in detached mode.

### Stopping
#### Stop the log

```sh
docker compose -f ./cmd/conformance/mysql/docker/compose.yaml down
```

## Manual
### Manual

#### Prerequisites

### Prerequisites
You need to have a MySQL database configured to run on port `3306`, accepting
password auth for `root` with the password set to `root`, and a DB instance
called `test_tessera`.

Assume you have the MySQL database ready. An alternative way is to run a MySQL database via [Docker](https://docs.docker.com/engine/install/).
You can start one using [Docker](https://docs.docker.com/engine/install/).

```sh
docker run --name test-mysql -p 3306:3306 -e MYSQL_ROOT_PASSWORD=root -e MYSQL_DATABASE=test_tessera -d mysql:8.4
```

### Starting
#### Start the log

```sh
go run ./cmd/conformance/mysql --mysql_uri="root:root@tcp(localhost:3306)/test_tessera" --init_schema_path="./storage/mysql/schema.sql" --private_key_path="./cmd/conformance/mysql/docker/testdata/key"
```

### Stopping
#### Stop the log

<kbd>Ctrl</kbd> <kbd>C</kbd>

## Using the log
## Add entries to the log

### Manually

Head over to the [codelab](../#codelab) to manually add entries to the log, and inspect the log.

### Using the hammer

In this example, we're running 256 writers against the log to add 1024 new leaves within 1 minute.

Expand Down
31 changes: 22 additions & 9 deletions cmd/conformance/posix/README.md
Original file line number Diff line number Diff line change
@@ -1,25 +1,35 @@
# conformance-posix

This command runs an HTTP web server that accepts POST HTTP requests to a `/add` endpoint.
This binary runs an HTTP web server that accepts POST HTTP requests to an `/add` endpoint.
This endpoint takes arbitrary data and adds it to a file-based log.

## Using in the hammer
## Bring up a log
This will create a directory in your filesystem to store a log, and start a personality binary
that can add entries to this log.

First, define a few environment vaiables:

First bring up a new log in one terminal:
```shell
export LOG_PRIVATE_KEY="PRIVATE+KEY+example.com/log/testdata+33d7b496+AeymY/SZAX0jZcJ8enZ5FY1Dz+wTML2yWSkK+9DSF3eg"
export LOG_PUBLIC_KEY="example.com/log/testdata+33d7b496+AeHTu4Q3hEIMHNqc6fASMsq3rKNx280NI+oO5xCFkkSx"
export LOG_DIR=/tmp/mylog2
export LOG_DIR=/tmp/mylog
```

Then, start the personality:

# Initialize a new log
```shell
go run ./cmd/conformance/posix \
--storage_dir=${LOG_DIR} \
--initialise \
--listen=:2025 \
--v=2
```

In another terminal, run the hammer against the log.
## Add entries to the log
### Manually
Head over to the [codelab](../#codelab) to manually add entries to the log, and inspect the log.

### Using the hammer
In another terminal, run the [hammer](./internal/hammer) against the log.
In this example, we're running 32 writers against the log to add 128 new leaves within 1 minute.

```shell
Expand All @@ -38,6 +48,9 @@ Optionally, inspect the log on the filesystem using the woodpecker tool to see t
Note that this reads only from the files on disk, so none of the commands above need to be running for this to work.

```shell
go run github.com/mhutchinson/woodpecker@main --custom_log_type=tiles --custom_log_url=file:///${LOG_DIR}/ --custom_log_origin=example.com/log/testdata --custom_log_vkey=${LOG_PUBLIC_KEY}
go run github.com/mhutchinson/woodpecker@main \
--custom_log_type=tiles \
--custom_log_url=file:///${LOG_DIR}/ \
--custom_log_origin=example.com/log/testdata \
--custom_log_vkey=${LOG_PUBLIC_KEY}
```

0 comments on commit ef536ca

Please sign in to comment.