Skip to content
This repository has been archived by the owner on Jul 10, 2023. It is now read-only.

Home

Jump to bottom
Mario Ranftl edited this page Mar 11, 2021 · 19 revisions

Walkthrough: go-beer-punk-proxy

This is a (guided) walkthrough of a typical dev workflow with allaboutapps/go-starter.

Our goal

We are going to reimplement the Beer PUNK API v2 🍺🍻 (and protect its routes with our default OAuth2 flow).

Ressources

The source-code of this project is available at majodev/go-beer-punk-proxy. However, we expect you to start from scratch with a brand new cloned/forked go-starter project and write (or copy) the referenced files in the upcoming sections yourself.

If you have questions about the general setup, ensure you visit our allaboutapps/go-starter FAQ. Otherwise, feel free to raise an issue at majodev/go-beer-punk-proxy/issues.

It's recommended to clone this wiki (yes, separately, it does not come with the main repo!) by issuing git clone https://github.com/majodev/go-beer-punk-proxy.wiki.git. Through that, you also get a copy of the _walkthrough/** directory which holds all files used in the next sections.

Additionally the completed service is available at beer-workshop-dev.allaboutapps.at for you to play around with.

0. Checkout & Setup VSCode

go-starter: Quickstart

Running on Windows? WSL2 setup done? No, then head to FAQ: WSL2 (Windows only).

🍻🍻🍻 -> Create a new GitHub repo from the go-starter template project <- 🍻🍻🍻.

# Windows WSL 2 specific:
# ensure you are at ~ (not in /mnt/c) in your WSL2 environment
# ensure git is installed and your config is also set in the WSL2 environment
# apt-get install git

# ---

# Also see https://github.com/allaboutapps/go-starter#quickstart
git clone <your-project-repo> go-beer-punk-proxy

cd go-beer-punk-proxy

# (optional, if you want to easily merge changes later from the upstream go-starter project)
make git-merge-go-starter

./docker-helper.sh --up

# You are now attached to the container in an interactive shell (exit via CMD+D).

# It will be easier if we all use the **same** modulename, but you may customize it of course (you will need to slightly adapt some go import paths when copying over our files)
make set-module-name
# github.com/majodev/go-beer-punk-proxy

# Fully initialize the project, build it, test it.
make all

code . # Start VSCode

You may now attach to the devContainer through VSCode. See this guide: FAQ: How does our VSCode setup work?

vscode

Showcase:

1. API Spec

See our defined dev workflow at FAQ: How does our dev workflow work? - 1. API specification (Swagger-first) and the whole FAQ: Swagger section.

First, we are going to spec this new api through OpenAPI v2 / Swagger files.

swagger

Showcase:

  • Type (or copy/paste if you are lazy) from the following urls (or from this wiki's /_walkthrough/01_api_spec/**) to the respective spot in your project:
  • make swagger, make watch-swagger (interactive changes), make (will yield that these routes are not implemented yets)
  • Try to add errors into your definitions or path files and see how make swagger behaves
  • Inspect the auto-generated /internal/types
  • Completion criteria: Head to your local swagger UI at localhost:8081 and check if all 3 routes are available there:
    • GET /api/v1/beers
    • GET /api/v1/beers/{id}
    • GET /api/v1/beers/random

2. ERM Spec

See our defined dev workflow at FAQ: How does our dev workflow work? - 2. ERM specification and migrations (SQL-first)

We will need a place for the beers in our database, thus we are going to add a beers table through a migration.

schema

sql-migrate new create-beers
# Created migration migrations/20201105145136-create-beers.sql

Showcase:

  • Type (or copy/paste if you are lazy) from the following urls (or from this wiki's /_walkthrough/02_erm_spec/**) to the respective spot in your project:
  • make sql, make watch-sql (interactive changes)
  • Inspect the auto-generated /internal/models
  • Toy around with VSCode pgFormatter (try to format your migration.sql file badly) and try to make errors to see how make sql behaves (checkout our sql related make targets (make help | grep sql))
  • FAQ: What's in your /migrations?
  • Completion criteria:
    • Ensure make and make test execute successfully.
    • You may try psql spec and SELECT * FROM beers; to see if the model got actually applied (there will be 0 rows).

3. Types and models

See our defined dev workflow at FAQ: How does our dev workflow work? - 3. Types and models

We will now (in a TDD driven manner) try to validate that the generated SQLBoiler models and go-swagger types work as expected.

vscode_debug_test

Showcase:

  • Type (or copy/paste if you are lazy) from the following urls (or from this wiki's /_walkthrough/03_types_and_models/**) to the respective spot in your project:
  • make, make test
  • Inspect code and ensure all tests execute as expected.
  • VSCode test debugging
  • FAQ: Why should I use this?
  • Completion criteria:
    • Ensure make and make test work (with the new test files)
    • Execute a single one of the above tests via VSCode (go test runs automatically in -v verbose mode there) and copy the integresql database name that got used.
      • e.g. the test logs: upsertable_beers_test.go:18: WithTestDatabase: "integresql_test_7f74ebb2bc5c148cc943fb12e6fc4685_643"
      • psql integresql_test_7f74ebb2bc5c148cc943fb12e6fc4685_643
      • SELECT id, name FROM beers ORDER BY id; should show you all the beers your upserted in your test.

4. Implement handlers / TDD

See our defined dev workflow at FAQ: How does our dev workflow work? - 4. Implement handlers / TDD

We will implement and test the actual beer handlers we have specced and use the models/types we have generated and already tested.

For this we will:

  • add a new route group to our server and apply the (O)Auth middleware for it
  • make beers available in all integration tests that utilize test DBs by configuring our IntegreSQL template database creation mechanism.
  • implement the handlers
  • test the handlers

You need to make the following changes to some *.go files:

// File: internal/api/server.go
// https://github.com/majodev/go-beer-punk-proxy/blob/dev/internal/api/server.go
type Router struct {
	// [...]
    APIV1Beers *echo.Group
    // [...]
}
// [...]

// File: internal/api/router/router.go
// https://github.com/majodev/go-beer-punk-proxy/blob/dev/internal/api/router/router.go
	// [...]
	// Your other endpoints, typically secured by bearer auth, available at /api/v1/**
	APIV1Beers: s.Echo.Group("/api/v1/beers", middleware.Auth(s)),
	// [...]

// File: internal/test/test_database.go
// https://github.com/majodev/go-beer-punk-proxy/blob/dev/internal/test/test_database.go
	// [...]
	// we will compute a db template hash over the following dirs/files
	// [...]
	beersJSONFile = filepath.Join(pUtil.GetProjectRootDir(), "/docs/beers.json")
	// [...]

func initTestDatabaseHash(t *testing.T) {
	// [...]
	h, err := util.GetTemplateHash(migDir, fixFile, selfFile, beersJSONFile)
	// [...]
}
// [...]
func insertFixtures(ctx context.Context, t *testing.T, db *sql.DB) error {
	// [...]
	beers, err := data.GetUpsertableBeerModels()

	if err != nil {
		return err
	}

	for _, beer := range beers {
		if err := beer.Insert(ctx, tx, boil.Infer()); err != nil {
			if err := tx.Rollback(); err != nil {
				return err
			}

			return err
		}
	}

	// Typically we do not use serial ids (auto-incrementing), resetting the sequence after bulk-importing is neccessary
	// https://stackoverflow.com/questions/4448340/postgresql-duplicate-key-violates-unique-constraint/21639138
	_, err = tx.ExecContext(ctx, `SELECT setval('beers_id_seq', (SELECT MAX(id) FROM beers)+1);`)

	if err != nil {
		return err
	}
	// [...]
}
// [...]

vscode_handlers_test

Showcase:

5. Running/debugging the compiled binary locally

We now have tested our new endpoints, let's run our server!

The only thing that's still missing currently is a way to seed (upsert) the beers into our actual development database from the compiled app binary. This binary is going to be the entrypoint of the final Dockerfile (stage app)).

You need to do the following change to a single *.go file:

// File: cmd/db_seed.go
// https://github.com/majodev/go-beer-punk-proxy/blob/dev/cmd/db_seed.go
func applyFixtures() error {
	// [...]
	beers, err := data.GetUpsertableBeerModels()

	if err != nil {
		return err
	}

	for _, beer := range beers {
		if err := beer.Upsert(ctx, tx, true, nil, boil.Infer(), boil.Infer()); err != nil {
			if err := tx.Rollback(); err != nil {
				return err
			}

			return err
		}
	}

	// Typically we do not use serial ids (auto-incrementing), resetting the sequence after bulk-importing is neccessary
	// https://stackoverflow.com/questions/4448340/postgresql-duplicate-key-violates-unique-constraint/21639138
	_, err = tx.ExecContext(ctx, `SELECT setval('beers_id_seq', (SELECT MAX(id) FROM beers)+1);`)

	if err != nil {
		return err
	}

	// [...]
}

Do the following to start the server via the compiled app binary and automatically migrate up the development database and seed all fixtures.

make # compile up
app --help
app server --migrate --seed # or "app db migrate && app db seed && app server"

vscode_running

swagger_exe

Showcase:

6. Posting beers from an admin route

Let's allow to POST to /api/v1/admin/beers to add new beers into our database. We only want to allow that for an administration user (management key auth) and will thus bypass our typical OAuth flow...

// File: internal/api/server.go
type Router struct {
	// [...]
	APIV1Admin *echo.Group
	// [...]
}

// File: internal/api/router/router.go
APIV1Admin: s.Echo.Group("/api/v1/admin", echoMiddleware.KeyAuthWithConfig(echoMiddleware.KeyAuthConfig{
			KeyLookup: "query:mgmt-secret",
			Validator: func(key string, c echo.Context) (bool, error) {
				return key == s.Config.Management.Secret, nil
			},
		})),


// File: internal/test/helper_request.go
// Optional for testing basePayload overwrites only

// Deep copies GenericPayload (map[string]interface{})
// Supports nested GenericPayload and []GenericPayload
// Based on ideas from https://stackoverflow.com/questions/23057785/how-to-copy-a-map
func (g GenericPayload) Copy() GenericPayload {
	cp := make(GenericPayload)
	for k, v := range g {
		if vm, ok := v.(GenericPayload); ok {
			cp[k] = vm.Copy()
		} else if vm, ok := v.([]GenericPayload); ok {
			slice := make([]GenericPayload, 0, len(vm))
			for _, vv := range vm {
				slice = append(slice, vv.Copy())
			}
			cp[k] = slice
		} else {
			// Note that we expect this to be a copyable primitive type
			cp[k] = v
		}
	}
	return cp
}

Showcase:

  • Implement the POST /api/v1/admin/beers endpoint, so we can actually add something to the database from a REST endpoint.
  • Type (or copy/paste if you are lazy) from the following urls (or from this wiki's /_walkthrough/06_posting_beers_as_admin/**) to the respective spot in your project. These are the most important ones:
  • Try to play with testing this endpoint (basePayload / copy payload), inspect the database isolation mechanics while running these tests.
  • Try to use the SwaggerUI to post new beers
  • Reinspect the created snapshots
  • Completion criteria:
    • Ensure make and make test work (with the new test files)
    • Investigate the created snapshots (.golden files)
    • Toogle code coverage in VSCode via the command line palette within internal/api/handlers/admin package to see if we still need to increase the test coverage there.
    • Also inspect and try out other endpoints that use the management authentication:

7. Preemptive scheduling

Here is a very naive endpoint that sums up a received count e.g.:

1: 1=1
2: 1+2=3
3: 1+2+3=6

// /internal/api/handlers/common/get_sum.go
package common

import (
	"fmt"
	"net/http"

	"github.com/labstack/echo/v4"
	"github.com/majodev/go-beer-punk-proxy/internal/api"
)

func GetSumRoute(s *api.Server) *echo.Route {
	return s.Router.Management.GET("/sum/:count", getSumHandler(s))
}

func getSumHandler(s *api.Server) echo.HandlerFunc {
	return func(c echo.Context) error {

		var cnt int64 = 1

		if err := echo.PathParamsBinder(c).Int64("count", &cnt).BindError(); err != nil {
			return c.String(400, "Please provide a integer.")
		}

		if cnt == 0 {
			return c.String(400, "Count cannot be 0")
		}

		var sum int64 = 1

		fmt.Printf("Received: %v: %v", cnt, &cnt)

		var i int64 = 1
		for ; i != cnt; i++ {
			sum += (i + 1)
			fmt.Println(sum)
		}

		return c.String(http.StatusOK, fmt.Sprintf("%v", sum))
	}
}

Showcase:

  • Try to add testcases (0 and for positive numbers) for this piece of code without fixing anything yet
  • Inspect how this piece of code does perform while running with app server (e.g. http://localhost:8080/-/sum/21?mgmt-secret=mgmtpass).
    • What happens to others users, if we have a malicious user who is requesting the sum of -1?
  • Whats preemptive scheduling (see Cooperative vs. Preemptive: a quest to maximize concurrency power and Go: Asynchronous Preemption.
  • In which ways does that differ to async programming in other languages (which mostly use cooperative scheduling of their async work)?
  • Completion criteria:

8. Going Live / Ops (docs only)

This is typically a guided tour into our CI and Kubernetes architecture at all about apps, so I'm afraid, we can only provide some docs here.

See:

Showcase (mostly all about apps internal):

Clone this wiki locally