My note for buf.build

Prerequisite

protoc-gen-go
protoc-gen-go-grpc
protoc-gen-grpc-gateway
protoc-gen-validate
protoc-gen-openapiv2
protoc-gen-gofast

What is buf?

Buf’s long-term goal is to enable Schema-Driven Development: a future where APIs are defined consistently, in a way that service owners and clients can depend on.

Defining APIs using an IDL provides a number of benefits over simply exposing JSON/REST services, and today, Protobuf is the most stable, widely-adopted IDL in the industry. However, as it stands, using Protobuf is much more difficult than using JSON as your data transfer format.

Buf is building tooling to make Protobuf reliable and easy to use for service owners and clients, while keeping it the obvious choice on the technical merits. Your organization should not have to reinvent the wheel to create, maintain, and consume Protobuf APIs efficiently and effectively. We'll handle your Protobuf management strategy for you, so you can focus on what matters.

You can find it more at here

Buf tour

This repository is used Buf Tour described here and then modified it to use another api to learn.

The tour introduces you to the buf CLI and the Buf Schema Registry (BSR). Along the way, you will enforce lint standards, detect breaking changes, generate code, create a module, manage a non-trivial dependency graph, and publish the module to the BSR so that it can be consumed by others. The tour takes approximately 20 minutes to complete.

What is buf.yaml?

The buf.yaml file defines a module, and is placed at the root of the Protobuf source files it defines. The placement of the buf.yaml configuration tells buf where to search for .proto files, and how to handle imports.

This file contains lint and breaking change detection rules, and if applicable, the name of your module and a list of dependencies.

Create buf.yaml

# cd to the dir that is the root of user proto dir
cd userapis

# create default buf.yaml file
buf mod init
buf build
# The above command should have exit code 0 and no output. This means that all of the .proto files defined in the current directory successfully compile.

Edit buf.yaml

version: v1 # version of buf, only accept v1 and v1beta1
deps: # all of dependency public from buf.build go here, you do not need to keep all 3rd proto on project
  - buf.build/envoyproxy/protoc-gen-validate
  - buf.build/googleapis/googleapis
  - buf.build/grpc-ecosystem/grpc-gateway
lint: # linter to make sure user.v1.user.proto is valid
  use: # define what module that linter will use
    - DEFAULT
breaking: # same as git, allow to check last state of buf.yaml file and checkout it
  use: # define what module that breaking will use
    - FILE

What is buf.lock?

If your buf.yaml declares any deps, it will be accompanied by a buf.lock file that contains your module's dependency manifest. This manifest represents a single, reproducible build of your module.

You can create or update a buf.lock file for your module by running the buf mod update command

Create buf.lock

# Still in userapis dir
buf mod update
buf build
cat buf.lock

# Generated by buf. DO NOT EDIT.
version: v1
deps:
  - remote: buf.build
    owner: envoyproxy
    repository: protoc-gen-validate
    branch: main
    commit: bb405eae115246f0b5ccf8997136e3d8
    digest: b1-BJUoAjBMVXaIMUNFJ5kdGr-be0QDwhDSTEDQpoxXzys=
    create_time: 2021-08-20T15:05:15.02577Z
  - remote: buf.build
    owner: googleapis
    repository: googleapis
    branch: main
    commit: 169fc074fd8f4100b4c7f552afc3bedc
    digest: b1-hJiYnkothgtMT1aW3EfjtnAMErBk8uOB1UpXX9ck_iQ=
    create_time: 2021-09-15T19:44:01.730106Z
  - remote: buf.build
    owner: grpc-ecosystem
    repository: grpc-gateway
    branch: main
    commit: de24a84aed3d4d84806740946b55d947
    digest: b1-Q-mEW3P3NR5UL8gm0pr3YWFSo5DAtrBqqgvkwH2WFWw=
    create_time: 2021-09-08T00:32:28.231758Z

What is buf.gen.yaml?

The buf.gen.yaml file defines a local generation template, and is used by the buf generate command to generate code for the language(s) of your choice. This file is often used with a module (or another input), and is typically placed next to your buf.work.yaml.

# You need to cd to root folder
cd ..
touch buf.gen.yaml
# Generate pb automation
buf generate userapis
# New dir name "gen" will exists if your buf.gen.yaml is correct

version: v1
managed:
  enabled: true
  go_package_prefix:
    default: buf/gen/go # this is go_option, you don't need to specific it in proto anymore
    except: # except 3rd plugins
      - buf.build/googleapis/googleapis
      - buf.build/envoyproxy/protoc-gen-validate
      - buf.build/grpc-ecosystem/grpc-gateway
plugins:
  - name: go # name is path of protoc-gen-<name>, ex: this is protoc-gen-go
    out: gen/go # dir that storage the output
    opt: paths=source_relative # option of protoc-gen-go
  - name: go-grpc # protoc-gen-go-grpc
    out: gen/go
    opt:
      - paths=source_relative
      - require_unimplemented_servers=false
  - name: grpc-gateway # protoc-gen-grpc-gateway
    out: gen/go
    opt:
      - paths=source_relative
      # - generate_unbound_methods=true
  - name: validate # protoc-gen-validate
    out: gen/go
    opt:
      - paths=source_relative
      - lang=go
  - name: openapiv2 # protoc-gen-openapiv2
    out: gen/go

What is buf.work.yaml?

The buf.work.yaml file is used to define a workspace, where one or more modules can coexist and interoperate within a common directory. Workspaces make it possible for local modules to import Protobuf files from other local modules, and unlock other powerful use cases that operate on multiple modules at the same time.

Create buf.work.yaml

# Same level with buf.gen.yaml
touch buf.gen.yaml

version: v1
directories:
  - userapis # directory path you want to use

Take note

buf is deprecated v1beta1 and recommend to use v1 version

How to validate Protobuf struct?

My main purpose that I want to learn how to inject my own tags (using buf.build and gogoproto) and validate it with my custom validate function implement from validator v10 libary.

I use google error models to return multiple error to client.

All knowledge gathred from internet: Google errors, Advanced gRPC Error Usage, gRPC-Gateway Error Handler, Input validation in GoLang

How validate look from gRPC client?

λ go run client\main.go
2021/09/17 14:44:13 Connected to 127.0.0.1:8080
The "username" field was wrong:
        Username is a required field
The "password" field was wrong:
        Password is a required field
The "email" field was wrong:
        email must be a valid email!
The "role" field was wrong:
        role only accept 'user' value!
2021/09/17 14:44:13 server validate error: rpc error: code = InvalidArgument desc = invalid input
exit status 1

How validate look from HTTP client?

HTTP.status=400
{
    "code": 3,
    "message": "invalid input",
    "details": {
        "email": "Email is a required field",
        "password": "Password is a required field",
        "role": "role only accept 'user' value!",
        "username": "Username is a required field"
    }
}