feat: gnovm benchmarking tool (#2241)

<!-- please provide a detailed description of the changes made in this
pull request. -->

<details><summary>Contributors' checklist...</summary>

- [ x] Added new tests, or not needed, or not feasible
- [ x] Provided an example (e.g. screenshot) to aid review or the PR is
self-explanatory
- [ x] Updated the official documentation or not needed
- [ x] No breaking changes were made, or a `BREAKING CHANGE: xxx`
message was included in the description
- [ x] Added references to related issues and PRs
- [ x] Provided any useful hints for running manual tests
- [ ] Added new benchmarks to [generated
graphs](https://gnoland.github.io/benchmarks), if any. More info
[here](https://github.com/gnolang/gno/blob/master/.benchmarks/README.md).
</details>

We build this tool mainly for the following issues

#1826
#1828
#1281
#1973


We could also use it in the following cases. 

#1973
#2222




### `gnobench` benchmarks the time consumed for each VM CPU OpCode and
persistent access to the store, including marshalling and unmarshalling
of realm objects.

## Design consideration

### Minimum Overhead and Footprint

- Constant build flags enable benchmarking.
- Encode operations and measurements in binary.
- Dump to a local file in binary.
- No logging, printout, or network access involved.

### Accuracy

- Pause the timer for storage access while performing VM opcode
benchmarking.
- Measure each OpCode execution in nanoseconds.
- Store access includes the duration for Amino marshalling and
unmarshalling.


It is built on top of @deelawn's design and framework with @jaekwon's
input.
#2073



## Usage

### Simple mode

The benchmark only involves the GnoVM and the persistent store. It
benchmarks the bare minimum components, and the results are isolated
from other components. We use standardize gno contract to perform the
benchmarking.

This mode is the best for benchmarking each major release and/or changes
in GnoVM.

    make opcode
    make storage

### Production mode

It benchmarks the node in the production environment with minimum
overhead.
We can only benchmark with standardize the contract but also capture the
live usage in production environment.
It gives us a complete picture of the node perform.


  1. Build the production node with benchmarking flags:

`go build -tags "benchmarkingstorage benchmarkingops"
gno.land/cmd/gnoland`

2. Run the node in the production environment. It will dump benchmark
data to a benchmark.bin file.

3. call the realm contracts at `gno.land/r/x/benchmark/opcodes` and
`gno.land/r/x/benchmark/storage`

  4. Stop the server after the benchmarking session is complete.

  5. Run the following command to convert the binary dump:

  `gnobench -bin path_to_benchmark_bin`

    it converts the binary dump to results.csv and results_stats.csv.


## Results ( Examples )

The benchmarking results are stored in two files:
  1. The raw results are saved in results.csv.

  | Operation       | Elapsed Time | Disk IO Bytes |
  |-----------------|--------------|---------------|
  | OpEval          | 40333        | 0             |
  | OpPopBlock      | 208          | 0             |
  | OpHalt          | 167          | 0             |
  | OpEval          | 500          | 0             |
  | OpInterfaceType | 458          | 0             |
  | OpPopBlock      | 166          | 0             |
  | OpHalt          | 125          | 0             |
  | OpInterfaceType | 21125        | 0             |
  | OpEval          | 541          | 0             |
  | OpEval          | 209          | 0             |
  | OpInterfaceType | 334          | 0             |



2. The averages and standard deviations are summarized in
results_stats.csv.

  | Operation      | Avg Time | Avg Size | Time Std Dev | Count |
|----------------|----------|----------|--------------|-------|
| OpAdd          | 101      | 0        | 45           | 300   |
| OpAddAssign    | 309      | 0        | 1620         | 100   |
| OpArrayLit     | 242      | 0        | 170          | 700   |
| OpArrayType    | 144      | 0        | 100          | 714   |
| OpAssign       | 136      | 0        | 95           | 2900  |
| OpBand         | 92       | 0        | 30           | 100   |
| OpBandAssign   | 127      | 0        | 62           | 100   |
| OpBandn        | 97       | 0        | 54           | 100   |
| OpBandnAssign  | 125      | 0        | 113          | 100   |
| OpBinary1      | 128      | 0        | 767          | 502   |
| OpBody         | 127      | 0        | 145          | 13700 |

---------

Co-authored-by: Morgan Bazalgette <[email protected]>

examples/gno.land/r/x/benchmark/storage/boards.gno

@@ -0,0 +1,97 @@
+package storage
+
+import (
+	"strconv"
+
+	"gno.land/p/demo/avl"
+)
+
+var boards avl.Tree
+
+type Board interface {
+	AddPost(title, content string)
+	GetPost(id int) (Post, bool)
+	Size() int
+}
+
+// posts are persisted in an avl tree
+type TreeBoard struct {
+	id    int
+	posts *avl.Tree
+}
+
+func (b *TreeBoard) AddPost(title, content string) {
+	n := b.posts.Size()
+	p := Post{n, title, content}
+	b.posts.Set(strconv.Itoa(n), p)
+}
+
+func (b *TreeBoard) GetPost(id int) (Post, bool) {
+	p, ok := b.posts.Get(strconv.Itoa(id))
+	if ok {
+		return p.(Post), ok
+	} else {
+		return Post{}, ok
+	}
+}
+
+func (b *TreeBoard) Size() int {
+	return b.posts.Size()
+}
+
+// posts are persisted in a map
+type MapBoard struct {
+	id    int
+	posts map[int]Post
+}
+
+func (b *MapBoard) AddPost(title, content string) {
+	n := len(b.posts)
+	p := Post{n, title, content}
+	b.posts[n] = p
+}
+
+func (b *MapBoard) GetPost(id int) (Post, bool) {
+	p, ok := b.posts[id]
+	if ok {
+		return p, ok
+	} else {
+		return Post{}, ok
+	}
+}
+
+func (b *MapBoard) Size() int {
+	return len(b.posts)
+}
+
+// posts are persisted in a slice
+type SliceBoard struct {
+	id    int
+	posts []Post
+}
+
+func (b *SliceBoard) AddPost(title, content string) {
+	n := len(b.posts)
+	p := Post{n, title, content}
+	b.posts = append(b.posts, p)
+}
+
+func (b *SliceBoard) GetPost(id int) (Post, bool) {
+	if id < len(b.posts) {
+		p := b.posts[id]
+
+		return p, true
+	} else {
+		return Post{}, false
+	}
+}
+
+func (b *SliceBoard) Size() int {
+	return len(b.posts)
+}
+
+type Post struct {
+	id      int
+	title   string
+	content string
+}

examples/gno.land/r/x/benchmark/storage/forum.gno

@@ -0,0 +1,64 @@
+package storage
+
+import (
+	"strconv"
+
+	"gno.land/p/demo/avl"
+)
+
+func init() {
+	// we write to three common data structure for persistence
+	// avl.Tree, map and slice.
+	posts0 := avl.NewTree()
+	b0 := &TreeBoard{0, posts0}
+	boards.Set(strconv.Itoa(0), b0)
+
+	posts1 := make(map[int]Post)
+	b1 := &MapBoard{1, posts1}
+	boards.Set(strconv.Itoa(1), b1)
+
+	posts2 := []Post{}
+	b2 := &SliceBoard{2, posts2}
+	boards.Set(strconv.Itoa(2), b2)
+}
+
+// post to all boards.
+func AddPost(title, content string) {
+	for i := 0; i < boards.Size(); i++ {
+		boardId := strconv.Itoa(i)
+		b, ok := boards.Get(boardId)
+		if ok {
+			b.(Board).AddPost(title, content)
+		}
+	}
+}
+
+func GetPost(boardId, postId int) string {
+	b, ok := boards.Get(strconv.Itoa(boardId))
+	var res string
+
+	if ok {
+		p, ok := b.(Board).GetPost(postId)
+		if ok {
+			res = p.title + "," + p.content
+		}
+	}
+	return res
+}
+
+func GetPostSize(boardId int) int {
+	b, ok := boards.Get(strconv.Itoa(boardId))
+	var res int
+
+	if ok {
+		res = b.(Board).Size()
+	} else {
+		res = -1
+	}
+
+	return res
+}
+
+func GetBoardSize() int {
+	return boards.Size()
+}

examples/gno.land/r/x/benchmark/storage/gno.mod

@@ -0,0 +1 @@
+module gno.land/r/x/benchmark/storage

gnovm/Makefile

@@ -61,6 +61,26 @@ fmt:
 imports:
 	$(rundep) golang.org/x/tools/cmd/goimports $(GOIMPORTS_FLAGS) .
 
+# Benchmarking the VM's opcode and storage access
+.PHONY: opcode storage build_opcode build_storage
+build.bench.opcode:
+	go build -tags "benchmarkingops" -o build/gnobench ./cmd/benchops
+
+build.bench.storage:
+	go build -tags "benchmarkingstorage" -o build/gnobench ./cmd/benchops
+
+# Extract the latest commit hash
+COMMIT_HASH := $(shell git rev-parse --short=7 HEAD)
+
+# Run target
+run.bench.opcode: build.bench.opcode
+	./build/gnobench -out opcode_results_$(COMMIT_HASH).csv
+
+# Run target
+run.bench.storage: build.bench.storage
+	./build/gnobench -out store_results_$(COMMIT_HASH).csv
+
+
 ########################################
 # Test suite
 .PHONY: test
@@ -96,7 +116,6 @@ _test.pkg:
 _test.stdlibs:
 	go run ./cmd/gno test -v ./stdlibs/...
 
-
 _test.filetest:;
 	go test pkg/gnolang/files_test.go -test.short -run 'TestFiles$$/' $(GOTEST_FLAGS)
 

gnovm/cmd/benchops/README.md

@@ -0,0 +1,90 @@
+# `gnobench`  the time consumed for GnoVM OpCode execution and store access
+
+`gnobench` benchmarks the time consumed for each VM CPU OpCode and persistent access to the store, including marshalling and unmarshalling of realm objects.
+
+## Usage
+
+### Simple mode
+
+The benchmark only involves the GnoVM and the persistent store. It benchmarks the bare minimum components, and the results are isolated from other components. We use standardize gno contract to perform the benchmarking.
+
+This mode is the best for benchmarking each major release and/or changes in GnoVM.
+
+    make opcode
+    make storage
+
+### Production mode
+
+It benchmarks the node in the production environment with minimum overhead.
+We can not only benchmark with standardize the contract but also capture the live usage in production environment.
+It gives us a complete picture of the node perform.
+
+
+  1. Build the production node with benchmarking flags:
+
+  `go build -tags "benchmarkingstorage benchmarkingops" gno.land/cmd/gnoland`
+
+  2. Run the node in the production environment. It will dump benchmark data to a benchmarks.bin file.
+
+  3. call the realm contracts at `gno.land/r/x/benchmark/opcodes` and `gno.land/r/x/benchmark/storage`
+
+  4. Stop the server after the benchmarking session is complete.
+
+  5. Run the following command to convert the binary dump:
+
+  `gnobench -bin path_to_benchmarks.bin`
+
+    it converts the binary dump to results.csv and results_stats.csv.
+
+
+## Results
+
+The benchmarking results are stored in two files:
+  1. The raw results are saved in results.csv.
+
+  | Operation       | Elapsed Time | Disk IO Bytes |
+  |-----------------|--------------|---------------|
+  | OpEval          | 40333        | 0             |
+  | OpPopBlock      | 208          | 0             |
+  | OpHalt          | 167          | 0             |
+  | OpEval          | 500          | 0             |
+  | OpInterfaceType | 458          | 0             |
+  | OpPopBlock      | 166          | 0             |
+  | OpHalt          | 125          | 0             |
+  | OpInterfaceType | 21125        | 0             |
+  | OpEval          | 541          | 0             |
+  | OpEval          | 209          | 0             |
+  | OpInterfaceType | 334          | 0             |
+
+
+
+  2. The averages and standard deviations are summarized in results_stats.csv.
+
+  | Operation      | Avg Time | Avg Size | Time Std Dev | Count |
+|----------------|----------|----------|--------------|-------|
+| OpAdd          | 101      | 0        | 45           | 300   |
+| OpAddAssign    | 309      | 0        | 1620         | 100   |
+| OpArrayLit     | 242      | 0        | 170          | 700   |
+| OpArrayType    | 144      | 0        | 100          | 714   |
+| OpAssign       | 136      | 0        | 95           | 2900  |
+| OpBand         | 92       | 0        | 30           | 100   |
+| OpBandAssign   | 127      | 0        | 62           | 100   |
+| OpBandn        | 97       | 0        | 54           | 100   |
+| OpBandnAssign  | 125      | 0        | 113          | 100   |
+| OpBinary1      | 128      | 0        | 767          | 502   |
+| OpBody         | 127      | 0        | 145          | 13700 |
+
+## Design consideration
+
+### Minimum Overhead and Footprint
+
+- Constant build flags enable benchmarking.
+- Encode operations and measurements in binary.
+- Dump to a local file in binary.
+- No logging, printout, or network access involved.
+
+### Accurate
+
+- Pause the timer for storage access while performing VM opcode benchmarking.
+- Measure each OpCode execution in nanoseconds.
+- Store access includes the duration for Amino marshalling and unmarshalling.

gnovm/cmd/benchops/main.go

@@ -0,0 +1,55 @@
+package main
+
+import (
+	"flag"
+	"log"
+	"os"
+	"path/filepath"
+
+	bm "github.com/gnolang/gno/gnovm/pkg/benchops"
+)
+
+var (
+	outFlag   = flag.String("out", "results.csv", "the out put file")
+	benchFlag = flag.String("bench", "./pkg/benchops/gno", "the path to the benchmark contract")
+	binFlag   = flag.String("bin", "", "interpret the existing benchmarking file.")
+)
+
+// We dump the benchmark in bytes for speed and minimal overhead.
+const tmpFile = "benchmark.bin"
+
+func main() {
+	flag.Parse()
+	if *binFlag != "" {
+		binFile, err := filepath.Abs(*binFlag)
+		if err != nil {
+			log.Fatal("unable to get absolute path for the file", err)
+		}
+		stats(binFile)
+		return
+	}
+	bm.Init(tmpFile)
+	bstore := benchmarkDiskStore()
+	defer bstore.Delete()
+
+	dir, err := filepath.Abs(*benchFlag)
+	if err != nil {
+		log.Fatal("unable to get absolute path for storage directory.", err)
+	}
+
+	// load  stdlibs
+	loadStdlibs(bstore)
+
+	if bm.OpsEnabled {
+		benchmarkOpCodes(bstore.gnoStore, dir)
+	}
+	if bm.StorageEnabled {
+		benchmarkStorage(bstore, dir)
+	}
+	bm.Finish()
+	stats(tmpFile)
+	err = os.Remove(tmpFile)
+	if err != nil {
+		log.Printf("Error removing tmp file: %v", err)
+	}
+}