Skip to content

Commit

Permalink
Docs/examples (#881)
Browse files Browse the repository at this point in the history
* docs: examples
* docs: readme
  • Loading branch information
0xJepsen authored Feb 13, 2024
1 parent a0bc9e0 commit cc509b1
Show file tree
Hide file tree
Showing 3 changed files with 59 additions and 21 deletions.
37 changes: 16 additions & 21 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -11,14 +11,16 @@
**Arbiter** is a framework for stateful Ethereum smart-contract simulation.
The framework features an [`ethers-rs`](https://github.com/gakonst/ethers-rs) middleware built on top of [revm](https://github.com/bluealloy/revm) which allows the end user to interact with a sandboxed `revm` instance as if it were an Ethereum node.
This provides a familiar interface for interacting with the Ethereum Virtual Machine (EVM), but with unrivaled speed.
Furthermore, Arbiter provides containment and management for simulations. For a running list of vulnerabilities found with Arbiter, please see the [Vulnerability Corpus](./documentation/src/contributing/vulnerability_corpus.md).
Furthermore, Arbiter provides containment and management for simulations. For a running list of vulnerabilities found with Arbiter, please see the [Vulnerability Corpus](https://primitivefinance.github.io/arbiter/vulnerability_corpus.html).

The Arbiter workspace has three crates:
The Arbiter workspace has five crates:
- `arbiter`: The binary crate that exposes a command line interface for initializing simulations via a templated repository and generating contract bindings needed for the simulation.
- `arbiter-core`: The lib crate that contains the core logic for the Arbiter framework including the `RevmMiddleware` discussed before, the `Environment` which envelopes simulations, and the `Manager` who controls a collection of environments.
- `arbiter-engine`: The lib crate that provides abstractions for building simulations and more.
- `arbiter-core`: The lib crate that contains the core logic for the Arbiter framework including the `ArbiterMiddleware` discussed before, and the `Environment` which envelopes simulations.
- `arbiter-engine`: The lib crate that provides abstractions for building simulations, agents, and behaviors.
- `arbiter-bindings`: The lib crate that contains the bindings for utility smart contracts that are used for testing and development.
- `arbiter-macros`: The lib crate that contains the macros used in the `arbiter-engine` crate.

The purpose of Arbiter is to provide a toolset to construct arbitrary agents (defined in Rust, by smart contracts, or even other foreign function interfaces) and have these agents interact with an Ethereum-like environment of your design.
The purpose of Arbiter is to provide a toolset to construct arbitrary agents that interact with an Ethereum-like environment of your design.
All contract bytecode is run directly using a blazing-fast EVM instance `revm` (which is used in live RPC nodes such as [`reth`](https://github.com/paradigmxyz/reth)) so that your contracts are tested in the exact same type of environment that they are deployed in.

## Docs
Expand Down Expand Up @@ -129,39 +131,32 @@ To see the documentation for the Arbiter crates, please visit the following:
- [`arbiter`](https://docs.rs/crate/arbiter/)
- [`arbiter-bindings`](https://docs.rs/crate/arbiter-bindings/)
- [`arbiter-core`](https://docs.rs/arbiter-core/)
- [`arbiter-derive`](https://docs.rs/arbiter-derive/)
- [`arbiter-macros`](https://docs.rs/arbiter-macros/)
- [`arbiter-engine`](https://docs.rs/arbiter-engine/)

You will also find each of these on crates.io.
You will find each of these on crates.io.

## Benchmarks

In `arbiter-core`, we have a a small benchmarking suite that compares the `RevmMiddleware` implementation over the `Environment` to the [Anvil](https://github.com/foundry-rs/foundry/tree/master/crates/anvil) local testnet chain implementation.
In `arbiter-core`, we have a a small benchmarking suite that compares the `ArbiterMiddleware` implementation over the `Environment` to the [Anvil](https://github.com/foundry-rs/foundry/tree/master/crates/anvil) local testnet chain implementation.
The biggest reasons why we chose to build Arbiter was to gain more control over the EVM environment and to have a more robust simulation framework, but we also wanted to gain in speed which is why we chose to build our own interface over `revm` as opposed to using Anvil (which does use `revm` under the hood).
For the following, Anvil was set to mine blocks for each transaction as opposed to setting an enforced block time and the `Environment` was set with a block rate of 10.0 (this was chosen somewhat arbitrarily as we will add in more block control in the future).
Preliminary benchmarks of the `RevmMiddleware` interface over `revm` against Anvil are given in the following table.
Preliminary benchmarks of the `ArbiterMiddleware` interface over `revm` against Anvil are given in the following table.

to run the benchmarking code yourself, you can run:
```bash
cargo bench --package arbiter-core
```

bench from 10/24/23 arbiter-core v0.6.3
| Operation | RevmMiddleware | Anvil | Relative Difference |
| Operation | ArbiterMiddleware | Anvil | Relative Difference |
|-----------------|-----------------|--------------|---------------------|
| Deploy | 238.975µs | 7712.436µs | ~32.2729x |
| Lookup | 565.617µs | 17880.124µs | ~31.6117x |
| Stateless Call | 1402.524µs | 10397.55µs | ~7.413456x |
| Stateful Call | 2043.88µs | 154553.225µs | ~75.61756x |


bench from 06/??/23ish arbiter-core v0.4.??
| Operation | RevmMiddleware | Anvil | Relative Difference |
|-----------------|-----------------|--------------|---------------------|
| Deploy | 241.819µs | 8.215446ms | ~33.97x |
| Lookup | 480.319µs | 13.052063ms | ~27.17x |
| Stateless Call | 4.03235ms | 10.238771ms | ~2.53x |
| Stateful Call | 843.296µs | 153.102478ms | ~181.55x |

The above can be described by:
- Deploy: Deploying a contract to the EVM.
We deployed both `ArbiterToken` and `ArbiterMath` in this method, so you can divide the time by two to get an estimate for the time it takes to deploy a single contract.
Expand All @@ -182,9 +177,9 @@ The benchmarking code can be found in the `arbiter-core/benches/` directory and
The above was achieved running `cargo bench --package arbiter-core` which will automatically run with the release profile.
Times were achieved on an Apple Macbook Pro M1 Max with 8 performance and 2 efficiency cores, and with 32GB of RAM.

Of course, the use cases of Anvil and the `RevmMiddleware` can be different.
Anvil represents a more realistic environment with networking and mining, while the `RevmMiddleware` is simpler environment with the bare essentials to running stateful simulations.
Anvil also mines blocks for each transaction, while the `RevmMiddleware` does not.
Of course, the use cases of Anvil and the `ArbiterMiddleware` can be different.
Anvil represents a more realistic environment with networking and mining, while the `ArbiterMiddleware` is simpler environment with the bare essentials to running stateful simulations.
Anvil also mines blocks for each transaction, while the `ArbiterMiddleware` does not.
We hope to improve our API to allow the end user to be able to interface with their own choice of EVM environment to suit what ever their needs may be!

Please let us know if you find any issues with these benchmarks or if you have any suggestions on how to improve them!
Expand Down
1 change: 1 addition & 0 deletions documentation/src/SUMMARY.md
Original file line number Diff line number Diff line change
@@ -1,6 +1,7 @@
# Summary
[Arbiter](./index.md)
- [Getting Started](./getting_started/index.md)
- [Examples](./getting_started/examples.md)
# Usage
- [Overview](./usage/index.md)
- [Arbiter Core](./usage/arbiter_core/index.md)
Expand Down
42 changes: 42 additions & 0 deletions documentation/src/getting_started/examples.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,42 @@
# Examples

We have a few examples to help you get started with Arbiter. These examples are designed to be simple and easy to understand. They are also designed to be easy to run and modify. We hope you find them helpful!

Our examples are in the [examples](https://github.com/primitivefinance/arbiter/tree/main/examples) directory. There are two examples: one for building a simulation and one fork forking the mainnet state.

## Simulation

You can run them with the following command:

```bash
cargo run --example project simulate examples/project/configs/example.toml
```

This will run the minimal counter-simulation. The simulation is very minimal and is designed to be easy to understand. It uses an arbiter main macro to derive the `incrementer` behavior for a single agent. Our design philosophy is that the users of Arbiter should only need to define behaviors and a configuration toml for the behaviors. You can see how the behaviors were represented in this simulation in the [behaviors](https://github.com/primitivefinance/arbiter/tree/main/examples/project/behaviors) module. We implement a single behavior for the incrementer struct that deploys the counter on startup and then on the increment event will increment the count.

For more information on the behavior trait please see the section on [behaviors](https://primitivefinance.github.io/arbiter/usage/arbiter_engine/behaviors.html)


## Forking

You can run the fork example with the following command:

```bash
arbiter fork examples/fork/weth_config.toml
```

This will fork the state specified in the `weth_config.toml` file. If you would like to fork a different state, you can modify the `weth_config.toml` file to point to include additional EOAs or contract storage. Once you have forked the state you want, you can start your simulation with the forked state by loading it into a memory revm instance like so:

```rust ignore
use arbiter_core::{database::Fork::*, Environment, ArbiterMiddleware};

let fork = Fork::from_disk("tests/fork.json").unwrap();

// Get the environment going
let environment = Environment::builder().with_db(fork.db).build();

// Create a client
let client = ArbiterMiddleware::new(&environment, Some("name")).unwrap();
```


0 comments on commit cc509b1

Please sign in to comment.