Skip to content

Commit

Permalink
document avail funnels (#49)
Browse files Browse the repository at this point in the history
* document avail funnels

* Fix typo

---------

Co-authored-by: Sebastien Guillemot <[email protected]>
  • Loading branch information
ecioppettini and SebastienGllmt authored Jul 8, 2024
1 parent d90534b commit d2275cf
Show file tree
Hide file tree
Showing 3 changed files with 90 additions and 5 deletions.
23 changes: 18 additions & 5 deletions docs/home/300-react-to-events/3-funnel-types/200-configuration.md
Original file line number Diff line number Diff line change
Expand Up @@ -15,13 +15,17 @@ Each entry should have a `type` variable, which can be one of
- `evm-main`
- `evm-other`
- `cardano`
- `mina`
- `avail-main`
- `avail-other`

There can be only one entry with the `evm-main` type. This network is the one
that has the Paima contract deployed. Currently also there should only be one
entry of type `cardano`.
There can be only one entry of the `main` type. In the case of `evm-main`, this
network is the one that has the Paima contract deployed.

There can be multiple entries of the `evm-other` type. A funnel is instantiated
for each one of these.
Currently also there should only be one entry of type `cardano` and `mina`.

There can be multiple entries of the `evm-other` or `avail-other` types. A
funnel is instantiated for each one of these.

## Variables

Expand Down Expand Up @@ -95,6 +99,15 @@ Cardano:
network: preview
confirmationDepth: 10
paginationLimit: 2

Avail:
type: avail-other
lightClient: 'http://localhost:7007'
rpc: ws://127.0.0.1:9944
delay: 200
confirmationDepth: 10
funnelBlockGroupSize: 100
presyncStepSize: 1000
```
## Extensions
Expand Down
Original file line number Diff line number Diff line change
@@ -0,0 +1,58 @@
import Search from "./parallel-search.png";

# Parallel Avail funnel

## Configuration

```yaml
Avail:
type: avail-other
lightClient: 'http://localhost:7007'
rpc: ws://127.0.0.1:9944
delay: 200
confirmationDepth: 10
funnelBlockGroupSize: 100
presyncStepSize: 1000
```
## Conceptually
This funnel has the following steps:
1. Use the [Avail light
client](https://docs.availproject.org/docs/operate-a-node/run-a-light-client/Overview)
API to get submitted data in a certain block range.
2. Fetch the headers for the blocks that have data. Use this to build a
deterministic mapping from blocks from this chain to the main chain.
3. Fetch the latest block the light client has processed.
4. Merge the submitted data with the one coming from the base funnel by
appending to it.
## Finding parallel chain blocks
We achieve this in 2 steps:
### 1) Find key blocks during startup with binary search
When booting up Paima engine we need to know what's the earliest parallel chain
block whose events might be included in the next main chain block we have to
sync.
Syncing blocks one-by-one until we find these points would be too slow. To
solve this, we instead use [binary
search](https://simple.wikipedia.org/wiki/Binary_search) to find and cache this
block on boot.
<center>
<img src={Search} />
*Fig*: A binary search looking for the block with a timestamp of 4
</center>
### 2) Fetching blocks in chunks
We fetch parallel chain blocks in chunks of
[funnelBlockGroupSize](./200-configuration.md). We keep doing this until we
have a block with a timestamp greater than the latest one we've seen from the
main chain. This allows us to know how to map the parallel chain block number
of any event we find to the right block in the main chain.
Original file line number Diff line number Diff line change
@@ -0,0 +1,14 @@
# Avail block funnel

## Configuration

```yaml
Avail:
type: avail-main
lightClient: 'http://localhost:7007'
rpc: ws://127.0.0.1:9944
delay: 200
confirmationDepth: 10
funnelBlockGroupSize: 100
presyncStepSize: 1000
```

0 comments on commit d2275cf

Please sign in to comment.