Skip to content

Commit

Permalink
Update Changelog and minor improvements (#400)
Browse files Browse the repository at this point in the history
  • Loading branch information
enoldev authored Feb 20, 2024
1 parent 7cf5f54 commit 8310934
Show file tree
Hide file tree
Showing 5 changed files with 117 additions and 56 deletions.
2 changes: 1 addition & 1 deletion docs/new/common/authentication.md
Original file line number Diff line number Diff line change
@@ -1,5 +1,5 @@
---
description: Learn how to authenticate with a Substreams Provider
description: StreamingFast Substreams authentication reference
---

Running a Substreams involves sending your package (`.spkg`) to a a Substreams provider for execution. Usually, Substreams provider will require you to authenticate to avoid abuses on the service.
Expand Down
1 change: 1 addition & 0 deletions docs/new/common/intro-solana.md
Original file line number Diff line number Diff line change
Expand Up @@ -33,6 +33,7 @@ If you want to deep dive into the code, you can follow one or several of the Sol
- The [Explore Solana Tutorial](../tutorials/solana/explore-solana/explore-solana.md) will teach you the most basic operations you can perform in a Solana Substreams.
- The [Solana Token Tracker Tutorial](../tutorials/solana/token-tracker/token-tracker.md) will teach how to track an SPL token of your choice.
- The [NFT Trades Tutorial](../tutorials/solana/top-ledger/nft-trades.md) will help you in extracting data from different NFT exchanges.
- The [DEX Trades Tutorial](../tutorials/solana/top-ledger/dex-trades.md) will help you in extracting data from different decentralized exchanges.

## The Solana Data Model

Expand Down
58 changes: 19 additions & 39 deletions docs/new/develop/init-project.md
Original file line number Diff line number Diff line change
@@ -1,49 +1,29 @@

Getting started with Substreams is very easy! If you have the Substreams CLI installed in your computer, you can use the `substreams init` command to initialize a Substreams project from a smart contract.
Getting started with Substreams is very easy! Depending on the blockchain that you want to use, the best way to get started might change:

Given a smart contract address, the `substreams init` command will download the contract's ABI, inspect it and create a Substreams that extracts all the events.
{% tabs %}
{% tab title="Solana" %}
If you want to extract data from Solana, you can take a look at the [Tutorials](../tutorials/solana/solana.md) section, which covers the development of several useful Substreams (SPL tokens, NFT trades, DEX trades...). The [Substreams for Solana Developers](../common/intro-solana.md) section is really useful if this is your first time using Substreams.

{% embed url="https://www.youtube.com/watch?v=vWYuOczDiAA&" %}
Initialize a Substreams project
{% endembed %}

## Initialize the Project

1. Run `substreams init`:

```bash
substreams init
```

2. Provide a name for the project:
The [Substreams Explorer](../tutorials/solana/explore-solana/explore-solana.md) teaches you to most basic extractions you can perform on Solana. More tooling will be developed for Solana soon.
{% endtab %}

```bash
✗ Project name (lowercase, numbers, undescores):
```
{% tab title="EVM" %}
If you have a specific smart contract that you want to extract data from, the `substreams init` initializes a Substreams project that extract the events from the smart contract.

3. Select what protocol you want to use (e.g. Ethereum):
## Tracking a Smart Contract

```bash
? Select protocol:
▸ Ethereum
Other
```

4. Select what chain to use (Ethereum Mainnet, BNB, Polygon...):
{% embed url="https://www.youtube.com/watch?v=vWYuOczDiAA" %}
Initialize a Substreams project
{% endembed %}

```bash
? Select Ethereum chain:
▸ Mainnet
BNB
Polygon
Goerli
↓ Mumbai
```
## Tracking a Factory Smart Contract (Dynamic Datasource)

5. Input the smart contrat address. If you do not provider an addressm the "Bored Ape Yacht Club" smart contract will be used:
The `substreams init` command also offers you the possibility to easily track a factory smart contract (in Substreams terminology, a _dynamic datasource_). The following video covers how easy it is to get started with factory contracts on Substreams.

```bash
✔ Contract address to track (leave empty to use "Bored Ape Yacht Club"):
```
{% embed url="https://www.youtube.com/watch?v=Vn11ovfSpNU" %}
Initialize a Substreams project
{% endembed %}

After providing all the previous information, a new Substreams project will be generated. You can now start coding!
{% endtab %}
{% endtabs %}
93 changes: 89 additions & 4 deletions docs/new/references/change-log.md
Original file line number Diff line number Diff line change
@@ -1,20 +1,105 @@
# Change log
All notable changes to the Substreams project will be documented in this file.

The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.0.0/), and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0.html).

{% hint style="info" %}
Substreams builds upon Firehose.\
Keep track of [Firehose releases and Data model updates](https://firehose.streamingfast.io/release-notes/change-logs) in the Firehose documentation.
{% endhint %}

All notable changes to this project will be documented in this file.
## v1.3.5

The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.0.0/), and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0.html).
### Code generation

* Added `substreams init` support for creating a substreams with data from fully-decoded Calls instead of only extracting events.

## v1.3.4

### Code generation

* Added `substreams init` support for creating a substreams with the "Dynamic DataSources" pattern (ex: a `Factory` contract creating `pool` contracts through the `PoolCreated` event)
* Changed `substreams init` to always add prefixes the tables and entities with the project name
* Fixed `substreams init` support for unnamed params and topics on log events

## v1.3.3

* Fixed `substreams init` generated code when dealing with Ethereum ABI events containing array types.

> [!NOTE]
> For now, the generated code only works with Postgres, an upcoming revision is going to lift that constraint.
## v1.3.2

* Fixed `store.has_at` Wazero signature which was defined as `has_at(storeIdx: i32, ord: i32, key_ptr: i32, key_len: i32)` but should have been `has_at(storeIdx: i32, ord: i64, key_ptr: i32, key_len: i32)`.
* Fixed the local `substreams alpha service serve` ClickHouse deployment which was failing with a message regarding fork handling.
* Catch more cases of WASM deterministic errors as `InvalidArgument`.
* Added some output-stream info to logs.

## v1.3.1

### Server

* Fixed error-passing between tier2 and tier1 (tier1 will not retry sending requests that fail deterministicly to tier2)
* Tier1 will now schedule a single job on tier2, quickly ramping up to the requested number of workers after 4 seconds of delay, to catch early exceptions
* "store became too big" is now considered a deterministic error and returns code "InvalidArgument"

## v1.3.0

### Highlights

* Support new `networks` configuration block in `substreams.yaml` to override modules' *params* and *initial_block*. Network can be specified at run-time, avoiding the need for separate spkg files for each chain.
* [BREAKING CHANGE] Remove the support for the `deriveFrom` overrides. The `imports`, along with the new `networks` feature, should provide a better mechanism to cover the use cases that `deriveFrom` tried to address.

> [!NOTE]
> These changes are all handled in the substreams CLI, applying the necessary changes to the package before sending the requests. The Substreams server endpoints do not need to be upgraded to support it.
### Added

* Added `networks` field at the top level of the manifest definition, with `initialBlock` and `params` overrides for each module. See the substreams.yaml.example file in the repository or https://substreams.streamingfast.io/reference-and-specs/manifests for more details and example usage.
* The networks `params` and `initialBlock`` overrides for the chosen network are applied to the module directly before being sent to the server. All network configurations are kept when packing an .spkg file.
* Added the `--network` flag for choosing the network on `run`, `gui` and `alpha service deploy` commands. Default behavior is to use the one defined as `network` in the manifest.
* Added the `--endpoint` flag to `substreams alpha service serve` to specify substreams endpoint to connect to
* Added endpoints for Antelope chains
* Command 'substreams info' now shows the params

### Removed

* Removed the handling of the `DeriveFrom` keyword in manifest, this override feature is going away.
* Removed the `--skip-package-validation`` option only on run/gui/inspect/info

### Changed

* Added the `--params` flag to `alpha service deploy` to apply per-module parameters to the substreams before pushing it.
* Renamed the `--parameters` flag to `--deployment-params` in `alpha service deploy`, to clarify the intent of those parameters (given to the endpoint, not applied to the substreams modules)
* Small improvement on `substreams gui` command: no longer reads the .spkg multiple times with different behavior during its process.

## v1.2.0

### Client

* Fixed bug in `substreams init` with numbers in ABI types

### Backend

* Return the correct GRPC code instead of wrapping it under an "Unknown" error. "Clean shutdown" now returns CodeUnavailable. This is compatible with previous substreams clients like substreams-sql which should retry automatically.
* Upgraded components to manage the new block encapsulation format in merged-blocks and on the wire required for firehose-core v1.0.0

## v1.1.22

### alpha service deployments

* Fix fuzzy matching when endpoint require auth headers
* Fix panic in "serve" when trying to delete a non-existing deployment
* Add validation check of substreams package before sending deploy request to server

## Unreleased
## v1.1.21

### Changed

* Codegen: substreams-database-change to v1.3, properly generates primary key to support chain reorgs in postgres sink.
* Sink server: support for deploying sinks with DBT configuration, so that users can deploy their own DBT models (supported on postgres and clickhouse sinks). Example manifest file segment:
* Sink server commands all moved from `substreams alpha sink-*` to `substreams alpha service *`
* Sink server: support for deploying sinks with DBT configuration, so that users can deploy their own DBT models (supported on postgres and clickhouse sinks). Example manifest file segment:

```yaml
[...]
Expand Down
19 changes: 7 additions & 12 deletions docs/new/references/manifests.md
Original file line number Diff line number Diff line change
Expand Up @@ -163,23 +163,18 @@ The `binaries[name].file` field references a locally compiled [WASM module](http
**Tip**: The WASM file referenced by the `binary` field is picked up and packaged into an `.spkg` when invoking the [`pack`](https://substreams.streamingfast.io/reference-and-specs/command-line-interface#pack) and [`run`](https://substreams.streamingfast.io/reference-and-specs/command-line-interface#run) commands through the [`substreams` CLI](command-line-interface.md).
{% endhint %}

### `deriveFrom`
### network

It is possible to override an existing substreams by pointing to an override file in the `run` or `gui` command. This override manifest will have a `deriveFrom` field which points to the original Substreams which is to be overriden. This is useful to port a substreams to one network to another. Example of an override manifest:
The `network` field specifies the blockchain where the Substreams will be executed.

```yaml
network: solana
```
deriveFrom: path/to/mainnet-substreams.spkg # This can also be a remote url
package:
name: "polygon-substreams"
version: "100.0.0"

network: polygon
or

initialBlocks:
module1: 17500000
params:
module1: "address=2a75ca72679cf1299936d6104d825c9654489058"
```yaml
network: ethereum
```

### sink
Expand Down

0 comments on commit 8310934

Please sign in to comment.