Make Docs Solana-friendly (#384)

docs/README.md

@@ -1,5 +1,5 @@
 Substreams is a powerful indexing technology, which allows you to:
-- Extract data from several blockchains (Ethereum, Polygon, BNB, Solana...).
+- Extract data from several blockchains (Solana, Ethereum, Polygon, BNB...).
 - Apply custom transformations to the data.
 - Send the data to a place of your choice (for example, a Postgres database or a file).
 
@@ -21,6 +21,37 @@ If you can't find a Substreams package that retrieves exactly the data you need,
 
 You can write your own Rust function to extract data from the blockchain:
 
+{% tabs %}
+{% tab title="Solana" %}
+The following function extracts Solana instructions from a specific program ID.
+```rust
+fn map_filter_instructions(params: String, blk: Block) -> Result<Instructions, Vec<substreams::errors::Error>> {
+    let filters = parse_filters_from_params(params)?;
+
+    let mut instructions : Vec<Instruction> = Vec::new();
+
+    blk.transactions.iter().for_each(|tx| {
+        let msg = tx.transaction.clone().unwrap().message.unwrap();
+        let acct_keys = msg.account_keys.as_slice();
+        let insts : Vec<Instruction> = msg.instructions.iter()
+            .filter(|inst| apply_filter(inst, &filters, acct_keys.to_vec()))
+            .map(|inst| {
+            Instruction {
+                program_id: bs58::encode(acct_keys[inst.program_id_index as usize].to_vec()).into_string(),
+                accounts: inst.accounts.iter().map(|acct| bs58::encode(acct_keys[*acct as usize].to_vec()).into_string()).collect(),
+                data: bs58::encode(inst.data.clone()).into_string(),
+            }
+        }).collect();
+        instructions.extend(insts);
+    });
+
+    Ok(Instructions { instructions })
+}
+```
+{% endtab %}
+
+{% tab title="EVM" %}
+The following function extracts USDT transaction from EVM blockchains.
 ```rust
 fn get_usdt_transaction(block: eth::Block) -> Result<Vec<Transaction>, substreams:error:Error> {
     let my_transactions = block.transactions().
@@ -30,8 +61,9 @@ fn get_usdt_transaction(block: eth::Block) -> Result<Vec<Transaction>, substream
     Ok(my_transactions)
 }
 ```
+{% endtab %}
+{% endtabs %}
 
-<figure><img src=".gitbook/assets/intro/develop-flow.png" width="100%" /></figure>
 
 ## How Does It Work?
 

docs/SUMMARY.md

@@ -1,6 +1,8 @@
 # Table of contents
 
-* [Substreams](README.md)
+* [What is Substreams?](README.md)
+* [Substreams for Solana Developers](new/common/intro-solana.md)
+
 
 ## Documentation
 
@@ -50,8 +52,14 @@
 * [EVM](new/tutorials/evm/evm.md)
   * [Exploring Ethereum](new/tutorials/evm/exploring-ethereum/exploring-ethereum.md)
     * [Mapping Blocks](new/tutorials/evm/exploring-ethereum/map_block_meta_module.md)
-    * [Filtering Transactions](new/tutorials/evm/exploring-ethereum/map_filter_transactions_module.md)
-    * [Retrieving Events of a Smart Contract](new/tutorials/evm/exploring-ethereum/map_contract_events_module.md)
+    * [Filter Transactions](new/tutorials/evm/exploring-ethereum/map_filter_transactions_module.md)
+    * [Retrieve Events of a Smart Contract](new/tutorials/evm/exploring-ethereum/map_contract_events_module.md)
+* [Solana](new/tutorials/solana/solana.md)
+  * [Explore Solana](new/tutorials/solana/explore-solana/explore-solana.md)
+    * [Filter Instructions](new/tutorials/solana/explore-solana/filter-instructions.md)
+    * [Filter Transactions](new/tutorials/solana/explore-solana/filter-transactions.md)
+
+
 * [Rust](new/tutorials/rust/rust.md)
   * [Option struct](new/tutorials/rust/option.md)
   * [Result struct](new/tutorials/rust/result.md)

docs/new/common/intro-evm.md

@@ -0,0 +1,13 @@
+Substreams is available for several EVM-compatible chains, such as Ethereum, Polygon or Avalanche.
+
+
+## Getting Started with Substreams
+
+If you intention is to consume an already developed Substreams package...
+
+
+## The EVM Data Model
+
+## Eth Calls
+
+## Tutorials

docs/new/common/intro-solana.md

@@ -0,0 +1,59 @@
+Substreams allows you to easily extract data from the the Solana blockchain. With Substreams, you can retrieve transactions, instructions or accounts, taking advantage of its powerful streaming technology. It's super fast!
+
+<figure><img src="../../.gitbook/assets/intro/solana-logo.png" width="100%" /></figure>
+
+## Getting Started
+
+First, you must consider whether you want to develop your own Substreams or consume a ready-to-use Substreams. It is possible that someone has already built a Substreams package to extract the data you want; you can explore Substreams packages in the [Substreams Registry](https://substreams.dev).
+
+**If you have found a Substreams package that fits your needs**, then explore the [Consume Substreams](../consume/consume.md) section. At the most basic level you should cover:
+
+- [Install the Substreams CLI](./installing-the-cli.md)
+- [Authentication](./authentication.md)
+- [Packages](./packages.md)
+- Choose how you want to consume the data:
+    - [Send the data to a SQL database.](./../consume/sql/sql.md)
+    - [Stream the data from your application.](../consume/stream/stream.md)
+    - [Send the data to a subgraph.]((./../consume/subgraph/subgraph.md))
+
+**If you can't find a Substreams package that fits your needs**, then you can go ahead and develop your own Substreams. The [Develop Substreams](../develop/develop.md) section of the documentation covers everything you need to know about building a Substreams from scratch. At the most basic level, you should cover:
+
+- [Install the Substreams CLI](./installing-the-cli.md)
+- [Authentication](./authentication.md)
+<!-- - [Initialize a project (Quickstart)](./../develop/init-project.md) -->
+- [Manifest & Modules](./../common/manifest-modules.md)
+- [Protobuf defitions](./../develop/creating-protobuf-schemas.md)
+- [Packages](./packages.md)
+- [Run a Substreams](./running-substreams.md)
+- [Choose how you want to consume the data](./../consume/consume.md)
+
+## Tutorials
+
+If you want to deep dive into the code, you can follow one or several of the Solana Tutorials available in the documentation. A good way to start is to complete the [Explore Solana Tutorial](../tutorials/solana/explore-solana/explore-solana.md).
+
+## The Solana Data Model
+
+A Substreams module is, essentially, a Rust function that extracts data from the blockchain. In order to specify which data you want to retrieve, Substreams gives you access to the abstraction of a full [Solana block](https://github.com/streamingfast/firehose-solana/blob/develop/proto/sf/solana/type/v1/type.proto#L9).
+
+In the following example, a Solana block (`solana::Block`) is passed as a parameter. Then, a custom object defined by the user, `BlockMeta`, is emitted as output, containing some relevant fields (`slot`, `hash`, `parent_hash`):
+
+```rust
+fn map_block_meta(block: solana::Block) -> Result<BlockMeta, substreams::errors::Error> {
+    Ok(BlockMeta {
+        slot: blk.slot,
+        hash: blk.blockhash,
+        parent_hash: blk.previous_blockhash,
+    })
+}
+```
+
+The [Block](https://github.com/streamingfast/firehose-solana/blob/develop/proto/sf/solana/type/v1/type.proto#L9) object holds other important data, such as:
+- [block.transactions_owned()](https://github.com/streamingfast/substreams-solana/blob/1f66cc3081f61ad1189dc814cb82096ae5ac4b3b/core/src/block_view.rs#L15)
+- [block.rewards](https://github.com/streamingfast/firehose-solana/blob/develop/proto/sf/solana/type/v1/type.proto#L64)
+- [block.parent_slot](https://github.com/streamingfast/firehose-solana/blob/develop/proto/sf/solana/type/v1/type.proto#L12)
+
+## The Account Address Tables
+
+In Solana, the _account_ concept plays a very important role. However, the original Solana data model imposes a restriction on the number of accounts that a single transaction can have. The [Account Address Tables](https://docs.solana.com/developing/lookup-tables) is a way to overcome this restriction, allowing developers to increase the number of accounts per transaction.
+
+The [resolved_accounts()](https://github.com/streamingfast/substreams-solana/blob/1f66cc3081f61ad1189dc814cb82096ae5ac4b3b/core/src/lib.rs#L9) method of the [ConfirmedTransaction](https://github.com/streamingfast/substreams-solana/blob/1f66cc3081f61ad1189dc814cb82096ae5ac4b3b/core/src/lib.rs#L8) object includes a ready-to-use array of accounts, including accounts from the Lookup Table.

docs/new/tutorials/evm/exploring-ethereum/exploring-ethereum.md

@@ -17,7 +17,7 @@ $ git clone https://github.com/streamingfast/substreams-explorers
 
 ### The Substreams CLI
 
-The Substreams CLI allows you to run, package, and visualize your Substreams. Make sure you have the CLI installed by following this [simple tutorial](/getting-started/installing-the-cli.md).
+The Substreams CLI allows you to run, package, and visualize your Substreams. Make sure you have the CLI installed by following this [simple tutorial](../../../common/installing-the-cli.md).
 
 ### Substreams Basics
 
@@ -59,7 +59,7 @@ In the following sections, you will go through every module, run the correspondi
 
 ### The Project Structure
 
-<figure><img src="../../.gitbook/assets/eth-explorer-structure.png" /><figcaption><p>Ethereum Explorer Project Structure</p></figcaption></figure>
+<figure><img src="../../../../.gitbook/assets/tutorials/eth-explorer-structure.png" /><figcaption><p>Ethereum Explorer Project Structure</p></figcaption></figure>
 
 1. The `proto` folder contains the Protobuf definitions for the transformations.
 In this example, there are three Protobuf objects, which are the outputs of the Substreams module mentioned in the previous section: BlockMeta (which represents the information of an Ethereum block), Transaction (which is an abstraction for an Ethereum transaction), and Event (an abstraction for an Ethereum event).
@@ -70,7 +70,7 @@ In this example, there are three Protobuf objects, which are the outputs of the
 
 Let's take a closer look at the Substreams manifest (`substreams.yml`):
 
-<figure><img src="../../.gitbook/assets/eth-explorer-manifest.png" width="100%" /><figcaption><p>Ethereum Explorer Manifest</p></figcaption></figure>
+<figure><img src="../../../../.gitbook/assets/tutorials/eth-explorer-manifest.png" width="100%" /><figcaption><p>Ethereum Explorer Manifest</p></figcaption></figure>
 
 1. The `protobuf` section specifies the location of the Protobuf files used in the Substreams (i.e. where are the files defining the objects that you are going to use as output). In this example, the files are under the `proto` folder.
 2. When you run Substreams, you are really executing a Rust application inside a WASM container. Therefore, Substreams needs to know where is the WASM executable. The `binaries` section specifies the location of the WASM executable.

docs/new/tutorials/solana/explore-solana/explore-solana.md

@@ -0,0 +1,30 @@
+In this tutorial, you will learn the basics of developing Solana Substreams through examples. The Solana Substreams Explorer contains several modules performing basic and advanced aggregations on Solana data.
+
+## Before You Begin
+
+Before you start coding, there are several dependencies you must install on your computer.
+
+### The GitHub Repository
+The `https://github.com/streamingfast/substreams-explorers` GitHub repository contains all the Substreams Explorers currently available. You can simply clone the repository:
+
+```
+$ git clone https://github.com/streamingfast/substreams-explorers
+```
+
+### The Substreams CLI
+
+The Substreams CLI allows you to run, package, and visualize your Substreams. Make sure you have the CLI installed by following this [simple tutorial](../../../common/installing-the-cli.md).
+
+### Substreams Basics
+
+You should be familiar with the basic Substreams terminology, which includes:
+- Manifest & Modules (understanding the difference between a `map` and a `store` module)
+- Protobufs
+- Packages
+
+Take a look at the _Develop Substreams_ section for more information on how to start developing Substreams.
+
+## The Solana Explorer
+
+The Solana explorer includes several modules showcasing what Solana data you can extract with Substreams (it's easy and fast!). In the following sections, you will find out about the different functions you can use to easily get started with Solana.
+

docs/new/tutorials/solana/explore-solana/filter-instructions.md

@@ -0,0 +1,95 @@
+The `map_filter_instructions` module of the Solana Substreams Explorer extracts instruction of a given Program ID. For example, consider that you want to extract all the `Stake11111111111111111111111111111111111111` instructions.
+
+## Run the Substreams
+
+### Run From Source Code
+
+In the `substreams-explorer` project, move to the `solana-explorer` folder, which contains the source of the Solana Substreams. Then, build the Rust code:
+
+```bash
+make build
+```
+
+Now, you can run the Substreams by using the `substreams gui` command. To avoid iterating over the whole blockchain, the following command extracts instructions from the Stake Program only at block `243830383`:
+
+```bash
+substreams gui ./substreams.yaml \
+    map_filter_instructions -e mainnet.sol.streamingfast.io:443 \
+    --start-block 243830383 --stop-block +1
+```
+
+In the `Output` screen of the GUI, you can see two `Stake11111111111111111111111111111111111111` instructions were retrieved at block `243830383`:
+
+<figure><img src="../../../../.gitbook/assets/tutorials/solana-filter-instructions-output.png" width="100%" /></figure>
+
+The `map_filter_instructions` allows you to filter any Program ID, and this is configurable as a parameter in the Substreams Manifest (`substreams.yaml`):
+
+```yaml
+params:
+  map_filter_instructions: "program_id=Stake11111111111111111111111111111111111111"
+```
+
+You can replace `Stake11111111111111111111111111111111111111` by any instruction of your choice.
+
+### Run the Package From the Substreams Registry
+
+The Solana Explorer package is also available on the [Substreams Registry](https://substreams.dev). You can run it by using the following command, achieving the same result:
+
+```bash
+substreams gui https://spkg.io/streamingfast/solana-explorer-v0.2.0.spkg \
+    map_filter_instructions -e mainnet.sol.streamingfast.io:443 \
+    --start-block 243830383 --stop-block +1
+```
+
+## Inspect the Code
+
+The `map_filter_instructions.rs` file contains the source of the module. The output of the Substreams module is the `Instructions` object, which is defined in the `/proto/transactions.proto` file of the project. This is a custom object defined by the user, and you can modify at your will.
+
+```protobuf
+message Instructions {
+  repeated Instruction instructions = 1;
+}
+
+message Instruction {
+  string program_id = 1;
+  repeated string accounts = 2;
+  string data = 3;
+}
+```
+
+Let's inspect the module function:
+
+```rust
+#[substreams::handlers::map]
+fn map_filter_instructions(params: String, blk: Block) -> Result<Instructions, Vec<substreams::errors::Error>> {
+    let filters = parse_filters_from_params(params)?; // 1.
+
+    let mut instructions : Vec<Instruction> = Vec::new();
+
+    blk.transactions_owned().into_iter().for_each(|tx| { // 2.
+        let msg = tx.transaction.clone().unwrap().message.unwrap(); // 3.
+        let acct_keys = tx.resolved_accounts();  // 4.
+        let insts : Vec<Instruction> = msg.instructions.iter() // 5.
+            .filter(|inst| apply_filter(inst, &filters, &acct_keys)) // 6.
+            .map(|inst| {
+            Instruction { // 7.
+                program_id: bs58::encode(acct_keys[inst.program_id_index as usize].to_vec()).into_string(),
+                accounts: inst.accounts.iter().map(|acct| bs58::encode(acct_keys[*acct as usize].to_vec()).into_string()).collect(),
+                data: bs58::encode(inst.data.clone()).into_string(),
+            }
+        }).collect();
+        instructions.extend(insts);
+    });
+
+    Ok(Instructions { instructions })
+}
+```
+1. The `parse_filters_from_params` function parses the parameters passed to the module.
+In this example, the parameter passed is defined in the `substreams.yaml` file as `program_id=Stake11111111111111111111111111111111111111`.
+2. Iterate over the transactions of the blocks.
+3. Extract the [Message](https://github.com/streamingfast/firehose-solana/blob/develop/proto/sf/solana/type/v1/type.proto#L32) object, which contains relevant information, such as the instructions of the trasaction.
+4. Get accounts of the transaction (the `resolved_accounts()` method contains also accounts stored in the [Address Lookup Tables](https://docs.solana.com/developing/lookup-tables)).
+5. Iterave over the instructions.
+6. Use the `apply_filter` function to only keep instruction where `program_id=Stake11111111111111111111111111111111111111`.
+7. Create an `Instruction` object, which will be the output of the Substreams.
+This object is declared as a Protobuf in the `proto` folder of the project.