Skip to content

Commit

Permalink
subgraph migration (#515)
Browse files Browse the repository at this point in the history
* subgraph migration

* Subgraph migration
  • Loading branch information
jamesbayly authored May 8, 2024
1 parent 2528b1e commit 7dfa05d
Show file tree
Hide file tree
Showing 3 changed files with 43 additions and 20 deletions.
Binary file added docs/.vuepress/public/assets/img/build/graph.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
63 changes: 43 additions & 20 deletions docs/indexer/build/graph-migration.md
Original file line number Diff line number Diff line change
@@ -1,5 +1,7 @@
# The Graph Migration

![Graph Migration](/assets/img/build/graph.png)

SubQuery provides a superior developer experience to The Graph, while maintaining a similar development workflow that makes migration quick and easy. Migrating to SubQuery brings the following benefits:

- **Performance improvements** - SubQuery is up to 3.9x faster for common projects over The Graph (e.g the standard [Ethereum POAP subgraph](https://github.com/poap-xyz/poap-subgraph)) thanks to multi-threading and a store cache layer
Expand All @@ -11,7 +13,7 @@ SubQuery provides a superior developer experience to The Graph, while maintainin
- **Managed Service hosting** - We have no plans to sunset our [Managed Service](https://managedservice.subquery.network), which provides enterprise-level infrastructure hosting and handles over hundreds of millions of requests each day
- **A decentralised network supporting all chains** - Our [decentralised network](https://app.subquery.network) supports all chains that SubQuery support, there is no _second-class_ chain support in the SubQuery ecosystem

![Competitor Comparison](/assets/img/build/competitor_comparison.jpg)
![Competitor Comparison](/assets/img/build/graph_comparison.jpg)

The migration from a SubGraph to a SubQuery Project is (by design) easy and quick. It may take you an hour or two to complete the migration, depending on the complexity of your SubGraph.

Expand All @@ -25,27 +27,48 @@ Reach out to our team at [[email protected]](mailto:professi

## Migration Overview

SubQuery makes it easy to migrate your existing SubGraph to SubQuery in the shortest time possible, we even have an automated tool support you with migrating most of your project's boilerplate code. At a high level:

- Both SubGraph and SubQuery use the same `schema.graphql` file to define schema entities. In addition, both have similar sets of supported scalars and entity relationships (SubQuery adds support for JSON types though).
- The manifest file shows the most differences but you can easily overcome these differences once you understand them.
- In addition, Mapping files are also quite similar with an intentionally equivalent set of commands, which are used to access the Graph Node store and the SubQuery Project store.

### Recommended Migration Steps

This is the recommended process that we use at SubQuery whenever we migrate projects from a SubGraph to SubQuery:
This is the recommended process that we use at SubQuery whenever we migrate projects from a SubGraph to SubQuery.

Firstly, use the SubQuery CLI tool to migrate an existing SubGraph by using the `subql migrate` command. This will:

- Intialise a new SubQuery project in your chosen directory for the matching target chain
- Copy over basic information like project name and other metadata
- Enable ts strict mode to assist you in identifying potential issues when migrating your project.
- Copy over the existing `schema.graphql` to save you time (you will need to edit this)
- Copy over the existing `project.ts` mapping to save you time (you will need to edit this)

An example of using this tool for the Graph Gravatar project is as follows

- `-f` should point to a local project, or a subgraph on GitHub (if so include `/tree/<branchname>`)
- `-d` is optional and used when the GitHub project is in a subdirectory
- `-o` is the output directory

```sh
subql migrate -f https://github.com/graphprotocol/graph-tooling/tree/main -d examples/ethereum-gravatar -o ~/subquery/gravatar-starter
```

Once this is done, follow along and complete the remaining steps:

1. [Initialise a new SubQuery project](./introduction.md) for the same network using the `subql init` command. When migrating an existing SubGraph, it's not necessary to perform code scaffolding. It also ensures that you are using TS `strict` mode, which will help you identify any potential bugs.
2. Copy over your `schema.graphql` and replace any `Bytes` and `BigDecimals`. [More info](#graphql-schema).
3. Copy over relevant abi contracts to the `abis` directory and update the `project.manifest`. [More info](#manifest-file).
4. Migrate your data sources in the `project.manifest`, specifically the `handlers` (retain the same handler names). [More info](#manifest-file).
5. Perform code generation using the `yarn codegen`, this will generate GraphQL entity types, and generate types from ABIs. [More info](#codegen).
6. Copy over the `mappings` directory, and then go through one by one to migrate them across. The key differences:
1. Review and edit the copied `schema.graphql` and replace any `Bytes` and `BigDecimals`. [More info](#graphql-schema).
2. Copy over relevant abi contracts to the `abis` directory and update the `project.manifest`. [More info](#manifest-file).
3. Review and edit the copied data sources in the `project.ts` manifest, specifically the `handlers` (retain the same handler names). [More info](#manifest-file).
4. Perform code generation using the `yarn codegen`, this will generate GraphQL entity types, and generate types from ABIs. [More info](#codegen).
5. Copy over the `mappings` directory, and then go through one by one to migrate them across. The key differences:
- Imports will need to be updated
- Store operations are asynchronous, e.g. `<entityName>.load(id)` should be replaced by `await <entityName>.get(id)` and `<entityName>.save()` to `await <entityName>.save()` (note the `await`).
- With strict mode, you must construct new entities with all the required properties. You may want to replace `new <entityName>(id)` with `<entityName>.create({ ... })`
- [More info](#mapping).
7. Test and update your clients to follow the GraphQL api differences and take advantage of additional features. [More info](#graphql-query-differences)
6. Test and update your clients to follow the GraphQL api differences and take advantage of additional features. [More info](#graphql-query-differences)

## GraphQL Schema
### Differences in the GraphQL Schema

Both SubGraphs and SubQuery projects use the same `schema.graphql` to define entities and includes both similar [scalar types](./graphql.md#supported-scalar-types) as well as [full text search](./graphql.md#full-text-search).

Expand All @@ -57,7 +80,7 @@ Notable differences include:
- SubQuery has the additional scalar types of `Float`, `Date`, and `JSON` (see [JSON type](./graphql.md#json-type)).
- Comments are added to SubQuery Project GraphQL files using hashes (`#`).

## Manifest File
### Differences in the Manifest File

The manifest file contains the largest set of differences, but once you understand those they can be easily overcome. Most of these changes are due to the layout of this file, you can see the [full documentation of this file here](./manifest/ethereum.md).

Expand Down Expand Up @@ -227,7 +250,7 @@ dataSources:
:::
## Codegen
### Differences in Codegen
The `codegen` command is also intentionally similar between SubQuery and SubGraphs

Expand All @@ -246,7 +269,7 @@ import {
} from "../types/abi-interfaces/Gravity";
```

## Mapping
### Differences in Mapping Functions

Mapping files are also quite identical to an intentionally equivalent set of commands, which are used to access the Graph Node store and the SubQuery Project store.

Expand Down Expand Up @@ -329,7 +352,7 @@ export async function handleUpdatedGravatar(

:::

## Querying Contracts
### Differences in Querying Contracts

We globally provide an `api` object that implements an [Ethers.js Provider](https://docs.ethers.io/v5/api/providers/provider/). This will allow querying contract state at the current block height being indexed. The easiest way to use the `api` is with [Typechain](https://github.com/dethcrypto/TypeChain), with this you can generate typescript interfaces that are compatible with this `api` that make it much easier to query your contracts.

Expand All @@ -346,11 +369,11 @@ const balance = await erc20.balanceOf(address);

The above example assumes that the user has an ABI file named `erc20.json`, so that TypeChain generates `ERC20__factory` class for them. Check out [this example](https://github.com/dethcrypto/TypeChain/tree/master/examples/ethers-v5) to see how to generate factory code around your contract ABI using TypeChain

## GraphQL Query Differences
### Differences in the GraphQL Query Interface

There are minor differences between the default GraphQL query service for SubQuery, and that of the Graph.

### Query format
#### Query format

Note the additional nesting of entity properties under the `nodes` syntax.

Expand Down Expand Up @@ -382,7 +405,7 @@ Note the additional nesting of entity properties under the `nodes` syntax.

:::

### Filters
#### Filters

Instead of `where`, SubQuery uses `filter`. The Graph also uses a list of suffixes on the end of the entity field for filtering, when using SubQuery, you will need to add an extra layer on the GraphQL to specify the operator for the filters.

Expand Down Expand Up @@ -414,7 +437,7 @@ Instead of `where`, SubQuery uses `filter`. The Graph also uses a list of suffix

:::

### Sorting
#### Sorting

Instead of using `orderBy` and `orderDirection`, SubQuery creates directional values for the `orderBy` property (e.g. `CREATED_AT_ASC` and `CREATED_AT_DSC`)

Expand Down Expand Up @@ -446,7 +469,7 @@ Instead of using `orderBy` and `orderDirection`, SubQuery creates directional va

:::

### Historical queries
#### Historical queries

There is no difference when querying [historical data](../run_publish/historical.md).

Expand Down Expand Up @@ -478,7 +501,7 @@ There is no difference when querying [historical data](../run_publish/historical

:::

### Metadata
#### Metadata

SubQuery does not support historical metadata querying. However `deployments` will still show the deployments with their heights and other key metrics

Expand Down

0 comments on commit 7dfa05d

Please sign in to comment.