Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[WIP] Add b-sdk for trade queries #333

Merged
merged 91 commits into from
Dec 8, 2023
Merged
Show file tree
Hide file tree
Changes from 4 commits
Commits
Show all changes
91 commits
Select commit Hold shift + click to select a range
d90b321
Basic addition of b-sdk.
johngrantuk May 3, 2023
8f62a6c
Uses pools from Prisma db.
johngrantuk May 8, 2023
298eba2
Add TradeResults Prisma for recording results.
johngrantuk May 8, 2023
56b4a8d
Update SOR service to have placeholders for compare, etc.
johngrantuk May 8, 2023
1de2adc
Add placeholder for SORV1 service call to make more obvious.
johngrantuk May 8, 2023
3101589
Move SOR_QUERIES in to temp file as only used there and can potential…
johngrantuk May 9, 2023
3884a90
Update Phantom/Composable pool comments.
johngrantuk May 9, 2023
4e539c0
Add pool ids to ignore in to network config.
johngrantuk May 9, 2023
8b2ae68
WIP SORv1 API query.
johngrantuk May 10, 2023
92d269c
Use cache for pools.
johngrantuk May 10, 2023
1f9bcff
Query API (Currently Balancers) for SORV1.
johngrantuk May 19, 2023
1725a25
Add basic compare, WIP.
johngrantuk May 19, 2023
5c3b6aa
Remove Prisma trade result in favour of cloudwatch implementation.
johngrantuk May 19, 2023
243d855
Find best and log result via cloudwatch.
johngrantuk May 19, 2023
5cc1667
Change response to be in API format. Use generated types.
johngrantuk May 19, 2023
be6bcfa
Remove temp.
johngrantuk May 19, 2023
a01a9fb
Implement mapResultToCowSwap.
johngrantuk May 19, 2023
554a26e
Compare correctly for ExactOut.
johngrantuk May 19, 2023
388772e
Use cloneDeep for b-sdk as it mutates pools.
johngrantuk May 19, 2023
3578f01
Remove TODO on swapOptions as I believe we dont need it.
johngrantuk May 19, 2023
abdee5e
Merge branch 'v3-canary' into sor-v2
johngrantuk May 19, 2023
a0ec5c3
Remove old gql.
johngrantuk May 19, 2023
65f84ad
Refactor to common Swap result interface.
johngrantuk May 25, 2023
946a6ed
Update SDK package.
johngrantuk May 25, 2023
f14a203
Implement onchain query method for V2 service.
johngrantuk May 25, 2023
311b82d
Implement onchain query method for V1 service.
johngrantuk May 25, 2023
5d04f38
Handle empty responses.
johngrantuk May 30, 2023
633c674
Return correct empty response.
johngrantuk May 30, 2023
ac5cda0
Merge branch 'v3-canary' into sor-v2
johngrantuk May 30, 2023
fbc8553
Use prisma pool version.
johngrantuk May 30, 2023
67fce9a
Handle invalid swap case correctly.
johngrantuk May 30, 2023
a91db31
Filtering out Linear pools with 0 price rate which causes issues on b…
johngrantuk May 31, 2023
969e2ae
Merge branch 'v3-canary' into sor-v2
franzns Jun 2, 2023
c844d9b
Merge branch 'sor-v2' of github.com:beethovenxfi/beethovenx-backend i…
franzns Jun 2, 2023
b0b1409
Update logs with chainId. Fix prettier.
johngrantuk Jun 7, 2023
c5f1ed5
Initial restructure for Balancer/Beets separation.
johngrantuk Jun 7, 2023
87a6b70
Split resolvers.
johngrantuk Jun 7, 2023
1dafb8e
Add getBeetsSwapResponse type.
johngrantuk Jun 8, 2023
aa06361
Add reusable formatResponse function.
johngrantuk Jun 8, 2023
bdf7591
Add mapResultToBeetsSwap - still need to add routes.
johngrantuk Jun 8, 2023
78e46ad
Add sorV1Beets service and hook up to resolver.
johngrantuk Jun 8, 2023
c9f75cd
Best swap for beets endpoint. Zero response.
johngrantuk Jun 9, 2023
585bda3
Map Routes for SingleSwap.
johngrantuk Jun 9, 2023
bd036a2
Route mapping.
johngrantuk Jun 9, 2023
4e6a2ad
Route mapping for batchSwap with paths > 1.
johngrantuk Jun 9, 2023
be0516a
Wider catch in V2 service. (Catches b-sdk/Fantom issue so response is…
johngrantuk Jun 14, 2023
ad90592
Remove balancerQueryTest query.
johngrantuk Jun 14, 2023
2b7821a
Remove union type in sorGetSwaps query.
johngrantuk Jun 14, 2023
fefc348
fix schema exclusion
franzns Jun 14, 2023
16fdead
Return marketSp of 0 for beets.
johngrantuk Jun 20, 2023
0c2cf05
Merge branch 'sor-v2' of https://github.com/beethovenxfi/beethovenx-b…
johngrantuk Jun 20, 2023
915ae7b
Merge canary.
johngrantuk Aug 28, 2023
1221d7d
Revert commented out section in codegen.yml.
johngrantuk Aug 29, 2023
ae35e83
exclude all in protocol specific folder
franzns Aug 29, 2023
de6ea10
Revert "exclude all in protocol specific folder"
johngrantuk Sep 1, 2023
1688a41
Change to use 2 different query types for Beets/Cow swaps. Add missin…
johngrantuk Sep 1, 2023
fd52d50
fix: Filter out Linear pools. Filter out pools from vulnerability/mit…
johngrantuk Sep 4, 2023
061e524
chore: Remove unneccesary Migration folders.
johngrantuk Sep 4, 2023
2ea2442
chore: Update sdk to 0.1.1 and add missing RawPool fields.
johngrantuk Sep 4, 2023
aae9bd8
feat: Add poolGetGyroPools query.
johngrantuk Sep 5, 2023
5d6ade7
chore: Add new Gyro fields for SOR calcs.
johngrantuk Sep 5, 2023
048699d
Merge branch 'v3-canary' into sor-v2
johngrantuk Sep 7, 2023
6390fe4
chore: More Gyro fields for SOR.
johngrantuk Sep 7, 2023
abbc61e
feat: Add Gyro pools for sorV2 service.
johngrantuk Sep 7, 2023
08767d1
Merge branch 'v3-canary' into sor-v2
johngrantuk Sep 7, 2023
a3e4a10
bump typescript version
franzns Sep 15, 2023
a22b7dc
Merge branch 'v3-canary' into sor-v2
franzns Sep 18, 2023
9aef5d0
Merge branch 'v3-canary' into sor-v2
franzns Sep 18, 2023
c7d44b7
formatting and output
franzns Sep 18, 2023
ca4865d
adding cloudwatch metrics
franzns Sep 18, 2023
0aca520
logging
franzns Sep 18, 2023
caca5e3
chore: Bump b-sdk.
johngrantuk Oct 12, 2023
026bf85
chore: Add Gyro3 to mapRawPoolType.
johngrantuk Oct 13, 2023
33e39d5
fix: Handle amount scaling correctly across various services by using…
johngrantuk Oct 13, 2023
d5f7af5
chore: Remove debug statements.
johngrantuk Oct 13, 2023
54d90d2
fix: For Balancer resolver use raw scale for swapAmount.
johngrantuk Oct 17, 2023
b7e3bff
Merge branch 'v3-canary' into sor-v2
gmbronco Nov 13, 2023
d99f22f
chain aware SOR queries
gmbronco Nov 13, 2023
23aad1b
gyro pool data updates
gmbronco Nov 13, 2023
056c178
excluding unsupported pools
gmbronco Nov 14, 2023
14f2393
sor metrics
gmbronco Nov 14, 2023
920600c
sor bench logging
gmbronco Nov 15, 2023
d12e892
last review and styling
gmbronco Nov 21, 2023
f256592
updating b-sdk
gmbronco Nov 21, 2023
f7ac2dd
Merge branch 'v3-canary' into sor-v2
gmbronco Dec 6, 2023
3792147
prettier
gmbronco Dec 7, 2023
c62793e
V2 options
gmbronco Dec 7, 2023
22abdd7
Merge branch 'v3-canary' into sor-v2
gmbronco Dec 7, 2023
61cec2b
node18 in github action
gmbronco Dec 7, 2023
b50d476
sor args cleaup
gmbronco Dec 7, 2023
0aa8c08
less profiling
gmbronco Dec 7, 2023
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
33 changes: 33 additions & 0 deletions modules/sor/sor.gql
Original file line number Diff line number Diff line change
@@ -0,0 +1,33 @@
extend type Query {
sorGetSwapsNew(
johngrantuk marked this conversation as resolved.
Show resolved Hide resolved
tokenIn: String!
tokenOut: String!
swapType: GqlSorSwapType!
swapAmount: BigDecimal! #expected in raw amount
): GqlSorGetSwapsResponseNew!
tradeResults: [GqlTradeResults]!
}

enum GqlSorSwapType {
EXACT_IN
EXACT_OUT
}

type GqlSorGetSwapsResponseNew {
tokenIn: String!
tokenOut: String!
result: String!
}

type GqlTradeResults {
id: String!
timestamp: Int!
chain: String!
tokenIn: String!
tokenOut: String!
swapAmount: String!
swapType: String!
sorV1Result: String!
sorV2Result: String!
isSorV1: Boolean!
}
14 changes: 14 additions & 0 deletions modules/sor/sor.prisma
Original file line number Diff line number Diff line change
@@ -0,0 +1,14 @@
model PrismaTradeResult {
johngrantuk marked this conversation as resolved.
Show resolved Hide resolved
@@id([id, chain])

id String
timestamp Int
chain Chain
tokenIn String
tokenOut String
swapAmount String
swapType String
sorV1Result String
sorV2Result String
isSorV1 Boolean
}
17 changes: 17 additions & 0 deletions modules/sor/sor.resolvers.ts
Original file line number Diff line number Diff line change
@@ -0,0 +1,17 @@
import { Resolvers } from '../../schema';
import { sorService as sorService } from './sor.service';
import { getTrades, TradeResults } from './trade.service';
import { tokenService } from '../token/token.service';

const sorResolvers: Resolvers = {
johngrantuk marked this conversation as resolved.
Show resolved Hide resolved
Query: {
sorGetSwapsNew: async (parent, args, context) => {
return sorService.getSwaps({ ...args });
},
tradeResults: async () => {
return await getTrades();
}
},
};

export default sorResolvers;
60 changes: 60 additions & 0 deletions modules/sor/sor.service.ts
Original file line number Diff line number Diff line change
@@ -0,0 +1,60 @@
import { GqlSorGetSwapsResponseNew, GqlSorSwapType } from '../../schema';
import { networkContext } from '../network/network-context.service';
import { sorV2Service } from './sorV2/sorV2.service';
import { prisma } from '../../prisma/prisma-client';

export interface GetSwapsInput {
tokenIn: string;
tokenOut: string;
swapType: GqlSorSwapType;
swapAmount: string;
}

export class SorService {
public async getSwaps({
tokenIn,
tokenOut,
swapType,
swapAmount,
}: GetSwapsInput): Promise<GqlSorGetSwapsResponseNew> {
const timestamp = Math.floor(Date.now() / 1000);

// TODO - SORV1 result - via API call to current API or using SORV1/pools directly?
const sorV1Result = 'TODO';

const sorV2Result = await sorV2Service.getSwaps({
tokenIn,
tokenOut,
swapType,
swapAmount,
});

johngrantuk marked this conversation as resolved.
Show resolved Hide resolved
let isSorV1 = false;
// TODO - Compare V1 vs V2 result and return/log best

// Update db with best result so we can track performace
await prisma.prismaTradeResult.create({
johngrantuk marked this conversation as resolved.
Show resolved Hide resolved
data: {
id: `${timestamp}-${tokenIn}-${tokenOut}`,
timestamp,
chain: networkContext.chain,
tokenIn,
tokenOut,
swapAmount,
swapType,
sorV1Result,
sorV2Result: sorV2Result.result,
isSorV1
}
});

// TODO - Return in current CowSwap format so its plug and play
return {
tokenIn,
tokenOut,
result: isSorV1 ? sorV1Result : sorV2Service.mapResultToCowSwap(sorV2Result.result)
}
johngrantuk marked this conversation as resolved.
Show resolved Hide resolved
}
}

export const sorService = new SorService();
1 change: 1 addition & 0 deletions modules/sor/sorV1/sorV1.service.ts
Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
// This will hold logic for querying SORV1 trade. Either via call to existing API or SORV1 package directly.
johngrantuk marked this conversation as resolved.
Show resolved Hide resolved
243 changes: 243 additions & 0 deletions modules/sor/sorV2/sorV2.service.ts
Original file line number Diff line number Diff line change
@@ -0,0 +1,243 @@
import {
BasePool,
ChainId,
johngrantuk marked this conversation as resolved.
Show resolved Hide resolved
sorGetSwapsWithPools,
Token,
Address,
SwapKind,
sorParseRawPools,
RawStablePool,
RawLinearPool,
RawWeightedPool,
RawComposableStablePool,
RawMetaStablePool,
Swap
} from '@balancer/sdk';
import { GqlSorGetSwapsResponseNew, GqlSorSwapType } from '../../../schema';
import { PrismaPoolType, PrismaToken } from '@prisma/client';
import { GetSwapsInput } from '../sor.service';
import { tokenService } from '../../token/token.service';
import { networkContext } from '../../network/network-context.service';
import { prisma } from '../../../prisma/prisma-client';
import { PrismaPoolWithDynamic, prismaPoolWithDynamic } from '../../../prisma/prisma-types';
import { RawPool } from '@balancer/sdk';
import { HumanAmount, SupportedRawPoolTypes } from '@balancer/sdk';

import { getSwapCompare } from './temp';

// TODO - Check if this has been deployed to same address across networks?
export const SOR_QUERIES = '0x1814a3b3e4362caf4eb54cd85b82d39bd7b34e41';
johngrantuk marked this conversation as resolved.
Show resolved Hide resolved

export class SorV2Service {
public async getSwaps({
tokenIn,
tokenOut,
swapType,
swapAmount,
}: GetSwapsInput): Promise<GqlSorGetSwapsResponseNew> {
// TODO - This takes ~1.262s on my local machine. Is this fast enough? Any obvious ways to improve?
console.time('poolsFromDb');
const poolsFromDb = await this.getBasePoolsFromDb();
johngrantuk marked this conversation as resolved.
Show resolved Hide resolved
console.timeEnd('poolsFromDb');
const chainId = networkContext.chainId as unknown as ChainId;
const tIn = await this.getToken(tokenIn as Address, chainId);
const tOut = await this.getToken(tokenOut as Address, chainId);
const swap = await sorGetSwapsWithPools(
tIn,
tOut,
this.mapSwapType(swapType),
swapAmount,
poolsFromDb,
// swapOptions, // TODO - Handle properly
johngrantuk marked this conversation as resolved.
Show resolved Hide resolved
);

if (!swap) throw new Error('Swap is undefined');
console.log(`Swap (db pools)`);
console.log(poolsFromDb.length);
console.log(swap.swaps);
console.log(swap.outputAmount.amount.toString());
console.log(`------------------`);

// Just using to compare results against onchain pool calls
// await getSwapCompare(tIn, tOut, swapAmount);

// TODO - Update with proper required result data
return {
tokenIn,
tokenOut,
result: swap.outputAmount.amount.toString()
}
}

public mapResultToCowSwap(swap: string): string {
// TODO - match existing CowSwap SOR API format so its plug and play
return swap;
}

/**
* Gets a b-sdk Token based off tokenAddr.
* @param tokenAddr
* @param chainId
* @returns
*/
private async getToken(tokenAddr: Address, chainId: ChainId): Promise<Token> {
const tokens = await tokenService.getTokens();
const prismaToken = this.getPrismaToken(tokenAddr, tokens);
return new Token(
chainId,
tokenAddr,
prismaToken.decimals,
prismaToken.symbol,
);
}

private getPrismaToken(tokenAddress: string, tokens: PrismaToken[]): PrismaToken {
tokenAddress = tokenAddress.toLowerCase();
const match = tokens.find((token) => token.address === tokenAddress);

if (!match) {
throw new Error('Unknown token: ' + tokenAddress);
}
return match;
}

private mapSwapType(swapType: GqlSorSwapType): SwapKind {
return swapType === "EXACT_IN" ? SwapKind.GivenIn : SwapKind.GivenOut;
}

/**
* Fetch pools from Prisma and map to b-sdk BasePool.
* @returns
*/
private async getBasePoolsFromDb(): Promise<BasePool[]> {
const pools = await prisma.prismaPool.findMany({
where: {
chain: networkContext.chain,
dynamicData: {
totalSharesNum: {
gt: 0.000000000001,
},
swapEnabled: true,
},
},
include: prismaPoolWithDynamic.include
});
const rawPools = this.mapToRawPools(pools);
// TODO - The pools below cause issue with b-sdk maths. Both have a token with balance = 0 so maybe that's issue?
const linearPoolsToIgnore = [
"0xbfa413a2ff0f20456d57b643746133f54bfe0cd20000000000000000000004c3",
"0xdc063deafce952160ec112fa382ac206305657e60000000000000000000004c4",
]
johngrantuk marked this conversation as resolved.
Show resolved Hide resolved

const basePools = this.mapToBasePools(rawPools.filter(p => {
if (p.poolType == 'Linear') {
return !linearPoolsToIgnore.includes(p.id);
} else return true;
}));
// const basePools = this.mapToBasePools(rawPools);
return basePools;
}

/**
* Map Prisma pools to b-sdk RawPool.
* @param pools
* @returns
*/
private mapToRawPools(pools: PrismaPoolWithDynamic[]): RawPool[] {
return pools.map(prismaPool => {
// b-sdk: src/data/types.ts
let rawPool: RawPool = {
id: prismaPool.id as Address,
address: prismaPool.address as Address,
poolType: this.mapRawPoolType(prismaPool.type),
poolTypeVersion: 1, // TODO - Can we add this to Prisma??
johngrantuk marked this conversation as resolved.
Show resolved Hide resolved
tokensList: prismaPool.tokens.map(t => t.address as Address),
swapEnabled: prismaPool.dynamicData!.swapEnabled,
swapFee: prismaPool.dynamicData!.swapFee as unknown as HumanAmount,
totalShares: prismaPool.dynamicData!.totalShares as unknown as HumanAmount,
liquidity: prismaPool.dynamicData!.totalLiquidity as unknown as HumanAmount,
tokens: prismaPool.tokens.map(t => {
return {
address: t.token.address as Address,
index: t.index,
symbol: t.token.symbol,
name: t.token.name,
decimals: t.token.decimals,
balance: t.dynamicData?.balance as unknown as HumanAmount
}
}),
};
if(["Weighted", "Investment", "LiquidityBootstrapping"].includes(rawPool.poolType)) {
rawPool = {
...rawPool,
tokens: rawPool.tokens.map((t, i) => { return {...t, weight: prismaPool.tokens[i].dynamicData?.weight} }),
} as RawWeightedPool;
}
if(rawPool.poolType === "Stable") {
rawPool = {
...rawPool,
amp: prismaPool.stableDynamicData?.amp,
} as RawStablePool;
}
if(["MetaStable", "ComposableStable"].includes(rawPool.poolType)) {
rawPool = {
...rawPool,
amp: prismaPool.stableDynamicData?.amp.split('.')[0], // Taken b-sdk onChainPoolDataEnricher.ts
tokens: rawPool.tokens.map((t, i) => { return {...t, priceRate: prismaPool.tokens[i].dynamicData?.priceRate} }),
} as RawMetaStablePool;
}
if(rawPool.poolType === "Linear") {
rawPool = {
...rawPool,
mainIndex: prismaPool.linearData?.mainIndex,
wrappedIndex: prismaPool.linearData?.wrappedIndex,
lowerTarget: prismaPool.linearDynamicData?.lowerTarget,
upperTarget: prismaPool.linearDynamicData?.upperTarget,
tokens: rawPool.tokens.map((t, i) => { return {...t, priceRate: prismaPool.tokens[i].dynamicData?.priceRate} }),
} as RawLinearPool;
}
return rawPool;
});
}

/**
* Map b-sdk RawPools to BasePools.
* @param pools
* @returns
*/
private mapToBasePools(pools: RawPool[]): BasePool[] {
const chainId = networkContext.chainId as unknown as ChainId;
return sorParseRawPools(chainId, pools);
}

/**
* Map Prisma pool type to b-sdk Raw pool type.
* @param type
* @returns
*/
private mapRawPoolType(type: PrismaPoolType): SupportedRawPoolTypes | string {
// From b-sdk:
// - type LinearPoolType = `${string}Linear`;
// - LinearPoolType | 'Weighted' | 'Investment' | 'LiquidityBootstrapping' | 'Stable' | 'MetaStable' | 'ComposableStable' | 'StablePhantom' | 'Element';
switch(type) {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I guess this also needs gyro type?

case PrismaPoolType.WEIGHTED:
return 'Weighted';
case PrismaPoolType.INVESTMENT:
return 'Investment';
case PrismaPoolType.LIQUIDITY_BOOTSTRAPPING:
return 'LiquidityBootstrapping';
case PrismaPoolType.STABLE:
return 'Stable'; // TODO - Is there a ComposableStable/Stable differentiation in Prisma?
johngrantuk marked this conversation as resolved.
Show resolved Hide resolved
case PrismaPoolType.META_STABLE:
return 'MetaStable';
case PrismaPoolType.PHANTOM_STABLE:
return 'ComposableStable'; // b-sdk just treats these as ComposableStable
case PrismaPoolType.LINEAR:
return 'Linear';
default:
return type;
}
}
}

export const sorV2Service = new SorV2Service();
Loading