APR Services - Add some yield tokens APR to linear pools #396

lgahdl · 2023-07-03T13:04:36Z

Adding APR Service to fetch token yields APR on https://yield-tokens.balancer.workers.dev/ and calculate the APR for boosted pools with nested pools that contain the fetched tokens.

…-tokens-apr

gmbronco · 2023-07-06T10:31:21Z

What are "Non Reaper Boosted pools"?

lgahdl · 2023-07-06T18:14:48Z

What are "Non Reaper Boosted pools"?

I thought it would be necessary to update boosted pools in the same APR Service that updates the linear pools with "non reaper tokens" yield (Which John called Non Reaper Boosted Pools, because there's another service that updates the APR for pools with wrapped tokens from Reaper Crypt) but there's already a service that updates all boosted pools, so it's not necessary to update the "Non Reaper Boosted Pools", because they're already being updated by the "boosted-pool.apr.service"

… instead of use the yield-tokens cloudflare service; Removed the boosted pools apr, and let only the yield tokens APR update, boosted pools are updated in another service;

franzns · 2023-07-07T07:02:33Z

modules/pool/lib/apr-data-sources/ib-tokens-apr.service.ts

+              title: `${ token.token.symbol || token.address } APR`,
+              apr: aprs.get(token.address) ?? 0,
+              group: null,
+              type: 'IB_YIELD',


This needs to be different, based on the pool. If it's a wrapped asset from a linear pool, it needs to be LINEAR_BOOSTED, else IB_YIELD

franzns · 2023-07-07T07:04:27Z

modules/pool/lib/apr-data-sources/ib-tokens-apr.service.ts

+  private async fetchYieldTokensApr(): Promise<Map<string, number>> {
+    const res = await axios.get("https://yield-tokens.balancer.workers.dev/");
+    const aprs = new Map<string, number>(Object.entries(res.data as {
+      [key: string]: number
+    }).map(([key, apr]) => [key, apr * 0.0001]));
+    return aprs;
+  }


I assume this will change to be sourced locally from the sources you added?

…er-yield-tokens-apr Grouping APRs by source;

franzns · 2023-07-21T08:50:27Z

modules/token/lib/token-apr-handler/fetch-all.ts

+import { tokens } from "./tokens";
+import { TokenApr } from "./types";
+
+export const fetchAllAprs = async (): Promise<TokenApr[]> => {


Please create a proper function for this

franzns · 2023-07-21T08:55:47Z

modules/token/lib/token-apr-handler/sources/reaper.ts

+
+
+const jsonRpcProvider = new JsonRpcProvider('https://arb1.arbitrum.io/rpc', 42161 /*ARBITRUM*/)
+const provider = MulticallWrapper.wrap(jsonRpcProvider)


why is this multicall wrapper needed?

I used it so calls performed simultaneously can be automatically batched. Like in this case:

const getAprs = () => { const calls = Object.keys(strategiesMap).map(async (coin) => { const contract = new Contract(strategiesMap[coin], abi, provider) return contract.averageAPRAcrossLastNHarvests(3) }) return Promise.all(calls) }

on viem, it was this:

const getAprs = () => { const contracts = Object.keys(strategiesMap).map((coin) => ({ address: strategiesMap[coin], abi, functionName: 'averageAPRAcrossLastNHarvests', args: [3], })) return client.multicall({ contracts }) }

franzns · 2023-07-21T09:04:48Z

modules/token/lib/token-apr-handler/fetch-all.ts

+import { TokenApr } from "./types";
+
+export const fetchAllAprs = async (): Promise<TokenApr[]> => {
+  const res = await Promise.all(


any of these fetches can fail for various reasons. I'd rather do that sequential (no need to optimize in the cron) and log proper errors to sentry

franzns · 2023-07-21T09:11:01Z

modules/token/lib/token-apr-handler/fetch-all.ts

+    tokens.map(async ({ name, group, fetchFn }) => {
+      const fetchedResponse: {[key:string]:number} = await fetchFn()
+      return Object.entries(fetchedResponse).map(([address, aprValue]) => ({
+        val: aprValue * 0.0001 /*Values come in BPS 10000=>100% */,


maybe the apr services should already provide values in %

franzns · 2023-07-21T09:11:09Z

Looked through the thing now. We will need to redesign it a bit and make it multichain aware. Currently, some APR sources are multichain aware and others arent.
I would even redesign the whole thing, introducing interfaces for the APR sources, similar to what we did for the other APR services.
In addition, that thing should live in the pools module with the other APR code and not in the token module.

…rks, tokens and urls;

franzns · 2023-07-31T09:20:40Z

modules/pool/lib/apr-data-sources/ib-yield-apr-handlers/ib-yield-apr-handlers.ts

+  getHandlersAprs = async (): Promise<TokenApr[]> => {
+    const aprPromises = this.handlers.map(async (handler) => {
+      const fetchedResponse: { [key: string]: number } = await handler.getAprs()
+      return Object.entries(fetchedResponse).map(([address, aprValue]) => ({
+        val: aprValue,
+        group: handler.group,
+        address
+      }))
+    });
+    const res = Array(this.handlers.length)
+    for (const [index, aprPromise] of aprPromises.entries()) {
+      res[index] = await aprPromise
+    }
+    return res.flat();
+  }
+}


Make this a proper function of this class

franzns · 2023-07-31T09:23:10Z

modules/pool/lib/apr-data-sources/ib-yield-apr-handlers/sources/index.ts

+export const aprHandlers: AprHandler[] = [
+  aaveV2MainnetAprHandler,
+  aaveV2PolygonAprHandler,
+  aaveV3MainnetAprHandler,
+  aaveV3PolygonAprHandler,
+  aaveV3ArbitrumAprHandler,
+  ankrEthMainnetAprHandler,
+  stMaticAprHandler,
+  cbEthAprHandler,
+  sfrxEthAprHandler,
+  rETHAprHandler,
+  USDRAprHandler,
+  MATICXAprHandler,
+  wbETHAprHandler,
+  swETHAprHandler,
+  wjAURAAprHandler,
+  eulerMainnetAprHandler,
+  gearboxMainnetAprHandler,
+  idleMainnetAprHandler,
+  overnightMainnetAprHandler,
+  ovixZkEVMAprHandler,
+  reaperArbitrumAprHandler,
+  tesseraApePoolAprHandler,
+  tetuAprHandler,
+  tranchessMainnetAprHandler,
+]


Should these better be initialized in the ib-yield-apr-handlers.ts instead of an additional file?

franzns · 2023-07-31T09:24:04Z

modules/pool/lib/apr-data-sources/ib-yield-apr-handlers/types.ts

+}
+
+export interface AprHandler {
+  readonly group: string | undefined;


Should this ever be undefined?

Changing arrow functions to normal functions;

…er-yield-tokens-apr # Conflicts: # yarn.lock

Adding fixed apr tokens; Removing logs

# Conflicts: # modules/network/arbitrum.ts # modules/network/avalanche.ts # modules/network/fantom.ts # modules/network/gnosis.ts # modules/network/mainnet.ts # modules/network/network-config-types.ts # modules/network/optimism.ts # modules/network/polygon.ts # modules/network/zkevm.ts # modules/pool/lib/apr-data-sources/fantom/yearn-vault-apr.service.ts # modules/pool/lib/apr-data-sources/reaper-crypt-apr.service.ts

…oo, because idk what to do with the spooky apr service yet;

franzns

Thanks, still a few things. Also, I think Optimism is missing reaper single strat configs (maybe also fantom).

Also, there are two reaper apr handlers, I believe only one is needed.

modules/pool/lib/apr-data-sources/ib-tokens-apr.service.ts

franzns · 2023-08-30T12:01:00Z

modules/pool/lib/apr-data-sources/ib-linear-apr-handlers/sources/reaper-crypt-apr-handler.ts

+            let tokenApr = parseFloat(apr) / APR_PERCENT_DIVISOR;
+            if (token.isSftmX) {
+                tokenApr = tokenApr * (1 + sFTMxBaseApr);
+            }
+            if (token.isWstETH) {
+                tokenApr = tokenApr * (1 + this.wstETHBaseApr);
+            }


is there a possibility to make this more general? Like if any of the IB-token-apr handlers for linear pools have an IB token, this needs to apply.

franzns · 2023-08-30T12:01:23Z

modules/pool/lib/apr-data-sources/reaper-crypt-apr.service.ts

@@ -1,181 +0,0 @@
-import { isSameAddress } from '@balancer-labs/sdk';


remove alltogether

franzns · 2023-08-30T12:02:26Z

modules/pool/lib/apr-data-sources/fantom/yearn-vault-apr.service.ts

@@ -1,59 +0,0 @@
-import { PoolAprService } from '../../../pool-types';


remove file

franzns · 2023-08-30T12:02:54Z

modules/pool/lib/apr-data-sources/apr-types.ts

+import { Dictionary } from 'lodash'
+


still there :)

Fixing default apr handler error; testing on all networks;

timjrobinson · 2023-09-05T15:08:02Z

I've been testing this out with balancer/frontend-v2#4068 running it all locally, and found a few issues:

All pools are missing BAL rewards

Base

USDbC rewards on cbETH/WETH (0xfb4c2e6e6e27b5b4a07a36360c89ede29bb3c9b6000000000000000000000026) and USDbC/axlUSDC (0xe58ca65f418d4121d6c70d4c133e60cf6fda363c000000000000000000000013) pools on base are way too low (like orders of magnitude lower than expected, seems like a scaling issue maybe because the reward token has 6 decimals rather than 18)
Pool liquidity values are incorrect (different to prod)
GOLD rewards APR are much lower than prod (40% vs 60%)
There's no APR on 0x18f150c43598cd822d39711c55bd90407a8b8ad700020000000000000000003b when in prod it has 12% APR - rewards from the subpool 0x2db50a0e0310723ef0c2a165cb9a9f80d772ba2f00020000000000000000000d

Ethereum

0x1e19cf2d73a72ef1332c882f20534b6519be0276000200000000000000000112 is missing rETH APR while 0x42ed016f826165c2e5976fe5bc3df540c5ad0af700000000000000000000058b has the token APR's including rETH

For some things (like missing BAL rewards) I'm thinking maybe I haven't run the right jobs to get the data needed?

It would be good to get this merged to canary or hosted somewhere where it can continually update with the latest data and anyone compare current Balancer prod with pulling pools from the beets API and report any differences.

franzns · 2023-09-05T15:28:10Z

Thanks for testing further! We are in the final phase to test everything locally. Next phase is to merge to canary and have it running there to investigate differences between prod and canary, like you already did.

The missing/wrong BAL rewards is something we already noticed and @gmbronco is looking into that separately.

lgahdl · 2023-09-05T17:20:15Z

0x1e19cf2d73a72ef1332c882f20534b6519be0276000200000000000000000112
This rETH token included in the APR's is the stafi rETH (0x9559aaa82d9649c7a7b220e7c461d2e74c9a3593), not the rETH you're referring to (0xae78736cd615f374d3085123a210448e74fc6393), that's why this pool is not getting the rETH APR, thanks for seeing that.
I'm not sure if the source we're using for the stafi rETH could be used for the rETH, but I can add it

lgahdl · 2023-09-05T17:20:59Z

I've been testing this out with balancer/frontend-v2#4068 running it all locally, and found a few issues:

All pools are missing BAL rewards

Base

USDbC rewards on cbETH/WETH (0xfb4c2e6e6e27b5b4a07a36360c89ede29bb3c9b6000000000000000000000026) and USDbC/axlUSDC (0xe58ca65f418d4121d6c70d4c133e60cf6fda363c000000000000000000000013) pools on base are way too low (like orders of magnitude lower than expected, seems like a scaling issue maybe because the reward token has 6 decimals rather than 18)

Pool liquidity values are incorrect (different to prod)

GOLD rewards APR are much lower than prod (40% vs 60%)

There's no APR on 0x18f150c43598cd822d39711c55bd90407a8b8ad700020000000000000000003b when in prod it has 12% APR - rewards from the subpool 0x2db50a0e0310723ef0c2a165cb9a9f80d772ba2f00020000000000000000000d

Ethereum

0x1e19cf2d73a72ef1332c882f20534b6519be0276000200000000000000000112 is missing rETH APR while 0x42ed016f826165c2e5976fe5bc3df540c5ad0af700000000000000000000058b has the token APR's including rETH

For some things (like missing BAL rewards) I'm thinking maybe I haven't run the right jobs to get the data needed?

It would be good to get this merged to canary or hosted somewhere where it can continually update with the latest data and anyone compare current Balancer prod with pulling pools from the beets API and report any differences.

This rETH token included in the APR's is the stafi rETH (0x9559aaa82d9649c7a7b220e7c461d2e74c9a3593), not the rETH you're referring to (0xae78736cd615f374d3085123a210448e74fc6393), that's why this pool(0x1e19cf2d73a72ef1332c882f20534b6519be0276000200000000000000000112) is not getting the rETH APR, thanks for seeing that.
I'm not sure if the source we're using for the stafi rETH could be used for the rETH, but I can add it
Note.: The rETH APR of the pool 0x42ed016f826165c2e5976fe5bc3df540c5ad0af700000000000000000000058b is probably not from this service

Luiz Gustavo Abou Hatem de Liz added 3 commits June 30, 2023 02:01

Created a service to fetch yield tokens APR and push into DB

e6a65b3

Merge remote-tracking branch 'origin/v3-canary' into non-reaper-yield…

1f18d5c

…-tokens-apr

Adding Apr Service to Mainnet network

27ca26d

lgahdl requested a review from gmbronco July 3, 2023 13:04

lgahdl self-assigned this Jul 3, 2023

lgahdl marked this pull request as draft July 3, 2023 13:04

Added yield tokens fetch funcionality directly on beethovenx-backend,…

3d84c4b

… instead of use the yield-tokens cloudflare service; Removed the boosted pools apr, and let only the yield tokens APR update, boosted pools are updated in another service;

franzns reviewed Jul 7, 2023

View reviewed changes

Adapting contracts from apr handler to ethers instead of viem;

6120e3a

lgahdl changed the title ~~APR Services - Non Reaper Boosted pools~~ APR Services - Add some yield tokens APR to linear pools Jul 8, 2023

Changing apr type if pool type is linear;

65211ed

lgahdl requested a review from franzns July 8, 2023 02:08

Luiz Gustavo Abou Hatem De Liz added 3 commits July 7, 2023 23:15

removing viem

304988e

Merge commit 'fccc29b6e65ffd3df41b7b7de0c2e407e8de2c59' into non-reap…

5cc075b

…er-yield-tokens-apr Grouping APRs by source;

Fixing Ovix APR;

11cd519

franzns reviewed Jul 21, 2023

View reviewed changes

Luiz Gustavo Abou Hatem De Liz added 2 commits July 28, 2023 01:01

Redesigning Ib Apr Handlers to be more scalable and open to new netwo…

a1a20dd

…rks, tokens and urls;

adding interface to default, ankr and aave apr handlers;

b3d5575

franzns reviewed Jul 31, 2023

View reviewed changes

Luiz Gustavo Abou Hatem De Liz added 3 commits August 1, 2023 19:25

Adding multichain support;

65047bb

Changing arrow functions to normal functions;

Types files for ankr and default-fetch; (Missing in the last commit)

19a48fc

Merge commit 'a0eb08ebd453441e8f566fb03cc684ec8b4e324d' into non-reap…

bfe33c1

…er-yield-tokens-apr # Conflicts: # yarn.lock

Luiz Gustavo Abou Hatem De Liz added 8 commits August 29, 2023 00:00

Removing Reaper Crypt Apr Service;

c509044

Removing services that are already in the ib-yield service;

3d568ca

Adding fixed apr tokens; Removing logs

Removing test route;

ae0d54c

fixing base network incompatibilities;

98e9df6

fixing optional beefy property in network config;

fd7f136

Removing all unused functions of liquid staked base apr, left just xB…

a836720

…oo, because idk what to do with the spooky apr service yet;

Renaming AprConfig to IbAprConfig;

114386d

franzns marked this pull request as ready for review August 30, 2023 11:11

franzns self-requested a review August 30, 2023 11:11

franzns reviewed Aug 30, 2023

View reviewed changes

franzns and others added 9 commits September 1, 2023 11:57

clean up types

9ac7a10

simplify ib-tokens-apr service

a5ca109

change default handler, adapt token lists

97a795f

fix percentage

ea59577

add try catch

1a9352e

Changing the ib yield filter to network config;

c174806

Fixing default apr handler error; testing on all networks;

ibyield flag

b00f57d

cleanup

21cd9ae

dont use nested query for apr

1261ec6

franzns added 3 commits September 5, 2023 17:31

Merge branch 'v3-canary' into non-reaper-yield-tokens-apr

5563895

lower case all addresses in configs

e68f06a

Merge branch 'v3-canary' into non-reaper-yield-tokens-apr

c511a9b

adding rocket eth apr

57099bf

franzns merged commit 4b000ae into v3-canary Sep 6, 2023

franzns deleted the non-reaper-yield-tokens-apr branch September 6, 2023 11:52

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

APR Services - Add some yield tokens APR to linear pools #396

APR Services - Add some yield tokens APR to linear pools #396

lgahdl commented Jul 3, 2023

gmbronco commented Jul 6, 2023

lgahdl commented Jul 6, 2023

franzns Jul 7, 2023

franzns Jul 7, 2023

franzns Jul 21, 2023

franzns Jul 21, 2023

lgahdl Jul 21, 2023 •

edited

Loading

franzns Jul 21, 2023

franzns Jul 21, 2023

franzns commented Jul 21, 2023

franzns Jul 31, 2023

franzns Jul 31, 2023

franzns Jul 31, 2023

franzns left a comment

franzns Aug 30, 2023

franzns Aug 30, 2023

franzns Aug 30, 2023

franzns Aug 30, 2023

timjrobinson commented Sep 5, 2023 •

edited

Loading

franzns commented Sep 5, 2023

lgahdl commented Sep 5, 2023

lgahdl commented Sep 5, 2023 •

edited

Loading

Base

Ethereum



		const jsonRpcProvider = new JsonRpcProvider('https://arb1.arbitrum.io/rpc', 42161 /ARBITRUM/)
		const provider = MulticallWrapper.wrap(jsonRpcProvider)

		@@ -1,181 +0,0 @@
		import { isSameAddress } from '@balancer-labs/sdk';

		@@ -1,59 +0,0 @@
		import { PoolAprService } from '../../../pool-types';

APR Services - Add some yield tokens APR to linear pools #396

APR Services - Add some yield tokens APR to linear pools #396

Conversation

lgahdl commented Jul 3, 2023

gmbronco commented Jul 6, 2023

lgahdl commented Jul 6, 2023

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

lgahdl Jul 21, 2023 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

franzns commented Jul 21, 2023

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

franzns left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

timjrobinson commented Sep 5, 2023 • edited Loading

Base

Ethereum

franzns commented Sep 5, 2023

lgahdl commented Sep 5, 2023

lgahdl commented Sep 5, 2023 • edited Loading

Base

Ethereum

lgahdl Jul 21, 2023 •

edited

Loading

timjrobinson commented Sep 5, 2023 •

edited

Loading

lgahdl commented Sep 5, 2023 •

edited

Loading