[Multi-Chain] Enable payments on other chains #425

fhenneke · 2024-11-11T13:36:36Z

This is a PR for the current design of payments on other chains.

The current design allows for creation of csv files on Gnosis and Arbitrum One. The script needs to be invoked once for each chain with a different .env file.

fhenneke · 2024-11-22T11:11:38Z

I compared results from this PR to values computed in cowprotocol/gnosis_chain_solver_payouts#13

Results for batch and quote rewards seem to coincide:

For the accounting week 2024-11-12 to 2024-11-19, the other script returns a breakdown of

prod-1inch: 325.21550040000005, 2251.494,
prod-Barter: 2.5696692, 245.3724,
prod-Enso: 1.4641452000000001, 53.928000000000004,
prod-Furucombo: 8773.5759804, 4316.9364000000005,
prod-Baseline: 522.5164812, 48.5352,
prod-FractalSolver: 72.07477200000001, 0.0,
prod-SeaSolver: 220.68416159999998, 2076.228,
prod-Balancer: 350.18416440000004, 202.23000000000002,
prod-CopiumCapital: 726.6851928000001, 2634.3828000000003,
prod-Quasimodo: 56.759220000000006, 83.58840000000001,
barn-Quasimodo: 0.2076228, 18.8748,
barn-Baseline: 0.026964000000000002, 5.3928,
barn-Balancer: 0.0053928000000000005, 8.0892,
barn-Enso: 0.0026964000000000003, 0.0,
barn-1inch: 0.0, 5.3928,
barn-Barter: 0.0, 10.7856,

The code from from this PR returns reward transfers as follows:

0x056545021b39790efb0a074827e7fddcb751a179,325.2123999448295
0x056545021b39790efb0a074827e7fddcb751a179,2251.4812949103393
0x0a360134553feed49fe5eb273074d80b6e45941f,2.5700961371414333
0x0a360134553feed49fe5eb273074d80b6e45941f,245.37101537346211
0x12c53cdd1ef150e1cd291dd210b761acfada6b9c,1.4649489548145511
0x12c53cdd1ef150e1cd291dd210b761acfada6b9c,53.9276956864752
0x227fda1d5970df605d785bf5f2f8899d5fdf8624,8773.39375939779
0x227fda1d5970df605d785bf5f2f8899d5fdf8624,4316.912039702339
0x71b2ea049f4247bf0c0265dfc156483c0df58056,522.5127537424969
0x71b2ea049f4247bf0c0265dfc156483c0df58056,48.53492611782767
0x727eb77c6f84ef148403f641aa32d75b7f6902a7,72.07455166464771
0xe3068acb5b5672408eadad4417e7d3ba41d4febe,220.68222342810193
0xe3068acb5b5672408eadad4417e7d3ba41d4febe,2076.216283929295
0xf671d28fef15e5eafc21898c2814b1b4cd88bc9a,350.1820506673969
0xf671d28fef15e5eafc21898c2814b1b4cd88bc9a,202.22885882428199
0xfabbdf8a77005c00edbe0000bdc000644c024322,726.4381790869907
0xfabbdf8a77005c00edbe0000bdc000644c024322,2634.3679342843134
0xfb88fc234278a2c9cceffbacdb01b8a993307a22,56.755341349764265
0xfb88fc234278a2c9cceffbacdb01b8a993307a22,83.58792831403655
0x027af765554bd4f768415e299ee610c90e9bed55,18.874693490266317
0x46d5948b6d2ddd46138de1232738459ab77fd10e,5.39276956864752
0x4930a9012e8677ae764e44f2b46af8087a1f9f8e,8.089154352971278
0xc088aea818de69cec696054682098aba804b4fc9,5.39276956864752
0xc4dd6355fbc6eb108fd1c100389789c5b1a9a980,10.78553913729504

So rewards seem to work fine.

fhenneke · 2024-11-26T12:38:25Z

Using this dune query, the rewards and total protocol fee seem to coincide with what this PR proposes.

What has not been checked yet is the split of protocol fees into separate transfers to the protocol and to partners. The margin for error should be small there, as the logic around getting amounts and addresses does not depend on the chain.

(cherry picked from commit cad8433) (cherry picked from commit 45ecb53)

(cherry picked from commit 0db3b35) # Conflicts: # src/fetch/dune.py # Conflicts: # src/fetch/transfer_file.py

(cherry picked from commit 01c1b47)

(cherry picked from commit b54b3ab)

# Conflicts: # src/fetch/payouts.py

harisang · 2024-12-02T16:20:52Z

src/config.py

@@ -80,7 +104,7 @@ class ProtocolFeeConfig:
    def from_network(network: Network) -> ProtocolFeeConfig:
        """Initialize protocol fee config for a given network."""
        match network:
-            case Network.MAINNET:
+            case Network.MAINNET | Network.GNOSIS | Network.ARBITRUM_ONE:


@fhenneke This probably needs updating? I think we have different addresses per chain

I added appropriate addresses for each chain.

harisang · 2024-12-02T16:31:01Z

src/config.py

                payment_safe_address = Web3.to_checksum_address(
                    os.environ.get(
-                        "SAFE_ADDRESS", "0xA03be496e67Ec29bC62F01a428683D7F9c204930"
+                        "SAFE_ADDRESS_MAINNET",


Just realized that the name is not very descriptive. Something like PAYOUTS_SAFE would be more descriptive

I changed it to PAYOUTS_SAFE_ADDRESS_${NETWORK}.

harisang

Looks good overall! Haven't tested it yet but code makes sense.

Added some small comments

MartinquaXD · 2024-12-04T07:19:37Z

src/config.py

+            case Network.MAINNET:
+                prod_db_url = os.environ.get("PROD_DB_URL_MAINNET", "")
+                barn_db_url = os.environ.get("BARN_DB_URL_MAINNET", "")
+            case Network.GNOSIS:
+                prod_db_url = os.environ.get("PROD_DB_URL_GNOSIS", "")
+                barn_db_url = os.environ.get("BARN_DB_URL_GNOSIS", "")
+            case Network.ARBITRUM_ONE:
+                prod_db_url = os.environ.get("PROD_DB_URL_ARBITRUM", "")
+                barn_db_url = os.environ.get("BARN_DB_URL_ARBITRUM", "")


Came here from https://github.com/cowprotocol/infrastructure/pull/2349.
This seems like it complicates the configuration. Why not continue to always read the URLs from PROD_DB_URL and BARN_DB_URL and expect the job to be run with the correctly configured env variables?

Same comment applies to NODE_URL.

In principle it is possible to have one config file with all environment variables per chain, and remove all post-fixes. With our infrastructure it would be one config file and one secret file per chain then, right?

At some point we had planned to compute all payments in one run and then all secrets would need to be available from the same environment. But this is not planned anymore.

With our infrastructure it would be one config file and one secret file per chain then, right?

Yeah, I think so. Both approaches can definitely work. Just seemed surprising to me that you'd basically have 2 places where you branch based on the network. First when you decide which ENV variable to write your values into and later when you decide which ENV variable to read from. 🤷‍♂️

.github/workflows/pull-request.yaml

harisang · 2024-12-05T15:30:12Z

.env.sample

@@ -4,7 +4,7 @@ FILE_OUT_PATH=./out
 DUNE_API_KEY=

 # Safe Transaction Service Requirements.
-SAFE_ADDRESS=0xA03be496e67Ec29bC62F01a428683D7F9c204930
+PAYOUTS_SAFE_ADDRESS=0xA03be496e67Ec29bC62F01a428683D7F9c204930
 NETWORK=mainnet


Could you add a comment with the options here?
Is it {mainnet, xdai, arbitrum_one} or {ethereum, gnosis, arbitrum}?

I added a comment.

harisang

Looks good overall!

Added some small comments

add comment on possible values of NETWORK reorder variables

fhenneke force-pushed the other_chains branch 4 times, most recently from a96802a to cb7fdcd Compare November 21, 2024 11:22

fhenneke changed the title ~~[Multi-Chain] [Draft] Enable payments on other chains~~ [Multi-Chain] Enable payments on other chains Nov 22, 2024

fhenneke marked this pull request as ready for review November 22, 2024 09:17

fhenneke requested review from harisang and bram-vdberg November 22, 2024 09:17

fhenneke mentioned this pull request Nov 22, 2024

[Do not merge] Code changes for testing rewards cowprotocol/gnosis_chain_solver_payouts#13

Open

fhenneke added 7 commits November 26, 2024 13:39

add additional tokens to price fetching

5966dba

(cherry picked from commit cad8433) (cherry picked from commit 45ecb53)

add blockchain parameter to dune fetcher

cdf9019

(cherry picked from commit 0db3b35) # Conflicts: # src/fetch/dune.py # Conflicts: # src/fetch/transfer_file.py

rename weth to wrapped native token

1ce5d8e

(cherry picked from commit 01c1b47)

add config for other chains

9f5eaee

(cherry picked from commit b54b3ab)

change pull request node environment variable name

26ffae0

fix bug with missing data

9222211

# Conflicts: # src/fetch/payouts.py

fix gnosis cap

504ad05

fhenneke force-pushed the other_chains branch from 3647445 to 504ad05 Compare November 26, 2024 12:41

fhenneke added 3 commits December 2, 2024 11:54

use new vouching query

e48581c

fix time range

36d79d2

add exception for missing vouches

f9ec687

harisang reviewed Dec 2, 2024

View reviewed changes

harisang requested changes Dec 2, 2024

View reviewed changes

fhenneke added 3 commits December 3, 2024 16:55

add log for native token exchange rate

6a045cf

change name of payouts safe environment variable

52afbd5

make protocol fee safe chain dependent

9606888

MartinquaXD reviewed Dec 4, 2024

View reviewed changes

update sample .env file

3d6c205

fhenneke added 9 commits December 4, 2024 14:57

fix bug in arbitrum payment network

b2b587e

use chain agnostic environment variables

5761990

change vouching query id to old query

cb2be14

Merge branch 'main' into other_chains

e2dc829

fix CI

ca7d981

update payout safe addresss for testing

299e963

change pull request env variable formating

a58bf93

add network to transfer file

98323d7

lint fix

8d7cd14

harisang reviewed Dec 5, 2024

View reviewed changes

.github/workflows/pull-request.yaml Show resolved Hide resolved

harisang reviewed Dec 5, 2024

View reviewed changes

harisang approved these changes Dec 5, 2024

View reviewed changes

fhenneke added 2 commits December 5, 2024 16:31

add network variable to github env variables

b35b106

update .env.sample

424e89a

add comment on possible values of NETWORK reorder variables

fhenneke merged commit b4727c4 into main Dec 5, 2024
5 checks passed

fhenneke deleted the other_chains branch December 5, 2024 15:51

github-actions bot locked and limited conversation to collaborators Dec 5, 2024

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

[Multi-Chain] Enable payments on other chains #425

[Multi-Chain] Enable payments on other chains #425

fhenneke commented Nov 11, 2024 •

edited

Loading

fhenneke commented Nov 22, 2024 •

edited

Loading

fhenneke commented Nov 26, 2024

harisang Dec 2, 2024 •

edited

Loading

fhenneke Dec 3, 2024

harisang Dec 2, 2024

fhenneke Dec 3, 2024

harisang left a comment

MartinquaXD Dec 4, 2024

fhenneke Dec 4, 2024

MartinquaXD Dec 4, 2024

harisang Dec 5, 2024

fhenneke Dec 5, 2024

harisang left a comment

[Multi-Chain] Enable payments on other chains #425

[Multi-Chain] Enable payments on other chains #425

Conversation

fhenneke commented Nov 11, 2024 • edited Loading

fhenneke commented Nov 22, 2024 • edited Loading

fhenneke commented Nov 26, 2024

harisang Dec 2, 2024 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

harisang left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

harisang left a comment

Choose a reason for hiding this comment

fhenneke commented Nov 11, 2024 •

edited

Loading

fhenneke commented Nov 22, 2024 •

edited

Loading

harisang Dec 2, 2024 •

edited

Loading