Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Bridges of LI.FI across chains, the contract is LiFiDiamond_v2 #7172

Merged
merged 46 commits into from
Dec 9, 2024

Conversation

lequangphu
Copy link
Contributor

Thank you for contributing to Spellbook 🪄

Please open the PR in draft and mark as ready when you want to request a review.

Description:

Extract bridge data from event LiFiDiamond_v2_evt_LiFiTransferStarted on each chain, then union across chains.


quick links for more information:

@lequangphu lequangphu marked this pull request as draft November 22, 2024 08:29
@jeff-dude jeff-dude added WIP work in progress dbt: daily covers the Daily dbt subproject labels Nov 25, 2024
@jeff-dude jeff-dude self-assigned this Nov 25, 2024
@jeff-dude
Copy link
Member

before we dig into the failing test, i think we need to redesign this pipeline a bit.

  • flip your materialization strategy approach -- i.e. materialize each chain incrementally, then have the crosschain be a view on top
  • looks like the code is the same per chain -- in this case, i'd use a dbt macro for the logic and call the macro for each chain while passing in the blockchain parameter to differentiate. there are tons of examples in spellbook to use as reference, one to search could be the uniswap models in dex subproject (but plenty others too!)
  • when joining from decoded to base table to get additional columns, lets use this shared macro. an example to call the macro is here
  • when using the if is incremental logic, let's use the latest macro approach like here rather than hardcode last 7 days
  • for incremental chain models, i would expect config to contain all of this
  • looks like you removed the surrogate key logic, i'd put that back in and use as unique key. then update the schema yml to run unique test on this column, not the combination of columns that make up the surrogate key

let's leave it at that for now, and revisit once all this in place 🙏

@lequangphu
Copy link
Contributor Author

thanks, i will update the pr based on your recommendation.

great, just tag me again when you are ready for next iteration of review 🙏

it's green check, so i guess i'm ready.

@jeff-dude jeff-dude marked this pull request as ready for review November 27, 2024 16:48
@jeff-dude jeff-dude added in review Assignee is currently reviewing the PR and removed WIP work in progress labels Nov 27, 2024
Copy link
Member

@jeff-dude jeff-dude left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

really clean code here, much appreciated!

left a few minor final changes below, please do implement when ready.

last thing: have you queried the data output yet, from CI pipeline?
https://github.com/duneanalytics/spellbook/actions/runs/12054789316/job/33613636277?pr=7172#step:12:32
you can query these tables on dune and ensure it looks as you expect, prior to merge 🙏

@lequangphu
Copy link
Contributor Author

lequangphu commented Nov 28, 2024

really clean code here, much appreciated!

left a few minor final changes below, please do implement when ready.

last thing: have you queried the data output yet, from CI pipeline? https://github.com/duneanalytics/spellbook/actions/runs/12054789316/job/33613636277?pr=7172#step:12:32 you can query these tables on dune and ensure it looks as you expect, prior to merge 🙏

I applied your suggestion.

I checked the data output of CI pipeline, they were as expected.

I want to add amount_usd to the output as what I do from line 223-427 in this query: https://dune.com/queries/4182530. The steps are:

  1. convert address of eth to weth, variants of usdc/usdt to usdc/usdt (without this step, there are many null records of amount_usd)
  2. join prices.usd to get amount_usd + add chain name based on chain id

How would you apply them in the models?

@jeff-dude
Copy link
Member

How would you apply them in the models?

for pricing, i would handle the way you are now. CTE which reads raw decoded data and pass into macro, then pass that CTE into new one with join to prices. i was thinking you could use a macro for the join to prices and have each model use it, but if your manual mapping of token addresses differs per chain, it may be more complex.

to summarize:

  • first CTE reads only decoded data ✅
  • second CTE (todo) maps addresses as needed per chain
  • third CTE joins to prices ✅
  • final macro call to add columns

@lequangphu
Copy link
Contributor Author

  • first CTE reads only decoded data ✅
  • second CTE (todo) maps addresses as needed per chain
  • third CTE joins to prices ✅
  • final macro call to add columns

Done, please review!

@lequangphu
Copy link
Contributor Author

@jeff-dude the test fails because decoded table notfound, which is weird.

Screenshot 2024-12-04 at 10 08 25

Copy link
Member

@jeff-dude jeff-dude left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

this is looking clean now 🔥
thank you for the patience and iterating on changes, i'll get it prepped for merge!

@jeff-dude jeff-dude added ready-for-merging and removed in review Assignee is currently reviewing the PR labels Dec 6, 2024
@jeff-dude jeff-dude merged commit dfd137c into duneanalytics:main Dec 9, 2024
2 checks passed
@github-actions github-actions bot locked and limited conversation to collaborators Dec 9, 2024
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
dbt: daily covers the Daily dbt subproject ready-for-merging
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants