Only keep validators with an assigned index in staking_ethereum.flows

[hildobby]

…_ethereum.flows

Thank you for contributing to Spellbook 🪄

Please open the PR in draft and mark as ready when you want to request a review.

Description:

[...]

---

quick links for more information:

• README.md
• spellbook docs
• CONTRIBUTING.md

[0xRobin]

@hildobby can you give more details on the external dataset you're using here?

[hildobby]

@hildobby can you give more details on the external dataset you're using here?

This dataset only contains validator_index and their corresponding pubkey values. This is because the BLS key check happens offchain due to it being quite compute heavy. So this dataset was fetched from beaconcha.in and using it as an inner join ensures that validator_index is correct while excluding non-valid deposits (there are a few, currently throwing off the validator_index values the current spell calculates by counting incrementally).

TLDR; this fixes a bug I've had for a while and makes the data accurate

[0xRobin]

Ok, thanks for explaining, how is the dataset maintained?

[hildobby]

Ok, thanks for explaining, how is the dataset maintained?

Currently I have local scripts, one for fetching from beaconcha.in (worked with them on this) and one for uploading and i run those very regularly but once this is merged and all working I'll be moving those to a dedicated server with a cron job every 4h or less. Updates aren't as time-sensitive since there already is the deposit queue anyways which is ~8h long.

[hildobby]

I defined the source (+ included it in a sources.yml) like on here:

spellbook/dbt_subprojects/tokens/models/prices/prices_solana_day.sql

Line 25 in 588ca27

{{ source("dune", "prices_solana_day_raw", database="dune") }}

But it's still failing, is there something I'm missing?

[0xRobin]

@hildobby you duplicated the full model below, which uses incorrect syntax for the source (hence the compile error)

[jeff-dude]

minor note: we are also in sync in TG group for design patterns related to the source in this PR