Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[DRAFT] Eigen Incredible Squaring Blueprint Integration Test #304

Closed
wants to merge 80 commits into from
Closed
Show file tree
Hide file tree
Changes from 57 commits
Commits
Show all changes
80 commits
Select commit Hold shift + click to select a range
5d55fd2
Move executor to SDK
drewstone Aug 26, 2024
4bfe7bd
nit: make struct var public
drewstone Aug 26, 2024
9d035db
fix: fix crate paths for executor, change incredible-squaring avs
drewstone Aug 26, 2024
83972eb
fmt
drewstone Aug 26, 2024
c4fb791
Fix eigen event macro
drewstone Aug 26, 2024
603fe8b
Fix: SDK build fix, blueprint WIP
Tjemmmic Aug 27, 2024
bfe6cd0
Merge remote-tracking branch 'origin/drew/move-executor-to-sdk' into …
Tjemmmic Aug 27, 2024
275ed88
Fix: Blueprint build fixes and WIP
Tjemmmic Aug 28, 2024
a82ed1e
Merge branch 'refs/heads/main' into drew/move-executor-to-sdk
Tjemmmic Aug 28, 2024
3e62221
Deps: Cleanup and reorganize
Tjemmmic Aug 28, 2024
26d15cf
Fix: incredible-squaring-blueprint fixes WIP
Tjemmmic Aug 29, 2024
fe6f739
Fix: Blueprint Fix WIP
Tjemmmic Aug 29, 2024
5568658
Incredible Squaring Fixes
Tjemmmic Aug 30, 2024
545d3fb
WIP: EL Incredible Squaring and Tangle AVS Blueprints
Tjemmmic Sep 4, 2024
844d3e3
WIP: Adding BLS BN254 to Keystore and some refactoring
Tjemmmic Sep 4, 2024
f5e8690
Feat: Bls Bn254 added to keystore
Tjemmmic Sep 5, 2024
4f9f920
WIP: Getting Eigenlayer Incredible Squaring Blueprint working
Tjemmmic Sep 6, 2024
be5b9ef
WIP: Eigen IS AVS Blueprint
Tjemmmic Sep 11, 2024
06a12d9
Merge branch 'refs/heads/main' into drew/move-executor-to-sdk
Tjemmmic Sep 11, 2024
ee827b0
Debugging, post-merge Cleanup, and fixes
Tjemmmic Sep 11, 2024
390dfc0
Fixing merge conflicts and pulling in blueprint testing logic
Tjemmmic Sep 12, 2024
3f896b1
More cleanup and fixes
Tjemmmic Sep 12, 2024
684892d
Work on getting testing passing with testnet (#239)
tbraun96 Sep 12, 2024
0043c0b
Keystore and blueprint fixes for Eigen IS
Tjemmmic Sep 12, 2024
e070cba
Merge remote-tracking branch 'origin/drew/move-executor-to-sdk' into …
Tjemmmic Sep 12, 2024
6c649cd
fmt
Tjemmmic Sep 12, 2024
93db9fe
Cleanup and refactor
tbraun96 Sep 13, 2024
06e6328
pass proper args
tbraun96 Sep 13, 2024
5976ce3
WIP: Getting Eigen IS Blueprint Compiling
Tjemmmic Sep 13, 2024
dbbc1a8
continue debugging
tbraun96 Sep 13, 2024
a4e6bcd
Merge branch 'drew/move-executor-to-sdk' of https://github.com/webb-t…
tbraun96 Sep 13, 2024
d8e2b1a
pull in origin/HEAD
tbraun96 Sep 13, 2024
d0a6d0b
Sync
tbraun96 Sep 13, 2024
35afe91
Fix the test, it passes
tbraun96 Sep 13, 2024
cbe6186
Fix: Builds without errors
Tjemmmic Sep 13, 2024
c805443
Fix: Merge, cleanup, and addressing PR comments
Tjemmmic Sep 13, 2024
7461ad5
Merge remote-tracking branch 'origin/drew/move-executor-to-sdk' into …
Tjemmmic Sep 13, 2024
68c107e
Merge branch 'refs/heads/main' into drew/move-executor-to-sdk
Tjemmmic Sep 13, 2024
528a0fb
Fix: Pull in changes from main, resolve conflicts
Tjemmmic Sep 13, 2024
2c64dd1
Fix: Add Forge to Workflow
Tjemmmic Sep 14, 2024
e913725
Merge in Main and pull generic blueprint code into SDK
Tjemmmic Sep 16, 2024
abed344
Merge branch 'refs/heads/main' into drew/move-executor-to-sdk
Tjemmmic Sep 16, 2024
2feaf6d
Fix: Workflow fix
Tjemmmic Sep 16, 2024
ca3cb14
Fix: Additional Workflow Fix
Tjemmmic Sep 16, 2024
c4b4a2f
Fix: Workflow debugging
Tjemmmic Sep 16, 2024
f2ace9c
Fix: Workflow debugging
Tjemmmic Sep 16, 2024
c8f8cd9
Fix: More debugging of workflow
Tjemmmic Sep 16, 2024
5f1d93f
Debugging Foundry workflow installation
Tjemmmic Sep 16, 2024
92ec028
Fix: Forge install path fix
Tjemmmic Sep 16, 2024
0a219b6
Fix: Final workflow fix
Tjemmmic Sep 16, 2024
a7cb57b
Fix: Fix feature-related errors
Tjemmmic Sep 17, 2024
298bab8
Merge branch 'refs/heads/main' into drew/move-executor-to-sdk
Tjemmmic Sep 17, 2024
64d7975
Fix: no-std fixes and cleanup
Tjemmmic Sep 17, 2024
d0d3d58
Starting Eigen IS Blueprint Test
Tjemmmic Sep 17, 2024
d4ed6fd
Merge branch 'refs/heads/main' into donovan/eigen-blueprint-test
Tjemmmic Sep 18, 2024
843aba2
Debugging Eigen Incredible Squaring
Tjemmmic Sep 19, 2024
a9cb990
Fixes: Updating from alloy and eigensdk-rs, fixing configs for eigen
Tjemmmic Sep 24, 2024
fdd4a8a
nit: make struct var public
drewstone Aug 26, 2024
92b2805
Fix: Blueprint build fixes and WIP
Tjemmmic Aug 28, 2024
b63e6b2
Deps: Cleanup and reorganize
Tjemmmic Aug 28, 2024
ab42d53
Incredible Squaring Fixes
Tjemmmic Aug 30, 2024
0bf59bf
WIP: Adding BLS BN254 to Keystore and some refactoring
Tjemmmic Sep 4, 2024
1755edb
Debugging, post-merge Cleanup, and fixes
Tjemmmic Sep 11, 2024
4102b55
Fixing merge conflicts and pulling in blueprint testing logic
Tjemmmic Sep 12, 2024
a8561e8
More cleanup and fixes
Tjemmmic Sep 12, 2024
2639e31
Keystore and blueprint fixes for Eigen IS
Tjemmmic Sep 12, 2024
3c28aa0
Work on getting testing passing with testnet (#239)
tbraun96 Sep 12, 2024
3c92968
Fix: Merge, cleanup, and addressing PR comments
Tjemmmic Sep 13, 2024
0c59449
Sync
tbraun96 Sep 13, 2024
dffb5c4
Fix: Pull in changes from main, resolve conflicts
Tjemmmic Sep 13, 2024
47bd3c0
Merge in Main and pull generic blueprint code into SDK
Tjemmmic Sep 16, 2024
c4e5227
Fix: Fix feature-related errors
Tjemmmic Sep 17, 2024
aa2cc2b
Fix: no-std fixes and cleanup
Tjemmmic Sep 17, 2024
20360b8
Starting Eigen IS Blueprint Test
Tjemmmic Sep 17, 2024
0184814
Debugging Eigen Incredible Squaring
Tjemmmic Sep 19, 2024
5566d62
Merge remote-tracking branch 'origin/donovan/eigen-blueprint-test' in…
Tjemmmic Sep 24, 2024
ec5ff15
Fixes: Fix Macro errors and cleanup after rebase
Tjemmmic Sep 24, 2024
a25d108
Fixes: Eigen Event handler fix
Tjemmmic Sep 24, 2024
0bf0dbb
Merge branch 'refs/heads/main' into donovan/eigen-blueprint-test
Tjemmmic Sep 24, 2024
5bcd861
Fixes: Cleanup after merging in Main
Tjemmmic Sep 24, 2024
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
3,536 changes: 3,262 additions & 274 deletions Cargo.lock

Large diffs are not rendered by default.

25 changes: 13 additions & 12 deletions Cargo.toml
Original file line number Diff line number Diff line change
Expand Up @@ -147,17 +147,17 @@ typed-builder = "0.19"
url = { git = "https://github.com/domenukk/rust-url", branch = "no_std", default-features = false }
w3f-bls = { version = "0.1.4", default-features = false }

alloy-primitives = "0.7.7"
alloy-json-abi = "0.7.7"
alloy-sol-types = "0.7.7"
alloy-rpc-types = { version = "0.2.1" }
alloy-provider = { version = "0.2.1", default-features = false, features = ["reqwest", "ws"] }
alloy-pubsub = { version = "0.2.1" }
alloy-signer = { version = "0.2.1" }
alloy-signer-local = { version = "0.2.1" }
alloy-network = { version = "0.2.1" }
alloy-contract = { version = "0.2.1" }
alloy-transport = { version = "0.2.1" }
alloy-primitives = "0.8.3"
alloy-json-abi = "0.8.3"
alloy-sol-types = "0.8.3"
alloy-rpc-types = { version = "0.3.6" }
alloy-provider = { version = "0.3.6", default-features = false, features = ["reqwest", "ws"] }
alloy-pubsub = { version = "0.3.6" }
alloy-signer = { version = "0.3.6" }
alloy-signer-local = { version = "0.3.6" }
alloy-network = { version = "0.3.6" }
alloy-contract = { version = "0.3.6" }
alloy-transport = { version = "0.3.6" }

ark-bn254 = "0.4.0"
ark-ec = "0.4.0"
Expand All @@ -176,7 +176,8 @@ secp256k1 = "0.29.1"

# Eigenlayer
# TODO: Remove this branch, point to main once https://github.com/webb-tools/eigensdk-rs/pull/20 is merged
eigensdk-rs = { git = "https://github.com/webb-tools/eigensdk-rs.git", branch = "donovan/maintenance" }
#eigensdk-rs = { git = "https://github.com/webb-tools/eigensdk-rs.git", branch = "donovan/maintenance" }
eigensdk-rs = { path = "../eigensdk-rs" }

[profile.dev.package.backtrace]
opt-level = 3
Expand Down
1 change: 1 addition & 0 deletions blueprint-test-utils/Cargo.toml
Original file line number Diff line number Diff line change
Expand Up @@ -70,6 +70,7 @@ uuid = { workspace = true, features = ["v4"] }
schnorrkel = { workspace = true }
subxt = { workspace = true, features = ["substrate-compat"] }
color-eyre = { workspace = true }
eigensdk-rs = { workspace = true }

# Pallets
pallet-timestamp = { workspace = true }
Expand Down
95 changes: 95 additions & 0 deletions blueprint-test-utils/src/lib.rs
Original file line number Diff line number Diff line change
Expand Up @@ -551,4 +551,99 @@ mod tests_standard {
})
.await
}

#[tokio::test(flavor = "multi_thread")]
async fn test_eigen_incredible_squaring_blueprint() {
let mut base_path = std::env::current_dir().expect("Failed to get current directory");

base_path.push("../blueprints/incredible-squaring");
base_path
.canonicalize()
.expect("File could not be normalized");

let manifest_path = base_path.join("Cargo.toml");
let blueprint_test_file = base_path.join("blueprint-test.json");
let blueprint_file = base_path.join("blueprint.json");

// cp the blueprint-test-file into the blueprint-file
tokio::fs::copy(&blueprint_test_file, &blueprint_file)
.await
.expect("Failed to copy blueprint-test.json to blueprint.json");
Tjemmmic marked this conversation as resolved.
Show resolved Hide resolved

let opts = Opts {
pkg_name: Some("incredible-squaring-blueprint".to_string()),
rpc_url: "ws://127.0.0.1:9944".to_string(),
manifest_path,
signer: None,
signer_evm: None,
};
// --ws-external
const INPUT: u64 = 10;
const OUTPUT: u64 = INPUT.pow(2);

// Start Local Anvil Testnet in the background
let _ = eigensdk_rs::test_utils::anvil::testnet::incredible_squaring::run_incredible_squaring_testnet().await;

new_test_ext_blueprint_manager::<5, 1, (), _, _>((), opts, run_test_blueprint_manager)
.await
.execute_with_async(move |client, handles, logger| async move {
// At this point, blueprint has been deployed, every node has registered
// as an operator for the relevant services, and, all gadgets are running

// What's left: Submit a job, wait for the job to finish, then assert the job results
let keypair = handles[0].sr25519_id().clone();

// TODO: Important! The tests can only run serially, not in parallel, in order to not cause a race condition in IDs
Copy link
Contributor

@drewstone drewstone Sep 24, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can we explain this more? How can we get to parallel test execution? We want to mimic the real-world as close as possible if we possibly can. CC @tbraun96

let service_id = get_next_service_id(client)
.await
.expect("Failed to get next service id")
.saturating_sub(1);
let call_id = get_next_call_id(client)
.await
.expect("Failed to get next job id")
.saturating_sub(1);

logger.info(format!(
"Submitting job with params service ID: {service_id}, call ID: {call_id}"
));

// Pass the arguments
let mut job_args = Args::new();
let input =
api::runtime_types::tangle_primitives::services::field::Field::Uint64(INPUT);
job_args.push(input);

// Next step: submit a job under that service/job id
if let Err(err) = submit_job(
client,
&keypair,
service_id,
Job::from(call_id as u8),
job_args,
)
.await
{
logger.error(format!("Failed to submit job: {err}"));
panic!("Failed to submit job: {err}");
}

// Step 2: wait for the job to complete
let job_results = wait_for_completion_of_tangle_job(
client,
service_id,
call_id,
handles[0].logger(),
)
.await
.expect("Failed to wait for job completion");

// Step 3: Get the job results, compare to expected value(s)
let expected_result =
api::runtime_types::tangle_primitives::services::field::Field::Uint64(OUTPUT);
assert_eq!(job_results.service_id, service_id);
assert_eq!(job_results.call_id, call_id);
assert_eq!(job_results.result[0], expected_result);
})
.await
}
}
95 changes: 58 additions & 37 deletions blueprints/incredible-squaring/src/eigenlayer.rs
Original file line number Diff line number Diff line change
Expand Up @@ -4,31 +4,30 @@ use alloy_network::Ethereum;
use alloy_network::EthereumWallet;
use alloy_primitives::{Address, ChainId, FixedBytes};
use alloy_primitives::{Bytes, U256};
use alloy_provider::ProviderBuilder;
use alloy_provider::Provider;
use alloy_provider::ProviderBuilder;
use alloy_signer::k256::ecdsa::SigningKey;
use alloy_signer::k256::elliptic_curve::SecretKey;
use alloy_signer_local::PrivateKeySigner;
use alloy_sol_types::private::alloy_json_abi::JsonAbi;
use alloy_sol_types::sol;
use alloy_transport::Transport;
use color_eyre::{eyre::eyre, eyre::OptionExt, Result};
use gadget_sdk::job;
use gadget_sdk::{
events_watcher::{
evm::{Config, EventWatcher},
},
events_watcher::evm::{Config, EventWatcher},
keystore::Backend,
network::setup::{start_p2p_network, NetworkConfig},
run::GadgetRunner,
};
use std::convert::Infallible;
use std::ops::Deref;
use std::sync::OnceLock;
use alloy_signer::k256::elliptic_curve::SecretKey;
use alloy_transport::Transport;
use IncredibleSquaringTaskManager::{
respondToTaskCall, G1Point, G2Point,
NonSignerStakesAndSignature, Task, TaskResponse,
};
use IBLSSignatureChecker::NonSignerStakesAndSignature;
use IIncredibleSquaringTaskManager::{Task, TaskResponse};
use IncredibleSquaringTaskManager::respondToTaskCall;
use BN254::{G1Point, G2Point};

use gadget_sdk::{
tangle_subxt::tangle_testnet_runtime::api::{
runtime_types::{sp_core::ecdsa, tangle_primitives::services},
Expand All @@ -40,9 +39,7 @@ use eigensdk_rs::eigen_utils::*;
use eigensdk_rs::incredible_squaring_avs::operator::*;
use sp_core::Pair;
use subxt_signer::ecdsa::SecretKeyBytes;
use gadget_sdk::config::{
GadgetConfiguration, StdGadgetConfiguration,
};
use gadget_sdk::config::GadgetConfiguration;
use gadget_sdk::network::gossip::GossipHandle;
use gadget_sdk::tangle_subxt::tangle_testnet_runtime::api;
use gadget_sdk::tangle_subxt::tangle_testnet_runtime::api::runtime_types::tangle_primitives::services::PriceTargets;
Expand Down Expand Up @@ -77,8 +74,6 @@ pub async fn xsquare_eigen(
quorum_threshold_percentage: u32,
) -> Result<respondToTaskCall, Infallible> {
// TODO: Send our task response to Aggregator RPC server
// TODO: OR we use the gossip protocol to send the response to peers
// TODO: Where is by BLS key?

// // 1. Calculate the squared number and save the response
// let my_msg = MyMessage {
Expand Down Expand Up @@ -166,7 +161,7 @@ pub fn convert_event_to_inputs(
impl GadgetRunner for EigenlayerGadgetRunner<parking_lot::RawRwLock> {
type Error = color_eyre::eyre::Report;

fn config(&self) -> &StdGadgetConfiguration {
fn config(&self) -> &GadgetConfiguration<parking_lot::RawRwLock> {
todo!()
}

Expand Down Expand Up @@ -252,15 +247,15 @@ impl GadgetRunner for EigenlayerGadgetRunner<parking_lot::RawRwLock> {
.clone()
.boxed();

let _ws_provider = ProviderBuilder::new()
let ws_provider = ProviderBuilder::new()
.with_recommended_fillers()
.wallet(wallet)
.on_http(self.env.rpc_endpoint.parse()?)
.root()
.clone()
.boxed();

let _operator_info_service = OperatorInfoService::new(
let operator_info_service = OperatorInfoService::new(
types::OperatorInfo {
socket: "0.0.0.0:0".to_string(),
pubkeys: OperatorPubkeys {
Expand All @@ -278,27 +273,27 @@ impl GadgetRunner for EigenlayerGadgetRunner<parking_lot::RawRwLock> {
.await
.map_err(|e| OperatorError::HttpEthClientError(e.to_string()))?;

let _signer = EigenGadgetSigner::new(
let signer = EigenGadgetSigner::new(
PrivateKeySigner::from_signing_key(ecdsa_signing_key),
Some(ChainId::from(chain_id)),
);

// This creates and registers an operator with the given configuration
// let operator = Operator::<NodeConfig, OperatorInfoService>::new_from_config(
// node_config.clone(),
// EigenGadgetProvider {
// provider: http_provider,
// },
// EigenGadgetProvider {
// provider: ws_provider,
// },
// operator_info_service,
// signer,
// )
// .await
// .map_err(|e| eyre!(e))?;
let operator = Operator::<NodeConfig, OperatorInfoService>::new_from_config(
node_config.clone(),
EigenGadgetProvider {
provider: http_provider.clone(),
},
EigenGadgetProvider {
provider: ws_provider.clone(),
},
operator_info_service.clone(),
signer.clone(),
)
.await
.map_err(|e| eyre!(e))?;

// self.set_operator(operator);
self.set_operator(operator);

tracing::info!("Registered operator for Eigenlayer");
Ok(())
Expand All @@ -308,7 +303,7 @@ impl GadgetRunner for EigenlayerGadgetRunner<parking_lot::RawRwLock> {
todo!()
}

async fn run(&self) -> Result<()> {
async fn run(&mut self) -> Result<()> {
// Tangle Portion of Run
let _client = self.env.client().await.map_err(|e| eyre!(e))?;
let signer = self.env.first_signer().map_err(|e| eyre!(e))?;
Expand Down Expand Up @@ -376,19 +371,45 @@ impl GadgetRunner for EigenlayerGadgetRunner<parking_lot::RawRwLock> {

let _network: GossipHandle =
start_p2p_network(network_config).map_err(|e| eyre!(e.to_string()))?;

// let x_square_eigen = blueprint::XsquareEigenEventHandler {
// ctx: blueprint::MyContext { network, keystore },
// };
//
// let contract: IncredibleSquaringTaskManager::IncredibleSquaringTaskManagerInstance = IncredibleSquaringTaskManager::IncredibleSquaringTaskManagerInstance::new(contract_address, provider);

//
// EventWatcher::run(
// &EigenlayerEventWatcher,
// contract,
// // Add more handler here if we have more functions.
// vec![Box::new(x_square_eigen)],
// )
// .await?;
// .await?;'

let operator = self.operator.ok_or(eyre!("Run Error - No Operator Found"))?;

let instance = IncredibleSquaringTaskManagerInstanceWrapper::new(
IncredibleSquaringTaskManager::new(
operator.incredible_squaring_contract_manager.task_manager_addr.clone(),
operator.incredible_squaring_contract_manager.eth_client_http.clone(),
),
);

let event_watcher: EigenlayerEventWatcher<NodeConfig> = EigenlayerEventWatcher::new();
event_watcher
.run(
instance,
vec![
// Box::new(x_square_eigen)
],
)
.await?;

// self.operator.unwrap().start().await?;
// if let Some(operator) = &self.operator {
// operator.start().await?;
// } else {
// return Err(eyre!("Run Error - No Operator Found"));
// }

Ok(())
}
Expand Down
10 changes: 2 additions & 8 deletions blueprints/incredible-squaring/src/main.rs
Original file line number Diff line number Diff line change
Expand Up @@ -15,12 +15,6 @@ use structopt::StructOpt;
use gadget_sdk::run::GadgetRunner;
use gadget_sdk::tangle_subxt::tangle_testnet_runtime::api::runtime_types::tangle_primitives::services::PriceTargets;

// #[async_trait::async_trait]
// trait GadgetRunner {
// async fn register(&self) -> Result<()>;
// async fn run(&self) -> Result<()>;
// }

struct TangleGadgetRunner {
env: GadgetConfiguration<parking_lot::RawRwLock>,
}
Expand Down Expand Up @@ -73,7 +67,7 @@ impl GadgetRunner for TangleGadgetRunner {
todo!()
}

async fn run(&self) -> Result<()> {
async fn run(&mut self) -> Result<()> {
let client = self.env.client().await.map_err(|e| eyre!(e))?;
let signer = self.env.first_signer().map_err(|e| eyre!(e))?;
let logger = self.env.logger.clone();
Expand Down Expand Up @@ -124,7 +118,7 @@ async fn create_gadget_runner(
async fn main() -> Result<()> {
// Load the environment and create the gadget runner
// TODO: Place protocol in the config
let protocol = Protocol::Tangle;
let protocol = Protocol::Eigenlayer;
let config = ContextConfig::from_args();

let (env, mut runner) = create_gadget_runner(protocol, config.clone()).await;
Expand Down
Loading