Skip to content

Commit

Permalink
Removing Covalent API and retrieving balances through provider (#116)
Browse files Browse the repository at this point in the history
  • Loading branch information
sifnoc authored Aug 2, 2023
1 parent c18cc23 commit 7e39b84
Show file tree
Hide file tree
Showing 12 changed files with 238 additions and 392 deletions.
1 change: 0 additions & 1 deletion .github/workflows/rust.yml
Original file line number Diff line number Diff line change
Expand Up @@ -29,7 +29,6 @@ jobs:

- name: Test backend
env:
COVALENT_API_KEY: ${{ secrets.COVALENT_API_KEY }}
SIGNATURE_VERIFICATION_MESSAGE: "Summa proof of solvency for CryptoExchange"
run: |
cd backend
Expand Down
57 changes: 39 additions & 18 deletions backend/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -12,42 +12,63 @@ Furthermore, the `Snapshot` struct contains the following methods:

- `generate_solvency_verifier` -> write the Solidity Verifier contract (for the `SolvencyProof`) to a file
- `generate_proof_of_solvency` -> generate the `SolvencyProof` for the current snapshot to be verified on-chain
- `generate_inclusion_proof` -> generate the `MstInclusionProof` for a specific user for the current snapshot to be verified off-chain
- `get_account_onwership_proof` -> generate the `AccountOwnership` for a specific user for the current snapshot to be verified off-chain
- `generate_proof_of_inclusion` -> generate the `MstInclusionProof` for a specific user for the current snapshot to be verified off-chain
- `get_proof_of_account_ownership` -> generate the `AccountOwnership` for a specific user for the current snapshot to be verified off-chain

## Prerequisites

In order to initialize the Snapshot, you need to download the Powers of Tau files. These are the trusted setup parameters needed to build the zk circuits. You can find such files at https://github.com/han0110/halo2-kzg-srs, download it
The `ptau` file, containing the Powers of Tau trusted setup parameters needed to build the zk circuits, is already included. However, if you wish to test or run the code with a higher number of entries, you may choose to download a different `ptau` file.

You can find the necessary files at https://github.com/han0110/halo2-kzg-srs. To download a specific file, you can use:

```
wget https://trusted-setup-halo2kzg.s3.eu-central-1.amazonaws.com/hermez-raw-11
```

and pass the path to the file to the `Snapshot::new` method.
After downloading, pass the path to the desired file to the `Snapshot::new` method. If you are using the included `ptau` file, no additional steps are necessary.

Furthermore, the `generate_proof_of_solvency` method requires to fetch data about the balances of the wallets of the CEX. This data is fetched using the Covalent API. In order to use this method, you need to create an `.env` file and store the `COVALENT_API_KEY` there. You can get an API key at https://www.covalenthq.com/platform/.
## Important Notices

## Usage
### For Proof of Solvency

To build the binary executable and test it
As of the current implementation, the `generate_proof_of_solvency` method does not directly fetch data about the balances of the wallets of the CEX. Instead, you can use the `fetch_asset_sums` function to retrieve balance information from the blockchain. Here's an example of how you might utilize it:

```Rust
let asset_sums = fetch_asset_sums(client, token_contracts, exchange_addresses).await?;
```
cargo build
SIGNATURE_VERIFICATION_MESSAGE="Summa proof of solvency for CryptoExchange" cargo test --release -- --nocapture

Please note that the first element in the `asset_sums` array represents the ETH balance.

Alternatively, you can create your own custom fetcher to retrieve the balances.

### For Proof of Ownership

To generate a signed message, you must first initialize the `SummaSigner` and use the `generate_signatures` method:

```Rust
let signatures = signer.generate_signatures().await.unwrap();
```

The contract Rust interfaces are built by the [buildscript](./build.rs) from the JSON ABI. The [Summa contract ABI json](./src/contracts/Summa.json) is updated when the contract is deployed from the [contracts subproject](./../contracts/README.md).
The content of the message can be specified with the local variable `SIGNATURE_VERIFICATION_MESSAGE`.

## Example
### For Generating Solvency Verifier

In the `example/verifying_inclusion.rs`, you can find code for verifying the `inclusion_proof` on the user side. On top of that the user has to verify that:
- the public input `leaf_hash` matches the combination `H(username, balances[])`
- the public input `root_hash` matches the commited by the CEX
The provided verifier found at `src/contracts/Verifier.json` is based on the trusted setup, `hermez-raw-11`. If you are working with a higher number of entries, you will need to generate a new verifier contract by using the `generate_solvency_verifier` method.

You can run the command like this:
Here's a brief example of how you might invoke this method:
```Rust
Snapshot::generate_solvency_verifier("SolvencyVerifier.yul", "SolvencyVerifier.sol");
```

This method creates two files, `SolvencyVerifier.yul` and `SolvencyVerifier.sol`, which will be used in `Summa.sol`.

`> cargo run --release --example verifying_inclusion`
## Usage

To build the binary executable and test it

The example will generate a proof for the user at index 0, and then it will pass only the `proof_inclusion` data to the user side (within the same example file).
```
cargo build
SIGNATURE_VERIFICATION_MESSAGE="Summa proof of solvency for CryptoExchange" cargo test --release -- --nocapture
```

On the user side, the proof is verified using as public input the `leaf_hash` that is generated by the user themselves.
The contract Rust interfaces are built by the [buildscript](./build.rs) from the JSON ABI. The [Summa contract ABI json](./src/contracts/Summa.json) is updated when the contract is deployed from the [contracts subproject](./../contracts/README.md).
17 changes: 10 additions & 7 deletions backend/build.rs
Original file line number Diff line number Diff line change
Expand Up @@ -7,21 +7,22 @@ fn main() {
.unwrap()
.join("src/contracts/generated/summa_contract.rs");
if contract_out_file.exists() {
std::fs::remove_file(&contract_out_file);
std::fs::remove_file(&contract_out_file).unwrap();
}

Abigen::new("Summa", "./src/contracts/Summa.json")
.unwrap()
.format(true)
.generate()
.unwrap()
.write_to_file(contract_out_file);
.write_to_file(contract_out_file)
.unwrap();

let mod_out_file: PathBuf = std::env::current_dir()
.unwrap()
.join("src/contracts/generated/mod.rs");
if mod_out_file.exists() {
std::fs::remove_file(&mod_out_file);
std::fs::remove_file(&mod_out_file).unwrap();
}

let mut mod_file = OpenOptions::new()
Expand All @@ -38,27 +39,29 @@ fn main() {
.unwrap()
.join("src/contracts/generated/mock_erc20.rs");
if contract_out_file.exists() {
std::fs::remove_file(&contract_out_file);
std::fs::remove_file(&contract_out_file).unwrap();
}

Abigen::new("MockERC20", "./src/contracts/MockERC20.json")
.unwrap()
.format(true)
.generate()
.unwrap()
.write_to_file(contract_out_file);
.write_to_file(contract_out_file)
.unwrap();

let contract_out_file = std::env::current_dir()
.unwrap()
.join("src/contracts/generated/verifier.rs");
if contract_out_file.exists() {
std::fs::remove_file(&contract_out_file);
std::fs::remove_file(&contract_out_file).unwrap();
}

Abigen::new("SolvencyVerifier", "./src/contracts/Verifier.json")
.unwrap()
.format(true)
.generate()
.unwrap()
.write_to_file(contract_out_file);
.write_to_file(contract_out_file)
.unwrap();
}
196 changes: 65 additions & 131 deletions backend/src/apis/fetch.rs
Original file line number Diff line number Diff line change
@@ -1,151 +1,85 @@
use base64::encode;
use reqwest::header::{HeaderMap, HeaderValue, AUTHORIZATION, CONTENT_TYPE};
use serde::Deserialize;
use std::error::Error;

#[derive(Deserialize, Debug)]
/// This is the root level of the JSON response. It contains a data field, which corresponds to the "data" field in the JSON.
struct Response {
data: Data,
}
/// This struct represents the "data" field of the JSON. It contains an items field, which corresponds to the "items" array in the JSON.
#[derive(Deserialize, Debug)]
struct Data {
items: Vec<Item>,
}

/// This struct represents each object in the "items" array. It contains a balance field and a contract_address field. These fields correspond to the "balance" and "contract_address" fields in each item of the "items" array in the JSON.
#[derive(Deserialize, Debug)]
struct Item {
balance: String,
contract_address: String,
use futures::future::try_join_all;
use std::{error::Error, sync::Arc};

use ethers::{
abi::Address,
contract::builders::ContractCall,
prelude::SignerMiddleware,
providers::{Http, Middleware, Provider},
signers::LocalWallet,
types::{H160, U256},
};

use crate::contracts;

pub trait TokenBalance<M: Middleware> {
fn get_token_balance(&self, account: Address) -> ContractCall<M, U256>;
}

/// This function takes a list of asset contracts addresses, a list of addresses and returns the aggregated balance of these address PER EACH asset
pub fn fetch_asset_sums(
addresses: Vec<String>,
asset_contract_addresses: Vec<String>,
) -> Result<Vec<u64>, Box<dyn Error>> {
// create asset sums vector
let mut asset_sums: Vec<u64> = Vec::new();

// for each address in addresses vector call fetch_balances_per_addr and increment the asset_sums vector
for address in addresses {
let balances = fetch_balances_per_addr(address.clone(), asset_contract_addresses.clone())?;
for (i, balance) in balances.iter().enumerate() {
if asset_sums.len() <= i {
asset_sums.push(*balance);
} else {
let sum = asset_sums[i] + balance;
asset_sums[i] = sum;
}
}
/// This function takes a list of token contracts, addresses and returns the balances of that address for the queried contracts.
/// The first balance returned is the Ether (ETH) balance, followed by the balances of other specified token contracts.
///
pub async fn fetch_asset_sums<'a, M: Middleware + 'a>(
client: Arc<SignerMiddleware<Provider<Http>, LocalWallet>>,
token_contracts: Vec<Box<dyn TokenBalance<M> + Send>>,
exchange_addresses: Vec<H160>,
) -> Result<Vec<U256>, Box<dyn Error>> {
let mut result: Vec<U256> = Vec::new();

let mut get_balance_futures = Vec::new();
for addr in exchange_addresses.clone() {
get_balance_futures.push(client.get_balance(addr, None));
}

Ok(asset_sums)
}

/// This function takes a list of token contracts, an address and returns the balances of that address for the queried contracts.
#[tokio::main]
async fn fetch_balances_per_addr(
address: String,
asset_contract_addresses: Vec<String>,
) -> Result<Vec<u64>, Box<dyn Error>> {
// Create a header map
let mut headers = HeaderMap::new();

// Load .env file
dotenv::dotenv().ok();

// Access API key from the environment
let api_key = std::env::var("COVALENT_API_KEY").unwrap();

// Add `Content-Type` header
headers.insert(CONTENT_TYPE, HeaderValue::from_static("application/json"));

// Add `Authorization` header
let auth_value = format!("Basic {}", encode(api_key));
headers.insert(AUTHORIZATION, HeaderValue::from_str(&auth_value)?);

// Form URL
let url = format!(
"https://api.covalenthq.com/v1/eth-mainnet/address/{}/balances_v2/",
address
);

// Send a GET request to the API
let res = reqwest::Client::new()
.get(&url)
.headers(headers)
.send()
.await?
.json::<Response>()
.await?;

// Get balances for the specific tokens
let filtered_balances =
filter_balances_by_token_contracts(asset_contract_addresses, &res).unwrap();

Ok(filtered_balances)
}

/// This function filters out only the balances corresponding to an asset listed in asset_contract_addresses.
fn filter_balances_by_token_contracts(
asset_contract_addresses: Vec<String>,
response: &Response,
) -> Result<Vec<u64>, &'static str> {
let mut balances = Vec::new();
for contract_adddress in asset_contract_addresses {
if let Some(item) = response.data.items.iter().find(|&item| {
item.contract_address.to_ascii_lowercase() == contract_adddress.to_ascii_lowercase()
}) {
match item.balance.parse::<u64>() {
Ok(num) => balances.push(num),
Err(e) => println!("Failed to parse string: {}", e),
}
} else {
balances.push(0 as u64);
let get_balances = try_join_all(get_balance_futures).await?;
let sum_eth_balance = get_balances
.into_iter()
.reduce(|acc, balance| acc + balance)
.unwrap();
result.push(sum_eth_balance);

// Most in case, the number of contracts is less than the number of asset addresses
// Iterating over contracts first is more efficient
let mut sum_token_balance = U256::zero();
for contract in &token_contracts {
for addr in exchange_addresses.clone() {
let token_balance = contract.get_token_balance(addr).call().await.unwrap();
sum_token_balance = sum_token_balance + token_balance;
}
result.push(sum_token_balance)
}
Ok(balances)

Ok(result)
}

#[cfg(test)]
mod tests {
use super::*;

#[test]
#[ignore]
fn test_fetch_balances_per_addr() {
// this is an address with 0 usdc and 0.010910762665574143 ETH
let address = "0xe4D9621321e77B499392801d08Ed68Ec5175f204".to_string();
use crate::contracts::tests::initialize_anvil;
use contracts::generated::mock_erc20::MockERC20;

let usdc_contract = "0xa0b86991c6218b36c1d19d4a2e9eb0ce3606eb48".to_string();
let eth_contract = "0xeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeee".to_string();

let balances =
fetch_balances_per_addr(address, [usdc_contract, eth_contract].to_vec()).unwrap();

assert_eq!(balances[0], 0); // balance for usdc
assert_eq!(balances[1], 10910762665574143); // balance for wei
}

#[test]
#[ignore]
fn test_fetch_asset_sums() {
let address_1 = "0xe4D9621321e77B499392801d08Ed68Ec5175f204".to_string(); // this is an address with 0 usdc and 0.010910762665574143 ETH
let address_2 = "0x90F8bf6A479f320ead074411a4B0e7944Ea8c9C1".to_string(); // this is an address with 0.000001 USDC usdc and 1 wei
#[tokio::test]
async fn test_fetch_asset_sums() {
// Necessary to implement `get_balance_from_contract` for the `contracts` parameter by following trait
impl<M: Middleware> TokenBalance<M> for MockERC20<M> {
fn get_token_balance(&self, account: Address) -> ContractCall<M, U256> {
self.balance_of(account)
}
}

let usdc_contract = "0xA0b86991c6218b36c1d19D4a2e9Eb0cE3606eB48".to_string();
let eth_contract = "0xeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeee".to_string();
let (anvil, cex_addr_1, cex_addr_2, client, mock_erc20) = initialize_anvil().await;

let asset_sums = fetch_asset_sums(
[address_1, address_2].to_vec(),
[usdc_contract, eth_contract].to_vec(),
client.clone(),
vec![Box::new(mock_erc20)],
[cex_addr_1, cex_addr_2].to_vec(),
)
.await
.unwrap();

assert_eq!(asset_sums[0], 1); // asset sum for usdc
assert_eq!(asset_sums[1], 10910762665574144); // asset sum for wei
assert_eq!(asset_sums[0], U256::from(556864));
assert_eq!(asset_sums[1], U256::from(556863));

drop(anvil);
}
}
Loading

0 comments on commit 7e39b84

Please sign in to comment.