Skip to content

Commit

Permalink
feat(sdk): latest network proto + error handling + fix examples (#1884)
Browse files Browse the repository at this point in the history
  • Loading branch information
mattstam authored Dec 20, 2024
1 parent 653e8c5 commit a1e3623
Show file tree
Hide file tree
Showing 43 changed files with 510 additions and 286 deletions.
6 changes: 3 additions & 3 deletions book/docs/generating-proofs/proof-types.md
Original file line number Diff line number Diff line change
Expand Up @@ -13,7 +13,7 @@ the size of the execution. Use this in settings where you don't care about **ver

```rust,noplayground
let client = ProverClient::from_env();
client.prove(&pk, stdin).run().unwrap();
client.prove(&pk, &stdin).run().unwrap();
```

## Compressed
Expand All @@ -23,7 +23,7 @@ care about **verification cost / proof size**, but not onchain verification. Com

```rust,noplayground
let client = ProverClient::from_env();
client.prove(&pk, stdin).compressed().run().unwrap();
client.prove(&pk, &stdin).compressed().run().unwrap();
```

## Groth16 (Recommended)
Expand All @@ -45,5 +45,5 @@ PLONK does not require a trusted setup and reuses contributions from the Aztec I

```rust,noplayground
let client = ProverClient::from_env();
client.prove(&pk, stdin).plonk().run().unwrap();
client.prove(&pk, &stdin).plonk().run().unwrap();
```
2 changes: 1 addition & 1 deletion book/docs/generating-proofs/prover-network/usage.md
Original file line number Diff line number Diff line change
Expand Up @@ -10,7 +10,7 @@ To use the prover network to generate a proof, you can run your script that uses
// Generate the proof for the given program.
let client = ProverClient::from_env();
let (pk, vk) = client.setup(ELF);
let mut proof = client.prove(&pk, stdin).run().unwrap();
let mut proof = client.prove(&pk, &stdin).run().unwrap();
```

```sh
Expand Down
20 changes: 13 additions & 7 deletions book/docs/writing-programs/cycle-tracking.mdx
Original file line number Diff line number Diff line change
Expand Up @@ -8,7 +8,7 @@ When writing a program, it is useful to know how many RISC-V cycles a portion of

To track the number of cycles spent in a portion of the program, you can either put `println!("cycle-tracker-start: block name")` + `println!("cycle-tracker-end: block name")` statements (block name must be same between start and end) around the portion of your program you want to profile or use the `#[sp1_derive::cycle_tracker]` macro on a function. An example is shown below:

<Example/>
<Example />

Note that to use the macro, you must add the `sp1-derive` crate to your dependencies for your program.

Expand Down Expand Up @@ -56,25 +56,28 @@ fn main() {

This will log the cycle count for `block name` and include it in the `ExecutionReport` in the `cycle_tracker` map.

### Profiling a ZKVM program
### Profiling a ZKVM program

Profiling a zkVM program produces a useful visualization ([example profile](https://share.firefox.dev/3Om1pzz)) which makes it easy to examine program performance and see exactly where VM cycles are being spent without needing to modify the program at all.

To profile a program, you need to:

1. Enable the profiling feature for `sp1-sdk` in `script/Cargo.toml`
2. Set the env variable `TRACE_FILE=trace.json` and then call `ProverClient::execute()` in your script.

If you're executing a larger program (>100M cycles), you should set `TRACE_SAMPLE_RATE` to reduce the size of the trace file. A sample rate of `1000` means that 1 in every 1000 VM cycles is sampled. By default, every cycle is sampled.

Many examples can be found in the repo, such as this ['fibonacci'](https://github.com/succinctlabs/sp1/blob/dev/examples/fibonacci/script/src/main.rs#L22) script.

Once you have your script it should look like the following:
```rs
Once you have your script it should look like the following:

```rs
// Execute the program using the `ProverClient.execute` method, without generating a proof.
let (_, report) = client.execute(ELF, stdin.clone()).run().unwrap();
let (_, report) = client.execute(ELF, &stdin).run().unwrap();
```

As well you must enable the profiling feature on the SDK:

```toml
sp1-sdk = { version = "3.0.0", features = ["profiling"] }
```
Expand All @@ -83,11 +86,13 @@ The `TRACE_FILE` env var tells the executor where to save the profile, and the `
A larger sample rate will give you a smaller profile, it is the number of instructions in between each sample.

The full command to profile should look something like this

```sh
TRACE_FILE=output.json TRACE_SAMPLE_RATE=100 cargo run ...
```

To view these profiles, we recommend [Samply](https://github.com/mstange/samply).

```sh
cargo install --locked samply
samply load output.json
Expand All @@ -97,7 +102,8 @@ Samply uses the Firefox profiler to create a nice visualization of your programs
![An example screenshot of the Firefox Profiler](@site/static/profiling.png)

#### Interpreting the Profile
- The "time" measurement in the profiler is actually the number of cycles spent,
in general the less cycles for a given callframe the better.

- The "time" measurement in the profiler is actually the number of cycles spent,
in general the less cycles for a given callframe the better.

- The CPU usage of the program will always be constant, as its running in the VM which is single threaded.
Original file line number Diff line number Diff line change
Expand Up @@ -16,7 +16,7 @@ fn main() {
// Generate the constant-sized proof for the given program and input.
let client = ProverClient::new();
let (pk, vk) = client.setup(ELF);
let mut proof = client.prove(&pk, stdin).compressed().run().unwrap();
let mut proof = client.prove(&pk, &stdin).compressed().run().unwrap();

println!("generated proof");
// Read and verify the output.
Expand Down
2 changes: 1 addition & 1 deletion book/static/examples_fibonacci_script_bin_execute.rs.mdx
Original file line number Diff line number Diff line change
Expand Up @@ -16,7 +16,7 @@ fn main() {

// Only execute the program and get a `SP1PublicValues` object.
let client = ProverClient::new();
let (mut public_values, execution_report) = client.execute(ELF, stdin).run().unwrap();
let (mut public_values, execution_report) = client.execute(ELF, &stdin).run().unwrap();

// Print the total number of cycles executed and the full execution report with a breakdown of
// the RISC-V opcode and syscall counts.
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -20,7 +20,7 @@ fn main() {
println!("vk: {:?}", vk.bytes32());

// Generate the Groth16 proof.
let proof = client.prove(&pk, stdin).groth16().run().unwrap();
let proof = client.prove(&pk, &stdin).groth16().run().unwrap();
println!("generated proof");

// Get the public values as bytes.
Expand Down
4 changes: 2 additions & 2 deletions book/static/examples_fibonacci_script_src_main.rs.mdx
Original file line number Diff line number Diff line change
Expand Up @@ -20,12 +20,12 @@ fn main() {
let client = ProverClient::new();

// Execute the program using the `ProverClient.execute` method, without generating a proof.
let (_, report) = client.execute(ELF, stdin.clone()).run().unwrap();
let (_, report) = client.execute(ELF, &stdin).run().unwrap();
println!("executed program with {} cycles", report.total_instruction_count());

// Generate the proof for the given program and input.
let (pk, vk) = client.setup(ELF);
let mut proof = client.prove(&pk, stdin).run().unwrap();
let mut proof = client.prove(&pk, &stdin).run().unwrap();

println!("generated proof");

Expand Down
4 changes: 2 additions & 2 deletions book/static/examples_groth16_script_src_main.rs.mdx
Original file line number Diff line number Diff line change
Expand Up @@ -29,7 +29,7 @@ fn generate_fibonacci_proof() -> (Vec<u8>, Vec<u8>, String) {
// Generate the groth16 proof for the Fibonacci program.
let (pk, vk) = client.setup(FIBONACCI_ELF);
println!("vk: {:?}", vk.bytes32());
let proof = client.prove(&pk, stdin).groth16().run().unwrap();
let proof = client.prove(&pk, &stdin).groth16().run().unwrap();
(proof.bytes(), proof.public_values.to_vec(), vk.bytes32())
}

Expand All @@ -50,7 +50,7 @@ fn main() {
let client = ProverClient::new();

// Execute the program using the `ProverClient.execute` method, without generating a proof.
let (_, report) = client.execute(GROTH16_ELF, stdin.clone()).run().unwrap();
let (_, report) = client.execute(GROTH16_ELF, &stdin).run().unwrap();
println!("executed groth16 program with {} cycles", report.total_instruction_count());
println!("{}", report);
}
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -13,7 +13,7 @@ the size of the execution. Use this in settings where you don't care about **ver

```rust
let client = ProverClient::new();
client.prove(&pk, stdin).run().unwrap();
client.prove(&pk, &stdin).run().unwrap();
```

## Compressed
Expand All @@ -23,7 +23,7 @@ care about **verification cost / proof size**, but not onchain verification. Com

```rust
let client = ProverClient::new();
client.prove(&pk, stdin).compressed().run().unwrap();
client.prove(&pk, &stdin).compressed().run().unwrap();
```

## Groth16 (Recommended)
Expand All @@ -34,7 +34,7 @@ The trusted setup for the Groth16 circuit keys uses the [Aztec Ignition ceremony

```rust
let client = ProverClient::new();
client.prove(&pk, stdin).groth16().run().unwrap();
client.prove(&pk, &stdin).groth16().run().unwrap();
```

## PLONK
Expand All @@ -45,5 +45,5 @@ PLONK does not require a trusted setup.

```rust
let client = ProverClient::new();
client.prove(&pk, stdin).plonk().run().unwrap();
client.prove(&pk, &stdin).plonk().run().unwrap();
```
Original file line number Diff line number Diff line change
Expand Up @@ -10,7 +10,7 @@ To use the prover network to generate a proof, you can run your script that uses
// Generate the proof for the given program.
let client = ProverClient::new();
let (pk, vk) = client.setup(ELF);
let mut proof = client.prove(&pk, stdin).run().unwrap();
let mut proof = client.prove(&pk, &stdin).run().unwrap();
```

```sh
Expand Down
4 changes: 2 additions & 2 deletions crates/perf/src/main.rs
Original file line number Diff line number Diff line change
Expand Up @@ -177,13 +177,13 @@ fn main() {
let (_, _) = time_operation(|| prover.execute(&elf, &stdin));

let (proof, _) = time_operation(|| {
prover.prove(&pk, stdin.clone()).groth16().skip_simulation(true).run().unwrap()
prover.prove(&pk, &stdin).groth16().skip_simulation(true).run().unwrap()
});

let (_, _) = time_operation(|| prover.verify(&proof, &vk));

let (proof, _) = time_operation(|| {
prover.prove(&pk, stdin).plonk().skip_simulation(true).run().unwrap()
prover.prove(&pk, &stdin).plonk().skip_simulation(true).run().unwrap()
});

let (_, _) = time_operation(|| prover.verify(&proof, &vk));
Expand Down
60 changes: 52 additions & 8 deletions crates/sdk/src/network/client.rs
Original file line number Diff line number Diff line change
Expand Up @@ -20,14 +20,14 @@ use tonic::{

use super::utils::Signable;
use crate::network::proto::artifact::{
artifact_store_client::ArtifactStoreClient, CreateArtifactRequest,
artifact_store_client::ArtifactStoreClient, ArtifactType, CreateArtifactRequest,
};
use crate::network::proto::network::{
prover_network_client::ProverNetworkClient, CreateProgramRequest, CreateProgramRequestBody,
CreateProgramResponse, FulfillmentStatus, FulfillmentStrategy, GetNonceRequest,
GetProgramRequest, GetProgramResponse, GetProofRequestStatusRequest,
GetProofRequestStatusResponse, MessageFormat, ProofMode, RequestProofRequest,
RequestProofRequestBody, RequestProofResponse,
CreateProgramResponse, FulfillmentStatus, FulfillmentStrategy, GetFilteredProofRequestsRequest,
GetFilteredProofRequestsResponse, GetNonceRequest, GetProgramRequest, GetProgramResponse,
GetProofRequestStatusRequest, GetProofRequestStatusResponse, MessageFormat, ProofMode,
RequestProofRequest, RequestProofRequestBody, RequestProofResponse,
};

/// A client for interacting with the network.
Expand Down Expand Up @@ -105,7 +105,8 @@ impl NetworkClient {
) -> Result<CreateProgramResponse> {
// Create the program artifact.
let mut store = self.artifact_store_client().await?;
let program_uri = self.create_artifact_with_content(&mut store, &elf).await?;
let program_uri =
self.create_artifact_with_content(&mut store, ArtifactType::Program, &elf).await?;

// Serialize the verifying key.
let vk_encoded = bincode::serialize(&vk)?;
Expand All @@ -130,6 +131,44 @@ impl NetworkClient {
.into_inner())
}

/// Get all the proof requests that meet the filter criteria.
#[allow(clippy::too_many_arguments)]
pub async fn get_filtered_proof_requests(
&self,
version: Option<String>,
fulfillment_status: Option<i32>,
execution_status: Option<i32>,
minimum_deadline: Option<u64>,
vk_hash: Option<Vec<u8>>,
requester: Option<Vec<u8>>,
fulfiller: Option<Vec<u8>>,
from: Option<u64>,
to: Option<u64>,
limit: Option<u32>,
page: Option<u32>,
mode: Option<i32>,
) -> Result<GetFilteredProofRequestsResponse> {
let mut rpc = self.prover_network_client().await?;
let res = rpc
.get_filtered_proof_requests(GetFilteredProofRequestsRequest {
version,
fulfillment_status,
execution_status,
minimum_deadline,
vk_hash,
requester,
fulfiller,
from,
to,
limit,
page,
mode,
})
.await?
.into_inner();
Ok(res)
}

/// Get the status of a given proof.
///
/// # Details
Expand Down Expand Up @@ -190,7 +229,8 @@ impl NetworkClient {

// Create the stdin artifact.
let mut store = self.artifact_store_client().await?;
let stdin_uri = self.create_artifact_with_content(&mut store, &stdin).await?;
let stdin_uri =
self.create_artifact_with_content(&mut store, ArtifactType::Stdin, &stdin).await?;

// Send the request.
let mut rpc = self.prover_network_client().await?;
Expand Down Expand Up @@ -248,10 +288,14 @@ impl NetworkClient {
pub(crate) async fn create_artifact_with_content<T: Serialize>(
&self,
store: &mut ArtifactStoreClient<Channel>,
artifact_type: ArtifactType,
item: &T,
) -> Result<String> {
let signature = self.signer.sign_message_sync("create_artifact".as_bytes())?;
let request = CreateArtifactRequest { signature: signature.as_bytes().to_vec() };
let request = CreateArtifactRequest {
artifact_type: artifact_type.into(),
signature: signature.as_bytes().to_vec(),
};
let response = store.create_artifact(request).await?.into_inner();

let presigned_url = response.artifact_presigned_url;
Expand Down
39 changes: 39 additions & 0 deletions crates/sdk/src/network/error.rs
Original file line number Diff line number Diff line change
@@ -0,0 +1,39 @@
use thiserror::Error;
use tonic::Status;

/// An error that can occur when interacting with the prover network.
#[derive(Error, Debug)]
pub enum Error {
/// The program execution failed.
#[error("Program simulation failed")]
SimulationFailed,

/// The proof request is unexecutable.
#[error("Proof request 0x{} is unexecutable", hex::encode(.request_id))]
RequestUnexecutable {
/// The ID of the request that cannot be executed.
request_id: Vec<u8>,
},

/// The proof request is unfulfillable.
#[error("Proof request 0x{} is unfulfillable", hex::encode(.request_id))]
RequestUnfulfillable {
/// The ID of the request that cannot be fulfilled.
request_id: Vec<u8>,
},

/// The proof request timed out.
#[error("Proof request 0x{} timed out", hex::encode(.request_id))]
RequestTimedOut {
/// The ID of the request that timed out.
request_id: Vec<u8>,
},

/// An error occurred while interacting with the RPC server.
#[error("RPC error")]
RpcError(#[from] Status),

/// An unknown error occurred.
#[error("Other error: {0}")]
Other(#[from] anyhow::Error),
}
13 changes: 11 additions & 2 deletions crates/sdk/src/network/mod.rs
Original file line number Diff line number Diff line change
Expand Up @@ -4,19 +4,28 @@
pub mod client;
pub mod prover;
mod sign_message;
#[rustfmt::skip]
#[allow(missing_docs)]
#[allow(clippy::default_trait_access)]
#[allow(clippy::too_many_lines)]
pub mod proto;
pub mod builder;
mod error;
pub mod prove;
pub mod utils;

pub use error::*;

pub use crate::network::client::NetworkClient;
pub use crate::network::proto::network::FulfillmentStrategy;

pub(crate) const DEFAULT_PROVER_NETWORK_RPC: &str = "https://rpc.production.succinct.tools/";
/// The default RPC URL for the prover network.
pub(crate) const DEFAULT_NETWORK_RPC_URL: &str = "https://rpc.production.succinct.tools/";

/// The default timeout for the prover network (4 hours).
pub(crate) const DEFAULT_TIMEOUT_SECS: u64 = 14400;

/// The default cycle limit for the prover network (100M cycles).
///
/// This will only be used if both simulation is skipped and the cycle limit is not explicitly set.
pub(crate) const DEFAULT_CYCLE_LIMIT: u64 = 100_000_000;
Loading

0 comments on commit a1e3623

Please sign in to comment.