- The tracer binary makes use of goblin, capstone and rustc-demangle to disassemble the riscV ELF and build a symbol table
- The execution trace for the rollup ELF is generated by running the
demo-proverwith theROLLUP_TRACEenvironment variable set to the desired path - The command should be run from the sovereign root repo
BLOCKS=1 TXNS_PER_BLOCK=10 ROLLUP_TRACE=rollup.trace cargo bench --bench prover_bench --features benchrollup.tracenow contains the program counter values for the run of prover_bench (how many ever blocks, blobs and transactions we configured it for)
$ find . -name rollup.trace
./examples/demo-rollup/rollup.trace- The corresponding ELF used for the trace is found at
$ find . -name mock_da
./target/riscv-guest/riscv32im-risc0-zkvm-elf/release/mock_da
$ file ./target/riscv-guest/riscv32im-risc0-zkvm-elf/release/mock_da
./target/riscv-guest/riscv32im-risc0-zkvm-elf/release/mock_da: ELF 32-bit LSB executable, UCB RISC-V, soft-float ABI, version 1 (SYSV), statically linked, with debug_info, not stripped- Both the above files are passed as input to the tracer
- cd into the tracer sub folder
$ cd utils/zk-cycle-utils/tracer/
$ cargo run --release -- --rollup-elf ../../../target/riscv-guest/riscv32im-risc0-zkvm-elf/release/mock_da --rollup-trace ../../../examples/demo-rollup/rollup.trace -h
Finished release [optimized] target(s) in 0.09s
Running `target/release/tracer --rollup-elf ../../../target/riscv-guest/riscv32im-risc0-zkvm-elf/release/mock_da --rollup-trace ../../../examples/demo-rollup/rollup.trace -h`
Usage: tracer [OPTIONS] --rollup-elf <ROLLUP_ELF> --rollup-trace <ROLLUP_TRACE>
Options:
-t, --top <TOP> Include the "top" number of functions [default: 30]
--no-stack-counts Don't print stack aware instruction counts
--no-raw-counts Don't print raw (stack un-aware) instruction counts
--rollup-elf <ROLLUP_ELF> Path to the riscv32 elf
--rollup-trace <ROLLUP_TRACE> Path to the rollup trace. File must be one u64 program counter per line
-s, --strip-hashes Strip the hashes from the function name while printing
-f, --function-name <FUNCTION_NAME> Function name to target for getting stack counts
-e, --exclude-view <EXCLUDE_VIEW> Exclude functions matching these patterns from display usage: -e func1 -e func2 -e func3
-h, --help Print help
-V, --version Print version
$
- Important note: The paths of
rollup.traceand themock_daELF used need to be adjusted to be reachable from the tracer folder
elf: ../../../target/riscv-guest/riscv32im-risc0-zkvm-elf/release/mock_da
trace: ../../../examples/demo-rollup/rollup.trace
- Running it with the minimum options produces 2 tables
$ cargo run --release -- --rollup-elf ../../../target/riscv-guest/riscv32im-risc0-zkvm-elf/release/mock_da --rollup-trace ../../../examples/demo-rollup/rollup.trace
Finished release [optimized] target(s) in 0.14s
Running `target/release/tracer --rollup-elf ../../../target/riscv-guest/riscv32im-risc0-zkvm-elf/release/mock_da --rollup-trace ../../../examples/demo-rollup/rollup.trace`
[00:00:07] [########################################] 11364536/11364536 (0s)
Total instructions in trace: 11364536
Instruction counts considering call graph
+--------------------------------------------------------------------------------------------+-------------------+
| Function Name | Instruction Count |
+============================================================================================+===================+
| __start | 12043541 |
+--------------------------------------------------------------------------------------------+-------------------+
| main | 12042478 |
+--------------------------------------------------------------------------------------------+-------------------+
| sov_modules_stf_blueprint::<impl | 10353814 |
| sov_rollup_interface::state_machine::stf::StateTransitionFunction<Vm,Da> for | |
| sov_modules_stf_blueprint::stf_blueprint::StfBlueprint<C,Da,Vm,RT,K>>::apply_slot::h97c58a | |
| f7db410f5f | |
+--------------------------------------------------------------------------------------------+-------------------+
| sov_modules_stf_blueprint::stf_blueprint::StfBlueprint<C,Da,Vm,RT,K>::apply_blob::he0c919b | 6780985 |
| cb192ee1f | |
+--------------------------------------------------------------------------------------------+-------------------+
| <sov_modules_api::default_signature::DefaultSignature as | 6098836 |
| sov_modules_core::common::key::Signature>::verify::h4001d1ebfb6fbfa5 | |
+--------------------------------------------------------------------------------------------+-------------------+
| <curve25519_dalek::backend::serial::risc0::field::FieldElementR0 as | 2191143 |
| core::ops::arith::AddAssign<&curve25519_dalek::backend::serial::risc0::field::FieldElement | |
| R0>>::add_assign::h803b2e4d149c8486 | |
+--------------------------------------------------------------------------------------------+-------------------+
| curve25519_dalek::backend::serial::curve_models::ProjectivePoint::double::hec30a33dc70ebee | 2130578 |
| c | |
+--------------------------------------------------------------------------------------------+-------------------+
.
.
.
+--------------------------------------------------------------------------------------------+-------------------+
Instruction counts ignoring call graph
+--------------------------------------------------------------------------------------------+-------------------+
| Function Name | Instruction Count |
+============================================================================================+===================+
| <curve25519_dalek::backend::serial::risc0::field::FieldElementR0 as | 2128983 |
| core::ops::arith::AddAssign<&curve25519_dalek::backend::serial::risc0::field::FieldElement | |
| R0>>::add_assign::h803b2e4d149c8486 | |
+--------------------------------------------------------------------------------------------+-------------------+
| memcpy | 1780887 |
+--------------------------------------------------------------------------------------------+-------------------+
| serde::de::impls::<impl serde::de::Deserialize for | 1584576 |
| alloc::vec::Vec<T>>::deserialize::hdea67e98bd7defc3 | |
+--------------------------------------------------------------------------------------------+-------------------+
| <sov_modules_api::default_signature::DefaultSignature as | 1226230 |
| sov_modules_core::common::key::Signature>::verify::h4001d1ebfb6fbfa5 | |
+--------------------------------------------------------------------------------------------+-------------------+
| sha2::sha256::compress256::h7aa33e4ad80197be | 944318 |
+--------------------------------------------------------------------------------------------+-------------------+
| curve25519_dalek::backend::serial::curve_models::ProjectivePoint::double::hec30a33dc70ebee | 916777 |
| c | |
+--------------------------------------------------------------------------------------------+-------------------+
| <T as jmt::SimpleHasher>::finalize::h16335de3188b5a1f | 444077 |
+--------------------------------------------------------------------------------------------+-------------------+
| sha2::sha512::compress512::hf06958a07bd42930 | 313566 |
+--------------------------------------------------------------------------------------------+-------------------+
.
.
.
+--------------------------------------------------------------------------------------------+-------------------+-
The complete tables have been omitted for readability
-
By default the tracer only picks the top 30 functions by count (This can be changed with a flag)
-
Instruction counts ignoring call graphonly considers the ELF and not the trace -
Instruction counts considering call graphanalyzes the trace by using the symbol tables built from analyzing the ELF -
The hashes mangled at the end of the function names can be removed using
-s, --strip-hashesoption (only for readability). For the next section regarding detailed stack traces however, we should provide the hash as well -
In order to see the detailed stack trace of a single function, it can be passed as an argument. Lets say we're interested in this particular function
<sov_modules_api::default_signature::DefaultSignature as sov_modules_core::common::key::Signature>::verify::h4001d1ebfb6fbfa5
- It can be passed in as an argument using
-f. Additionally, if we don't want to see the two primary tables, we can exclude them using--no-raw-counts --no-stack-counts
$ cargo run --release -- --rollup-elf ../../../target/riscv-guest/riscv32im-risc0-zkvm-elf/release/mock_da --rollup-trace ../../../examples/demo-rollup/rollup.trace -f "<sov_modules_api::default_signature::DefaultSignature as sov_modules_core::common::key::Signature>::verify::h4001d1ebfb6fbfa5" --no-raw-counts --no-stack-counts
[00:00:09] [########################################] 11364536/11364536 (0s)
Total instructions in trace: 11364536
Stack patterns for function '<sov_modules_api::default_signature::DefaultSignature as sov_modules_core::common::key::Signature>::verify::h4001d1ebfb6fbfa5'
+--------------------------------------------------------------------------------------------+-------------------+
| Function Stack | Instruction Count |
+============================================================================================+===================+
| (1) <sov_modules_api::default_signature::DefaultSignature as | 6098836 |
| sov_modules_core::common::key::Signature>::verify::h4001d1ebfb6fbfa5 | |
| (2) | |
| sov_modules_stf_blueprint::stf_blueprint::StfBlueprint<C,Da,Vm,RT,K>::apply_blob::he0c919b | |
| cb192ee1f | |
| (3) sov_modules_stf_blueprint::<impl | |
| sov_rollup_interface::state_machine::stf::StateTransitionFunction<Vm,Da> for | |
| sov_modules_stf_blueprint::stf_blueprint::StfBlueprint<C,Da,Vm,RT,K>>::apply_slot::h97c58a | |
| f7db410f5f | |
| (4) main | |
| (5) __start | |
+--------------------------------------------------------------------------------------------+-------------------+
- If we want to try the
memcpyfunction, since its used in a lot more places
cargo run --release -- --rollup-elf ../../../target/riscv-guest/riscv32im-risc0-zkvm-elf/release/mock_da --rollup-trace ../../../examples/demo-rollup/rollup.trace -f "memcpy" --no-raw-counts --no-stack-counts
Finished release [optimized] target(s) in 0.15s
Running `target/release/tracer --rollup-elf ../../../target/riscv-guest/riscv32im-risc0-zkvm-elf/release/mock_da --rollup-trace ../../../examples/demo-rollup/rollup.trace -f memcpy --no-raw-counts --no-stack-counts`
[00:00:07] [########################################] 11364536/11364536 (0s)
Total instructions in trace: 11364536
Stack patterns for function 'memcpy'
+--------------------------------------------------------------------------------------------+-------------------+
| Function Stack | Instruction Count |
+============================================================================================+===================+
| (1) memcpy | 506613 |
| (2) <sov_modules_api::default_signature::DefaultSignature as | |
| sov_modules_core::common::key::Signature>::verify::h4001d1ebfb6fbfa5 | |
| (3) | |
| sov_modules_stf_blueprint::stf_blueprint::StfBlueprint<C,Da,Vm,RT,K>::apply_blob::he0c919b | |
| cb192ee1f | |
| (4) sov_modules_stf_blueprint::<impl | |
| sov_rollup_interface::state_machine::stf::StateTransitionFunction<Vm,Da> for | |
| sov_modules_stf_blueprint::stf_blueprint::StfBlueprint<C,Da,Vm,RT,K>>::apply_slot::h97c58a | |
| f7db410f5f | |
| (5) main | |
| (6) __start | |
+--------------------------------------------------------------------------------------------+-------------------+
| (1) memcpy | 193956 |
| (2) jmt::types::proof::SparseMerkleNode::hash::hdd0011ca380b940b | |
| (3) jmt::types::proof::definition::SparseMerkleProof<H>::root_hash::h7ad79865c71ff315 | |
| (4) sov_modules_stf_blueprint::<impl | |
| sov_rollup_interface::state_machine::stf::StateTransitionFunction<Vm,Da> for | |
| sov_modules_stf_blueprint::stf_blueprint::StfBlueprint<C,Da,Vm,RT,K>>::apply_slot::h97c58a | |
| f7db410f5f | |
| (5) main | |
| (6) __start | |
+--------------------------------------------------------------------------------------------+-------------------+
| (1) memcpy | 137459 |
| (2) jmt::types::proof::SparseMerkleInternalNode::hash::h45df4fe2c5f50a03 | |
| (3) jmt::types::proof::definition::SparseMerkleProof<H>::root_hash::h7ad79865c71ff315 | |
| (4) sov_modules_stf_blueprint::<impl | |
| sov_rollup_interface::state_machine::stf::StateTransitionFunction<Vm,Da> for | |
| sov_modules_stf_blueprint::stf_blueprint::StfBlueprint<C,Da,Vm,RT,K>>::apply_slot::h97c58a | |
| f7db410f5f | |
| (5) main | |
| (6) __start | |
+--------------------------------------------------------------------------------------------+-------------------+
| (1) memcpy | 132932 |
| (2) <jmt::types::proof::definition::SparseMerkleProof<H> as | |
| borsh::de::BorshDeserialize>::deserialize_reader::h0ba54d52b47c6c43 | |
| (3) sov_modules_stf_blueprint::<impl | |
| sov_rollup_interface::state_machine::stf::StateTransitionFunction<Vm,Da> for | |
| sov_modules_stf_blueprint::stf_blueprint::StfBlueprint<C,Da,Vm,RT,K>>::apply_slot::h97c58a | |
| f7db410f5f | |
| (4) main | |
| (5) __start | |
+--------------------------------------------------------------------------------------------+-------------------+
| (1) memcpy | 98317 |
| (2) jmt::types::proof::SparseMerkleNode::hash::hdd0011ca380b940b | |
| (3) | |
| jmt::types::proof::definition::SparseMerkleProof<H>::verify_nonexistence::h9f642ad03e92295 | |
| 1 | |
| (4) sov_modules_stf_blueprint::<impl | |
| sov_rollup_interface::state_machine::stf::StateTransitionFunction<Vm,Da> for | |
| sov_modules_stf_blueprint::stf_blueprint::StfBlueprint<C,Da,Vm,RT,K>>::apply_slot::h97c58a | |
| f7db410f5f | |
| (5) main | |
| (6) __start | |
+--------------------------------------------------------------------------------------------+-------------------+
- The above call prints all the places memcpy is used in descending order of the cycles (There are more tables since memcpy is used in a lot of functions, but excluded for readability)
- This uses the
mock_daELF becauseprover_bench.rsruns using it, but the tracer can be used with any risc0 ELF and the trace that it produces - The ELF used to produce the trace must be the same pairing used with the tracer. Recompiling the ELF and using an older trace would cause inconsistencies in the symbol table and thus produce garbage
- TODO: We could store a hash of the ELF in the trace and use that to exit early instead of producing garbage
mock_dais built using the flags from Cargo.toml.- TODO: We might see different results if we build it in non-release mode so that the compiler won't do any aggressive inlining
- This could possibly help us get a more accurate representation of the stack traces that would directly map to our code
- Running in
--releasemode speeds up the tracer since it performs a lot of analyzsis (analyzing larger workloads can be slow if not used in --release) - TODO: profiler needs to be added to CI as well. Fortunately, this can be merged with the part where prover_bench is already run (Just passing ROLLUP_TRACE so that the bench also produces the trace) as an additional step, so it should only take a minute or two extra