Skip to content

Commit

Permalink
Feat/dockerized application for local testing (#11)
Browse files Browse the repository at this point in the history
* added blocks verification logic tested on single mmr batch

* feat(publisher): implement block validity verification using STARK proofs

Add core functionality to verify L2 block headers using STARK proofs:
- Implement proof generation and verification logic
- Add error handling for image_id conversions
- Support optional proof generation skipping for testing
- Include test binary for verification flow validation

This change provides the foundational logic for verifying block header
validity and inclusion using zero-knowledge proofs, to be used by other
components of the system.

* Improve error messages for better clarity and context

- Enhanced error descriptions in PublisherError, AccumulatorError, and ValidatorError
- Added more detailed context and potential causes for errors
- Improved consistency in error message formatting
- Added debugging suggestions for critical errors
- Included potential causes for EmptyHeaders and InvalidProofsCount errors

The improved error messages provide better context for debugging and make
it easier to understand and resolve issues when they occur. Each error
now includes more specific information about what went wrong and, where
applicable, suggestions about potential causes or solutions.

* chore: cleaup logs

* feat(logging): enhance tracing across proof generation pipeline

- Add structured logging with spans for better context tracking
- Improve error logging with detailed context and error messages
- Add high-level progress tracking for proof generation
- Implement consistent logging patterns across MMR state management
- Optimize log verbosity to focus on key operational events
- Add debug logs for detailed troubleshooting capabilities

* Enhance input validation, error handling, and tracing

- Added input validation to prevent invalid states and improve robustness.
- Improved error handling by providing specific error messages and using error variants.
- Enhanced tracing with spans and detailed error messages for better debugging.
- Applied changes to ValidatorBuilder, BatchProcessor, MMRStateManager, and ProofGenerator.
- Ensured no panics occur from invalid inputs and all error cases are properly handled.

* feat(starknet-handler): enhance tracing and logging

- Add structured logging with tracing macros across all modules
- Implement span instrumentation for key operations
- Add debug and info level logs for important state changes
- Include meaningful context in log messages
- Skip sensitive data in instrumentation
- Add logging for:
  - MMR state operations
  - Account creation and verification
  - Provider RPC calls
  - Hex conversions

This improves observability and debugging capabilities across the
starknet-handler crate.

* fmt

* feat(validator): implement onchain MMR root verification

- Add verification of MMR roots against onchain state
- Extract MMR root verification into separate methods for better modularity:
  - verify_mmr_roots: orchestrates the verification process
  - verify_single_mmr_root: handles individual MMR root verification
- Create mapping between batch indexes and onchain roots for efficient lookup
- Add skip_proof option to bypass verification during testing
- Add tracing info for both verification and skip scenarios

This change ensures MMR roots match their onchain counterparts,
maintaining data integrity between L1 and L2. The skip_proof
option facilitates testing and development workflows.

* fix(publisher): correct InvalidBlockRange error construction

- Update error construction to use struct variant syntax with named fields
- Align with AccumulatorError enum definition where InvalidBlockRange expects
  {start_block, end_block} fields
- Fix compiler error in get_block_headers_by_block_range method

* fix: improve block integrity verification

- Fix hex conversion handling in starknet-handler for U256 values
- Update dependencies in common and ethereum crates
- Enhance proof generator validation checks
- Improve accumulator state handling
- Update messaging configuration

* refactor: reduce verbosity in light client tracing logs

- Simplify log messages to focus on essential information
- Remove redundant context from log statements
- Maintain critical state information in error scenarios

* refactor: remove unused docker images for anvil, ethereum, katana, and starknet

* cilppy

* ci: add Cairo workflow

* cairo ci fix

* ci fix

* feat(docker): add Dockerfiles for client and server

- Add Dockerfile.client for building the client binary
- Configure vendored dependencies for offline builds
- Set up multi-stage build for minimal runtime image
- Add proper file permissions and ownership handling

* feat(starknet): add garaga as git submodule

* chore(docker): setup local deployment infrastructure

WIP: client Dockerfile needs fixing for vendored dependencies

* feat: add build-mmr container

- Add new Dockerfile.build-mmr for building and running build-mmr binary
- Install Foundry and required dependencies in container
- Update docker-compose.yml to use new build-mmr container
- Add .env.local.docker to gitignore
- Configure container to use existing Anvil instance

* misc fixes

* rolled back starknet contracts to prev state

* removed debug logging

* feat: add retry mechanism for forge script deployment

- Add retry_command function with exponential backoff
- Implement retries for forge script to handle Anvil startup delays
- Add better error messaging for deployment attempts

Fixes issue with deployment failing due to Anvil not being ready

* fix(docker): resolve cargo patch resolution in offline mode

- Simplify cargo config to handle crates-io and vendored sources
- Update patch section to explicitly target crates.io-index
- Fix client binary path and volume mount conflicts
- Add cargo fetch step for git dependencies

Resolves build error with sha2 patch resolution and client binary location

* fix(ethereum): handle Anvil provider timeout gracefully

- Replace unwrapping provider setup with proper error handling
- Use try_on_anvil_with_wallet_and_config to return Result
- Add detailed error messages for retry mechanism
- Improve error logging for debugging purposes

Resolves panic from Anvil timeout errors by implementing proper
retry logic and error handling.

* refactor(docker): switch relayer to Alpine Linux base image

- Replace Ubuntu base image with Alpine Linux 3.19
- Simplify package installation using apk
- Remove GPG key management steps
- Use Alpine's SSL libraries for runtime dependencies

* refactor(docker): align relayer Dockerfile with client pattern

- Copy binary to /usr/local/bin instead of working directory
- Add executable permissions with chmod
- Switch to CMD from ENTRYPOINT
- Follow consistent pattern with client Dockerfile

* chore(docker): verify relayer service configuration

- Confirm command matches Dockerfile CMD instruction
- Ensure consistent service definition with other containers
- Maintain existing dependency chain

* fix(docker): optimize relayer build process

- Create minimal workspace with only required crates
- Configure static OpenSSL linking with openssl-libs-static
- Add necessary build dependencies (gcc, pkgconfig)
- Use specific Alpine mirror for reliability
- Fix dependency resolution by creating minimal workspace
- Optimize multi-stage build process

* chore ignore

* Clean up gitmodules

* Remove garaga from index

* Add garaga submodule at commit 5f3b232

* refactor: switch from bash to sh in Docker environment

- Modified Dockerfile.relayer to use sh shell
- Updated docker-compose.yml to use /bin/sh instead of /bin/bash
- Converted run_relayer.sh to use POSIX-compliant shell syntax

This change reduces the container size by avoiding the need to install bash
and makes the setup more compatible with Alpine Linux's default shell.

* format starknet contracts

* removed duplicate dep

* refactor: split deployment scripts

- Split deploy-contracts.sh into separate Ethereum and Starknet deployment scripts
- deploy-ethereum.sh handles L1 contract deployment and updates messaging config
- deploy-starknet.sh handles L2 contract deployment
- Updated docker-compose.yml to run deployments in correct order:
  * deploy-ethereum runs after anvil
  * katana runs after ethereum deployment
  * deploy-starknet runs after katana
  * other services depend on starknet deployment

* fix: json config update in ethereum deployment

- Fix anvil.messaging.json not being updated after ethereum deployment
- Add proper error handling for json updates
- Create temporary files in same directory to avoid permission issues

* feat/ Added chain_id to proving logic to handle sepolia test net

* removed hardcoded anvil account PK

* added submodule to ci to handle garaga dep

* added jq to deps

* chore updated

* fmt

* new deployments

* feat: add retry mechanism to ethereum deployment script

- Add deploy_contracts function with 3 retry attempts
- Add 10 second timeout between retries
- Improve error handling and feedback messages

* fix: adjust mmr-builder restart policy

- Change restart policy from 'always' to 'on-failure:3'
- Prevent continuous restart loop while maintaining resilience

* added source check for mmr update and min update time on store

* fix: remove entrypoint from relayer Dockerfile

- Remove ENTRYPOINT and CMD directives
- Allow docker-compose to control the container entry point
- Enable proper logging from run_relayer.sh script

* feat: improve relayer service reliability

- Add retry mechanism to run_relayer.sh
- Configure different retry settings for local/prod environments
- Remove restart policy from docker-compose
- Update healthcheck to monitor script process
- Improve logging and error handling

* chore removed redundant logging

* refactor: use RELAYER_INTERVAL environment variable

- Remove ENV_FILE check from run_relayer.sh
- Add RELAYER_INTERVAL environment variable with default 720 minutes
- Set 10-minute interval for local development in docker-compose
- Simplify configuration management
  • Loading branch information
ametel01 authored Dec 9, 2024
1 parent 96b55dc commit 515a0a4
Show file tree
Hide file tree
Showing 46 changed files with 1,176 additions and 587 deletions.
44 changes: 44 additions & 0 deletions .cargo/config.toml
Original file line number Diff line number Diff line change
@@ -0,0 +1,44 @@
# [source.crates-io]
# replace-with = "vendored-sources"

# [source."git+https://github.com/ametel01/eth-rlp-verify.git"]
# git = "https://github.com/ametel01/eth-rlp-verify.git"
# replace-with = "vendored-sources"

# [source."git+https://github.com/ametel01/garaga.git"]
# git = "https://github.com/ametel01/garaga.git"
# replace-with = "vendored-sources"

# [source."git+https://github.com/ametel01/rust-accumulators.git?branch=feat/sha2-hasher"]
# git = "https://github.com/ametel01/rust-accumulators.git"
# branch = "feat/sha2-hasher"
# replace-with = "vendored-sources"

# [source."git+https://github.com/lambdaclass/lambdaworks.git"]
# git = "https://github.com/lambdaclass/lambdaworks.git"
# replace-with = "vendored-sources"

# [source."git+https://github.com/risc0/RustCrypto-hashes.git?tag=sha2-v0.10.8-risczero.0"]
# git = "https://github.com/risc0/RustCrypto-hashes.git"
# tag = "sha2-v0.10.8-risczero.0"
# replace-with = "vendored-sources"

# [source."git+https://github.com/risc0/risc0-ethereum?tag=v1.1.4"]
# git = "https://github.com/risc0/risc0-ethereum"
# tag = "v1.1.4"
# replace-with = "vendored-sources"

# [source."git+https://github.com/xJonathanLEI/starknet-rs.git?branch=master"]
# git = "https://github.com/xJonathanLEI/starknet-rs.git"
# branch = "master"
# replace-with = "vendored-sources"

# [source."git+https://github.com/OpenZeppelin/openzeppelin-contracts.git"]
# git = "https://github.com/OpenZeppelin/openzeppelin-contracts.git"
# replace-with = "vendored-sources"

# [source.vendored-sources]
# directory = "vendor"

# [net]
# git-fetch-with-cli = true
69 changes: 67 additions & 2 deletions .dockerignore
Original file line number Diff line number Diff line change
@@ -1,3 +1,68 @@
# Build artifacts
target/
Dockerfile
.dockerignore
/target/
dist/

# Docker files
Dockerfile*
.dockerignore
docker/

# Version control
.git/
.gitignore
.github/

# IDE and editor files
.vscode/
.idea/
*.swp
*.swo
.DS_Store

# Debug files
*.log
*.log.*
debug/

# Development configs
.env*
*.local
.editorconfig
.eslintrc*
.prettierrc*

# Documentation
docs/
*.md
LICENSE

# Test files
tests/
*_test.go
*.test
coverage/

# Dependencies
node_modules/

# Temporary files
tmp/
temp/
*.tmp

# Cargo specific
Cargo.lock # unless it's a binary project
.cargo/
!vendor/ # Explicitly allow the vendor directory
!vendor/**/target/ # Explicitly allow target directories under vendor

# Foundry specific
cache/
out/
broadcast/

# Misc development files
.husky/
.lint*
.format*
3 changes: 2 additions & 1 deletion .github/workflows/cairo.yml
Original file line number Diff line number Diff line change
Expand Up @@ -6,7 +6,8 @@ jobs:
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v4
# - uses: foundry-rs/setup-snfoundry@v3
with:
submodules: true
- uses: software-mansion/setup-scarb@v1
with:
scarb-version: "2.8.5"
Expand Down
9 changes: 9 additions & 0 deletions .gitignore
Original file line number Diff line number Diff line change
Expand Up @@ -24,3 +24,12 @@ scripts/katana/deploy.log
scripts/katana/katana.log

*.pb

vendor/

.env.local
.env.sepolia
.env.mainnet
docker/rebuild-all.sh

scripts/deploy-contracts.bash.sh
3 changes: 3 additions & 0 deletions .gitmodules
Original file line number Diff line number Diff line change
Expand Up @@ -2,3 +2,6 @@
path = contracts/ethereum/lib/forge-std
url = https://github.com/foundry-rs/forge-std.git
shallow = true
[submodule "contracts/starknet/garaga"]
path = contracts/starknet/garaga
url = [email protected]:keep-starknet-strange/garaga.git
Loading

0 comments on commit 515a0a4

Please sign in to comment.