Skip to content

Commit

Permalink
Merge branch 'hyperledger:main' into issue-3810-no-empty-block
Browse files Browse the repository at this point in the history
  • Loading branch information
amsmota authored Jun 21, 2024
2 parents 26bfeed + 8c04d0a commit 3a6d41d
Show file tree
Hide file tree
Showing 1,741 changed files with 25,178 additions and 14,313 deletions.
4 changes: 2 additions & 2 deletions .circleci/config.yml
Original file line number Diff line number Diff line change
Expand Up @@ -6,7 +6,7 @@ orbs:
executors:
besu_executor_med: # 2cpu, 4G ram
docker:
- image: cimg/openjdk:17.0
- image: cimg/openjdk:21.0
resource_class: medium
working_directory: ~/project
environment:
Expand All @@ -24,7 +24,7 @@ executors:

besu_executor_xl: # 8cpu, 16G ram
docker:
- image: cimg/openjdk:17.0
- image: cimg/openjdk:21.0
resource_class: xlarge
working_directory: ~/project
environment:
Expand Down
35 changes: 35 additions & 0 deletions .github/ISSUE_TEMPLATE/release-checklist.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,35 @@
---
name: Release Checklist
about: items to be completed for each release
title: ''
labels: ''
assignees: ''

---

- [ ] Confirm anything outstanding for release with other maintainers on #besu-release in Discord
- [ ] Notify maintainers about updating changelog for in-flight PRs
- [ ] Update changelog if necessary, and merge a PR for it to main
- [ ] Optional: for hotfixes, create a release branch and cherry-pick, e.g. `release-<version>-hotfix`
- [ ] Optional: create a PR into main from the hotfix branch to see the CI checks pass
- [ ] On the appropriate branch/commit, create a calver tag for the release candidate, format example: `24.4.0-RC2`
- [ ] Sign-off with team; confirm tag is correct in #besu-release in Discord
- [ ] Consensys staff start burn-in using the proposed release <version-RCX> tag
- [ ] Sign off burn-in; convey burn-in results in #besu-release in Discord
- [ ] Using the same git sha, create a calver tag for the FULL RELEASE, example format `24.4.0`
- [ ] Using the FULL RELEASE tag, create a release in github to trigger the workflows. Once published:
- makes the release "latest" in github
- this is now public and notifies subscribed users
- publishes artefacts and version-specific docker tags
- publishes the docker `latest` tag variants
- [ ] Draft homebrew PR
- [ ] Draft documentation release
- [ ] Ensure binary SHAs are correct on the release page
- [ ] Docker release startup test:
- `docker run hyperledger/besu:<version>`
- `docker run hyperledger/besu:<version>-arm64`
- `docker run --platform linux/amd64 hyperledger/besu:<version>-amd64`
- `docker run --pull=always hyperledger/besu:latest` (check version is <version>)
- [ ] Merge homebrew PR
- [ ] Publish Docs Release
- [ ] Social announcements
70 changes: 70 additions & 0 deletions .github/workflows/BesuContainerVerify.sh
Original file line number Diff line number Diff line change
@@ -0,0 +1,70 @@
#!/bin/bash
##
## Copyright contributors to Hyperledger Besu.
##
## Licensed under the Apache License, Version 2.0 (the "License"); you may not use this file except in compliance with
## the License. You may obtain a copy of the License at
##
## http://www.apache.org/licenses/LICENSE-2.0
##
## Unless required by applicable law or agreed to in writing, software distributed under the License is distributed on
## an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the License for the
## specific language governing permissions and limitations under the License.
##
## SPDX-License-Identifier: Apache-2.0
##

CONTAINER_NAME=${CONTAINER_NAME:-besu}
VERSION=${VERSION}
TAG=${TAG}
CHECK_LATEST=${CHECK_LATEST}
RETRY=${RETRY:-10}
SLEEP=${SLEEP:-5}

# Helper function to throw error
log_error() {
echo "::error $1"
exit 1
}

# Check container is in running state
_RUN_STATE=$(docker inspect --type=container -f={{.State.Status}} ${CONTAINER_NAME})
if [[ "${_RUN_STATE}" != "running" ]]
then
log_error "container is not running"
fi

# Check for specific log message in container logs to verify besu started
_SUCCESS=false
while [[ ${_SUCCESS} != "true" && $RETRY -gt 0 ]]
do
docker logs ${CONTAINER_NAME} | grep -q "Ethereum main loop is up" && {
_SUCCESS=true
continue
}
echo "Waiting for the besu to start. Remaining retries $RETRY ..."
RETRY=$(expr $RETRY - 1)
sleep $SLEEP
done

# Log entry does not present after all retries, fail the script with a message
if [[ ${_SUCCESS} != "true" ]]
then
docker logs --tail=100 ${CONTAINER_NAME}
log_error "could not find the log message 'Ethereum main loop is up'"
else
echo "Besu container started and entered main loop"
fi

# For the latest tag check the version match
if [[ ${TAG} == "latest" && ${CHECK_LATEST} == "true" ]]
then
_VERSION_IN_LOG=$(docker logs ${CONTAINER_NAME} | grep "#" | grep "Besu version" | cut -d " " -f 4 | sed 's/\s//g')
echo "Extracted version from logs [$_VERSION_IN_LOG]"
if [[ "$_VERSION_IN_LOG" != "${VERSION}" ]]
then
log_error "version [$_VERSION_IN_LOG] extracted from container logs does not match the expected version [${VERSION}]"
else
echo "Latest Besu container version matches"
fi
fi
52 changes: 34 additions & 18 deletions .github/workflows/acceptance-tests.yml
Original file line number Diff line number Diff line change
Expand Up @@ -11,8 +11,8 @@ concurrency:
cancel-in-progress: true

env:
GRADLE_OPTS: "-Xmx6g -Dorg.gradle.daemon=false"
total-runners: 16
GRADLE_OPTS: "-Xmx6g"
total-runners: 12

jobs:
acceptanceTestEthereum:
Expand All @@ -24,7 +24,7 @@ jobs:
strategy:
fail-fast: true
matrix:
runner_index: [0,1,2,3,4,5,6,7,8,9,10,11,12,13,14,15]
runner_index: [0,1,2,3,4,5,6,7,8,9,10,11]
steps:
- name: Checkout Repo
uses: actions/checkout@b4ffde65f46336ab88eb53be808477a3936bae11
Expand All @@ -34,36 +34,52 @@ jobs:
uses: actions/setup-java@387ac29b308b003ca37ba93a6cab5eb57c8f5f93
with:
distribution: temurin
java-version: 17
java-version: 21
- name: Install required packages
run: sudo apt-get install -y xmlstarlet
- name: get acceptance test report
uses: dawidd6/action-download-artifact@e7466d1a7587ed14867642c2ca74b5bcc1e19a2d
with:
branch: main
name_is_regexp: true
name: 'acceptance-node-\d*\d-test-results'
path: tmp/junit-xml-reports-downloaded
if_no_artifact_found: true
- name: setup gradle
uses: gradle/actions/setup-gradle@9e899d11ad247ec76be7a60bc1cf9d3abbb9e7f1
with:
cache-disabled: true
- name: List unit tests
run: ./gradlew acceptanceTestNotPrivacy --test-dry-run -Dorg.gradle.parallel=true -Dorg.gradle.caching=true
- name: Extract current test list
run: mkdir tmp; find . -type f -name TEST-*.xml | xargs -I{} bash -c "xmlstarlet sel -t -v '/testsuite/@name' '{}'; echo ' acceptanceTestNotPrivacy'" | tee tmp/currentTests.list
- name: Get acceptance test reports
uses: dawidd6/action-download-artifact@e7466d1a7587ed14867642c2ca74b5bcc1e19a2d
continue-on-error: true
with:
branch: main
workflow: update-test-reports.yml
name: acceptance-test-results
path: tmp/junit-xml-reports-downloaded
if_no_artifact_found: ignore
- name: Split tests
run: .github/workflows/splitTestsByTime.sh tmp/junit-xml-reports-downloaded ${{env.total-runners}} ${{ matrix.runner_index }} > testList.txt
run: .github/workflows/splitTestsByTime.sh tmp/junit-xml-reports-downloaded "tmp/junit-xml-reports-downloaded/acceptance-node-.*-test-results" "TEST-" ${{env.total-runners}} ${{ matrix.runner_index }} > testList.txt
- name: format gradle args
# we do not need the module task here
run: cat testList.txt | cut -f 2- -d ' ' | tee gradleArgs.txt
- name: Upload Timing
uses: actions/upload-artifact@5d5d22a31266ced268874388b861e4b58bb5c2f3
if: matrix.runner_index == 0
with:
name: acceptance-tests-timing
path: 'tmp/timing.tsv'
- name: format gradle args
# insert --tests option between each.
run: cat testList.txt | sed -e 's/^\| / --tests /g' | tee gradleArgs.txt
- name: Upload Lists
uses: actions/upload-artifact@5d5d22a31266ced268874388b861e4b58bb5c2f3
if: matrix.runner_index == 0
with:
name: acceptance-tests-lists
path: 'tmp/*.list'
- name: Upload gradle test tasks
uses: actions/upload-artifact@5d5d22a31266ced268874388b861e4b58bb5c2f3
with:
name: test-args-${{ matrix.runner_index }}.txt
path: '*.txt'
- name: run acceptance tests
run: ./gradlew acceptanceTestNotPrivacy `cat gradleArgs.txt` -Dorg.gradle.parallel=true -Dorg.gradle.caching=true
- name: cleanup tempfiles
run: rm testList.txt gradleArgs.txt
- name: Remove downloaded test results
run: rm -rf tmp/junit-xml-reports-downloaded
- name: Upload Acceptance Test Results
uses: actions/upload-artifact@5d5d22a31266ced268874388b861e4b58bb5c2f3
with:
Expand Down
115 changes: 0 additions & 115 deletions .github/workflows/artifacts.yml

This file was deleted.

2 changes: 1 addition & 1 deletion .github/workflows/codeql.yml
Original file line number Diff line number Diff line change
Expand Up @@ -33,7 +33,7 @@ jobs:
uses: actions/setup-java@387ac29b308b003ca37ba93a6cab5eb57c8f5f93
with:
distribution: 'temurin'
java-version: 17
java-version: 21
# Initializes the CodeQL tools for scanning.
- name: Initialize CodeQL
uses: github/codeql-action/init@2f93e4319b2f04a2efc38fa7f78bd681bc3f7b2f
Expand Down
47 changes: 47 additions & 0 deletions .github/workflows/container-security-scan.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,47 @@
name: container security scan

on:
workflow_dispatch:
inputs:
tag:
description: 'Container image tag'
required: false
default: 'develop'
schedule:
# Start of the hour is the busy time. Scheule it to run 8:17am UTC
- cron: '17 8 * * *'

jobs:
scan-sarif:
runs-on: ubuntu-latest
permissions:
contents: read
security-events: write

steps:
- name: Checkout
uses: actions/checkout@b4ffde65f46336ab88eb53be808477a3936bae11

# Shell parameter expansion does not support directly on a step
# Adding a separate step to set the image tag. This allows running
# this workflow with a schedule as well as manual
- name: Set image tag
id: tag
run: |
echo "TAG=${INPUT_TAG:-develop}" >> "$GITHUB_OUTPUT"
env:
INPUT_TAG: ${{ inputs.tag }}

- name: Vulnerability scanner
id: trivy
uses: aquasecurity/trivy-action@595be6a0f6560a0a8fc419ddf630567fc623531d
with:
image-ref: hyperledger/besu:${{ steps.tag.outputs.TAG }}
format: sarif
output: 'trivy-results.sarif'

# Check the vulnerabilities via GitHub security tab
- name: Upload results
uses: github/codeql-action/upload-sarif@23acc5c183826b7a8a97bce3cecc52db901f8251
with:
sarif_file: 'trivy-results.sarif'
Loading

0 comments on commit 3a6d41d

Please sign in to comment.