Use vanilla Besu

[fab-10]

Goal

We want to get rid of the current Linea Besu repo and directly use the official Besu repo for Linea packages and as dependency for plugins.

Background

We use then intermediate Linea Besu repo, a fork of Besu repo, to build and publish Besu artifacts, initially this repo was useful to preview some features that were needed by Linea, but not yet available in the vanilla Besu plugin API, and to have a lightweight process to quickly implement and ship fixes, before they were fully implemented in the upstream Besu repo.

Current status

Now all the feature required to build Linea plugins are exposed in the Besu plugin API, so in the fork there is nothing that is currently needed to develop and run Linea plugins, and so the fork is only useful to create Besu releases to be used by Linea, this because the standard Besu release schedule is not aligned with Linea, and so it happens frequently that we need to ship Besu build to be consumed by Linea, between two major Besu releases, and they also follow two distinct QA phases.

So to sum up, we can get rid of the Linea Besu fork, but we need to find a way to keep creating Besu builds for Linea whenever they are needed, without impacting in any way upstream Besu repo and its release schedule.

Proposed solution

Trivial solution could be to just use the latest development version of Besu and use that where you use a Linea Besu delivery today, but this has the drawback that official Linea Besu packages will show development in the version and that is very difficult for humans to remember what's behind a hash, so a better approach is required.

As said before we need a way to be able to build a new Besu release that is only relevant for Linea at any time, because a new feature or fix has been pushed to the Besu repo, so basically what we need is to know at which commit to produce a new Linea release, then we can simply achieve that using a git submodule pointing to Besu repo at the specific commit putting the commit in the /versions/linea-*.env files so that each network could use a different version of Besu and keep the configuration in the same place, and use that commit to build and publish, with a nice versioning scheme, the Besu artifacts relative to the Linea release in this repo.

Workflow

1. update the commit used by the Besu submodule
2. a new CI job ¹ will build Besu setting version to something easy to read, for example adding -lineaX to the Besu latest official version, 24.12.2-linea1 (Note: this is a separate job, in addition to the current one that builds the Linea Besu packages) [
3. the artifacts built in the previous steps, need to be published somewhere, in particular the jars need to be published to a maven repo, because they need to be referred by Linea plugins

Migration

Current Linea plugins will need to replace Linea Besu maven repo with the new repo, and just use standard group from Besu dependencies

Footnotes

1. Note this is a separate job, in addition to the current one that builds the Linea Besu packages, so this repo will be used to first build and publish Besu versions for Linea, and then to build and publish Linea Besu Packages. ↩

[Gabriel-Trintinalia]

Thanks for proposing the plan. It looks good to me. so the plan is to trigger a new release of besu with a nice tag that is only relevant to Linea whenever we need to update the package distribution? This job is independent of the rest of the workflow and needs to be done first every time we need a new release of the package, correct?

[fab-10]

Thanks for proposing this plan. It looks good to me! One thing, instead of the git submodule, could we have the besu commit in the /versions/linea-*.env files so that each network could use a different version of Besu and keep the configuration in the same place? Would that even be possible?

Good point to have the possibility to configure the version by env, updating the proposal with your suggestion

[fab-10]

Thanks for proposing the plan. It looks good to me. so the plan is to trigger a new release of besu with a nice tag that is only relevant to Linea whenever we need to update the package distribution? This job is independent of the rest of the workflow and needs to be done first every time we need a new release of the package, correct?

Yes that's correct, and that part needs to be very clear, since there are two distinct phases:

1. Building and publishing of Besu release for Linea, the new workflow introduced by this proposal
2. Building and publishing of the Linea Besu Package, the existing workflow

Since it is possible that a new Besu release for Linea, requires changes in some plugins, this extended scenario will be:

1. Building and publishing of Besu release for Linea, the new workflow introduced by this proposal
2. Use the new Besu for Linea release to build and publish plugins
3. Update plugins version in linea-besu-package
4. Building and publishing of the Linea Besu Package, the existing workflow

[Gabriel-Trintinalia]

Thanks @fab-10 , the proposed release process seems clear to me