Workflow to auto deploy contracts when changed #704

mb1896 · 2024-02-14T00:53:06Z

Sample output file: 7d65801
Saved in cd/contracts branch https://github.com/consensus-shipyard/ipc/commits/cd/contracts/

Closes ENG-627

linear · 2024-02-14T00:53:30Z

ENG-627 Automated IPC contract deployment on Calibnet

raulk

We'll need to publish the addresses somewhere accessible. If we stop here, I understand we can do the following:

Open the GitHub Actions UI.
Find the last job where this workflow ran.
Scroll down the list to the last step.
Expand the logs.
Find the two addresses we care about in the firehose of logs
Copy each address.

However, that is wasteful for us devs and very poor UX for adopters.

Instead, let's parse the two addresses we care about, construct a .json file, and commit that .json file elsewhere public, associated with the commit. We can then use shields.io's JSON support to display those addresses in the repo's README.

Here's the cleanest and most scalable way of doing this, in a way that does not muddy the commit history of main:

Create a long lived cd/contracts branch (standing for: continuous deployment > contracts).
When contracts change, have the workflow merge the main branch into cd/contracts with a merge commit.
Execute the deployment, and extract the gateway and registry addresses.
Generate a deployments/r314159.json file with the two addresses + master commit deployed.

{
    "commit": "<sha1>",
    "gateway_addr": "<eth addr>",
    "registry_addr": "<eth addr>",
}

Push a commit from within the action, containing the merge and the updated deployments metadata.

This solution allows us to hit all checkboxes:

Tracks exactly which contracts commit was deployed (thanks to the merge preciding the metadata update commit).
Keeps an easiliy accessible history of contract deployments (in case someone wants to test with an early version of contracts, e.g. when bisecting a bug).
Addresses are always viewable by users, in a predictable URL.
Latest addresses are machine-processable, always in a predictable URL.
Repo stays lean in size.

raulk · 2024-02-14T11:12:51Z

.github/workflows/auto-deploy-contracts.yaml

+  deploy-contracts:
+    runs-on: ubuntu-latest
+    env:
+      RPC_URL: https://calibration.filfox.info/rpc/v1


Not necessary, but can we move the URL to a secret, and make sure that the subnet deployment automation uses the same URL? Trying to reduce entropy and consolidate to a single configuration spot, in case endpoints become unstable and we need to change them.

Note that github secrets do not propagate between github workflows unless you explicitly forward them with (secrets: inherit) which is what I had to do in this PR

I don't think the URL is a secret per se. Besides, it's often that a RPC URL fails or take too long to response. In that case I would just make a PR to switch it to a different provider. But if we keep track them in secrets, we probably already forgotten what URL we were using. That just make things more implicit for maintainers.

maciejwitowski · 2024-02-15T10:58:58Z

Just a note that we discussed it in sync, and aligned that we need the addresses to be available somewhere.
We haven't 100% decided where exactly as @mb1896 had reservations about the branch suggested by @raulk .
@mb1896 feel free to propose alternatives. Let's aim to reach the decision today since it'd be valuable to have it next week when the hackathon starts.

raulk · 2024-02-15T11:05:21Z

For the record, the alternative that @mb1896 suggested was placing the addresses in a file in main. This was not seen as a good solution because bots pushing to main causes all sorts of problems. The parallel branch with merges from main is cleaner. And it's easy to change if we feel it doesn't work for us. So let's just go do it.

raulk · 2024-02-15T11:09:19Z

@mb1896 this is the GitHub Action you should use: https://github.com/stefanzweifel/git-auto-commit-action It does not require a Personal Access Token, nor a bot account.

mb1896 · 2024-02-15T17:40:43Z

Just a note that we discussed it in sync, and aligned that we need the addresses to be available somewhere. We haven't 100% decided where exactly as @mb1896 had reservations about the branch suggested by @raulk . @mb1896 feel free to propose alternatives. Let's aim to reach the decision today since it'd be valuable to have it next week when the hackathon starts.

Thanks for the comments. I'm following @raulk 's direction now. Will for sure make this ready soon for external developers to check out.

maciejwitowski · 2024-02-16T10:50:18Z

@mb1896 just cross-posting from Slack:
I think it would help reviewers if:

the commits had descriptive names, not "test" ("test" is fine for local development but when you want someone to review, it's better to squash the commits and made them clearer)
there was a TODO included in the description to indicate what is left to code so reviewers know what to expect

raulk · 2024-02-17T10:19:16Z

.github/workflows/auto-deploy-contracts.yaml

+          echo "commit_hash: $commit_hash"
+          echo "gateway_address: $gateway_address"
+          echo "registry_address: $registry_address"
+      - name: Switch code repo to cd/contract branch


@mb1896 we're missing the merge from main into the cd branch, prior to committing the file. Can you please add it? This will (a) place the source and the addresses in one convenient branch, (b) create a nice merge graph tracking at the git history level what got deployed when.

Fixed. Please take another look.

maciejwitowski

LGTM, thanks!

mb1896 · 2024-02-20T03:09:27Z

@raulk Can you take another look?

mb1896 · 2024-02-20T04:45:19Z

.github/workflows/auto-deploy-contracts.yaml

+          ref: cd/contracts
+          submodules: recursive
+
+      - name: (Dry run) Try merge from main branch and update cd/contracts branch


Dry run pull merge is to verify that the condition of cd/contracts branch is able to merge with latest main branch. If failed, we will resolve whatever error manually.

Calling this a dry run is at bit confusing as its actually pulling and merging (so has side effects on the runner)... its just not pushing changes.

Also "... and update cd/contracts branch" does not seem to be right

Yes, the "dry" part mostly means that we will not modify remote code repos. This is useful because we can fail early when merging generate conflicts (rather than complain after deploying the contract).

I updated the step's name to better describe what it does.

fridrik01

had some minor comment, but otherwise this looks good to me

fridrik01 · 2024-02-20T09:17:21Z

.github/workflows/auto-deploy-contracts.yaml

+          ref: cd/contracts
+          submodules: recursive
+
+      - name: (Dry run) Try merge from main branch and update cd/contracts branch


Calling this a dry run is at bit confusing as its actually pulling and merging (so has side effects on the runner)... its just not pushing changes.

Also "... and update cd/contracts branch" does not seem to be right

maciejwitowski · 2024-02-20T10:44:58Z

@mb1896 When you're online, let's address Fridrik's comments, merge main and merge this PR. Raul's comments have been addressed so no need to block this PR on them. Thanks!

maciejwitowski · 2024-02-20T12:23:35Z

Btw this is great, I was already able to unblock a dev who encountered error because of contracts incompatibility by sending addresses from this commit to them..

mb1896 requested review from maciejwitowski, raulk and aakoshh February 14, 2024 03:04

raulk requested changes Feb 14, 2024

View reviewed changes

mb1896 marked this pull request as draft February 16, 2024 08:02

mb1896 force-pushed the 0213_auto_deploy_contract_when_change branch from 36ae1c9 to 77f6c82 Compare February 16, 2024 08:02

mb1896 force-pushed the 0213_auto_deploy_contract_when_change branch from 3ba6628 to c0357dd Compare February 16, 2024 15:45

mb1896 marked this pull request as ready for review February 16, 2024 15:45

mb1896 requested review from raulk and fridrik01 February 16, 2024 15:47

mb1896 force-pushed the 0213_auto_deploy_contract_when_change branch from 74fb86a to 08c9007 Compare February 17, 2024 01:37

raulk requested changes Feb 17, 2024

View reviewed changes

mb1896 force-pushed the 0213_auto_deploy_contract_when_change branch from 107f7ff to 5371767 Compare February 19, 2024 05:09

mb1896 requested a review from raulk February 19, 2024 06:52

maciejwitowski approved these changes Feb 19, 2024

View reviewed changes

mb1896 force-pushed the 0213_auto_deploy_contract_when_change branch from 3f95f0d to 7facea7 Compare February 20, 2024 03:04

mb1896 commented Feb 20, 2024

View reviewed changes

fridrik01 approved these changes Feb 20, 2024

View reviewed changes

mb1896 force-pushed the 0213_auto_deploy_contract_when_change branch from 9d8021a to d9ca09f Compare February 20, 2024 17:48

mb1896 added 2 commits February 20, 2024 10:34

Initial commit

36b81d4

test

80c2b0e

mb1896 added 25 commits February 20, 2024 10:34

test

88cd5f7

test

9c5e76f

switch to rebase to avoid conflict

5b0eee5

test

030f490

test

4d083e0

test

8bfad28

test

201e39e

test

6f387b7

test

f25a14c

test

c47c0e1

test

f59ae81

Ready to final test

74f6f89

test

7b82e9f

start try merge instead of rebase

134e27b

test

f6ef9bf

test

57367a3

test

ce8f443

Add dry run for pull from main

2207a0a

fix

e8bf207

Update step name

93aa6cf

Try rebase instead of merge

0ec8013

keep testing

321911d

keep testing

53cc5f5

keep testing

55b2a53

keep testing

0315908

mb1896 force-pushed the 0213_auto_deploy_contract_when_change branch from 8ea7d51 to 0315908 Compare February 20, 2024 18:35

raulk approved these changes Feb 20, 2024

View reviewed changes

fix: associate git commits with bot; extract to envs; use local merge.

0ddba22

mb1896 merged commit 9d1abfc into main Feb 21, 2024
14 checks passed

mb1896 deleted the 0213_auto_deploy_contract_when_change branch February 21, 2024 03:11

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Workflow to auto deploy contracts when changed #704

Workflow to auto deploy contracts when changed #704

mb1896 commented Feb 14, 2024 •

edited

Loading

linear bot commented Feb 14, 2024

raulk left a comment

raulk Feb 14, 2024

fridrik01 Feb 14, 2024

mb1896 Feb 17, 2024

maciejwitowski commented Feb 15, 2024

raulk commented Feb 15, 2024

raulk commented Feb 15, 2024 •

edited

Loading

mb1896 commented Feb 15, 2024 •

edited

Loading

maciejwitowski commented Feb 16, 2024

raulk Feb 17, 2024

mb1896 Feb 19, 2024

maciejwitowski left a comment

mb1896 commented Feb 20, 2024

mb1896 Feb 20, 2024

fridrik01 Feb 20, 2024

mb1896 Feb 20, 2024

fridrik01 left a comment

fridrik01 Feb 20, 2024

maciejwitowski commented Feb 20, 2024

maciejwitowski commented Feb 20, 2024

Workflow to auto deploy contracts when changed #704

Workflow to auto deploy contracts when changed #704

Conversation

mb1896 commented Feb 14, 2024 • edited Loading

linear bot commented Feb 14, 2024

raulk left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

maciejwitowski commented Feb 15, 2024

raulk commented Feb 15, 2024

raulk commented Feb 15, 2024 • edited Loading

mb1896 commented Feb 15, 2024 • edited Loading

maciejwitowski commented Feb 16, 2024

Choose a reason for hiding this comment

Choose a reason for hiding this comment

maciejwitowski left a comment

Choose a reason for hiding this comment

mb1896 commented Feb 20, 2024

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

fridrik01 left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

maciejwitowski commented Feb 20, 2024

maciejwitowski commented Feb 20, 2024

mb1896 commented Feb 14, 2024 •

edited

Loading

raulk commented Feb 15, 2024 •

edited

Loading

mb1896 commented Feb 15, 2024 •

edited

Loading