Move Swift publishing workflow to main repo #1033

cnixbtc · 2024-07-08T13:14:19Z

Swift Publishing Workflow Consolidation

This PR relocates the Swift publishing workflow from the breez/breez-sdk-swift repository to the main repository. This change simplifies the publishing process by integrating it with the main repo's CI workflow.

📖 Overview

Previously, the Swift publishing workflow required separate triggers, unlike other platforms managed centrally in the main repo via the publish-all-platforms workflow. This PR enhances the Swift package publishing by:

Streamlining the publishing process by integrating the Swift package into the main repo's CI workflow.
Improving build runtime and reliability by reusing the already built binary SDK and language bindings from the main repo's workflows.

One interesting point: Because our Flutter and React Native packages rely on the Swift package at runtime, the PR includes logic to ensure that the Flutter and React Native packages are only published if the Swift package is successfully published too, preventing issues with runtime dependencies. If the Swift package is not set to be published, then the Flutter and React Native packages are published nonetheless allowing us to release updates to those packages that depend on an already published Swift package. Special thanks to @JssDWt for the solution adapted from this PR.

❓ Outstanding Questions

Before merging this PR, we need to address two key points. The first is straightforward, while the second requires some feedback.

Setting CocoaPods Token
We need to set secrets.COCOAPODS_TRUNK_TOKEN in the main repo to publish to the CocoaPods trunk. This token is currently only set in the breez-sdk-swift repo (I believe we're using @roeierez's token there). The simplest solution is to just move the token from the Swift repo to the main repo.
Publishing breez_sdkFFI.xcframework
The Swift package and CocoaPods require a download link to the breez_sdkFFI.xcframework binary artifact. Currently, we create a GitHub release in the breez-sdk-swift repo and attach the XCFramework as a binary artifact. Moving the workflow to the main repo presents a few challenges for maintaining this setup. I believe options 1 and 3 are the most straightforward, with option 2 being a viable alternative that may need some experimentation.
- Option 1: Use a GitHub Personal Access Token to replicate the current setup. This would involve creating a release in the breez-sdk-swift repo from the CI workflow in the main repo. While simple, this requires using user-bound tokens with broad permissions, which isn't ideal.
- Option 2: Use a GitHub app to provide short-lived tokens for the workflow. This option offers better security with non-user-bound tokens but requires more setup.
- Option 3: Create the release in the main breez-sdk-greenlight repo instead of the breez-sdk-swift repo and attach the artifact there. This option is easy to set up and doesn't require special tokens, but it requires releases to be created on the main repo whenever we want to update or publish the Swift package. Not ideal, imo.
- Option 4: Find an alternative hosting solution outside of GitHub for the artifact, accessible to our users.

✋ Feedback Request

I'd appreciate your thoughts on these options, particularly regarding the second point. Looking forward to your feedback. 🙏

JssDWt

Looking good to me (always hard to say that with CI, you never know if it works)! I'd say if this is tested it's good to go.

.github/workflows/publish-swift.yml

JssDWt · 2024-08-16T11:50:17Z

2. Publishing breez_sdkFFI.xcframework
The Swift package and CocoaPods require a download link to the breez_sdkFFI.xcframework binary artifact. Currently, we create a GitHub release in the breez-sdk-swift repo and attach the XCFramework as a binary artifact. Moving the workflow to the main repo presents a few challenges for maintaining this setup. I believe options 1 and 3 are the most straightforward, with option 2 being a viable alternative that may need some experimentation.

We have a similar setup for go. There we pass a repo ssh key to provide commit access. https://github.com/breez/breez-sdk-greenlight/blob/main/.github/workflows/publish-golang.yml#L31. I think we can use the same construct here (so option 1).

dangeross · 2024-08-16T12:00:11Z

Also check out the https://github.com/breez/breez-sdk-liquid repo, it publishes from the main publishing flow

cnixbtc · 2024-08-20T17:15:17Z

Thanks for the feedback guys and the pointer to the other repos.

Will have a look!

cnixbtc · 2024-08-31T09:45:24Z

Alright, I've added an action-archival step for the XCFramework (the language bindings are already archived here).

That means we're now just missing the publishing token analogous to the publishing for the Swift publishing for breez-sdk-liquid.

@yaslama Is that ok with you? I know we've previously talked about trying not to use a token. Let me know what you think. 🙏

yaslama · 2024-08-31T17:13:27Z

@yaslama Is that ok with you? I know we've previously talked about trying not to use a token. Let me know what you think. 🙏

Let's use tokens for now and try to find an alternative.

JssDWt · 2024-09-01T15:21:16Z

Looks good it me, if it works, let's merge it. Did you test this with publish=false with some different configurations?

dangeross · 2024-09-03T11:11:06Z

.github/workflows/publish-swift.yml

+      - name: Archive xcframework
+        uses: actions/upload-artifact@v4
+        with:
+          name: breez_sdkFFI-${{ inputs.package-version || intouts.ref }}.xcframework


This is causing the publish CI to fail when testing

Suggested change

name: breez_sdkFFI-${{ inputs.package-version || intouts.ref }}.xcframework

name: breez_sdkFFI-${{ inputs.package-version || inputs.ref }}.xcframework

Damn, typo! Thanks for catching!

dangeross · 2024-09-03T11:15:16Z

.github/workflows/publish-swift.yml

+
+      - name: Release and attach XCFramework binary artifact
+        if: ${{ inputs.publish }}
+        uses: ncipollo/release-action@v1


I seem to remember this action had issues in the breez-sdk-liquid repo when using a token. There we use softprops/action-gh-release@v2 as the release action instead.

Good insight thanks! Will use softprops/action-gh-release@v2 here too. Good for consistency too! 👏

cnixbtc · 2024-09-03T17:47:21Z

Looks good it me, if it works, let's merge it. Did you test this with publish=false with some different configurations?

@JssDWt Thanks for your feedback! 🙇 I primarily tested the syncing logic to ensure that the React Native and Flutter publishing workflows behave as follows:

Both Swift and Android packages are published: React Native and Flutter are published only if both Swift and Android publishing succeed.
Either Swift or Android package is published: React Native and Flutter are published only if the respective (Swift or Android) publishing succeeds.
Neither Swift nor Android packages are published: React Native and Flutter are published regardless of Swift or Android status, allowing for RN/Flutter patches without re-publishing Swift or Android.

Here's my test notes:

Tests using dummy binaries (publish == false)
- Wokflow config: ["maven", "swift"]
  - https://github.com/breez/breez-sdk/actions/runs/9928882340
  - jobs dependent on maven and swift run after maven and swift as expected
- Wokflow config: ["maven"]
  - https://github.com/breez/breez-sdk/actions/runs/9928828401
  - jobs dependent on maven and swift run after maven as expected
- Wokflow config: ["swift"]
  - https://github.com/breez/breez-sdk/actions/runs/9928935630
  - jobs dependent on maven and swift run after swift as expected
- Wokflow config: ["maven", "swift"]
  - hardcoded a fail for swift
  - https://github.com/breez/breez-sdk/actions/runs/9928994716
  - jobs dependent on maven and swift don't run as expected
- Wokflow config: ["golang"]
  - https://github.com/breez/breez-sdk/actions/runs/9929068463
  - jobs dependent on maven and swift run after as expected
Tests using actual binaries (publish == false)
- Wokflow config: ["flutter", "swift"]
  - https://github.com/breez/breez-sdk/actions/runs/9929607744
  - jobs dependent on maven and swift run after maven and swift as expected

I did some minor updates and addressing feedback after testing these scenarios but I think those should be irrelevant and the tests still valid.

Of course, CI is hard to fully test so when we merge this and see that publishing fails feel free to revert and let me know and I will fix whatever problems there might be.

cnixbtc · 2024-09-03T17:49:00Z

⚠️ Before we merge this, we need to set the SWIFT_RELEASE_TOKEN for the https://github.com/breez/breez-sdk-swift repo as a GH actions secret here.

Who can help me with that?

JssDWt · 2024-09-05T19:21:46Z

Test run failed due to some (I think) unrelated issues with building the Kotlin lang bindings. But I saw that @JssDWt did some test runs earlier that were successful. I will be on holiday for the next 2 weeks (starting on the 7th). Let me know guys if you'd prefer to merge this before or after. :)

This is a problem that comes up after rust 1.80. I ran into this issue with cln as well, I downgraded to rust 1.79 there. Let me see how to fix this here.

JssDWt · 2024-09-05T19:28:45Z

@cnixbtc can you try a rebase?

Co-authored-by: Jesse de Wit <[email protected]>

Co-authored-by: Ross Savage <[email protected]>

cnixbtc · 2024-09-07T08:25:24Z

@cnixbtc can you try a rebase?

Rebased onto main. ✔️ Though I thin there's still issues with the bindings.

cnixbtc · 2024-09-07T08:45:47Z

Guys, as mentioned above, I will be off for 2 weeks now. I can take this over the finish line and when I'm back but of course feel free to merge it beforehand when you feel like it. Whatever it is thanks for the help so far and I'll report back in two weeks. :) If there is anything I can be reached on Slack but won't have a laptop with me to jump into code.

JssDWt · 2024-09-08T11:48:48Z

I've removed the rust caches, and now the build works. It was picking time 0.3.30, where it should have picked 0.3.36`. Or maybe the build didn't use your rebased version yet.

cnixbtc · 2024-09-30T17:46:18Z

Alright, so I did three more test runs. For all three I used the workflow from this branch and the repo code from the 0.6.1 tag.

["flutter", "react-native", "swift"]: swift runs first, then flutter and react native in parallel
["react-native", "flutter"]: both run in parallel without waiting for the (non existing) swift build
["swift"]: runs as expected

Good to go from my side.

JssDWt

LGTM

dangeross · 2024-09-30T19:11:09Z

Just to double check: Is the COCOAPODS_TRUNK_TOKEN already set as a secret in this repo? If not I think @roeierez will need to add that one too.

@roeierez

cnixbtc · 2024-10-16T19:44:53Z

@dangeross I checked with Roei, COCOAPODS_TRUNK_TOKEN is set.

SWIFT_RELEASE_TOKEN is set too and REPO_SSH_KEY is used by other jobs too so must be set already. 🙃

dangeross

LGTM

cnixbtc · 2024-10-23T17:39:35Z

Merging this now. Please ping me if you feel like something is not working when pushing out the next Swift release. We still have the workflow in the Swift repo as backup in case anything breaks with the new workflow in the main repo here.

cc @roeierez

cnixbtc self-assigned this Jul 8, 2024

cnixbtc force-pushed the move-swift-workflow-to-main-repo branch 11 times, most recently from a1e7a41 to add1399 Compare July 14, 2024 15:33

cnixbtc requested review from JssDWt, erdemyerebasmaz, roeierez and yaslama August 11, 2024 12:50

JssDWt reviewed Aug 16, 2024

View reviewed changes

.github/workflows/publish-swift.yml Show resolved Hide resolved

cnixbtc changed the title ~~[WIP] Move Swift publishing workflow to main repo~~ Move Swift publishing workflow to main repo Aug 31, 2024

JssDWt mentioned this pull request Sep 2, 2024

add ci for swift #620

Closed

dangeross reviewed Sep 3, 2024

View reviewed changes

cnixbtc marked this pull request as ready for review September 3, 2024 17:50

cnixbtc requested a review from dangeross September 3, 2024 17:50

cnixbtc and others added 15 commits September 7, 2024 10:20

Build universal binary for ios

804b69b

Add workflow to build and publish swift package

dadb62d

If swift is run, run react navtive and flutter after swift

e00a733

Run flutter and RN after swift

754ff40

Do some debug runs

4928565

Remove debug code

8ca7d8c

Fix asset download for ios universal bindings

7ddfd23

Fix lipo command

d927bee

Archive xcframework

d54743a

Rename swift release token

3aed24b

Fix typo

ac21299

Align release action with what is used in liquid CI

b87ca10

Update publish-swift.yml

ad5d662

Co-authored-by: Jesse de Wit <[email protected]>

Update publish-swift.yml

ccba7b8

Co-authored-by: Ross Savage <[email protected]>

Update publish-all-platforms.yml

0715a42

Co-authored-by: Ross Savage <[email protected]>

cnixbtc force-pushed the move-swift-workflow-to-main-repo branch from fde8e27 to 0715a42 Compare September 7, 2024 08:22

JssDWt approved these changes Sep 30, 2024

View reviewed changes

dangeross approved these changes Oct 17, 2024

View reviewed changes

cnixbtc merged commit 114eba1 into main Oct 23, 2024
151 of 157 checks passed

cnixbtc deleted the move-swift-workflow-to-main-repo branch October 23, 2024 17:39

cnixbtc mentioned this pull request Oct 23, 2024

Remove CI workflow breez/breez-sdk-swift#17

Draft

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Move Swift publishing workflow to main repo #1033

Move Swift publishing workflow to main repo #1033

cnixbtc commented Jul 8, 2024 •

edited

Loading

JssDWt left a comment

JssDWt commented Aug 16, 2024

dangeross commented Aug 16, 2024

cnixbtc commented Aug 20, 2024

cnixbtc commented Aug 31, 2024

yaslama commented Aug 31, 2024

JssDWt commented Sep 1, 2024

dangeross Sep 3, 2024 •

edited

Loading

cnixbtc Sep 3, 2024

dangeross Sep 3, 2024

cnixbtc Sep 3, 2024

cnixbtc commented Sep 3, 2024 •

edited

Loading

cnixbtc commented Sep 3, 2024 •

edited

Loading

JssDWt commented Sep 5, 2024

JssDWt commented Sep 5, 2024

cnixbtc commented Sep 7, 2024 •

edited

Loading

cnixbtc commented Sep 7, 2024 •

edited

Loading

JssDWt commented Sep 8, 2024

cnixbtc commented Sep 30, 2024

JssDWt left a comment

dangeross commented Sep 30, 2024

cnixbtc commented Oct 16, 2024

dangeross left a comment

cnixbtc commented Oct 23, 2024 •

edited

Loading

	name: breez_sdkFFI-${{ inputs.package-version \|\| intouts.ref }}.xcframework
	name: breez_sdkFFI-${{ inputs.package-version \|\| inputs.ref }}.xcframework

Move Swift publishing workflow to main repo #1033

Move Swift publishing workflow to main repo #1033

Conversation

cnixbtc commented Jul 8, 2024 • edited Loading

Swift Publishing Workflow Consolidation

📖 Overview

❓ Outstanding Questions

✋ Feedback Request

JssDWt left a comment

Choose a reason for hiding this comment

JssDWt commented Aug 16, 2024

dangeross commented Aug 16, 2024

cnixbtc commented Aug 20, 2024

cnixbtc commented Aug 31, 2024

yaslama commented Aug 31, 2024

JssDWt commented Sep 1, 2024

dangeross Sep 3, 2024 • edited Loading

Choose a reason for hiding this comment

cnixbtc Sep 3, 2024

Choose a reason for hiding this comment

dangeross Sep 3, 2024

Choose a reason for hiding this comment

cnixbtc Sep 3, 2024

Choose a reason for hiding this comment

cnixbtc commented Sep 3, 2024 • edited Loading

cnixbtc commented Sep 3, 2024 • edited Loading

JssDWt commented Sep 5, 2024

JssDWt commented Sep 5, 2024

cnixbtc commented Sep 7, 2024 • edited Loading

cnixbtc commented Sep 7, 2024 • edited Loading

JssDWt commented Sep 8, 2024

cnixbtc commented Sep 30, 2024

JssDWt left a comment

Choose a reason for hiding this comment

dangeross commented Sep 30, 2024

cnixbtc commented Oct 16, 2024

dangeross left a comment

Choose a reason for hiding this comment

cnixbtc commented Oct 23, 2024 • edited Loading

cnixbtc commented Jul 8, 2024 •

edited

Loading

dangeross Sep 3, 2024 •

edited

Loading

cnixbtc commented Sep 3, 2024 •

edited

Loading

cnixbtc commented Sep 3, 2024 •

edited

Loading

cnixbtc commented Sep 7, 2024 •

edited

Loading

cnixbtc commented Sep 7, 2024 •

edited

Loading

cnixbtc commented Oct 23, 2024 •

edited

Loading