Skip to content
This repository has been archived by the owner on Sep 15, 2023. It is now read-only.

F-Droid build failed #260

Closed
linsui opened this issue Sep 15, 2021 · 13 comments
Closed

F-Droid build failed #260

linsui opened this issue Sep 15, 2021 · 13 comments

Comments

@linsui
Copy link

linsui commented Sep 15, 2021 

FAILURE: Build failed with an exception.

* What went wrong:
A problem occurred configuring root project 'CovidCertificate'.
> Could not resolve all artifacts for configuration ':classpath'.
   > Could not resolve ch.ubique.gradle:ubdiag-android:7.0.2.
     Required by:
         project :
      > No matching variant of ch.ubique.gradle:ubdiag-android:7.0.2 was found. The consumer was configured to find a runtime of a library compatible with Java 8, packaged as a jar, and its dependencies declared externally, as well as attribute 'org.gradle.plugin.api-version' with value '7.0.2' but:
          - Variant 'apiElements' capability ch.ubique.gradle:ubdiag-android:7.0.2 declares a library, packaged as a jar, and its dependencies declared externally:
              - Incompatible because this component declares an API of a component compatible with Java 11 and the consumer needed a runtime of a component compatible with Java 8
              - Other compatible attribute:
                  - Doesn't say anything about org.gradle.plugin.api-version (required '7.0.2')
          - Variant 'runtimeElements' capability ch.ubique.gradle:ubdiag-android:7.0.2 declares a runtime of a library, packaged as a jar, and its dependencies declared externally:
              - Incompatible because this component declares a component compatible with Java 11 and the consumer needed a component compatible with Java 8
              - Other compatible attribute:
                  - Doesn't say anything about org.gradle.plugin.api-version (required '7.0.2')

Could you please take a look? Thanks!
@thgoebel
Copy link
Contributor

Yep, Java 11. I'm already making a PR, just fighting to get one of the many way to run F-Droid server working...

@goebelUB
Copy link
Contributor

C.f. https://gitlab.com/fdroid/fdroiddata/-/merge_requests/9775

@goebelUB
Copy link
Contributor

goebelUB commented Oct 1, 2021 •
edited
Loading

It would be great if the above PR could be merged soonish. Then F-Droid bot can copy a working build script when it picks up v2.6.0, which I am going to tag for the bot sometime late this Sunday (2021-10-03).

That said, the build for v2.5.0 will still fail because of the "Unknown Maven Repo". Hence the PR doesn't remove the disable: line for v2.5.0.
However the offending repo has been removed in v2.6.0 so unless I missed something (or there are other reproducibility issues...) that version has good chances of making it onto F-Droid again! 🤞

@goebelUB
Copy link
Contributor

goebelUB commented Oct 7, 2021

Here's a new MR: https://gitlab.com/fdroid/fdroiddata/-/merge_requests/9885
It doesn't quite reproduce. I would highly appreciate any help as to why one APK/ZIP is deflated, while the other is not. I can't quite see what is different between our's and F-Droid's build environment.

@goebelUB goebelUB mentioned this issue Oct 7, 2021
Closed
@goebelUB
Copy link
Contributor

goebelUB commented Oct 8, 2021

Here is the diffoscope as to why F-Droid's build pipeline fails:

$ diffoscope verifier-prod-2.6.0-2600-signed.apk ch.admin.bag.covidcertificate.verifier_2600.apk
--- apk-official-signed.apk
+++ apk-fdroid-unsigned.apk
├── zipinfo {}
│ @@ -1,8 +1,8 @@
│ -Zip file size: 9921218 bytes, number of entries: 808
│ +Zip file size: 9845778 bytes, number of entries: 805
│  -rw-rw-rw-  0.0 unx       55 b- defN 81-Jan-01 01:01 META-INF/com/android/build/gradle/app-metadata.properties
│  -rw-rw-rw-  0.0 unx  8419692 b- defN 81-Jan-01 01:01 classes.dex
│  -rw-rw-rw-  0.0 unx   471540 b- defN 81-Jan-01 01:01 classes2.dex
│  -rw-rw-rw-  0.0 unx  9249884 b- defN 81-Jan-01 01:01 classes3.dex
│  -rw-rw-rw-  0.0 unx    27241 b- defN 81-Jan-01 01:01 assets/faq/config.json
│  -rw-rw-rw-  0.0 unx     2898 b- defN 81-Jan-01 01:01 assets/impressum/de/impressum.html
│  -rw-rw-rw-  0.0 unx     2684 b- defN 81-Jan-01 01:01 assets/impressum/de/licence.html
│ @@ -800,11 +800,8 @@
│  -rw----     0.0 fat      424 b- defN 81-Jan-01 01:01 res/z1.xml
│  -rw----     0.0 fat      396 b- defN 81-Jan-01 01:01 res/z3.xml
│  -rw----     0.0 fat     1116 b- defN 81-Jan-01 01:01 res/zH.xml
│  -rw----     0.0 fat     1264 b- defN 81-Jan-01 01:01 res/zI.xml
│  -rw----     0.0 fat     1388 b- defN 81-Jan-01 01:01 res/zl.xml
│  -rw----     0.0 fat      840 b- defN 81-Jan-01 01:01 res/zq.xml
│  -rw----     0.0 fat  1046104 b- stor 81-Jan-01 01:01 resources.arsc
│ --rw-rw-rw-  0.0 unx    73262 b- defN 81-Jan-01 01:01 META-INF/CERT.SF
│ --rw-rw-rw-  0.0 unx     1297 b- defN 81-Jan-01 01:01 META-INF/CERT.RSA
│ --rw-rw-rw-  0.0 unx    73188 b- defN 81-Jan-01 01:01 META-INF/MANIFEST.MF
│ -808 files, 22263224 bytes uncompressed, 9820772 bytes compressed:  55.9%
│ +805 files, 22115477 bytes uncompressed, 9753990 bytes compressed:  55.9%
├── filetype from file(1)
│ @@ -1 +1 @@
│ -Zip archive data, at least v0.0 to extract, compression method=store
│ +Zip archive data, at least v0.0 to extract, compression method=deflate

The diff in the META-INF is expected since the F-Droid apk is unsigned.
The crucial part therefore is the diff on the ZIP:

│ -Zip archive data, at least v0.0 to extract, compression method=store
│ +Zip archive data, at least v0.0 to extract, compression method=deflate

@goebelUB
Copy link
Contributor

goebelUB commented Oct 8, 2021 •
edited
Loading

Here is my current progress on hunting this down. I'm posting this because maybe someone can give a pointer what else to try, or maybe it'll help someone in the future.

xxd confirms the different compression methods:

official: 00000000: 504b 0304 0000 0000 0000 2108 2102 0000  PK........!.!...
fdroid:   00000000: 504b 0304 0000 0000 0800 2108 2102 43bc  PK........!.!.C.

The bytes at offset 8 and 9 indicate the compression method. Bytes 16+17 are expected to be different since they are the CRC (which will differ since one zip contains the signature files).

More digging shows that this was introduced between v2.0.0 and v2.1.0 (the latter didn't make it to F-Droid since it contained a JAR):

$ xxd wallet-prod-2.0.0-2000-signed.apk | head -1
00000000: 504b 0304 0000 0000 0800 2108 2102 4611  PK........!.!.F.
$ xxd wallet-prod-2.1.0-2100-signed.apk | head -1
00000000: 504b 0304 0000 0000 0000 2108 2102 0000  PK........!.!...

With a git bisect I eventually land at this commit. And indeed: if I check out v2.6.0-2600-wallet and and build the wallet app locally, I get 0000 (STORE) but if I change the minSdkVersion back up to 24 and build the wallet again, I suddenly get 0800 (DEFLATE).
So it seems that a change in the minSdkVersion is changing something about how the APK is zipped. Weird!

To find out next:

  1. Why do different SDK levels result in different compression methods? (Out of curiosity)
  2. Why does F-Droid, when it builds v2.6.0-wallet arrive at DEFLATE while locally I arrive at STORE?

And by "locally" I mean: our Dockerfile as well as my local install of the Android SDK on both my MacOS and Ubuntu laptops -- all three arrive at STORE.
The Android build-tools are the same (30.0.2, as inherited from the Android Gradle Plugin), and it's the same Gradle version (7.0.2). What else could be different? Another part of the Android SDK?

@linsui
Copy link
Author

linsui commented Oct 10, 2021

The build failed. Could you please take a look? https://monitor.f-droid.org/builds/log/ch.admin.bag.covidcertificate.wallet/2600#site-footer Thanks!

@goebelUB
Copy link
Contributor

The error message in that build is caused by the missing Java 11.
https://gitlab.com/fdroid/fdroiddata/-/merge_requests/9885 fixes that, but does not fix the build entirely since it fails to reproduce (see the discussion above).

Until we can hunt down why on earth F-Droid zips it differently than my local development environment and our Docker container do, you can disable the build again. If you or someone else from community who know the F-Droid build environment better than I do have any guesses, please do let me know :)

@linsui
Copy link
Author

linsui commented Oct 11, 2021

Maybe @obfusk ? Could you please take a look?

@goebelUB
Copy link
Contributor

Here's a status update, comparing v2.6.1 the official and F-Droid's CI builds:

diffoscopeand apksigcopier compare --unsigned don't verify it. However, with our simple script the two APKs verify.
Python abstracts away the unzipping, thus all the files contained in the APK/ZIP seem to be reproducible, but the ZIP metadata isn't.

The F-Droid docs mention 2 potential culprits. However opting out from zipflinger is deprecated and will be removed from Android Gradle Plugin 8.0, and for "ZIP entry info" it describes only the problem not a potential solution. We'll continue to look into it.

goebelUB added a commit that referenced this issue Oct 29, 2021
@goebelUB
Workaround: Bump minSdkVersion back up to 24
ef9c9a7 
This is a potential workaround to help F-Droid reproduce our builds again.
See #260 (comment)
for some background.
@goebelUB
Copy link
Contributor

As a workaround, I created a release for F-Droid that is exactly the same as the main release but with minSdkVersion set to 24 (instead of 23). This allows F-Droid to reproduce it again... 🤷‍♂️ It's not ideal, I would still like to find out why this is happening.

This shouldn't impact F-Droid users much, since the last release on F-Droid also had minSdk 24.
Google Play and AppGallery will continue to get the normal release with minSdk 23.
Furthermore, unless you are on API level 23, you are still able to cross-install all APKs since they are signed by the same key.

See my PR here https://gitlab.com/fdroid/fdroiddata/-/merge_requests/10000. It would be awesome if someoneone could have a look!

@goebelUB goebelUB mentioned this issue Nov 1, 2021
Open
@goebelUB
Copy link
Contributor

goebelUB commented Nov 1, 2021 •
edited
Loading

Thanks again @linsui for merging my MR with lightning speed!

I'll track bringing F-Droid back into the main release (instead of separate releases) in #304.

@goebelUB goebelUB closed this as completed Nov 1, 2021
@linsui
Copy link
Author

linsui commented Nov 1, 2021

Thanks!

Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
3 participants
@thgoebel @linsui @goebelUB