F-Droid can't build - not RB

[licaon-kter]

ref: https://monitor.f-droid.org/builds/log/com.starry.myne/9#site-footer

/LE: fyi https://gitlab.com/fdroid/fdroiddata/-/commit/de76cb1a0c45616ec0afcea093ced38f5f1a9c4d

[licaon-kter]

Also 11 https://gitlab.com/fdroid/fdroiddata/-/commit/ef553af3b3a7c7102f9ee63cdc444dab93d5df7b

verification of APK with copied signature failed
Comparing reference APK to APK with copied signature...
Unexpected diff output:
Binary files /tmp/tmp9ao0125l/unsigned_binaries_com.starry.myne_11.binary/content/assets/dexopt/baseline.prof and /tmp/tmp9ao0125l/_tmp_tmp9ao0125l_sigcp_com.starry.myne_11/content/assets/dexopt/baseline.prof differ
Binary files /tmp/tmp9ao0125l/unsigned_binaries_com.starry.myne_11.binary/content/classes.dex and /tmp/tmp9ao0125l/_tmp_tmp9ao0125l_sigcp_com.starry.myne_11/content/classes.dex differ
diff -r /tmp/tmp9ao0125l/unsigned_binaries_com.starry.myne_11.binary/content/res/GM.json /tmp/tmp9ao0125l/_tmp_tmp9ao0125l_sigcp_com.starry.myne_11/content/res/GM.json
1,3362c1,3362
< {
<   "v": "5.7.8",
<   "fr": 24,
<   "ip": 0,
<   "op": 151,
<   "w": 1080,
<   "h": 1080,
<   "nm": "EMOJI_CORACAO",
...

[starry-shivam]

Thanks for notifying,

Also 11 https://gitlab.com/fdroid/fdroiddata/-/commit/ef553af3b3a7c7102f9ee63cdc444dab93d5df7b

verification of APK with copied signature failed
Comparing reference APK to APK with copied signature...
Unexpected diff output:
Binary files /tmp/tmp9ao0125l/unsigned_binaries_com.starry.myne_11.binary/content/assets/dexopt/baseline.prof and /tmp/tmp9ao0125l/_tmp_tmp9ao0125l_sigcp_com.starry.myne_11/content/assets/dexopt/baseline.prof differ
Binary files /tmp/tmp9ao0125l/unsigned_binaries_com.starry.myne_11.binary/content/classes.dex and /tmp/tmp9ao0125l/_tmp_tmp9ao0125l_sigcp_com.starry.myne_11/content/classes.dex differ
diff -r /tmp/tmp9ao0125l/unsigned_binaries_com.starry.myne_11.binary/content/res/GM.json /tmp/tmp9ao0125l/_tmp_tmp9ao0125l_sigcp_com.starry.myne_11/content/res/GM.json
1,3362c1,3362
< {
<   "v": "5.7.8",
<   "fr": 24,
<   "ip": 0,
<   "op": 151,
<   "w": 1080,
<   "h": 1080,
<   "nm": "EMOJI_CORACAO",
...

This looks like json file for some Lottie animation, I'll have to check mappings.txt file of the last build to confirm though, but I'm not sure why this would cause issues 🤔

[licaon-kter]

.json aside for 11 classes.dex and baseline differ, will need to diff them locally next

ref: https://f-droid.org/docs/Reproducible_Builds/#potential-sources-of-unreproducible-builds

[starry-shivam]

.json aside for 11 classes.dex and baseline differ, will need to diff them locally next

Thanks, please update me if you find something which i could change to fix it, as i don't really havs much knowledge/experience of reproducible builds 😅

[licaon-kter]

Here's the diff log:
my.zip

[starry-shivam]

Here's the diff log: my.zip

Thanks, but unfortunately I've no idea how to read/understand what went wrong using these logs, as i don't really know anything about reproducible builds. Can you link me some guide which i can read to check reproducibility of builds myself using whatever tool used by f-droid to ensure reproducibility.

[licaon-kter]

As linked the base prof thing can be seen here: https://f-droid.org/docs/Reproducible_Builds/#potential-sources-of-unreproducible-builds

@linsui is the rest of the diff log a know type already?

[linsui]

Looks like only the java part is different.

[starry-shivam]

@licaon-kter I've read your linked article but wasn't able to figure out what exactly was causing it, I'm using gradle 7.4.x so gradle issue as linked in that article shouldn't be a culprit, same goes for coreLibraryDesugaring as app doesn't uses core library desugaring, about windows newlines, we've talked about and fixed it in app's yml file in f-droid data repository when i opened a RFP issue for the listing and timestamp related issue was also resolved around same time in the merge request. And other reasons about NDK and build paths etc aren't related here as far as i can understand.

[licaon-kter]

fyi https://gitlab.com/fdroid/fdroiddata/-/commit/3d4883f3635f24dffd49e6e2e48df91634548e24

[licaon-kter]

@starry-shivam can you build an APK from commit: 6f9af6d with these two as false? https://github.com/Pool-Of-Tears/Myne/blob/main/app/build.gradle#L37-L38

[starry-shivam]

@starry-shivam can you build an APK from commit: 6f9af6d with these two as false? https://github.com/Pool-Of-Tears/Myne/blob/main/app/build.gradle#L37-L38

Sure, will do in the evening.

[starry-shivam]

@licaon-kter here you go, I've uploaded it on mediafire as i couldn't find any ways to upload it here directly.

[starry-shivam]

Hello, any updates on this?

[licaon-kter]

Better ;)

The baseline issue might be fixed as explained here: https://f-droid.org/docs/Reproducible_Builds/#potential-sources-of-unreproducible-builds

The rest is still in lottie.json: myne1.log

[starry-shivam]

The baseline issue might be fixed

Cool!

The rest is still in lottie.json: myne1.log

Any idea what can i do to solve this? I'm not sure why it'll cause the issue..

[linsui]

Maybe you can sort it before it's packaged?

[starry-shivam]

Maybe you can sort it before it's packaged?

Sort it like? It's basically a json file stored in the raw resource directory.

[linsui]

Oh, sorry, it's not a order problem. Maybe it's about the line end. Then we can fix it on our side.

[starry-shivam]

Oh, sorry, it's not a order problem. Maybe it's about the line end. Then we can fix it on our side.

Nice! Btw since we're here, can anyone check why the screenshots for both of my apps this and GreenStash are kinda mixed up with old screenshots (which I've already deleted from fastlane directory long ago) Idk why it happens but it looks pretty weird..

[linsui]

I thought it's a known issue. But I don't know how to fix it.

[licaon-kter]

We do know but not yet fixed :)

[starry-shivam]

alright, not a big deal.

[licaon-kter]

Trying 2.3.0

diff log: myne230.log

local APK (Java17): com.starry.myne_17.apk.zip (remove .ZIP extenstion)

upstream APK: https://github.com/Pool-Of-Tears/Myne/releases/tag/v2.3.0

@linsui @obfusk

[obfusk]

Looks like broken resources: https://issuetracker.google.com/issues/287967713
Also one annotation in classes.dex, and newlines in META-INF/services/.

@licaon-kter the broken resources are on your end this time, and so are the windows newlines in META-INF/services/; that's really weird. Anything change in your setup?

[licaon-kter]

@obfusk

Anything change in your setup?

Not that I know, and doesn't affect any other apps either. 🤷

[licaon-kter]

I've rebuilt in another instance and my builds are repro between them

VM images rebuilt April 20 and June 30, fyi

[obfusk]

Ive rebuilt in another instance and my builds are repro between them

That's good to know. I see Google increased the priority and severity of the bug. Hopefully they'll fix it or at least give us some idea of what's going on. I don't really know how to debug this.

[starry-shivam]

Any updates on this?

[licaon-kter]

2.6.0 is fine https://gitlab.com/fdroid/fdroiddata/-/commit/8dbf581aae28376a5ab70e7ad5ad51cb5877c739 :)

[linsui]

https://monitor.f-droid.org/builds/log/com.starry.myne/391#site-footer

Only the zip metadata is different.

[starry-shivam]

https://monitor.f-droid.org/builds/log/com.starry.myne/391#site-footer

Only the zip metadata is different.

Weird, I didn't even make many changes in the latest release. You can check for changes here: v3.9.0...v3.9.1

[linsui]

Please test if you can still reproduce 3.9.0 apk.

[starry-shivam]

Please test if you can still reproduce 3.9.0 apk.

How can i test it? Should i build an apk from v3.9.0 tag and send it here?

[linsui]

You can build and sign it and compare it with the apk in release with diff.

[starry-shivam]

You can build and sign it and compare it with the apk in release with diff.

I don't know which tool you use to compare apks. Should I just try to compare the SHA-256 hashes of both files or something like that?

[linsui]

Yes, the hash should be the same. If the hash is different, please also check the content of the apk with diff -r. You can use diffoscope to inspect it.

[starry-shivam]

@linsui The hashes are different, but I think it is most likely because the GitHub build is compiled in a Linux environment with gradle directly, while the one I built is compiled with Android Studio on Windows 11. There must be different line endings (CRLF?), and Android Studio itself might do some extra stuff when archiving the APK file, though I'm not sure about that. However, diff -r didn't show anything useful, like what exactly are the differences, other than just indicating they are different.

[linsui]

Please unzip them then run diff -r. :) If you want to check the apk directly you can use diffoscope but I thought it's not available on Windows.

[starry-shivam]

I checked the diff with Meld and it shows these three files which are present in the GitHub build of v3.9.0 but missing in the build I compiled just now with Android Studio. They seem to be some signing-related keys, though I’ve signed this build with the same signature that I use for the GitHub builds.

[linsui]

They are signature files. Hmmm, that shouldn't affect our reproducible build. Can you upload the unsigned apk here?

[starry-shivam]

They are signature files. Hmmm, that shouldn't affect our reproducible build. Can you upload the unsigned apk here?

Sure, but it seems GitHub doesn't support uploading APK files in comments. I'll upload it somewhere else and share the link in a moment.

[linsui]

You can zip it.

[starry-shivam]

You can zip it.

Oh, good idea. Here you go
Myne-v3.9.0.zip

[linsui]

Your build has other difference. I thought they are mainly caused by CRLF.

[starry-shivam]

Your build has other difference. I thought they are mainly caused by CRLF.

Hmm, maybe. Still, I'm not really sure what exactly caused v3.9.1 to become unreproducible, especially considering I didn't even make any major changes between v3.9.0 and v3.9.1 like I usually do with other releases.

[linsui]

Yep, we also see this problem on other apps. It seems a toolchain is changed but we don't know what it is yet.

[licaon-kter]

@starry-shivam ref: https://gitlab.com/fdroid/fdroiddata/-/issues/3299#note_1985568674

so around here, I guess: https://github.com/Pool-Of-Tears/Myne/blob/v3.9.1/.github/workflows/release.yml#L55 make it do apksigner sign --alignment-preserved true ... (you are using some other external actions so not sure how to inject that in https://github.com/r0adkll/sign-android-release/blob/f30bdd30588842ac76044ecdbd4b6d0e3e813478/lib/signing.js#L66 )

eg. others have another setup like: https://github.com/ImranR98/Obtainium/blob/v1.1.13/.github/workflows/release.yml#L75

[starry-shivam]

you are using some other external actions so not sure how to inject that in https://github.com/r0adkll/sign-android-release/blob/f30bdd30588842ac76044ecdbd4b6d0e3e813478/lib/signing.js#L66 )

Looking at the code, I don't think the GitHub Actions plugin allows users to pass additional arguments. I think I'll have to run APK signer manually. I'll try and see how it goes.

[starry-shivam]

I've been trying to manually sign (#192), but I keep getting a "command not found" error. The error message isn't helpful because it doesn't specify which command has failed or the correct line number in the workflow file. I suspect it doesn't find the apksigner command, but the ANDROID_HOME environment variable seems to point to the correct path: /usr/local/lib/android/sdk. It would be great if someone could look at this and see if I'm doing something wrong. I'm not particularly good with GitHub Actions, which is why I was using a third-party plugin to get the job done.

[linsui]

ls: cannot access 'app/build/outputs/apk/release/app-release-unsigned.apk': No such file or directory

[licaon-kter]

Based on the docs here: https://github.com/obfusk/apksigcopier#what-about-signatures-made-by-apksigner-from-build-tools--3500-rc1

I've updated the recipe instead: https://gitlab.com/fdroid/fdroiddata/-/commit/be7626377555aa92f37e9310a3e7b36bc04df195

feel free to star the Google issue https://issuetracker.google.com/issues/351408623

[licaon-kter]

so #192 can be closed

[starry-shivam]

Based on the docs here: https://github.com/obfusk/apksigcopier#what-about-signatures-made-by-apksigner-from-build-tools--3500-rc1

I've updated the recipe instead: https://gitlab.com/fdroid/fdroiddata/-/commit/be7626377555aa92f37e9310a3e7b36bc04df195

feel free to star the Google issue https://issuetracker.google.com/issues/351408623

Wow thanks! I'll close that PR now.