API 29 support (Android 10.0)

[dan-v]

Just wanted to open an issue for tracking this. Google released Android 10.0 on September 3rd with a new API version 29.

I played around with the Q beta 6 factory image last month and noted down some of my findings here: https://gist.github.com/dan-v/a366c2cff6ef5a2f7e0022a39b1d5b00. I bumped into a number of new issues that will require fixes or more research to get everything working.

[dan-v]

Just to provide an update. Most of the devices are now booting and functional on 10.0 with changes in chirayudesai's fork. I also have marlin support in my fork.

[kuleszdl]

Another update: Seems like there is another fork by the GrapheneOS guys that added support for sailfish yesterday:

https://github.com/GrapheneOS/android-prepare-vendor

[Mawth]

Unfortunately none of the above forks work for me. Here's the error:

[] Running as root - using loopback for image mounts
[] Extracting '/media/bluebird/aosp10/aosp_b_prod/sailfish/qp1a/sailfish-qp1a.190711.020-factory-0bb95222.zip'
[] Unzipping 'image-sailfish-qp1a.190711.020.zip'
[] Extracting '/media/bluebird/aosp10/aosp_b_prod/sailfish/qp1a/sailfish-qp1a.190711.020-factory-0bb95222.zip'
Traceback (most recent call last):
File "/media/bluebird/aosp10/aosp_b_prod/scripts/extract_android_ota_payload/extract_android_ota_payload.py", line 17, in
import update_metadata_pb2
File "/media/bluebird/aosp10/aosp_b_prod/scripts/extract_android_ota_payload/update_metadata_pb2.py", line 4, in
from google.protobuf import descriptor as _descriptor
ImportError: No module named google.protobuf
[-] OTA data extract failed

Any ideas how to resolve? Thanks.

[kuleszdl]

@Mawth which versions did you use exactly? In my case I used version 54998cb from the GrapheneOS work with the following blobs and this seems to have worked fine:

./execute-all.sh -k -d sailfish -a sailfish -b PQ3A.190801.002 -o sailfish-blobs

Maybe you are missing some dependencies? I am using the packages as described in this tutorial.

[Mawth]

Thanks for the reply. I added Branch "10" from GrapheneOS/android-prepare-vendor to my local_manifest. For sailfish, I believe this pulls in d1f9e96.

Here's the specific script I used:

sudo ./execute-all.sh -d sailfish -b qp1a.190711.020 -i /media/bluebird/aosp10/aosp_b_prod/sailfish/qp1a/sailfish-qp1a.190711.020-factory-0bb95222.zip -o ~/Desktop

In your example above, you're extracting blobs from a Pie image. I am trying to extract from 10. That might be why you were successful and I have not been? Anyway, I appreciate your time.

[dan-v]

@Mawth - in the forked branches for 10.0, there is a new dependency on protobuf (e.g. if on ubuntu just apt-get install python-protobuf) which is used by a new script that downloads OTA updates and extract missing partitions that previously came from the factory image. This OTA extraction mechanism may just be a temporary solution, but depends on if the October images look any different or not.

[shareefalis]

GrapheneOS-Archive#4
here is my blob list in the comment and a fix to product partition file not being added,

[Mawth]

@dan-v - OK, thanks for that. If I understand correctly, then this method is currently not possible: "As an alternative to download images from Google's website, script can also read factory images from file-system location using the -i|--img flag." With the current scripts, it's necessary to download from Google's website. Next time I have access to a high-speed connection, I will try this method.

[anestisb]

Android 10 factory images have removed certain images required for building OTA zips from factory image zip. They need to be extracted from bootloader.img & radio.img via https://github.com/anestisb/qc_image_unpacker. Have started working Android-10 support for existing devices. Pixel 4 devices will follow.

[CaseyBakey]

Android 10 factory images have removed certain images required for building OTA zips from factory image zip. They need to be extracted from bootloader.img & radio.img via https://github.com/anestisb/qc_image_unpacker. Have started working Android-10 support for existing devices. Pixel 4 devices will follow.

Can you provide more information on this?

Can you also explain how to proceed to create/fill the config.json as you do? I mean, spotting/diffing the files not builded by AOSP and that need to be extracted from the zip?

Also how to determine the things needed only by "full" configuration vs those needed by "naked" AND "full" configurations?

I would like to help but I just need some insights on this :)

Thanks by advance!

[peaceforall]

Hello,
It seems sargo + Android 10 is still broken on every android-prepare-vendor repo I have tested ( from anetisb , @chirayudesai , GrapheneOS , etc. ) This is the error I'm seeing, which lines up with anetisb's comment:

# yes | ./execute-all.sh -d sargo -b qq1a.200105.002 -o vendorOutput

...

[*] Running as root - using loopback for image mounts
[*] Extracting '/storage/vendortest/android-prepare-vendor/vendorOutput/sargo/qq1a.200105.002/sargo-qq1a.200105.002-factory-0c3e12c2.zip'
[*] Unzipping 'image-sargo-qq1a.200105.002.zip'
[-] Failed to locate 'abl.img' in factory image
[-] Factory images data extract failed

@anestisb can you please provide instructions on how to use qc_image_unpacker ? The documentation is pretty sparse, especially relating to how it should be used with android-prepare-vendor.

[CaseyBakey]

@anestisb how could we help to add "full" configuration for all the Pixel devices? Show us the way please. How do you find that THIS package is needed?

Do we need to diff the filesystem of a stock ROM vs an AOSP ROM for the same device?

Each missig file/folder will need to be added to the .json?

[sdh4]

I was able to get a "full" configuration working on pixel 2 (walleye) based on the GrapheneOS fork.

In my fork at https://github.com/sdh4/android-prepare-vendor I made some minor changes (python fixes), pulled in shareefalis's orphan bytecode fix and based my walleye api-29 "full" on the api-28 values. Basically I mounted the current Google factory system image and looked for the files listed for api-28. I eliminated those that were entirely missing and fixed up the paths for those which had moved (many moved into a new "product" subdirectory).

The result has worked well for me, including on Verizon Wireless -- usually the toughest carrier for compatibility. See my locked bootloader AOSP build instructions for the rest of the steps needed to get a working system.

I have not attempted a full configuration for other devices but I'd be happy to take pull requests in my fork

[dan-v]

@anestisb - just wanted to check in on your status with this project. Looked like you were online a bit in December and made some good progress on a tool to properly extract Qualcomm images. There's been a lot of different people making changes for 10.0 and Pixel 4 support for the past few months, but there isn't a common place to share these changes at the moment with upstream not being updated. For now CalyxOS / #!os / RattlesnakeOS, plan on consolidating on this fork https://github.com/AOSPAlliance/android-prepare-vendor so that we still have a common place to share updates, but hoping we can all consolidate back here in the future.

[anestisb]

@dan-v thanks for reaching out. Yes ideally I would like to port the glueing parts (issues with wrapped around tools) and resume the Android-10 support. At the moment though I don't have any free time to support anything. Mostly likely nothing will change until June time frames.

So the best I can offer here is link the project/space that you plan to work on your changes and I'll try to port/use them when resume working on this project.

[dan-v]

@anestisb - no worries and thanks for the update. We'll try to keep things moving along until you are back online.

[chirayudesai]

Hey, we just pushed an update to the above fork,
https://github.com/AOSPAlliance/android-prepare-vendor

Main inclusion:

• Pixel 4 support (naked only)
• Misc download fixes

[chirayudesai]

We now have Pixel 4a (sunfish) support as well,

AOSPAlliance#10