Add docker scripts to simplify setting up idevicerestore #562

cameroncros · 2023-02-01T19:25:27Z

This makes it easier to setup idevicerestore with all the latest libmobiledevice libraries.

I have not tested this through a full restore, but it does detect my device (which is as far as I got following the official instructions anyway)

Example output:

❯ ./docker/run.sh
Sending build context to Docker daemon  4.096kB
Step 1/11 : FROM ubuntu:18.04
 ---> e28a50f651f9
Step 2/11 : RUN apt-get update && apt-get install -y 	build-essential 	pkg-config 	checkinstall 	git 	autoconf 	automake 	libtool-bin 	libreadline-dev 	libusb-1.0-0-dev 	libcurl4-openssl-dev 	libssl-dev 	libzip-dev 	zlib1g-dev     python3     cython
 ---> Using cache
 ---> f4f246aec778
Step 3/11 : RUN git clone https://github.com/libimobiledevice/libplist.git &&     cd libplist &&     ./autogen.sh &&     make &&     make install &&     cd .. &&     rm libplist -rf
 ---> Using cache
 ---> bde5605a4e79
Step 4/11 : RUN git clone https://github.com/libimobiledevice/libimobiledevice-glue.git &&     cd libimobiledevice-glue &&     ./autogen.sh &&     make &&     make install &&     cd .. &&     rm libimobiledevice-glue -rf
 ---> Using cache
 ---> a18f0f0d312f
Step 5/11 : RUN git clone https://github.com/libimobiledevice/libusbmuxd.git &&     cd libusbmuxd &&     ./autogen.sh &&     make &&     make install &&     cd .. &&     rm libusbmuxd -rf
 ---> Using cache
 ---> 7487aaf95ef8
Step 6/11 : RUN git clone https://github.com/libimobiledevice/libimobiledevice.git &&     cd libimobiledevice &&     ./autogen.sh &&     make &&     make install &&     cd .. &&     rm libimobiledevice -rf
 ---> Using cache
 ---> d9914575dc50
Step 7/11 : RUN git clone https://github.com/libimobiledevice/libirecovery.git &&     cd libirecovery &&     ./autogen.sh &&     make &&     make install &&     cd .. &&     rm libirecovery -rf
 ---> Using cache
 ---> c3a4e07b521f
Step 8/11 : RUN git clone https://github.com/libimobiledevice/idevicerestore.git &&     cd idevicerestore &&     ./autogen.sh &&     make &&     make install &&     cd .. &&     rm idevicerestore -rf
 ---> Using cache
 ---> 2c151c072140
Step 9/11 : RUN ldconfig
 ---> Using cache
 ---> c9890a92e34b
Step 10/11 : WORKDIR /tmp
 ---> Using cache
 ---> f5aa930da9b3
Step 11/11 : CMD idevicerestore
 ---> Using cache
 ---> bd1c899c82a3
Successfully built bd1c899c82a3
Successfully tagged idevicerestore-docker:latest
idevicerestore 1.0.0-128-g7b89019
Found device in DFU mode
ECID: 5991944144224286
Identified device as j313ap, MacBookAir10,1
The following firmwares are currently being signed for MacBookAir10,1:
  [1] 13.2 (build 22D49)
...
  [27] 11.0.1 (build 20B29)
Select the firmware you want to restore:

remoosaad · 2024-09-22T18:03:31Z

This makes it easier to setup idevicerestore with all the latest libmobiledevice libraries.

I have not tested this through a full restore, but it does detect my device (which is as far as I got following the official instructions anyway)

Example output:


❯ ./docker/run.sh

Sending build context to Docker daemon  4.096kB

Step 1/11 : FROM ubuntu:18.04

 ---> e28a50f651f9

Step 2/11 : RUN apt-get update && apt-get install -y 	build-essential 	pkg-config 	checkinstall 	git 	autoconf 	automake 	libtool-bin 	libreadline-dev 	libusb-1.0-0-dev 	libcurl4-openssl-dev 	libssl-dev 	libzip-dev 	zlib1g-dev     python3     cython

 ---> Using cache

 ---> f4f246aec778

Step 3/11 : RUN git clone https://github.com/libimobiledevice/libplist.git &&     cd libplist &&     ./autogen.sh &&     make &&     make install &&     cd .. &&     rm libplist -rf

 ---> Using cache

 ---> bde5605a4e79

Step 4/11 : RUN git clone https://github.com/libimobiledevice/libimobiledevice-glue.git &&     cd libimobiledevice-glue &&     ./autogen.sh &&     make &&     make install &&     cd .. &&     rm libimobiledevice-glue -rf

 ---> Using cache

 ---> a18f0f0d312f

Step 5/11 : RUN git clone https://github.com/libimobiledevice/libusbmuxd.git &&     cd libusbmuxd &&     ./autogen.sh &&     make &&     make install &&     cd .. &&     rm libusbmuxd -rf

 ---> Using cache

 ---> 7487aaf95ef8

Step 6/11 : RUN git clone https://github.com/libimobiledevice/libimobiledevice.git &&     cd libimobiledevice &&     ./autogen.sh &&     make &&     make install &&     cd .. &&     rm libimobiledevice -rf

 ---> Using cache

 ---> d9914575dc50

Step 7/11 : RUN git clone https://github.com/libimobiledevice/libirecovery.git &&     cd libirecovery &&     ./autogen.sh &&     make &&     make install &&     cd .. &&     rm libirecovery -rf

 ---> Using cache

 ---> c3a4e07b521f

Step 8/11 : RUN git clone https://github.com/libimobiledevice/idevicerestore.git &&     cd idevicerestore &&     ./autogen.sh &&     make &&     make install &&     cd .. &&     rm idevicerestore -rf

 ---> Using cache

 ---> 2c151c072140

Step 9/11 : RUN ldconfig

 ---> Using cache

 ---> c9890a92e34b

Step 10/11 : WORKDIR /tmp

 ---> Using cache

 ---> f5aa930da9b3

Step 11/11 : CMD idevicerestore

 ---> Using cache

 ---> bd1c899c82a3

Successfully built bd1c899c82a3

Successfully tagged idevicerestore-docker:latest

idevicerestore 1.0.0-128-g7b89019

Found device in DFU mode

ECID: 5991944144224286

Identified device as j313ap, MacBookAir10,1

The following firmwares are currently being signed for MacBookAir10,1:

  [1] 13.2 (build 22D49)

...

  [27] 11.0.1 (build 20B29)

Select the firmware you want to restore:

cameroncros · 2024-09-23T21:16:45Z

Is this something that has value? What do you need from me to get this to a mergable state?

arkanoid87 · 2024-10-22T01:21:55Z

my device was not recognized with the packages in official ubuntu 24.04 repositories, but this docker image made it

thanks @cameroncros

arkanoid87 · 2024-10-22T02:13:15Z

well, not really.
I got showstopper ERROR: Device did not disconnect. Possibly invalid iBEC. Reset device and try again

running idevicerestore --erase -d iPad_Spring_2019_17.7_21H16_Restore.ipsw from RECOVERY mode:

...
Extracting iBEC.j217.RELEASE.im4p (Firmware/dfu/iBEC.j217.RELEASE.im4p)...
Personalizing IMG4 component iBEC...
Tag found
Sending iBEC (1479099 bytes)...
Waiting for device to disconnect...
ERROR: Device did not disconnect. Possibly invalid iBEC. Reset device and try again.

running idevicerestore --erase -d iPad_Spring_2019_17.7_21H16_Restore.ipsw from DFU mode:

...
Extracting iBSS.j217.RELEASE.im4p (Firmware/dfu/iBSS.j217.RELEASE.im4p)...
Personalizing IMG4 component iBSS...
Tag found
Sending iBSS (1479099 bytes)...
[==================================================] 100.0%
Waiting for device to disconnect...
ERROR: Device did not disconnect. Possibly invalid iBSS. Reset device and try again.
ERROR: Unable to place device into recovery mode from DFU mode

in both cases dmesg shows device disconnecting from source mode and reconnecting in Recovery Mode, but docker image seems not getting the events? Maybe some provileges are missing?

[36721.573835] usb 1-1: new high-speed USB device number 43 using xhci_hcd                                                                                                                    
[36721.700861] usb 1-1: New USB device found, idVendor=05ac, idProduct=1227, bcdDevice= 0.00                                                                                                  
[36721.700879] usb 1-1: New USB device strings: Mfr=2, Product=3, SerialNumber=4                                                                                                              
[36721.700887] usb 1-1: Product: Apple Mobile Device (DFU Mode)                                                                                                                               
[36721.700894] usb 1-1: Manufacturer: Apple Inc.                                                                                                                                              
[36721.700900] usb 1-1: SerialNumber: CPID:8020 CPRV:11 CPFM:03 SCEP:01 BDID:1C ECID:00054D3E2105002E IBFL:3C SRTG:[iBoot-3865.0.0.4.7]                                                                                                                                                                                              
[36739.714160] apple-mfi-fastcharge 1-1: USB disconnect, device number 43                                                                                                                     
[36743.744625] usb 1-1: new high-speed USB device number 44 using xhci_hcd                                                                                                                    
[36743.873487] usb 1-1: New USB device found, idVendor=05ac, idProduct=1281, bcdDevice= 0.00                                                                                                  
[36743.873525] usb 1-1: New USB device strings: Mfr=2, Product=3, SerialNumber=4                                                                                                              
[36743.873537] usb 1-1: Product: Apple Mobile Device (Recovery Mode)                                                                                                                          
[36743.873546] usb 1-1: Manufacturer: Apple Inc.                                                                                                                                              
[36743.873553] usb 1-1: SerialNumber: SDOM:01 CPID:8020 CPRV:11 CPFM:03 SCEP:01 BDID:1C ECID:00054D3E2105002E IBFL:3C SRNM:[F9FZG2HJLMPD]

[36935.136044] usb 1-1: new high-speed USB device number 45 using xhci_hcd
[36935.264985] usb 1-1: New USB device found, idVendor=05ac, idProduct=1281, bcdDevice= 0.00
[36935.265004] usb 1-1: New USB device strings: Mfr=2, Product=3, SerialNumber=4
[36935.265012] usb 1-1: Product: Apple Mobile Device (Recovery Mode)
[36935.265018] usb 1-1: Manufacturer: Apple Inc.
[36935.265024] usb 1-1: SerialNumber: SDOM:01 CPID:8020 CPRV:11 CPFM:03 SCEP:01 BDID:1C ECID:00054D3E2105002E IBFL:3D SRNM:[F9FZG2HJLMPD]
[36944.947433] apple-mfi-fastcharge 1-1: USB disconnect, device number 45
[36948.770186] usb 1-1: new high-speed USB device number 46 using xhci_hcd
[36948.899159] usb 1-1: New USB device found, idVendor=05ac, idProduct=1281, bcdDevice= 0.00
[36948.899177] usb 1-1: New USB device strings: Mfr=2, Product=3, SerialNumber=4
[36948.899185] usb 1-1: Product: Apple Mobile Device (Recovery Mode)
[36948.899192] usb 1-1: Manufacturer: Apple Inc.
[36948.899198] usb 1-1: SerialNumber: SDOM:01 CPID:8020 CPRV:11 CPFM:03 SCEP:01 BDID:1C ECID:00054D3E2105002E IBFL:3C SRNM:[F9FZG2HJLMPD]

arkanoid87 · 2024-10-22T03:22:36Z

I am pretty sure idevicerestore successfully sends command to put device into recovery store (both when starting from DFU or recovery) but is not getting the event device entered recovery mode.
But the device really is, as I can read it from lsusb, dmesg but also from irecovery -q from the very same image

for completeness, please note that I've slightly modified run.sh to make room for udev events and general commands and parameters

#!/bin/bash
set -e
cd "$(dirname "$0")"

docker build . -t idevicerestore-docker
docker run -it --net=host --privileged -v /run/udev/control:/run/udev/control -v "$(pwd):/tmp" idevicerestore-docker $@

arkanoid87 · 2024-10-22T03:46:51Z

Small improvement. Adding --net=host -v /run/udev/control:/run/udev/control made idevicerestore
detect disconnection, but yet fails to detect re-connection (that happens)

Waiting for device to disconnect...                                                                                                                                                           
irecv_event_cb: device 00054d3e2105002e (udid: N/A) disconnected                                                                                                                 
Waiting for device to reconnect in recovery mode...                                                                                                                                           
ERROR: Device did not reconnect in recovery mode. Possibly invalid iBEC. Reset device and try again.

I can see that the Waiting for device to reconnect in recovery mode... persists for 60 seconds, but Product: Apple Mobile Device (Recovery Mode) appears in dmesg after 5 seconds from irecv_event_cb: device 00054d3e2105002e (udid: N/A) disconnected. This means that this line is where things goes wrong:

idevicerestore/src/idevicerestore.c

Line 1449 in 151c680

    
           cond_wait_timeout(&client->device_event_cond, &client->device_event_mutex, 60000);

cameroncros · 2024-10-22T09:09:47Z

@arkanoid87 I've updated the readme and run script with your suggestions, but I cant test as I don't have a spare device to wipe.

Unfortunately, I think this is a waste of time, as it seems this project is uninterested in this contribution. (In fairness, it doesn't work, but I haven't heard any communication, not even a "This wont work due to XYZ, pls fix").

I'll leave the MR up until it gets closed, and hopefully someone with a device can actually test it and work out what is missing.

nikias · 2024-10-22T09:35:19Z

I would gladly add a docker script to the repository, if it works fine. I don't have the time to debug this at the moment. Something keeps the event from being received in the libirecovery code. On Linux the device detection is implemented via libusb's hotplug API. It works on a normal system so I don't know what would be wrong here...

cameroncros · 2024-10-22T09:44:32Z

That's all I needed to hear, but the otherwise radio silence for over a year was pretty disheartening. I will work out how to get this working correctly, and in the meantime, make this MR a draft

cameroncros · 2024-10-22T12:09:44Z

Look at that, motivated :D

...
Unmounting filesystems (29)
Checkpoint completed id: 0x642 (unmount_filesystems) result=0
Checkpoint started   id: 0x67C (cleanup_boot_command)
Checkpoint completed id: 0x67C (cleanup_boot_command) result=0
Checkpoint started   id: 0x1613 (cleanup_recovery_os_volume)
Checkpoint completed id: 0x1613 (cleanup_recovery_os_volume) result=0
Checkpoint started   id: 0x647 (cleanup_check_result)
Checkpoint completed id: 0x647 (cleanup_check_result) result=0
Checkpoint started   id: 0x6C2 (cleanup_send_crash_logs)
Checkpoint completed id: 0x6C2 (cleanup_send_crash_logs) result=0
Checkpoint started   id: 0x648 (cleanup_send_final_status)
Got status message
Status: Restore Finished
ReverseProxy[Ctrl]: (status=2) Terminated
DONE
[==================================================] 100.0%

Should now work correctly. @arkanoid87 can you validate that it now works? The run command is now ./run.sh --latest. It successfully restored my laptop.

arkanoid87 · 2024-10-22T12:31:02Z

I was hacking in the same direction! I was wrapping usbmuxd too :D

arkanoid87 · 2024-10-22T12:41:50Z

pulled your changes

almost there, but not yet

About to restore device... 
restore_is_current_device: Connected to com.apple.mobile.restored, version 15
Connecting now...
Connected to com.apple.mobile.restored, version 15
Device 00008020-00054D3E2105002E has successfully entered restore mode
Hardware Information:
BoardID: 28
ChipID: 32800
UniqueChipID: 1492304120840238
ProductionMode: true
Starting Reverse Proxy
ReverseProxy[Ctrl]: ERROR: Could not receive BeginCtrl plist reply, error -256
Device didn't accept new reverse proxy protocol, trying to use old one
idevice_event_cb: device 00054d3e2105002e (udid: 00008020-00054D3E2105002E) disconnected
Could not create Reverse Proxy
Device will not use nonce slots.
ERROR: Unable to start the restore process
ERROR: Unable to restore device

cameroncros · 2024-10-22T13:56:38Z

What are you restoring? I restored an m1 air, don't have any other devices to test with.

arkanoid87 · 2024-10-22T14:37:26Z

I was trying not with latest but with second available firmware (iPad_Spring_2019_17.7_21H16_Restore.ipsw) Now I tried with latest one (iPad_Spring_2019_18.0.1_22A3370_Restore.ipsw) and it succeeded!!!

Got status message
Status: Restore Finished
ReverseProxy[Ctrl]: Terminating
ReverseProxy[Ctrl]: (status=2) Terminated
DONE

Thanks thanks thanks

nikias · 2024-10-22T18:50:21Z

I feel like having a build.sh that (re-)creates the container and a separate run.sh that executes the actual idevicerestore procedure in the container would make more sense. Then run.sh should also allow passing command line arguments to the idevicerestore instance.

cameroncros · 2024-10-22T19:03:06Z

run.sh does already allow passing arguments, but I can make a build.sh instead.

3a229ed

cameroncros · 2024-10-22T19:30:29Z

Other future nice to haves (In case you would like them implemented now)

CI to validate that the images build successfully (Happy to discuss how to do this, github actions can work, but might waste credits)
Push "official" image to dockerhub, so others can just pull a known-good version

nikias · 2024-10-22T20:47:16Z

CI would be good, there are no credits for this project.
Not sure if I would want to maintain an official image, but I guess if it works it works...

arkanoid87 · 2024-10-22T23:10:33Z

just my 2 cents

latest LTS is Ubuntu 24.04. It might be worth using that as base image instead of 22.04
this docker image is not only relevant for idevicerestore, but for all the tools linked to libimodiledevice project. I am not aware if there is already an image for that, but this one is clean and it works nicely. The wrapper sh script may expose the whole toolset instead of just idevicerestore

# Conflicts: # docker/run.sh

cameroncros · 2024-10-23T00:00:08Z

Added CI, and bumped up to 24.04. I will retest a restore later this evening. Passing CI: https://github.com/cameroncros/idevicerestore/actions/runs/11470400877/job/31919496240

I'm hesitant to add further tools if not required, but if that is later requested, I am more than happy to be summoned to do that.

cameroncros · 2024-10-23T09:33:28Z

Successfully worked on laptop, this is all yours now @nikias unless there is something else.

nikias force-pushed the master branch from 4c7a6e5 to 4145e95 Compare September 20, 2024 22:13

remoosaad approved these changes Sep 30, 2024

View reviewed changes

cameroncros changed the title ~~Add docker scripts to simplify setting up idevicerestore~~ Draft: Add docker scripts to simplify setting up idevicerestore Oct 22, 2024

cameroncros marked this pull request as draft October 22, 2024 09:45

This comment was marked as outdated.

Sign in to view

cameroncros marked this pull request as ready for review October 22, 2024 15:51

cameroncros changed the title ~~Draft: Add docker scripts to simplify setting up idevicerestore~~ Add docker scripts to simplify setting up idevicerestore Oct 22, 2024

Cameron Cross and others added 4 commits October 23, 2024 10:37

Add docker scripts to simplify setting up idevicerestore

cde5a39

Update to include libtatsu, and a newer version of Ubuntu

aa54c3e

Update command used to restore, seems to work better.

4f25af8

# Conflicts: # docker/run.sh

Add usbmuxd, which was the final link. Cleaned up run script.

d76ddc7

Separate build and run stages.

3a229ed

cameroncros force-pushed the docker-container branch from fb2b335 to 3a229ed Compare October 22, 2024 23:37

cameroncros added 2 commits October 23, 2024 10:49

Add CI to build and archive docker image

4d915d7

Upgrade docker image base to 24.04

8d7c306

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Add docker scripts to simplify setting up idevicerestore #562

Add docker scripts to simplify setting up idevicerestore #562

cameroncros commented Feb 1, 2023 •

edited

Loading

remoosaad commented Sep 22, 2024

cameroncros commented Sep 23, 2024

arkanoid87 commented Oct 22, 2024

arkanoid87 commented Oct 22, 2024 •

edited

Loading

arkanoid87 commented Oct 22, 2024 •

edited

Loading

arkanoid87 commented Oct 22, 2024 •

edited

Loading

cameroncros commented Oct 22, 2024

nikias commented Oct 22, 2024

cameroncros commented Oct 22, 2024

This comment was marked as outdated.

cameroncros commented Oct 22, 2024

arkanoid87 commented Oct 22, 2024

arkanoid87 commented Oct 22, 2024

cameroncros commented Oct 22, 2024

arkanoid87 commented Oct 22, 2024

nikias commented Oct 22, 2024

cameroncros commented Oct 22, 2024 •

edited

Loading

cameroncros commented Oct 22, 2024

nikias commented Oct 22, 2024

arkanoid87 commented Oct 22, 2024 •

edited

Loading

cameroncros commented Oct 23, 2024 •

edited

Loading

cameroncros commented Oct 23, 2024

Add docker scripts to simplify setting up idevicerestore #562

Are you sure you want to change the base?

Add docker scripts to simplify setting up idevicerestore #562

Conversation

cameroncros commented Feb 1, 2023 • edited Loading

remoosaad commented Sep 22, 2024

cameroncros commented Sep 23, 2024

arkanoid87 commented Oct 22, 2024

arkanoid87 commented Oct 22, 2024 • edited Loading

arkanoid87 commented Oct 22, 2024 • edited Loading

arkanoid87 commented Oct 22, 2024 • edited Loading

cameroncros commented Oct 22, 2024

nikias commented Oct 22, 2024

cameroncros commented Oct 22, 2024

This comment was marked as outdated.

cameroncros commented Oct 22, 2024

arkanoid87 commented Oct 22, 2024

arkanoid87 commented Oct 22, 2024

cameroncros commented Oct 22, 2024

arkanoid87 commented Oct 22, 2024

nikias commented Oct 22, 2024

cameroncros commented Oct 22, 2024 • edited Loading

cameroncros commented Oct 22, 2024

nikias commented Oct 22, 2024

arkanoid87 commented Oct 22, 2024 • edited Loading

cameroncros commented Oct 23, 2024 • edited Loading

cameroncros commented Oct 23, 2024

cameroncros commented Feb 1, 2023 •

edited

Loading

arkanoid87 commented Oct 22, 2024 •

edited

Loading

arkanoid87 commented Oct 22, 2024 •

edited

Loading

arkanoid87 commented Oct 22, 2024 •

edited

Loading

cameroncros commented Oct 22, 2024 •

edited

Loading

arkanoid87 commented Oct 22, 2024 •

edited

Loading

cameroncros commented Oct 23, 2024 •

edited

Loading