ESP32C3 fails to boot with otadata partition and no factory app. (QEMU-160)

[Samureimer]

When I try to run the emulator with the following partitions table the program fails to enter the main function it seems.
When targeting the ESP32 it works just fine.

Link to repo that reproduces and documents this:
https://github.com/Samureimer/ESP32C3QemuProblem

QEMU version:
esp-qemu-riscv32-softmmu-develop_8.0.0_20230522
https://github.com/espressif/qemu/releases/tag/esp-develop-8.0.0-20230522

# Name,   Type, SubType, Offset,  Size, Flags
nvs,      data, nvs,     0x9000,  0x4000,
otadata,  data, ota,     0xd000,  0x2000,
phy_init, data, phy,     0xf000,  0x1000,
app0,     app,  ota_0,   0x10000, 0x1F0000,
app1,     app,  ota_1,   ,        0x1F0000,
nvs_key,  data, nvs_keys,,0x1000 ,          encrypted

Console output:

Adding SPI flash device
ESP-ROM:esp32c3-api1-20210111-dirty
Build:Sep 30 2022
rst:0x1 (POWERON),boot:0x8 (SPI_FAST_FLASH_BOOT)
SPIWP:0xee
mode:DIO, clock div:1
load:0x3fcd5820,len:0x1738
load:0x403cc710,len:0xae8
load:0x403ce710,len:0x2e70
entry 0x403cc71a
I (0) boot: ESP-IDF HEAD-HASH-NOTFOUND 2nd stage bootloader
I (0) boot: compile time Jul 25 2023 11:11:04
I (0) boot: chip revision: v0.3
I (0) boot.esp32c3: SPI Speed      : 80MHz
I (0) boot.esp32c3: SPI Mode       : SLOW READ
I (0) boot.esp32c3: SPI Flash Size : 4MB
I (1) boot: Enabling RNG early entropy source...
I (1) boot: Partition Table:
I (1) boot: ## Label            Usage          Type ST Offset   Length
I (1) boot:  0 nvs              WiFi data        01 02 00009000 00004000
I (1) boot:  1 otadata          OTA data         01 00 0000d000 00002000
I (1) boot:  2 phy_init         RF data          01 01 0000f000 00001000
I (2) boot:  3 app0             OTA app          00 10 00010000 001f0000
I (2) boot:  4 app1             OTA app          00 11 00200000 001f0000
I (2) boot:  5 nvs_key          NVS keys         01 04 003f0000 00001000
I (2) boot: End of partition table
I (2) boot: No factory image, trying OTA 0
I (2) esp_image: segment 0: paddr=00010020 vaddr=3c020020 size=08cc0h ( 36032) map
I (6) esp_image: segment 1: paddr=00018ce8 vaddr=3fc8a800 size=01120h (  4384) load
I (7) esp_image: segment 2: paddr=00019e10 vaddr=40380000 size=06208h ( 25096) load
I (10) esp_image: segment 3: paddr=00020020 vaddr=42000020 size=157f4h ( 88052) map
I (19) esp_image: segment 4: paddr=0003581c vaddr=40386208 size=04540h ( 17728) load
I (24) boot: Loaded app from partition at offset 0x10000

And then nothing.....

If I remove the otadata from the partitions table, or change the partition table to: Factory app, two OTA definitions, it runs just fine:

# Name,   Type, SubType, Offset,  Size, Flags
nvs,      data, nvs,     0x9000,  0x4000,
phy_init, data, phy,     0xf000,  0x1000,
app0,     app,  ota_0,   0x10000, 0x1F0000,
app1,     app,  ota_1,   ,        0x1F0000,
nvs_key,  data, nvs_keys,,0x1000 ,          encrypted

Console output:

Adding SPI flash device
ESP-ROM:esp32c3-api1-20210111-dirty
Build:Sep 30 2022
rst:0x1 (POWERON),boot:0x8 (SPI_FAST_FLASH_BOOT)
SPIWP:0xee
mode:DIO, clock div:1
load:0x3fcd5820,len:0x1738
load:0x403cc710,len:0xae8
load:0x403ce710,len:0x2e70
entry 0x403cc71a
I (0) boot: ESP-IDF HEAD-HASH-NOTFOUND 2nd stage bootloader
I (0) boot: compile time Jul 25 2023 11:14:29
I (0) boot: chip revision: v0.3
I (0) boot.esp32c3: SPI Speed      : 80MHz
I (0) boot.esp32c3: SPI Mode       : SLOW READ
I (0) boot.esp32c3: SPI Flash Size : 4MB
I (1) boot: Enabling RNG early entropy source...
I (1) boot: Partition Table:
I (1) boot: ## Label            Usage          Type ST Offset   Length
I (1) boot:  0 nvs              WiFi data        01 02 00009000 00004000
I (1) boot:  1 phy_init         RF data          01 01 0000f000 00001000
I (1) boot:  2 app0             OTA app          00 10 00010000 001f0000
I (2) boot:  3 app1             OTA app          00 11 00200000 001f0000
I (2) boot:  4 nvs_key          NVS keys         01 04 003f0000 00001000
I (2) boot: End of partition table
I (2) esp_image: segment 0: paddr=00010020 vaddr=3c020020 size=08cc0h ( 36032) map
I (6) esp_image: segment 1: paddr=00018ce8 vaddr=3fc8a800 size=01120h (  4384) load
I (7) esp_image: segment 2: paddr=00019e10 vaddr=40380000 size=06208h ( 25096) load
I (10) esp_image: segment 3: paddr=00020020 vaddr=42000020 size=157f4h ( 88052) map
I (18) esp_image: segment 4: paddr=0003581c vaddr=40386208 size=04540h ( 17728) load
I (23) boot: Loaded app from partition at offset 0x10000
I (24) boot: Disabling RNG early entropy source...
I (24) cpu_start: Unicore app
I (24) cpu_start: Pro cpu up.
I (33) cpu_start: Pro cpu start user code
I (33) cpu_start: cpu freq: 160000000 Hz
I (33) cpu_start: Application information:
I (33) cpu_start: Project name:     test
I (33) cpu_start: App version:      e2e517a-dirty
I (33) cpu_start: Compile time:     Jul 25 2023 11:14:22
I (33) cpu_start: ELF file SHA256:  24f6c40c6...
I (33) cpu_start: ESP-IDF:          HEAD-HASH-NOTFOUND
I (33) cpu_start: Min chip rev:     v0.3
I (33) cpu_start: Max chip rev:     v0.99 
I (33) cpu_start: Chip rev:         v0.3
I (33) heap_init: Initializing. RAM available for dynamic allocation:
I (33) heap_init: At 3FC8C790 len 0004FF80 (319 KiB): DRAM
I (33) heap_init: At 3FCDC710 len 00002950 (10 KiB): STACK/DRAM
I (34) heap_init: At 50000010 len 00001FD8 (7 KiB): RTCRAM
I (34) spi_flash: detected chip: gd
I (34) spi_flash: flash io: dio
I (34) sleep: Configure to isolate all GPIO pins in sleep state
I (34) sleep: Enable automatic switching of GPIO sleep configuration
I (34) app_start: Starting scheduler on CPU0
I (35) main_task: Started on CPU0
I (65) main_task: Calling app_main()
W (65) TEST: HELLO WORLD

[o-marshmallow]

Hello @Samureimer ,

Thank you for reporting the issue and sorry for the delay of reply. I was able to reproduce the issue you are seeing with esp-develop_8.0.0_20230522 version.

The good thing is this issue is now fixed on the esp-develop branch, which has the newest commits. Since, these commits are not part of any release at the moment, I would advise you to use build esp-develop from source and to use it in your development.

[Samureimer]

Hi @o-marshmallow.
Thank you for the tip.
I managed to build esp-develop and it did solve my problem.
I then tried to do a simple WiFi connect and two new problems appeared.

It first gets stuck trying to do a ADC calibration at: esp/idf/components/hal/adc_hal_common.c@136.

I "fixed" this by removing the while loop and just returning some raw ADC value.
Then the processor would get stuck in esp_wifi_start and trigger the watchdog.

I would expect the emulator to "just" connect to any WiFi and substitute the actual WiFi driver with another driver that uses my computers network interface. Is this incorrect?

[igrr]

and substitute the actual WiFi driver with another driver that uses my computers network interface. Is this incorrect?

No, there is no support for Wi-Fi emulation on ESP32 or ESP32-C3 in this QEMU fork. Some work on this has been done in #80, but it's not ready yet.

If you need to test anything network-related, you can initialize Ethernet, instead. The steps to do this for ESP32 are described here: https://github.com/espressif/esp-toolchain-docs/blob/main/qemu/esp32/README.md#ethernet-support. For ESP32-C3, this will be supported in the next release.

[lcgamboa]

Hi @Samureimer,

this fork of Espressif QEMU has support to WIFI for ESP32 and ESP32C3.

[Samureimer]

Hi @lcgamboa.
With your fork my program seems to run just fine and I'm able to initialize the WiFi driver without crashing.
I'm however not able to connect to any WiFi networks.
I might not understand how this is supposed to work or my setup is not supported.
I'm running the emulator in a Dev-container that is running in WSL2 on my Windows machine...

I use the Arduino framework for ESP-IDF (Shouldn't really matter I guess).
I have compiled this simple program and QEMU runs just fine:

#include <Arduino.h>
#include <WiFi.h>

#include "esp_mac.h"

void setup() {
  Serial0.begin(115200);

  uint8_t newMACAddress[] = {0x32, 0xAE, 0xA4, 0x07, 0x0D, 0x66};
  esp_iface_mac_addr_set(newMACAddress, ESP_MAC_WIFI_STA);
  WiFi.begin("asdasdasd");
}

void loop() {
  if (WiFi.status() == WL_CONNECTED) {
    Serial0.println("Connected");
  } else {
    Serial0.printf("Not connected %i\r\n", WiFi.status());
  }
  delay(1000);
}

The QEMU command I use to start the emulator:
172.17.0.0/16 is the IP of the network my dev container is reporting that it's running in.

qemu-system-riscv32 -S -M esp32c3-picsimlab -drive file=result.bin,if=mtd,format=raw \
  -drive file=qemu_efuse.bin,if=none,format=raw,id=efuse \
  -global driver=nvram.esp32c3.efuse,property=drive,value=efuse \
  -serial stdio -gdb tcp::1234 -icount shift=3,align=off,sleep=on \
  -global driver=timer.esp32c3.timg,property=wdt_disable,value=true \
  -nic user,model=esp32c3_wifi,net=172.17.0.0/16

The output of the application:

Adding SPI flash device
ESP-ROM:esp32c3-api1-20210207
Build:Feb  7 2021
rst:0x1 (POWERON),boot:0x8 (SPI_FAST_FLASH_BOOT)
SPIWP:0xee
mode:DIO, clock div:1
load:0x3fcd5820,len:0x123c
load:0x403cc710,len:0x838
load:0x403ce710,len:0x3438
entry 0x403cc776
I (86) cpu_start: Unicore app
I (86) cpu_start: Pro cpu up.
W (212) clk: 32 kHz XTAL not found, switching to internal 150 kHz oscillator
I (220) cpu_start: Pro cpu start user code
I (220) cpu_start: cpu freq: 160000000 Hz
I (220) cpu_start: Application information:
I (220) cpu_start: Project name:     app_main
I (220) cpu_start: App version:      VERSION_MAJOR 1
I (220) cpu_start: ELF file SHA256:  4087d5a6fe8877ff...
I (220) cpu_start: ESP-IDF:          v5.1.1-686-gb6a66b7d8c
I (220) cpu_start: Min chip rev:     v0.3
I (220) cpu_start: Max chip rev:     v0.99 
I (220) cpu_start: Chip rev:         v0.3
I (220) heap_init: Initializing. RAM available for dynamic allocation:
I (221) heap_init: At 3FC98EE0 len 00027120 (156 KiB): DRAM
I (221) heap_init: At 3FCC0000 len 0001C710 (113 KiB): DRAM/RETENTION
I (221) heap_init: At 3FCDC710 len 00002950 (10 KiB): DRAM/RETENTION/STACK
I (221) heap_init: At 50000010 len 00001FD8 (7 KiB): RTCRAM
I (221) spi_flash: detected chip: gd
I (221) spi_flash: flash io: dio
I (223) sleep: Configure to isolate all GPIO pins in sleep state
I (223) sleep: Enable automatic switching of GPIO sleep configuration
D (223) esp_apptrace: HW interface 0x00000000
I (223) esp_apptrace_uart: Initialized UART on CPU0
I (223) esp_core_dump_uart: Init core dump to UART
I (224) app_start: Starting scheduler on CPU0
Not connected 6
Not connected 6
Not connected 6
Not connected 1
Not connected 1
Not connected 1
Not connected 1
Not connected 1

I would expect the emulator to connect to any SSID and password combinations. Is this incorrect?

[lcgamboa]

No. In Station mode, you must use one of the hardcoded SSIDs (PICSimLabWifi, Espressif, or MasseyWifi) without a password. Sorry for the lack of documentation.

I don't know if the function esp_iface_mac_addr_set will work. If you don´t use one efuse file, the Qemu will set one hardcoded MAC address for you. The MAC address is defined in bytes from offset 0x18 to 0x1D in the efuse file.

[Samureimer]

@lcgamboa Thanks for the quick reply.
Using a valid SSID made things a whole lot better :)

When calling esp_iface_mac_addr_set I was unable to connect or get a valid IP. Not sure what the problem was.
After removing the mac set call it connected just fine but getting access to the rest of my home network or the internet through my dev containers docker network did not work.

I ended up running the container with the arg --network=host and QEMU with -nic user,model=esp32c3_wifi,net=192.168.1.0/24

Now I was able to connect to the SSID, obtain a valid IP and access the internet.

esp_wifi_get_mac however was returning all zeros when called.
I ended up adding a MAC to the efuse file. Now the get mac function also returns the correct MAC.

I only have some minor issues left.
Maybe you can point me in the right direction for a solution.

When the IP is returned from the DHCP request the gateway seems to be incorrect. I'm still able to resolve DNS and do HTTP requests so this is not a problem for me, just odd.
This is reported in the log:
I (4470) esp_netif_handlers: sta ip: 192.168.1.15, mask: 255.255.255.0, gw: 192.168.1.2
I expected the gateway to be 192.168.1.1

The RTC/time seems to be broken in the sense that I can't change the system time.
A NTP requet is completed just fine:

I (7262) RTC: Time Syncronized! 1702333450 
I (7263) RTC: The current date/time is: Thu Jan  1 00:00:06 1970

The NTP request is supposed to set the system time automatically I believe. NTP example code

After calling settimeofday with the time value from the NTP request the time is still incorrect.
Do you know what could cause this?

[lcgamboa]

HI @Samureimer ,

when using Qemu -nic user (slirp) the virtual network gateway has the default IP address x.x.x.2. I don't know if it's possible to change this.

The RTC peripheral isn't fully implemented yet. I will look and try to fix the example code to work as soon as possible.

[lcgamboa]

The RTC works for ESP32 Qemu but doesn´t work for ESP32C3 Qemu. The RTC timer support is implemented in esp32c3_timg.c, but it doesn't work correctly as expected for the sntp example.

Using my fork with WIFI or official Espressif with Opencores Ethernet MAC for ESP32C3 get the same date time error:

I (6020) example: The current date/time in New York is: Wed Dec 31 19:00:05 1969
I (6020) example: The current date/time in Shanghai is: Thu Jan  1 08:00:05 1970

I have used the command line:

./qemu-system-riscv32 -M esp32c3  -drive file=sntp.bin,if=mtd,format=raw 
-drive file=ESP32C3.efuse,if=none,format=raw,id=efuse 
-global driver=nvram.esp32c3.efuse,property=drive,value=efuse
-serial stdio -icount shift=3,align=off,sleep=on 
-global driver=timer.esp32c3.timg,property=wdt_disable,value=true 
-nic user,model=open_eth,id=u2,net=192.168.3.0/24

I think is better to open an issue to Espressif to fix the RTC for ESP32C3, as they wrote the code, they will be able to fix it faster than me, who would have to study and understand it before trying to fix it.

[lcgamboa]

I did more research and found that the sntp example always get 0 seconds from the server. The Arduino example SimpeTime has the same problem. But the time function from the ESP32 Qemu gets the local machine time and the example apparently works for the ESP32C3 the time function doesn't return the machine local time and the examples don´t work.

Using code that does not depend on IDF sntp and time functions works (as in the example below).

#include <WiFi.h>
#include <TimeLib.h>
#include <WiFiUdp.h>
const char* ssid     = "PICSimLabWifi";
const char* password = "";
static const char ntpServerName[] = "time.nist.gov";
const int timeZone = 0;// 0 is UTC, use -3 for Brasil
WiFiUDP Udp;
unsigned int localPort = 8888;
time_t getNtpTime();

void setup() {
  Serial.begin(115200);
  Serial.println("WiFi Begin... ");

  WiFi.begin(ssid, password);
  while (WiFi.status() != WL_CONNECTED) {
    Serial.println("Connecting... ");
    delay(200);
  }
  Serial.print("Connected to WiFi ");
  delay(200);
  Udp.begin(localPort);
  setSyncProvider(getNtpTime);
  setSyncInterval(300);
  Serial.println("Waiting for Time Sync.. ");
  delay(500);
  Udp.begin(localPort);
}

time_t prevDisplay = 0;
void loop() {
  if (timeStatus() != timeNotSet)  {
    if (now() != prevDisplay) { //update the display only if time has changed
      prevDisplay = now();
      digitalClockDisplay();
    }
  }
}
void digitalClockDisplay()
{
  Serial.print(hour());
  printDigits(minute());
  printDigits(second());
  Serial.print(" ");
  Serial.print(day());
  Serial.print("/");
  Serial.print(month());
  Serial.print("/");
  Serial.println(year());
}

void printDigits(int digits)
{
  // utility for digital clock display: prints preceding colon and leading 0
  Serial.print(":");
  if (digits < 10)
    Serial.print('0');
  Serial.print(digits);
}

/*-------- NTP code ----------*/

const int NTP_PACKET_SIZE = 48; // NTP time is in the first 48 bytes of message
byte packetBuffer[NTP_PACKET_SIZE]; //buffer to hold incoming & outgoing packets

time_t getNtpTime()
{
  IPAddress ntpServerIP; // NTP server's ip address

  while (Udp.parsePacket() > 0) ; // discard any previously received packets
  // get a random server from the pool
  WiFi.hostByName(ntpServerName, ntpServerIP);
  Serial.print(ntpServerName);
  Serial.print(": ");
  Serial.println(ntpServerIP);
  sendNTPpacket(ntpServerIP);
  uint32_t beginWait = millis();
  while (millis() - beginWait < 1500) {
    int size = Udp.parsePacket();
    if (size >= NTP_PACKET_SIZE) {
      Serial.println("Receive NTP Response");
      Udp.read(packetBuffer, NTP_PACKET_SIZE);  // read packet into the buffer
      unsigned long secsSince1900;
      // convert four bytes starting at location 40 to a long integer
      secsSince1900 =  (unsigned long)packetBuffer[40] << 24;
      secsSince1900 |= (unsigned long)packetBuffer[41] << 16;
      secsSince1900 |= (unsigned long)packetBuffer[42] << 8;
      secsSince1900 |= (unsigned long)packetBuffer[43];
      return secsSince1900 - 2208988800UL + timeZone * SECS_PER_HOUR;
    }
  }
  return 0; // return 0 if unable to get the time
}

// send an NTP request to the time server at the given address
void sendNTPpacket(IPAddress &address)
{
  // set all bytes in the buffer to 0
  memset(packetBuffer, 0, NTP_PACKET_SIZE);
  // Initialize values needed to form NTP request
  // (see URL above for details on the packets)
  packetBuffer[0] = 0b11100011;   // LI, Version, Mode
  packetBuffer[1] = 0;     // Stratum, or type of clock
  packetBuffer[2] = 6;     // Polling Interval
  packetBuffer[3] = 0xEC;  // Peer Clock Precision
  // 8 bytes of zero for Root Delay & Root Dispersion
  packetBuffer[12] = 49;
  packetBuffer[13] = 0x4E;
  packetBuffer[14] = 49;
  packetBuffer[15] = 52;
  // all NTP fields have been given values, now
  // you can send a packet requesting a timestamp:
  Udp.beginPacket(address, 123); //NTP requests are to port 123
  Udp.write(packetBuffer, NTP_PACKET_SIZE);
  Udp.endPacket();
}

[lcgamboa]

This new commit f5de023 fixes the return of the time function to ESP32C3. And get the correct date time from servers.

[o-marshmallow]

Hello @lcgamboa @Samureimer ,

The commit f5de023 should now fix the RTC on the ESP32-C3, please check and tell me if it fixed your issue

EDIT: I forgot to refresh this page before sending my reply 😄
Happy to hear that it works for you now!

[Samureimer]

Hi @o-marshmallow and @lcgamboa
Sorry sorry for the delayed response.

I pulled the latest changes from @lcgamboa WiFi branch and it just worked!
Thank you so much for your work!

[Samureimer]

Hi @lcgamboa
I have a small issue with your QEMU fork.
When downloading larger files using the ESP HTTP client, the program never returns from vPortYield after some random number of bytes downloaded.
The program works on a real device.

I have a sample program that I would like to share with you.
Is there a better way of contacting you or create issues on your fork (Issue creation seems to be disabled)?

[lcgamboa]

Hi @Samureimer,

I have enabled the issue creation on my fork.
I believe there is a problem with C3 interrupts treatment in Qemu. This is the same thing that happens when I run programs with the pre-compiled Arduino as reported in this issue.