Adding IPv6 support to arduino-esp32

[nuclearcat]

This patch partially depends on:
espressif/esp32-arduino-lib-builder#67
Without this patch we will get only Link local IPv6 (still useful for MDNS and etc).
With patch we will get also global IPv6 address by SLAAC.
By default IPv6 disabled, until it is properly tested.

Tested on BasicHttpClient by adding:
wifiMulti.IPv6(true);
before: wifiMulti.addAP() call

Enabling Core Debug Level: verbose
If IP6 obtained, in logs will be visible:
[ 8028][V][WiFiGeneric.cpp:380] _arduino_event_cb(): IF[0] Got IPv6: IP Index: 0, Zone: 2, fe80:0000:0000:0000:xxxx:xxxx:xxxx:xxxx
[ 8028][D][WiFiGeneric.cpp:852] _eventCallback(): Arduino Event: 8 - STA_GOT_IP6
[ 11028][V][WiFiGeneric.cpp:380] _arduino_event_cb(): IF[0] Got IPv6: IP Index: 1, Zone: 0, 2a0d:yyyy:0000:4000:yyyy:yyyy:yyyy:yyyy
[ 11028][D][WiFiGeneric.cpp:852] _eventCallback(): Arduino Event: 8 - STA_GOT_IP6

This is linked to: #6242

Signed-off-by: Denys Fedoryshchenko [email protected]

[CLAassistant]

All committers have signed the CLA.

[github-actions]

Unit Test Results

0 files  0 suites   0s ⏱️
0 tests 0 ✔️ 0 💤 0 ❌

Results for commit 01dd9c4.

♻️ This comment has been updated with latest results.

[nuclearcat]

Additional note, as we use this optional flag, it is addressing many users fears, that they dont want to get IPv6 address on their device. So by default it wont take it, unless they do IPv6(true).

[nuclearcat]

Any feedback? :)

[me-no-dev]

Hi @nuclearcat sorry for the late reply. I have merged the PR in lib-builder and IDF support will come in 2.0.4.
We will look into this after we release that, because this adds new functionality and modifies good portion of existing code.
At least with 2.0.4, you (and we) will be able to test everything properly.

[nuclearcat]

Thanks! Will keep an eye, maybe meanwhile i will bring some good use-cases for IPv6 for new code

[Meekdai]

I am going to use ESP32-S3 to run an IPV6 web server, and I can access index.html in ESP32-S3 FLASH on any networked device through IPV6 address. Can version 2.0.4 implement this function?

[nuclearcat]

Hi @Meekdai , with patches in this PR - yes. I am not sure vanilla 2.0.4 can run ipv6 web server properly.

[Meekdai]

Hi @nuclearcat
I don't know how to use this PR, I just installed the package containing esp32 2.0.4 on my windows PC.

[nuclearcat]

I am afraid it will be a bit difficult then to test ipv6 properly, without manual building, until this PR is merged.

[sgryphon]

This PR is really important... how soon can it be merged? (I'm going to see if I can reference the branch in PlatformIO).

Trying to get IPv6 working, and have the basic network (i.e. enable and get address) working, except then HTTPClient fails on something like http.begin("http://v6.ipv6-test.com/api/myip.php"); because the internals assume IPv4 addresses.

If you need any help testing, let me know.

[nuclearcat]

I might make docker image with this PR and arduino-cli to build and flash some test arduino programs, if that will be useful.
But thats a bit linux stuff, it will produce bin files at best

[VojtechBartoska]

@sgryphon we plan to merge this for next major release 2.1.0.

[fryefryefrye]

Is "libraries/WiFi/src/WiFiUdp.cpp" also need to be update ?
When people want to use IPv6 for UDP protocol.

[sgryphon]

Throwing a spanner into the works, I think this should be re-worked before merging it: #7114

I know a separate IPv6Address.h class has already been added to the core, but I think it is a bad design, and complicates things. e.g. It has meant adding duplicate methods everywhere, so there are both remoteIP() and remoteIP6() addresses, and it would be complicated for client code to have to check both and determine what to do.

The approach taken in ESP8266 leads to a lot simpler code: https://github.com/esp8266/Arduino/blob/master/cores/esp8266/IPAddress.h

This makes IPAddress a wrapper around the Espressif ip_addr_t union, so that it supports both IPv4 and IPv6.

This means you only need one remoteIP() method, that simply returns what is appropriate, and simplifies the whole implementation down to supporting the new feature, rather than modifying all the APIs.

The format of an IP address should be an implementation detail, hidden from the user.

Suggestion (and I'm happy to help with the coding changes);

• Implement IPAddress similar to ESP8266, so it supports both types.
• Remove all the unnecessary overloads with IPv6Address
• Retain the changes to implementation where relevant that actually use IPv6 addresses.
• Mark the existing IPv6Address as deprecated (for the future, as everything it does will be in the main IPAddress), as it is already used in some places (e.g. UDP).

[nuclearcat]

I implemented it as new classes/functions for better backward compatibility, to avoid breaking existing code.
But looks like esp8266-arduino project running quite a while without issue.
I will research this a bit more, and if project maintainers doesn't mind can send another PR with mentioned approach.

[sgryphon]

new classes/functions for better backward compatibility

Provided you are only adding new features, then it should not break any existing code. e.g. if the old IPAddress had constructors for 4 bytes, that will continue to work, even if you add a second constructor with 16 bytes. The new class would be fully backwards compatible (i.e. can do everything the old class can do).

There could be some corner cases where client code is relying on the implementation, rather than the interface, e.g. if it relies on the storage size being 4 bytes, which could be important for a resource constrained device. The ESP8266 approach had a nice solution in that if compiled with IPv6 turned off, then it falls back to the smaller memory footprint.

Maybe when considering forward compatibility, i.e.the ability of old code to handle the new features, such as if old code gets handed a new address. Certainly not allowing it at all (by adding a new class) might help in some corner cases, but also mean they are locked out of the new features, with is probably a larger negative.

In general I don't think people want to care about the low level details of which IP address they are using.

If you recompile with components that support IPv6, and then deploy into an IPv6 only network, the same code will just continue to work (except with slightly larger memory requirements).

[sgryphon]

If it is useful, I have put a proposal pull release in to the ArduinoCore-API to add IPv6 support to the standard API: arduino/ArduinoCore-API#169

The code is slightly different from ESP8266. The public interface changes are minimal:

• add a type() getter and corresponding IPType enum, with values IPv4 and IPv6
• add three new constructors: one for 16 octets, empty with type specifier, and byte pointer with type specifier
• add constant for IP6ADDR_ANY (similar function as IPADDR_NONE)

Instead of type(), the ESP8266 implementation has isV4() and isV6(); in ESP8266 there is also a constructor from ip_addr_t, which we would want in the ESP32 implementation (but is not relevant for ArduinoCore-API).

There are also a bunch of other constructs in ESP8266, which I have kept out of the minimal implementation, e.g. a raw6() method which would be the same as type() == IPv6 ? raw_address() : nullptr, or isLocal() which returns if the address is a link local address (in my opinion badly named as the local address means something different to me than a link local address). These might be useful, but are not directly part of IPv6 support.

[sgryphon]

IPv6 support is now in ArduinoCore-API. I've copied the changes into a branch in my fork, which could be integrated into this change.

https://github.com/sgryphon/arduino-esp32/tree/feature/ipaddress-v6-support

It means you can avoid things like WiFiClient::connect(ip_addr_t ip, uint16_t port, int32_t timeout), because int WiFiClient::connect(IPAddress ip, uint16_t port, int32_t timeout) would have ip.type() support both IPv4 and IPv6.

(With IPv4 work exactly the same as it did before, just new support for IPv6 directly in IPAddress rather than using ip_addr_t).

[nuclearcat]

Thanks! Since it became a standard approach, i think will work on it this weekend to update my changes

[sgryphon]

The build / github actions for this project are a bit more complicated than for ArduinoCore-API, so I'm not sure how to build/compile locally (or run tests). Is there some instructions?

[sgryphon]

One complexity with IPv6 is that with IPv4 usually you will only have one local IP address (public address, private network, or if there is no network a link-local 169.254 address), however with IPv6 you will almost always have multiple addresses -- at least both an autoconfigure link local (fe80: address) as well as an address with the network global prefix, and possibly a ULA prefix as well, and outgoing addresses with privacy enhancements may have rotating outgoing addresses.

One way I was handling this was with the signature IPAddress localIP(uint8_t index = 0), returning one value for each index (or NULL / none once they run out), with another signature int8_t getLocalIPs(IPAddress addresses[], uint8_t max) to get multiple addresses, filling an array up to the max (and returning the number of addresses).

Usually you don't want to do a lot with the local address except display it, but with IPv6 you want to check which one to display.

[nuclearcat]

maybe we can just add globalIP6 and localIP6? This will keep it simple for Arduino users. We can iterate and by prefix identify who is local and who is global.
Privacy enhancements as far as i know is not available in lwip, yet.

[sgryphon]

Be careful with terminology not to mix up the concepts of "local" and "link-local", as they are different things.

Local vs remote: Local means this end of the connection, i.e. the IP address of the device, as opposed to remote, which is the other end.

Link-local vs global are scopes: Link-local is confined to the local link only, global is a routeable.

You can have both.

With IPv4 you usually only have one local address, and it usually has a global scope. With IPv6 you have multiple local addresses, both link-local and global scopes.

Generally the global scoped addresses are more useful, e.g. to use as a server address to connect to a device, or when looking in logs. (Although not always, e.g. if the destination machines is on the same link, then IPv6 will use the link-local address.)

[sgryphon]

Examples, from a SIMCOM GSM NB-IoT library I am working on: https://github.com/sgryphon/AdvancedGsmClient/blob/main/src/SIM7020/SIM7020GsmModem.cpp#L470

When I connect the NB-IoT modem, it gets assigned 3 local addresses, plus it's address list includes a localhost address:

Local addresses:

1. 2001:8004:4810:0:719d:1439:899a:42d7 - local IPv6 with global scope
2. 10.88.134.167 - local IPv4 with global scope
3. fe80:0:0:0:719d:1439:899a:42d7 - local IPv6 with link-local scope
4. 127.0.0.1 - local IPv4 with link-local scope (loopback addresses count as link-local).

The actual order the addresses are reported by the device is fe80:, 2001:, 10.88, 127.0, however my getLocalIPs() method sorts them from largest scope (global) to smallest scope (link-local), and then within each scope according to the RFC 6724 precedence rules (which includes IPv6 before IPv4), to get the order listed above.

[sgryphon]

For my local desktop computer, sorting my addresses a similar way (but using the bytes; my simple GSM algorithm is using strings and so sorts ':' differently), to sort them in order of usefulness:

1. 2407:8800:bc61:1300::8e6 - local IPv6 with global scope
2. 2407:8800:bc61:1300:4a23:22d5:6c48:f0e8 - local IPv6 with global scope
3. 2407:8800:bc61:1300:4df2:1392:9dd7:d2d7 - local IPv6 with global scope
4. 2407:8800:bc61:1300:ee48:5c84:ad87:c65c - local IPv6 with global scope
5. 192.168.1.195 - local IPv4 with global scope
6. fd7c:e25e:67e8::8e6 - local IPv6 with global scope, in ULA range
7. fd7c:e25e:67e8:0:2c73:36d3:d325:93df - local IPv6 with global scope, in ULA range
8. fd7c:e25e:67e8:0:9ac7:63c9:5f26:b026 - local IPv6 with global scope, in ULA range
9. fd7c:e25e:67e8:0:a9ed:c900:59fa:5449 - local IPv6 with global scope, in ULA range
10. fe80::5f65:961a:14b4:6263 - local IPv6 with link-local scope

Global scope is sorted before link-local (because it is more useful to contact me as a server), with the precedence order IPv6, then IPv4, then IPv6 unique local address (ULA).

As a server the ::8e6 address is better, although outgoing logs will use the temporary addresses (probably not an issue for Arduino). If I was communicating with an IPv4 only server, then it would see the IPv4 address.

So, to really investigate I would need to list all of my addresses (e.g. in logs). Even the ULA addresses could be used, for local communications, and the IPv6 link-local address could even be used in some cases.

[sgryphon]

If you really want to stick with separate properties for IPv6 and IPv4, for only one of each type, then you want to try and return a global scope address if possible:

• localIP4() - Shows the local IPv4 address (this will usually be a global scope address); there is usually only one. To be consistent, in case there is more than one IPv4, you should also sort by scope (e.g. if a device has two IP addresses 169.254.1.2 and 10.1.2.3, then show 10.1.2.3 ... although that would be really weird as IPv4 doesn't work that way).
• localIP6() - shows the first IPv6 address, sorting global scope before link-local scope.
• localIP() - for a single address should show the biggest scope, highest precedence address. i.e. global IPv6 if you have one, otherwise global IPv4, and then finally (if you have no global addresses), then the link-local IPv6.

I do think you should have a getLocalIPs() that returns all of the addresses, which you could then use to select out the other values, e.g. for logging, although in the case of the GSM code I know it only ever gets max 4 addresses (i.e. no temporary, only one IPv6 prefix, etc), so I just log the first 2 address, which will be the one global IPv6 and the one global IPv4.

[nuclearcat]

I started to convert my patch to new IPAddress contributed by @sgryphon: https://github.com/nuclearcat/arduino-esp32/tree/ipv6-support-v2
Work still in progress, please comment if you have suggestions.

[sgryphon]

I'll see when I have a chance to look in more detail. In the meanwhile:

(1)
For web browsers the approach for dual stack is the "happy eyeballs" algorithm, which queries DNS for the first IPv6 address and the first IPv4, and then attempts connections to both simultaneously, aborting the loser once one responds. Part of this was due to early days misconfiguration of IPv6 given addresses that didn't work -- that is no longer really a consideration, but performance wise often both are used.

This may not be applicable for Arduino devices as limited capacity, not a lot of multithreading, can't abort, etc. Plus constraints we may just have to configure as known IPv6 or IPv4.

(2)
For outputting "localIP()", you may want something that can be used as a server address and/or matched to logs.

If you look at the code I have in another project: https://github.com/sgryphon/AdvancedGsmClient/blob/main/src/SIM7020/SIM7020GsmModem.cpp#L470

This sorts addresses based on largest scope (i.e. we want to show a global address), and then by the RFC 6724 precedence. (The sorting is done on strings, as IPAddress in ESP32 doesn't have IPv6 yet).

[nuclearcat]

Closed in favor of #7520