Error initializing Fast-DDS when the Ethernet link is not up

[i-and]

Is there an already existing issue for this?

• I have searched the existing issues

Expected behavior

The stack must be initialized successfully regardless of external conditions - in this case, the current state of the Ethernet link

Current behavior

The stack is not functional if the link was down during initialization.

Steps to reproduce

This error can be recreated using a physical Ethernet interface with its configuration in a state where the set IP address is not reset by the system when the link go down.
But it is more convenient to recreate the situation using the docker0 bridge (in my case, it has the address 172.17.0.1). As an application, you can use HelloWorldExample with the whitelist value set to 172.17.0.1. If you run the application on the host first and then in the container, then DDS interaction will not be performed. If in a different sequence, then everything will work successfully because launching the container will raise the link connected to docker0 from the host side.

Fast DDS version/commit

2.13.0

Platform/Architecture

Ubuntu Focal 20.04 amd64

Transport layer

UDPv4

Additional context

I offer two options for correction:

1. not_run_ifc_1.patch - The current default stack behavior does not change, i.e. UDPv4 transport uses only running interfaces. To activate the new feature, set the allow_not_running_interfaces field from SocketTransportDescriptor to true.

2. not_run_ifc_2.patch - The current default stack behavior changes if whitelist is set in UDPv4 transport. In this case, the interfaces specified in the whitelist that are not running are also used. No new parameters are introduced at the TransportDescriptor level.

XML configuration file

No response

Relevant log output

No response

Network traffic capture

No response

[i-and]

not_run_ifc_1.patch:

diff --git a/include/fastdds/rtps/transport/SocketTransportDescriptor.h b/include/fastdds/rtps/transport/SocketTransportDescriptor.h
index 0b03eabdb..b8bf19b85 100644
--- a/include/fastdds/rtps/transport/SocketTransportDescriptor.h
+++ b/include/fastdds/rtps/transport/SocketTransportDescriptor.h
@@ -51,6 +51,7 @@ struct SocketTransportDescriptor : public PortBasedTransportDescriptor
         , sendBufferSize(0)
         , receiveBufferSize(0)
         , TTL(s_defaultTTL)
+        , allow_not_running_interfaces(false)
     {
     }
 
@@ -89,6 +90,8 @@ struct SocketTransportDescriptor : public PortBasedTransportDescriptor
     std::vector<std::string> interfaceWhiteList;
     //! Specified time to live (8bit - 255 max TTL)
     uint8_t TTL;
+    //! Allow the use of interfaces that are not running.
+    bool allow_not_running_interfaces;
 };
 
 } // namespace rtps
diff --git a/include/fastrtps/utils/IPFinder.h b/include/fastrtps/utils/IPFinder.h
index d9958b869..26605e37c 100644
--- a/include/fastrtps/utils/IPFinder.h
+++ b/include/fastrtps/utils/IPFinder.h
@@ -81,7 +81,8 @@ public:
 
     RTPS_DllAPI static bool getIPs(
             std::vector<info_IP>* vec_name,
-            bool return_loopback = false);
+            bool return_loopback = false,
+            bool allow_not_run = false);
 
     /**
      * Get the IP4Adresses in all interfaces.
diff --git a/src/cpp/rtps/transport/UDPv4Transport.cpp b/src/cpp/rtps/transport/UDPv4Transport.cpp
index 2d1b1e0e4..05964baa7 100644
--- a/src/cpp/rtps/transport/UDPv4Transport.cpp
+++ b/src/cpp/rtps/transport/UDPv4Transport.cpp
@@ -40,9 +40,10 @@ using Log = fastdds::dds::Log;
 
 static void get_ipv4s(
         std::vector<IPFinder::info_IP>& locNames,
-        bool return_loopback = false)
+        bool return_loopback = false,
+        bool allow_not_run = false)
 {
-    IPFinder::getIPs(&locNames, return_loopback);
+    IPFinder::getIPs(&locNames, return_loopback, allow_not_run);
     auto new_end = remove_if(locNames.begin(),
                     locNames.end(),
                     [](IPFinder::info_IP ip)
@@ -58,9 +59,10 @@ static void get_ipv4s(
 
 static void get_ipv4s_unique_interfaces(
         std::vector<IPFinder::info_IP>& locNames,
-        bool return_loopback = false)
+        bool return_loopback = false,
+        bool allow_not_run = false)
 {
-    get_ipv4s(locNames, return_loopback);
+    get_ipv4s(locNames, return_loopback, allow_not_run);
     std::sort(locNames.begin(), locNames.end(),
             [](const IPFinder::info_IP&  a, const IPFinder::info_IP& b) -> bool
             {
@@ -106,7 +108,7 @@ UDPv4Transport::UDPv4Transport(
         const auto white_end = descriptor.interfaceWhiteList.end();
 
         std::vector<IPFinder::info_IP> local_interfaces;
-        get_ipv4s(local_interfaces, true);
+        get_ipv4s(local_interfaces, true, descriptor.allow_not_running_interfaces);
         for (const IPFinder::info_IP& infoIP : local_interfaces)
         {
             if (std::find_if(white_begin, white_end, [infoIP](const std::string& white_list_element)
@@ -279,7 +281,7 @@ void UDPv4Transport::get_ips(
         std::vector<IPFinder::info_IP>& locNames,
         bool return_loopback)
 {
-    get_ipv4s(locNames, return_loopback);
+    get_ipv4s(locNames, return_loopback, configuration_.allow_not_running_interfaces);
 }
 
 const std::string& UDPv4Transport::localhost_name()
@@ -397,7 +399,7 @@ bool UDPv4Transport::OpenInputChannel(
                 if (channelResource->interface() == s_IPv4AddressAny)
                 {
                     std::vector<IPFinder::info_IP> locNames;
-                    get_ipv4s_unique_interfaces(locNames, true);
+                    get_ipv4s_unique_interfaces(locNames, true, configuration_.allow_not_running_interfaces);
                     for (const auto& infoIP : locNames)
                     {
                         auto ip = asio::ip::address_v4::from_string(infoIP.name);
@@ -501,7 +503,7 @@ LocatorList UDPv4Transport::NormalizeLocator(
     if (IPLocator::isAny(locator))
     {
         std::vector<IPFinder::info_IP> locNames;
-        get_ipv4s(locNames);
+        get_ipv4s(locNames, false, configuration_.allow_not_running_interfaces);
         for (const auto& infoIP : locNames)
         {
             auto ip = asio::ip::address_v4::from_string(infoIP.name);
@@ -577,7 +579,7 @@ void UDPv4Transport::update_network_interfaces()
             if (channelResource->interface() == s_IPv4AddressAny)
             {
                 std::vector<IPFinder::info_IP> locNames;
-                get_ipv4s_unique_interfaces(locNames, true);
+                get_ipv4s_unique_interfaces(locNames, true, configuration_.allow_not_running_interfaces);
                 for (const auto& infoIP : locNames)
                 {
                     auto ip = asio::ip::address_v4::from_string(infoIP.name);
diff --git a/src/cpp/utils/IPFinder.cpp b/src/cpp/utils/IPFinder.cpp
index 23567eb94..86146a0ae 100644
--- a/src/cpp/utils/IPFinder.cpp
+++ b/src/cpp/utils/IPFinder.cpp
@@ -163,7 +163,8 @@ bool IPFinder::getIPs(
 
 bool IPFinder::getIPs(
         std::vector<info_IP>* vec_name,
-        bool return_loopback)
+        bool return_loopback,
+        bool allow_not_run)
 {
     struct ifaddrs* ifaddr, * ifa;
     int family, s;
@@ -178,7 +179,7 @@ bool IPFinder::getIPs(
 
     for (ifa = ifaddr; ifa != NULL; ifa = ifa->ifa_next)
     {
-        if (ifa->ifa_addr == NULL || (ifa->ifa_flags & IFF_RUNNING) == 0)
+        if (ifa->ifa_addr == NULL || (!allow_not_run && ((ifa->ifa_flags & IFF_RUNNING) == 0)))
         {
             continue;
         }

[i-and]

not_run_ifc_2.patch:

diff --git a/include/fastrtps/utils/IPFinder.h b/include/fastrtps/utils/IPFinder.h
index d9958b869..26605e37c 100644
--- a/include/fastrtps/utils/IPFinder.h
+++ b/include/fastrtps/utils/IPFinder.h
@@ -81,7 +81,8 @@ public:
 
     RTPS_DllAPI static bool getIPs(
             std::vector<info_IP>* vec_name,
-            bool return_loopback = false);
+            bool return_loopback = false,
+            bool allow_not_run = false);
 
     /**
      * Get the IP4Adresses in all interfaces.
diff --git a/src/cpp/rtps/transport/UDPv4Transport.cpp b/src/cpp/rtps/transport/UDPv4Transport.cpp
index 2d1b1e0e4..7e0940ea4 100644
--- a/src/cpp/rtps/transport/UDPv4Transport.cpp
+++ b/src/cpp/rtps/transport/UDPv4Transport.cpp
@@ -40,9 +40,10 @@ using Log = fastdds::dds::Log;
 
 static void get_ipv4s(
         std::vector<IPFinder::info_IP>& locNames,
-        bool return_loopback = false)
+        bool return_loopback = false,
+        bool allow_not_run = false)
 {
-    IPFinder::getIPs(&locNames, return_loopback);
+    IPFinder::getIPs(&locNames, return_loopback, allow_not_run);
     auto new_end = remove_if(locNames.begin(),
                     locNames.end(),
                     [](IPFinder::info_IP ip)
@@ -106,7 +107,7 @@ UDPv4Transport::UDPv4Transport(
         const auto white_end = descriptor.interfaceWhiteList.end();
 
         std::vector<IPFinder::info_IP> local_interfaces;
-        get_ipv4s(local_interfaces, true);
+        get_ipv4s(local_interfaces, true, true);
         for (const IPFinder::info_IP& infoIP : local_interfaces)
         {
             if (std::find_if(white_begin, white_end, [infoIP](const std::string& white_list_element)
@@ -279,7 +280,7 @@ void UDPv4Transport::get_ips(
         std::vector<IPFinder::info_IP>& locNames,
         bool return_loopback)
 {
-    get_ipv4s(locNames, return_loopback);
+    get_ipv4s(locNames, return_loopback, !is_interface_whitelist_empty());
 }
 
 const std::string& UDPv4Transport::localhost_name()
@@ -501,7 +502,7 @@ LocatorList UDPv4Transport::NormalizeLocator(
     if (IPLocator::isAny(locator))
     {
         std::vector<IPFinder::info_IP> locNames;
-        get_ipv4s(locNames);
+        get_ipv4s(locNames, false, !is_interface_whitelist_empty());
         for (const auto& infoIP : locNames)
         {
             auto ip = asio::ip::address_v4::from_string(infoIP.name);
diff --git a/src/cpp/utils/IPFinder.cpp b/src/cpp/utils/IPFinder.cpp
index 23567eb94..86146a0ae 100644
--- a/src/cpp/utils/IPFinder.cpp
+++ b/src/cpp/utils/IPFinder.cpp
@@ -163,7 +163,8 @@ bool IPFinder::getIPs(
 
 bool IPFinder::getIPs(
         std::vector<info_IP>* vec_name,
-        bool return_loopback)
+        bool return_loopback,
+        bool allow_not_run)
 {
     struct ifaddrs* ifaddr, * ifa;
     int family, s;
@@ -178,7 +179,7 @@ bool IPFinder::getIPs(
 
     for (ifa = ifaddr; ifa != NULL; ifa = ifa->ifa_next)
     {
-        if (ifa->ifa_addr == NULL || (ifa->ifa_flags & IFF_RUNNING) == 0)
+        if (ifa->ifa_addr == NULL || (!allow_not_run && ((ifa->ifa_flags & IFF_RUNNING) == 0)))
         {
             continue;
         }

[bkueng]

I also ran into this. I used to re-create the participant in case the network was not available, but with the introduced caching in #4241, this does not work anymore.

As a work-around I'm now forcing the docker0 interface into a running state before starting my application with this:

ip link add veth_dummy0 type veth peer name veth_dummy1
ip link set veth_dummy1 master docker0
ip link set veth_dummy0 up
ip link set veth_dummy1 up

Would be good to have a better solution though, as neither of the 2 solutions proposed elsewhere work for me:

• Dynamic network interfaces do not work with whitelisting
• Checking the network interface before creating the participant is prone to race conditions

[Mario-DL]

Hi @i-and @bkueng,

We have just released Fast DDS v3.1.0 which solves some issues regarding Dynamic Network interfaces. Could you please check if the issue persists with that ?

[bkueng]

Thanks for the quick reply. So with 3.1.0 I can just call participant->set_qos(...) periodically, and it will re-enumerate, also with transport whitelisting?
And is there a way to query which interfaces are actually used?

[Mario-DL]

Yes, it should also take the transport whitelisting into account.
If you call participant->get_qos() you could infer it by looking at the updated metatraffic and default locator lists
Could you check whether it if issue is solved with that ?