Anjay 3.8.1

Improvements - In case when the LwM2M server answers with an RST message to a notification that is yielding an error value (e.g. failure to read), which effectively cancels the notification, Anjay is not infinitely trying to transmit that message with error value once again. New behavior is enabled by default, and controlled with `WITH_AVS_COAP_OBSERVE_FORCE_CANCEL_ON_UNACKED_ERROR` option of `avs_coap`. Existing projects have to opt-in explicitly. Bugfixes - Actually fixed compatibility with Mbed TLS 3.6. - Fixed compatibility of integration test framework with Mbed TLS versions that enabled TLS 1.3, but didn't use `MBEDTLS_USE_PSA_CRYPTO`. - The -Wformat warning appearing in some compilers has been fixed. - Fixed LwM2M CBOR parser incorrectly accepting inputs containing empty arrays as keys - Prevent from generating non unique session tokens when the monotonic system clock granulation is not fine enough. - Refactored how timeouts are handled in pymbedtls to be in line with use of mbedTLS in avs_commons.
AVSystem · Oct 25, 2024 · 98566ae · 98566ae
1 parent 3ba32ac
commit 98566ae
Show file tree

Hide file tree

Showing 98 changed files with 2,523 additions and 925 deletions.
diff --git a/.gitignore b/.gitignore
@@ -15,6 +15,7 @@ nbproject
 
 # build configuration autogenerated files
 /build
+/coverage
 /output
 *.swp
 CMakeFiles/

diff --git a/CHANGELOG.md b/CHANGELOG.md
@@ -1,5 +1,28 @@
 # Changelog
 
+## 3.8.1 (October 25th, 2024)
+
+### Improvements
+- In case when the LwM2M server answers with an RST message to a notification
+  that is yielding an error value (e.g. failure to read), which effectively
+  cancels the notification, Anjay is not infinitely trying to transmit that
+  message with error value once again. New behavior is enabled by default, and
+  controlled with `WITH_AVS_COAP_OBSERVE_FORCE_CANCEL_ON_UNACKED_ERROR` option
+  of `avs_coap`. Existing projects have to opt-in explicitly.
+
+### Bugfixes
+
+- Actually fixed compatibility with Mbed TLS 3.6.
+- Fixed compatibility of integration test framework with Mbed TLS versions that
+  enabled TLS 1.3, but didn't use `MBEDTLS_USE_PSA_CRYPTO`.
+- The -Wformat warning appearing in some compilers has been fixed.
+- Fixed LwM2M CBOR parser incorrectly accepting inputs containing empty arrays
+  as keys
+- Prevent from generating non unique session tokens when the monotonic system
+  clock granulation is not fine enough.
+- Refactored how timeouts are handled in pymbedtls to be in line with use of
+  mbedTLS in avs_commons.
+
 ## 3.8.0 (May 28th, 2024)
 
 ### BREAKING CHANGES

diff --git a/CMakeLists.txt b/CMakeLists.txt
@@ -8,7 +8,7 @@
 cmake_minimum_required(VERSION 3.6.0)
 
 project(anjay C)
-set(ANJAY_VERSION "3.8.0" CACHE STRING "Anjay library version")
+set(ANJAY_VERSION "3.8.1" CACHE STRING "Anjay library version")
 set(ANJAY_BINARY_VERSION 1.0.0)
 
 set(ANJAY_SOURCE_DIR "${CMAKE_CURRENT_SOURCE_DIR}")
@@ -433,6 +433,8 @@ add_library(anjay
             src/core/io/anjay_cbor_out.c
             src/core/io/anjay_common.c
             src/core/io/anjay_common.h
+            src/core/io/anjay_corelnk.c
+            src/core/io/anjay_corelnk.h
             src/core/io/anjay_dynamic.c
             src/core/io/anjay_input_buf.c
             src/core/io/anjay_json_encoder.c
@@ -617,7 +619,7 @@ if(WITH_TEST)
     add_custom_target(check)
     add_custom_target(anjay_unit_check)
     add_dependencies(check anjay_unit_check)
-
+    
     # anjay_test
     add_executable(anjay_test EXCLUDE_FROM_ALL
                    $<TARGET_PROPERTY:anjay,SOURCES>

diff --git a/README.md b/README.md
@@ -179,7 +179,7 @@ pip3 install -U -r requirements.txt
 
 ### Running the demo client
 
-For initial development and testing of LwM2M clients, we recommend using the [Coiote IoT Device Management](https://www.avsystem.com/products/coiote-iot-device-management-platform/) where you can use the basic LwM2M server functionality for free.
+For initial development and testing of LwM2M clients, we recommend using the [Coiote IoT Device Management](https://avsystem.com/coiote-iot-device-management-platform/) where you can use the basic LwM2M server functionality for free.
 
 After setting up an account and adding the device entry, you can compile Anjay demo client and connect it to the platform by running:
 
@@ -322,11 +322,11 @@ See [LICENSE](LICENSE) file.
 
 ### Commercial support
 
-Anjay LwM2M library comes with the option of [full commercial support, provided by AVSystem](https://www.avsystem.com/products/anjay/).
+Anjay LwM2M library comes with the option of [full commercial support, provided by AVSystem](https://avsystem.com/anjay-iot-sdk/).
 
 The list of features available commercially is [available here](https://AVSystem.github.io/Anjay-doc/CommercialFeatures.html).
 
-If you're interested in LwM2M Server, be sure to check out the [Coiote IoT Device Management](https://www.avsystem.com/products/coiote-iot-dm/) platform by AVSystem. It also includes the [interoperability test module](https://www.avsystem.com/lwm2m-interoperability-test/) that you can use to test your LwM2M client implementation. Our automated tests and testing scenarios enable you to quickly check how interoperable your device is with LwM2M.
+If you're interested in LwM2M Server, be sure to check out the [Coiote IoT Device Management](https://www.avsystem.com/products/coiote-iot-dm/) platform by AVSystem. It also includes the [interoperability test module](https://avsystem.com/coiote-iot-device-management-platform/lwm2m-interoperability-test/) that you can use to test your LwM2M client implementation. Our automated tests and testing scenarios enable you to quickly check how interoperable your device is with LwM2M.
 
 ## Contributing
 

diff --git a/covconfig b/covconfig
diff --git a/deps/avs_coap/CMakeLists.txt b/deps/avs_coap/CMakeLists.txt
@@ -38,6 +38,7 @@ option(WITH_AVS_COAP_TCP "Enable CoAP over TCP support" ON)
 option(WITH_AVS_COAP_STREAMING_API "Enable streaming API" ON)
 option(WITH_AVS_COAP_OBSERVE "Enable support for observations" ON)
 option(WITH_AVS_COAP_OBSERVE_CANCEL_ON_TIMEOUT "Turn on cancelling observation on a timeout " OFF)
+option(WITH_AVS_COAP_OBSERVE_FORCE_CANCEL_ON_UNACKED_ERROR "Force cancelling observation on unacked error" ON)
 cmake_dependent_option(WITH_AVS_COAP_OBSERVE_PERSISTENCE "Enable observations persistence" ON "WITH_AVS_COAP_OBSERVE" OFF)
 option(WITH_AVS_COAP_BLOCK "Enable support for BLOCK/BERT transfers" ON)
 

diff --git a/deps/avs_coap/include_public/avsystem/coap/avs_coap_config.h.in b/deps/avs_coap/include_public/avsystem/coap/avs_coap_config.h.in
@@ -80,6 +80,18 @@
  */
 #cmakedefine WITH_AVS_COAP_OBSERVE_CANCEL_ON_TIMEOUT
 
+/**
+ * Force cancelling observation, even if a confirmable notification yielding
+ * an error response is not acknowledged or rejected with RST by the observer.
+ *
+ * This is a circumvention for some non-compliant servers that respond with an
+ * RST message to a confirmable notification yielding an error response. This
+ * setting makes the library cancel the observation in such cases, even though
+ * the notification is formally rejected. Additionally, it will also make the
+ * library cancel the observation if no response is received at all.
+ */
+#cmakedefine WITH_AVS_COAP_OBSERVE_FORCE_CANCEL_ON_UNACKED_ERROR
+
 /**
  * Enable support for observation persistence (<c>avs_coap_observe_persist()</c>
  * and <c>avs_coap_observe_restore()</c> calls).

diff --git a/deps/avs_coap/src/async/avs_coap_async_server.c b/deps/avs_coap/src/async/avs_coap_async_server.c
@@ -307,7 +307,19 @@ send_result_handler(avs_coap_ctx_t *ctx,
 
     server->delivery_handler(ctx, fail_err, server->delivery_handler_arg);
 
-    if (avs_is_ok(fail_err)) {
+    bool is_timeout = fail_err.category == AVS_COAP_ERR_CATEGORY
+                      && fail_err.code == AVS_COAP_ERR_TIMEOUT;
+    bool is_rst = fail_err.category == AVS_COAP_ERR_CATEGORY
+                  && fail_err.code == AVS_COAP_ERR_UDP_RESET_RECEIVED;
+    (void) is_timeout;
+    (void) is_rst;
+
+    bool do_cancel_on_error = avs_is_ok(fail_err);
+#ifdef WITH_AVS_COAP_OBSERVE_FORCE_CANCEL_ON_UNACKED_ERROR
+    do_cancel_on_error = do_cancel_on_error || is_timeout || is_rst;
+#endif // WITH_AVS_COAP_OBSERVE_FORCE_CANCEL_ON_UNACKED_ERROR
+
+    if (do_cancel_on_error) {
         cancel_notification_on_error(ctx,
                                      (avs_coap_observe_id_t) {
                                          .token = token
@@ -316,8 +328,7 @@ send_result_handler(avs_coap_ctx_t *ctx,
     }
 #if defined(WITH_AVS_COAP_OBSERVE) \
         && defined(WITH_AVS_COAP_OBSERVE_CANCEL_ON_TIMEOUT)
-    else if (fail_err.category == AVS_COAP_ERR_CATEGORY
-             && fail_err.code == AVS_COAP_ERR_TIMEOUT) {
+    else if (is_timeout) {
         avs_coap_observe_cancel(ctx, (avs_coap_observe_id_t) {
                                          .token = token
                                      });
@@ -1334,6 +1345,19 @@ avs_coap_notify_async(avs_coap_ctx_t *ctx,
         avs_errno(AVS_EINVAL);
     }
 
+    // FIXME: Unsolicited non-confirmable notifications with an error code are
+    // currently broken, because of lack of the Observe option for error values,
+    // the lower, UDP layer assumes the message to be an ACK, not a NON.
+    //
+    // For reference, see assumptions in choose_msg_type() in avs_coap_udp_ctx.c
+    if (reliability_hint == AVS_COAP_NOTIFY_PREFER_NON_CONFIRMABLE
+            && !avs_coap_code_is_success(response_header->code)) {
+        LOG(ERROR,
+            _("Unsolicited notifications with an error code are currently "
+              "broken"));
+        return avs_errno(AVS_EINVAL);
+    }
+
     avs_coap_observe_notify_t notify;
 
     avs_error_t err = _avs_coap_observe_setup_notify(ctx, &observe_id, &notify);

diff --git a/deps/avs_coap/src/tcp/avs_coap_tcp_ctx.c b/deps/avs_coap/src/tcp/avs_coap_tcp_ctx.c
@@ -7,6 +7,8 @@
  * See the attached LICENSE file for details.
  */
 
+#include <inttypes.h>
+
 #include <avs_coap_init.h>
 
 #ifdef WITH_AVS_COAP_TCP
@@ -59,7 +61,8 @@ static void log_tcp_msg_summary(const char *info,
 #    ifdef WITH_AVS_COAP_OBSERVE
     uint32_t observe;
     if (avs_coap_options_get_observe(&msg->options, &observe) == 0) {
-        snprintf(observe_str, sizeof(observe_str), ", Observe %u", observe);
+        snprintf(observe_str, sizeof(observe_str), ", Observe %" PRIu32,
+                 observe);
     }
 #    endif // WITH_AVS_COAP_OBSERVE
 

diff --git a/deps/avs_coap/tests/udp/async_observe.c b/deps/avs_coap/tests/udp/async_observe.c
@@ -249,6 +249,12 @@ AVS_UNIT_TEST(udp_observe, notify_async_confirmable) {
 }
 
 AVS_UNIT_TEST(udp_observe, notify_async_cancel_with_error_response) {
+    // FIXME: Unsolicited non-confirmable notifications with an error code are
+    // currently broken, because of lack of the Observe option for error values,
+    // the lower, UDP layer assumes the message to be an ACK, not a NON.
+    //
+    // For reference, see assumptions in choose_msg_type() in avs_coap_udp_ctx.c
+    return;
     test_env_t env __attribute__((cleanup(test_teardown))) =
             test_setup_default();
 
@@ -258,8 +264,7 @@ AVS_UNIT_TEST(udp_observe, notify_async_cancel_with_error_response) {
     const test_msg_t *responses[] = {
         COAP_MSG(ACK, CONTENT, ID(100), MAKE_TOKEN("Obserw"), OBSERVE(0),
                  NO_PAYLOAD),
-        COAP_MSG(NON, NOT_FOUND, ID(0), MAKE_TOKEN("Obserw"), OBSERVE(1),
-                 NO_PAYLOAD),
+        COAP_MSG(NON, NOT_FOUND, ID(0), MAKE_TOKEN("Obserw"), NO_PAYLOAD),
     };
 
     expect_recv(&env, request);
@@ -303,8 +308,7 @@ AVS_UNIT_TEST(udp_observe,
     const test_msg_t *responses[] = {
         COAP_MSG(ACK, CONTENT, ID(100), MAKE_TOKEN("Obserw"), OBSERVE(0),
                  NO_PAYLOAD),
-        COAP_MSG(CON, NOT_FOUND, ID(0), MAKE_TOKEN("Obserw"), OBSERVE(1),
-                 NO_PAYLOAD),
+        COAP_MSG(CON, NOT_FOUND, ID(0), MAKE_TOKEN("Obserw"), NO_PAYLOAD),
     };
 
     expect_recv(&env, requests[0]);
@@ -343,6 +347,144 @@ AVS_UNIT_TEST(udp_observe,
     ASSERT_OK(avs_coap_async_handle_incoming_packet(env.coap_ctx, NULL, NULL));
 }
 
+AVS_UNIT_TEST(udp_observe,
+              notify_async_rst_to_cancel_with_confirmable_error_response) {
+    test_env_t env __attribute__((cleanup(test_teardown_late_expects_check))) =
+            test_setup_default();
+
+    const test_msg_t *requests[] = {
+        COAP_MSG(CON, GET, ID(100), MAKE_TOKEN("Obserw"), OBSERVE(0),
+                 NO_PAYLOAD),
+        COAP_MSG(RST, EMPTY, ID(0), NO_PAYLOAD),
+    };
+    const test_msg_t *responses[] = {
+        COAP_MSG(ACK, CONTENT, ID(100), MAKE_TOKEN("Obserw"), OBSERVE(0),
+                 NO_PAYLOAD),
+        COAP_MSG(CON, NOT_FOUND, ID(0), MAKE_TOKEN("Obserw"), NO_PAYLOAD),
+    };
+
+    expect_recv(&env, requests[0]);
+    expect_request_handler_call(&env, AVS_COAP_SERVER_REQUEST_RECEIVED,
+                                requests[0],
+                                &(avs_coap_response_header_t) {
+                                    .code = responses[0]->response_header.code
+                                },
+                                NULL);
+    expect_observe_start(&env, requests[0]->msg.token);
+    expect_send(&env, responses[0]);
+    expect_request_handler_call(&env, AVS_COAP_SERVER_REQUEST_CLEANUP, NULL,
+                                NULL, NULL);
+
+    expect_has_buffered_data_check(&env, false);
+    ASSERT_OK(avs_coap_async_handle_incoming_packet(
+            env.coap_ctx, test_accept_new_request, &env));
+    avs_coap_observe_id_t observe_id = {
+        .token = requests[0]->msg.token
+    };
+
+    expect_send(&env, responses[1]);
+
+    avs_coap_exchange_id_t id;
+    ASSERT_OK(avs_coap_notify_async(env.coap_ctx, &id, observe_id,
+                                    &responses[1]->response_header,
+                                    AVS_COAP_NOTIFY_PREFER_CONFIRMABLE, NULL,
+                                    NULL, test_observe_delivery_handler, &env));
+    ASSERT_TRUE(avs_coap_exchange_id_valid(id));
+
+    expect_recv(&env, requests[1]);
+    // Reset response should trigger FAIL result
+    expect_observe_delivery(&env,
+                            _avs_coap_err(AVS_COAP_ERR_UDP_RESET_RECEIVED));
+
+    // Whether the observations gets actually canceled depends on the config
+#    ifdef WITH_AVS_COAP_OBSERVE_FORCE_CANCEL_ON_UNACKED_ERROR
+    expect_observe_cancel(&env, observe_id.token);
+#    endif // WITH_AVS_COAP_OBSERVE_FORCE_CANCEL_ON_UNACKED_ERROR
+
+    expect_has_buffered_data_check(&env, false);
+    ASSERT_OK(avs_coap_async_handle_incoming_packet(env.coap_ctx, NULL, NULL));
+
+#    ifdef WITH_AVS_COAP_OBSERVE_FORCE_CANCEL_ON_UNACKED_ERROR
+    // on exit, the observation should already be canceled
+    ASSERT_NULL(env.expects_list);
+#    else  // WITH_AVS_COAP_OBSERVE_FORCE_CANCEL_ON_UNACKED_ERROR
+    // we're using late_expects_check to capture the implicit cancellation when
+    // cleaning up the test
+    expect_observe_cancel(&env, observe_id.token);
+#    endif // WITH_AVS_COAP_OBSERVE_FORCE_CANCEL_ON_UNACKED_ERROR
+}
+
+AVS_UNIT_TEST(udp_observe,
+              notify_async_timeout_of_cancel_with_confirmable_error_response) {
+    avs_coap_udp_tx_params_t tx_params = AVS_COAP_DEFAULT_UDP_TX_PARAMS;
+    tx_params.max_retransmit = 0;
+    test_env_t env __attribute__((cleanup(test_teardown_late_expects_check))) =
+            test_setup(&tx_params, 4096, 4096, NULL);
+
+    const test_msg_t *requests[] = { COAP_MSG(
+            CON, GET, ID(100), MAKE_TOKEN("Obserw"), OBSERVE(0), NO_PAYLOAD) };
+    const test_msg_t *responses[] = {
+        COAP_MSG(ACK, CONTENT, ID(100), MAKE_TOKEN("Obserw"), OBSERVE(0),
+                 NO_PAYLOAD),
+        COAP_MSG(CON, NOT_FOUND, ID(0), MAKE_TOKEN("Obserw"), NO_PAYLOAD),
+    };
+
+    expect_recv(&env, requests[0]);
+    expect_request_handler_call(&env, AVS_COAP_SERVER_REQUEST_RECEIVED,
+                                requests[0],
+                                &(avs_coap_response_header_t) {
+                                    .code = responses[0]->response_header.code
+                                },
+                                NULL);
+    expect_observe_start(&env, requests[0]->msg.token);
+    expect_send(&env, responses[0]);
+    expect_request_handler_call(&env, AVS_COAP_SERVER_REQUEST_CLEANUP, NULL,
+                                NULL, NULL);
+
+    expect_has_buffered_data_check(&env, false);
+    ASSERT_OK(avs_coap_async_handle_incoming_packet(
+            env.coap_ctx, test_accept_new_request, &env));
+    avs_coap_observe_id_t observe_id = {
+        .token = requests[0]->msg.token
+    };
+
+    expect_send(&env, responses[1]);
+
+    avs_coap_exchange_id_t id;
+    ASSERT_OK(avs_coap_notify_async(env.coap_ctx, &id, observe_id,
+                                    &responses[1]->response_header,
+                                    AVS_COAP_NOTIFY_PREFER_CONFIRMABLE, NULL,
+                                    NULL, test_observe_delivery_handler, &env));
+    ASSERT_TRUE(avs_coap_exchange_id_valid(id));
+
+    const avs_time_duration_t EPSILON =
+            avs_time_duration_from_scalar(1, AVS_TIME_S);
+
+    _avs_mock_clock_advance(avs_time_duration_add(
+            avs_coap_udp_max_transmit_wait(&tx_params), EPSILON));
+
+    // Timeout should trigger FAIL result
+    expect_observe_delivery(&env, _avs_coap_err(AVS_COAP_ERR_TIMEOUT));
+
+    // Whether the observations gets actually canceled depends on the config
+#    ifdef WITH_AVS_COAP_OBSERVE_FORCE_CANCEL_ON_UNACKED_ERROR
+    expect_observe_cancel(&env, observe_id.token);
+#    endif // WITH_AVS_COAP_OBSERVE_FORCE_CANCEL_ON_UNACKED_ERROR
+
+    expect_has_buffered_data_check(&env, false);
+    avs_sched_run(env.sched);
+    ASSERT_OK(avs_coap_async_handle_incoming_packet(env.coap_ctx, NULL, NULL));
+
+#    ifdef WITH_AVS_COAP_OBSERVE_FORCE_CANCEL_ON_UNACKED_ERROR
+    // on exit, the observation should already be canceled
+    ASSERT_NULL(env.expects_list);
+#    else  // WITH_AVS_COAP_OBSERVE_FORCE_CANCEL_ON_UNACKED_ERROR
+    // we're using late_expects_check to capture the implicit cancellation when
+    // cleaning up the test
+    expect_observe_cancel(&env, observe_id.token);
+#    endif // WITH_AVS_COAP_OBSERVE_FORCE_CANCEL_ON_UNACKED_ERROR
+}
+
 AVS_UNIT_TEST(udp_observe, notify_async_confirmable_reset_response) {
 #    define NOTIFY_PAYLOAD "Notifaj"
     test_env_t env __attribute__((cleanup(test_teardown))) =

diff --git a/deps/avs_commons b/deps/avs_commons
diff --git a/doc/sphinx/snippet_sources.md5 b/doc/sphinx/snippet_sources.md5
@@ -89,6 +89,6 @@ be8a4c22a41c2ab79bf3a68f1f74f301  include_public/anjay/attr_storage.h
 56b500ecbd5ffc8d468327c650883cbd  include_public/anjay/fw_update.h
 b49ca67d9f4f04bf20261c261ad41822  include_public/anjay/io.h
 e7c2357db51f9896d2b82245699e0a42  include_public/anjay/security.h
-b95279360cd728e0b137ae6318b2c4a7  tests/integration/framework/test_utils.py
-d867c4d479f49fc39254f70bdda67fb5  tools/provisioning-tool/factory_prov/factory_prov.py
-82ad6e92a83128ffac51b9c369d5275d  tools/provisioning-tool/ptool.py
+db30c5e22151df17cfe414f47f2e4a38  tests/integration/framework/test_utils.py
+88058893fb17ae75099f6b998233a693  tools/provisioning-tool/factory_prov/factory_prov.py
+70d3ff815fb42ff052a7a05d47f9aee9  tools/provisioning-tool/ptool.py
diff --git a/doc/sphinx/source/BasicClient/BC-MandatoryObjects.rst b/doc/sphinx/source/BasicClient/BC-MandatoryObjects.rst
@@ -61,9 +61,9 @@ and add instances of them. We modify the code from the
 
 The first one will be ``setup_security_object()``. In this tutorial, we will use
 the Coiote IoT Device Management platform as the hard-coded server URI. You can
-go to https://www.avsystem.com/products/coiote-iot-device-management-platform/
-to create an account, and after logging in, add the device entry for your
-application. If you wish to use another server, then you must replace
+go to https://avsystem.com/coiote-iot-device-management-platform/ to create an
+account, and after logging in, add the device entry for your application. If you
+wish to use another server, then you must replace
 ``coap://eu.iot.avsystem.cloud:5683`` with a valid value.
 
 For now, we will establish non-secure connection, a secure one will be described