Skip to content

Home

Jump to bottom
JaeYoung Hwang edited this page Sep 18, 2017 · 5 revisions

Welcome to oneM2M Conformance Testing Project Wiki Page!

  • oneM2MTester project is a oneM2M conformance testing project which was initialized by Korea Electronics Technology Institute (KETI) in January 2016 and collaborated with 10 international members Sejong University (SJU), Ericsson, Easy Global Market (EGM), Huawei, Korea Telecommunications Technology Association (TTA), DTNC, Sensinov, InterDigital, LAAS-CNRS, and InnoWireless.
  • The project aims to develop and distribute an open source oneM2M conformance testing tool and associated user manuals.
  • The oneM2MTester tool is developed based on open source Eclipse TITAN for oneM2M platform developers or testing engineers to test and evaluate their oneM2M implementations (CSE and AE) during or after the implementation development.

Members Logo

KETI logo SJU logo Ericsson logo EGM logo Huawei logo TTA logo DTNC logo Sensinov logo InterDigital logo LAAS-CNRS logo InnoWireless logo

Functionalities

  • The oneM2MTester implements system adapter and codec within a port to enable the communication between TTCN-3 test system and the implementation under test (IUT). The system adapter and codec are protocol dependent and they are responsible for encoding oneM2M request primitive message into protocol-specific formatted message before sending the messages to the IUT and vice versa.
  • The oneM2MTester makes use of Eclipse TITAN's TTCN-3 Test System environment which includes components of TITAN Designer, TITAN Executor, and TITAN LogViewer etc.
    • TITAN Designer : providing TTCN-3 editing and grammar checking functions and a graphical user interface for building an executable test suite;
    • TITAN Executor : providing a graphical user interface to execute testcases and generating corresponding verdict result;
    • TITAN LogViewer : logging the testcase running progress and providing a graphical presentation to assist user debugging their IUT. Besides, a group of log options are available for users to choose for different log purposes;
    • TITAN Runtime Configuration graphical user interface : providing protocol configuration for IUT and logging preferences for testers.

Software Architecture

oneM2MTester-architecture

  • The oneM2MTester extends the architecture of Eclipse TITAN by adding oneM2M system adapters and codecs for oneM2M binding protocols to enable that the TTCN-3 test system can communicate with and execute oneM2M Abstract Test Suite againstoneM2M implementations to achieve testing and evaluation purpose.

    oneM2M Port - Enhanced with translation capabilities

  • TTCN-3 language extensions: Configuration and Deployment Support defines an extended feature for ports which "allows to convert messages and/or addresses of one type into messages and/or addresses of different type during sending or receiving... It can be used e.g. in situations where the test behaviour is defined on one set of data types but the implementation under test (or connected component) actually communicates using a different set of data types, i.e. if the test system works on a different layer of the protocol stack than the implementation of the test."

oneM2M Abstract Test Suite

  • oneM2M Abstract Suite is a conformance testing test suite and fully contributed and maintained by oneM2M Task Force-001 supervised by oneM2M Test Work Group.
  • oneM2M Abstract Suite is developed based on oneM2M Testing Specification TS-0018 Test Suite Structure and Test Purpose.
  • oneM2M Abstract Suite can be downloaded and cloned through GitLab.

Runtime Environment

WE RECOMMEND USERS TO INSTALL RUNTIME ENVIRONMENT IN LINUX OPERATION SYSTEM. THE GUIDE FOR SETTING RUNTIME ENVIRONMENT ARE APPLIED FOR LINUX OPERATION SYSTEM FOR THE MOMENT!!

  • Eclipse Integrated Development Environment (IDE) * Eclipse IDE is required to install Eclipse TITAN environment. We recommend users to check and download the latest Eclipse IDE release from here. Note that Eclipse IDE MARS has been successfully evaluated with Eclipse TITAN.

  • Eclipse TITAN

    • Eclipse TITAN Core

      • TITAN Core is a core part of Eclipse TITAN, which is responsible for executing in a Unix/Linux-like environment and a set of Eclipse plug-ins.
      • The TITAN compiler builds an executable test suite from the TTCN-3, the test port code and the TITAN runtime library. It is written in C++ and is available for download in binary form from the Eclipse TITAN Project Download page. Or you can clone it from Eclipse TITAN GitHub project. Installation of TITAN core in Linux can be found here.

    • Eclipse TITAN Plugins

      • Eclipse TITAN plugins simplify to write TTCN-3 code, control test execution, log test execution progressing and generate test case execution verdict.
      • A group of Eclipse TITAN plugins is wrapped into one single package with its dependencies. The package Eclipse plug-ins plus dependencies is available for download from Eclipse TITAN project download page. Note: please check the latest version of plugins before downloading.

  • Configurations

    • TITAN Core Configuration

      • Several system variables need to be configured before starting to use.
        • TTCN3_DIR: This variable TTCN3_DIR contains the absolute location of the TITAN installation.
        • LD_LIBRARY_PATH: Here the user needs to specify the OpenSSL shared library, that is provided with TITAN. The LD_LIBRARY_PATH is set to $TTCN3_DIR/lib.
        • PATH: This variable should contain the location of TITAN binaries. The PATH is set to $TTCN3_DIR/bin. More detail regarding the configuration for system variables, please refer to Installation Guide for the TITAN TTCN-3 Test Executor included in Titan documentation package (pdf).

    • Eclipse Plugins Configuration

      • The TITAN installation path can be configured in TITAN Preferences.

Get Started

  • Download and Import oneM2MTester System Adapter Package into Eclipse TITAN Environment

    • The oneM2MTester source package can be cloned through below command: https://github.com/IoTKETI/oneM2MTester.git

  • Build oneM2MTester Project Build project via Command Line Mode (CLI) following

    • Export Titan project description .tpd (e.g. oneM2MTester_v_2_0_0.tpd) file via Titan TTCN-3 Editor GUI into the oneM2MTester project directory
    • Generate Makefile through .tpd file: Run command ~/oneM2MTester_v_2_0_0$ ttcn3_makefilegen -t oneM2MTester_v_2_0_0.tpd, then a new /bin directory will be generated containing all linked files associated with a makefile
    • Go to /bin directory, and build the oneM2MTester project using command make or you can choose how many CPU cores to compile the project in order to speed up the compiling by using command make -j8 in case your machine has at least an 8-core CPU.

** NOTE: If you encounter "out of memory allocating error" when you compile the oneM2MTester project, try to update the gcc compiler flag CXXFLAGS in the Makefile (under the /bin directory) as follows.**

CXXFLAGS = -Wall -O2 or,

CXXFLAGS = -Wall --param ggc-min-expand=10 -O2, or

CXXFLAGS = -Wall --param ggc-min-expand=10 -O1

  • Set IUT Configurations and TITAN Runtime Configurations

    • Set IUT configurations in Titan config.file

    • Set TITAN runtime configurations in terms of TIATN parallel launcher configuration through Titan Runtime Configuration GUI including

      • Basic main controller options,
      • Host controller,
      • Testsets,
      • Basic performance setting, and
      • Environment setting

  • Check and Select Testcase List for Running

    • Before running executable test suite against your SUT using oneM2MTester, check the testcase list table in the wiki page to select testcases related to functions that are supported by your SUT.

  • Run Testcases Against IUT

    • In TITAN Executor graphical user interface, testcases can be started by TITAN Execution Controller. The TITAN Executor controls the test execution and generates final verdict for each executed testcase while TITAN LogViewer generates logs during the testcase execution.

    • If users prefer using Command Line (CLI) to execute testcase, go following steps:

      • Step-1: Add testcase list into the [EXECUTE] section in a Titan configuration (.cfg) file, e.g.

      [EXECUTE]

      # In this section you can specify what parts of your test suite you want to execute.

OneM2M_Testcases.TC_CSE_GEN_DEL_001_CSR;

OneM2M_Testcases.TC_CSE_GEN_UPD_001_CSR;

      • Step-2: Run following command in CLI to execute testcases

        ttcn3_start ./[EXECUTABLE_TEST_SUITE_NAME] <TITAN_CONFIGURATION_FILENAME>.cfg

  • Check Verdict

    • The testers and test experts can check the verdict and evaluate the verdict is trusted or not. In the case of "fail" or "inconc" verdict, the testers can refer to the running logs to debug the IUT to find what caused that exception.

  • Debug

    • Eclipse TITAN provides a group of debugging options to handle different debug requirements. The debug preferences can be configured in oneM2MTester configuration .cfg file. The debug options include
      • DEBUG
      • PORTEVENT
      • TESTCASE
      • STATISTICS
      • MATCHING
      • PARALLEL
      • VERDICTOP
      • VERDICTOP_SETVERDICT More details for the debug options can be found in Programmer's Technical Reference Guide for the TITAN TTCN-3 Toolset which is available for downloading in Titan documentation package (pdf).

References

Clone this wiki locally