Merge pull request #1809 from dedis/work-karate-4xiom5-adding-more-tests

Minimal cross platform UI testing: Wallet and lao tests

tests/karate/docs/README.md

@@ -11,8 +11,7 @@ To create test cases, we handcraft messages with either valid or invalid message
 The mock components then send these messages to the component being tested.
 We then check that the responses the mock components receive are as expected.
 
-## Architecture
-### Features and Scenarios
+## Features and Scenarios
 Karate test cases are called scenarios and they are grouped within different feature files.
 Each feature file tests a different message type (i.e. electionOpen, createRollCall etc.).
 
@@ -24,7 +23,7 @@ For instance, `publish` creates a message of type publish that contains some hig
 - **Then**: Asserts that the action taken in the 'When' step has the expected outcome.
 - **And**: Connector that can be used after any of the other keywords.
 
-### Background section
+## Background section
 Code defined in the background section of a feature file runs before each scenario. This is especially useful for:
 
 - **Sharing scopes with other features**: The call to `read(classpath: "path/to/feature")` is used to make the current feature share the same scope as the feature that is called.
@@ -35,6 +34,7 @@ For instance, reading `mockClient.feature` exposes functions like `createMockFro
 - **Setting up previous steps necessary for a test**: For instance, before roll call messages can be tested, a LAO needs to be created first.
 `simpleScenarios.feature` contains many such useful setup steps.
 
+## Backend tests architecture
 ### Data model
 To generate valid message data for JSON payloads dynamically, a simplified version of the model is implemented in Java code.
 Mock components can create valid objects (for instance LAO, RollCall, Elections etc.), that can be used to handcraft messages.
@@ -59,7 +59,7 @@ This class provides the functions to create model data for LAOs, roll calls, ele
 For instance, here the created organizer and LAO are passed to the `createLaoScenario`.
 - The name tag `@createRollCall1` is used to call individual scenarios on the command line, see [Running the Tests](#running-the-tests)
 
-```
+```gherkin
 Feature: Create a Roll Call
 
   Background:
@@ -68,8 +68,8 @@ Feature: Create a Roll Call
     * call read(serverFeature)
     * call read(mockClientFeature)
     * def organizer = call createMockFrontend
-    * def lao = organizer.createValidLao()
-    * def validRollCall = organizer.createValidRollCall(lao)
+    * def lao = organizer.generateValidLao()
+    * def validRollCall = organizer.generateValidRollCall(lao)
     * call read(createLaoScenario) { organizer: '#(organizer)', lao: '#(lao)' }
 
   @createRollCall1
@@ -94,6 +94,37 @@ Feature: Create a Roll Call
     And match organizer.receiveNoMoreResponses() == true
 ```
 
+## Frontend tests architecture
+### Files
+* `utils`
+  * `constants.feature`: Contains all the necessary constants. Usually called at the beginning of any feature.
+  * `android.feature` and `web.feature`: Contain platform specific scenarios that will be used by the actual tests. Both files should implement the same scenarios.
+  * `platform.feature`: A simple wrapper around `android.feature` and `web.feature` that allows you to call the right scenario depending on the current env you are testing for. (i.e. if you set `karate.env=web`, it will call scenarios from `web.feature`)
+  * `mock_client.feature`: Allows you to create a mock client via `createMockClient`. Automatically stops all clients after each scenario.
+* `features`: The actual tests
+
+### Example: Lao join
+```gherkin
+Feature: LAO
+  Background:
+    # Get the needed utils
+    * call read('classpath:fe/utils/constants.feature')
+    * call read(MOCK_CLIENT_FEATURE)
+
+  @name=lao_join
+  Scenario: Manually connect to an existing LAO
+    # Use a mock client to create a random lao
+    Given def organizer = createMockClient()
+    And def lao = organizer.createLao()
+    # Call platform specific code with some parameters
+    # i.e. if karate.env=web, equals to call('classpath:fe/utils/web.feature@name=lao_join') { params: { lao: "#(lao)" } }
+    When call read(PLATFORM_FEATURE) { name: "#(JOIN_LAO)", params: { lao: "#(lao)" } }
+    # Actual test: The user should not have access to the button
+    Then assert !exists(event_create_button)
+    And screenshot()
+```
+
+
 ## First Setup
 
 ### All
@@ -173,16 +204,34 @@ mvn test -Dkarate.options="--tags @scenarioTagName"
 
 With the Karate plugin for IntelliJ, the full tests can also be run directly from inside IDE in the `BackEndTest` class.
 
+### Web Front-end
+Build the app with `npm run build-web` in the corresponding directory.
+
+Launch the Appium server (with `appium`).
+
+Finally run the tests:
+* All tests: `mvn test -Dkarate.env=web -Dtest=FrontEndTest#fullTest`
+* One feature: `mvn test -Dkarate.env=web -Dtest=FrontEndTest#fullTest -Dkarate.options="classpath:fe/features/<file_name>.feature"`
+* One scenario: `mvn test -Dkarate.env=web -Dtest=FrontEndTest#fullTest -Dkarate.options="classpath:fe/features/<file_name>.feature --tags=@name=<scenario_name>"`
+
+The following options are available (option names must be prefixed by `-D`).
+| Name         | Description                                    | Default                                   |
+|--------------|------------------------------------------------|-------------------------------------------|
+| browser      | One of 'chrome', 'safari', 'edge' or 'firefox' | 'chrome'                                  |
+| url          | URL of the web app                             | 'file:../../fe1-web/web-build/index.html' |
+| screenWidth  | Width of the browser                           | 1920                                      |
+| screenHeight | Height of the browser                          | 1080                                      |
+| serverURL    | Client URL of the backend server               | 'ws://localhost:9000/client' for the web and 'ws://10.0.2.2:9000/client' for android |
 
 ### Android Front-end
 Build the application by running `./gradlew assembleDebug` in the corresponding directory.
 
-Then, start an emulator from Android Studio and launch the Appium server (using the command `appium`).
+Launch the Appium server (with `appium`).
 
-Finally run the tests.
-```
-mvn test -Dkarate.env=android -Dtest=FrontEndTest#fullTest
-```
+Finally run the tests:
+* All tests: `mvn test -Dkarate.env=android -Dtest=FrontEndTest#fullTest`
+* One feature: `mvn test -Dkarate.env=android -Dtest=FrontEndTest#fullTest -Dkarate.options="classpath:fe/features/<file_name>.feature"`
+* One scenario: `mvn test -Dkarate.env=android -Dtest=FrontEndTest#fullTest -Dkarate.options="classpath:fe/features/<file_name>.feature --tags=@name=<scenario_name>"`
 
 In case you have multiple emulators running, you may specify one by avd id. To find the avd id of some emulator, go to the Device Manager (`Tools -> Device Manager`) and follow the steps in the image below.
 
@@ -194,20 +243,8 @@ mvn test -Dkarate.env=android -Davd=<avd_id> -Dtest=FrontEndTest#fullTest
 #e.g. mvn test -Dkarate.env=android -Davd=Galaxy_Note_9_API_29 -Dtest=FrontEndTest#fullTest
 ```
 
-### Web Front-end
-Build the app with `npm run build-web` in the corresponding directory.
-
-Launch the Appium server (with `appium`).
-
-Run the tests.
-```
-mvn test -Dkarate.env=web -Dtest=FrontEndTest#fullTest
-```
-
 The following options are available (option names must be prefixed by `-D`).
 | Name         | Description                                    | Default                                   |
 |--------------|------------------------------------------------|-------------------------------------------|
-| browser      | One of 'chrome', 'safari', 'edge' or 'firefox' | 'chrome'                                  |
-| url          | URL of the web app                             | 'file:../../fe1-web/web-build/index.html' |
-| screenWidth  | Width of the browser                           | 1920                                      |
-| screenHeight | Height of the browser                          | 1080                                      |
+| avd          | Name of the android emulator                   | Choosen automatically by appium           |
+| serverURL    | Client URL of the backend server               | 'ws://localhost:9000/client' for the web and 'ws://10.0.2.2:9000/client' for android |

tests/karate/src/test/java/be/features/LAO/create.feature

@@ -8,7 +8,7 @@ Feature: Create a pop LAO
     * call read(serverFeature)
     * call read(mockClientFeature)
     * def organizer = call createMockFrontend
-    * def validLao = organizer.createValidLao()
+    * def validLao = organizer.generateValidLao()
 
   @create1
   Scenario: Create Lao request with empty lao name should fail with an error response

tests/karate/src/test/java/be/features/LAO/update.feature

@@ -7,7 +7,7 @@ Feature: Update a LAO
     * call read(serverFeature)
     * call read(mockClientFeature)
     * def organizer = call createMockFrontend
-    * def lao = organizer.createValidLao()
+    * def lao = organizer.generateValidLao()
 
     # This call executes all the steps to create a valid lao on the server before every scenario
     # (lao creation, subscribe, catchup)

tests/karate/src/test/java/be/features/decentralizedCom/getMessagesById.feature

@@ -8,7 +8,7 @@ Feature: Request messages by id from other servers
     * call read(mockClientFeature)
     * def mockBackend = call createMockBackend
     * def mockFrontend = call createMockFrontend
-    * def lao = mockFrontend.createValidLao()
+    * def lao = mockFrontend.generateValidLao()
 
     # Create the template for heartbeat message
     # This is used in combination with 'eval' to dynamically resolve the channel keys in the heartbeat JSON
@@ -63,7 +63,7 @@ Feature: Request messages by id from other servers
   # trigger a getMessagesById anymore
   @getMessagesById4
   Scenario: Server should not request messages that it already has
-    Given def validRollCall = mockFrontend.createValidRollCall(lao)
+    Given def validRollCall = mockFrontend.generateValidRollCall(lao)
     And def validCreateRollCall =
       """
         {

tests/karate/src/test/java/be/features/decentralizedCom/heartbeat.feature

@@ -8,8 +8,8 @@ Feature: Send heartbeats to other servers
     * call read(mockClientFeature)
     * def mockBackend = call createMockBackend
     * def mockFrontend = call createMockFrontend
-    * def lao = mockFrontend.createValidLao()
-    * def validRollCall = mockFrontend.createValidRollCall(lao)
+    * def lao = mockFrontend.generateValidLao()
+    * def validRollCall = mockFrontend.generateValidRollCall(lao)
 
     # This call executes all the steps to create a valid lao on the server before every scenario
     # (lao creation, subscribe, catchup)

tests/karate/src/test/java/be/features/digitalCash/transaction.feature

@@ -7,8 +7,8 @@ Feature: Simple Transactions for digital cash
     * call read(mockClientFeature)
     * def organizer = call createMockFrontend
     * def recipient = call createMockFrontend
-    * def lao = organizer.createValidLao()
-    * def rollCall = organizer.createValidRollCall(lao)
+    * def lao = organizer.generateValidLao()
+    * def rollCall = organizer.generateValidRollCall(lao)
 
     # This call executes all the steps to set up a lao, complete a roll call and subscribe to the coin channel
     * call read(setupCoinChannelScenario) { organizer: '#(organizer)', lao: '#(lao)', rollCall: '#(rollCall)' }

tests/karate/src/test/java/be/features/election/castVote.feature

@@ -6,9 +6,9 @@ Feature: Cast a vote
     * call read(serverFeature)
     * call read(mockClientFeature)
     * def organizer = call createMockFrontend
-    * def lao = organizer.createValidLao()
-    * def rollCall = organizer.createValidRollCall(lao)
-    * def election = organizer.createValidElection(lao)
+    * def lao = organizer.generateValidLao()
+    * def rollCall = organizer.generateValidRollCall(lao)
+    * def election = organizer.generateValidElection(lao)
     * def question = election.createQuestion()
 
     # This call executes all the steps to set up a lao, complete a roll call and open an election with one question
@@ -71,7 +71,7 @@ Feature: Cast a vote
   # upon casting a vote
   @castVote3
   Scenario: Casting a valid vote on non existent election should return an error
-    Given def newElection = organizer.createValidElection(lao)
+    Given def newElection = organizer.generateValidElection(lao)
     And def newQuestion = newElection.createQuestion()
     And def newVote = newQuestion.createVote(0)
     And def newCastVote = newElection.castVote(newVote)

tests/karate/src/test/java/be/features/election/electionEnd.feature

@@ -6,9 +6,9 @@ Feature: Terminate an election
     * call read(serverFeature)
     * call read(mockClientFeature)
     * def organizer = call createMockFrontend
-    * def lao = organizer.createValidLao()
-    * def rollCall = organizer.createValidRollCall(lao)
-    * def election = organizer.createValidElection(lao)
+    * def lao = organizer.generateValidLao()
+    * def rollCall = organizer.generateValidRollCall(lao)
+    * def election = organizer.generateValidElection(lao)
     * def question = election.createQuestion()
 
     # This call executes all the steps to set up a lao, complete a roll call, open an election and cast a vote

tests/karate/src/test/java/be/features/election/electionOpen.feature

@@ -6,9 +6,9 @@ Feature: Open an Election
     * call read(serverFeature)
     * call read(mockClientFeature)
     * def organizer = call createMockFrontend
-    * def lao = organizer.createValidLao()
-    * def rollCall = organizer.createValidRollCall(lao)
-    * def election = organizer.createValidElection(lao)
+    * def lao = organizer.generateValidLao()
+    * def rollCall = organizer.generateValidRollCall(lao)
+    * def election = organizer.generateValidElection(lao)
     * def question = election.createQuestion()
 
     # This call executes all the steps to set up a lao, complete a roll call and create an election with one question
@@ -38,7 +38,7 @@ Feature: Open an Election
   # upon an open election message
   @electionOpen2
   Scenario: Opening the election without a setup should result in an error
-    Given def newElection = organizer.createValidElection(lao)
+    Given def newElection = organizer.generateValidElection(lao)
     And def newElectionOpen = newElection.open()
     And def validElectionOpen =
       """

tests/karate/src/test/java/be/features/election/electionSetup.feature

@@ -6,9 +6,9 @@ Feature: Setup an Election
     * call read(serverFeature)
     * call read(mockClientFeature)
     * def organizer = call createMockFrontend
-    * def lao = organizer.createValidLao()
-    * def rollCall = organizer.createValidRollCall(lao)
-    * def election = organizer.createValidElection(lao)
+    * def lao = organizer.generateValidLao()
+    * def rollCall = organizer.generateValidRollCall(lao)
+    * def election = organizer.generateValidElection(lao)
     * def question = election.createQuestion()
 
     # This call executes all the steps to set up a lao and complete a roll call, to get a valid pop token

tests/karate/src/test/java/be/features/rollCall/closeRollCall.feature

@@ -8,8 +8,8 @@ Feature: Close a Roll Call
     * call read(serverFeature)
     * call read(mockClientFeature)
     * def organizer = call createMockFrontend
-    * def lao = organizer.createValidLao()
-    * def rollCall = organizer.createValidRollCall(lao)
+    * def lao = organizer.generateValidLao()
+    * def rollCall = organizer.generateValidRollCall(lao)
 
     # This call executes all the steps to open a valid roll call on the server before every scenario
     # (lao creation, subscribe, catchup, roll call creation, roll call open)
@@ -78,7 +78,7 @@ Feature: Close a Roll Call
 
   @closeRollCall4
   Scenario: Closing a Roll Call that was not opened on the server returns an error
-    Given def newRollCall = organizer.createValidRollCall(lao)
+    Given def newRollCall = organizer.generateValidRollCall(lao)
     # This call creates the new roll call on the server without opening it
     And call read(createRollCallScenario) { organizer: '#(organizer)', lao: '#(lao)', rollCall: '#(newRollCall)' }
     And def closeNewRollCall = newRollCall.close()

tests/karate/src/test/java/be/features/rollCall/createRollCall.feature

@@ -8,8 +8,8 @@ Feature: Create a Roll Call
     * call read(serverFeature)
     * call read(mockClientFeature)
     * def organizer = call createMockFrontend
-    * def lao = organizer.createValidLao()
-    * def validRollCall = organizer.createValidRollCall(lao)
+    * def lao = organizer.generateValidLao()
+    * def validRollCall = organizer.generateValidRollCall(lao)
 
     # This call executes all the steps to create a valid lao on the server before every scenario
     # (lao creation, subscribe, catchup)
@@ -213,8 +213,8 @@ Feature: Create a Roll Call
   # in an error message from the backend.
   @createRollCall9
   Scenario: Roll Call Creation for non existent lao should return an error
-    Given def randomLao = organizer.createValidLao()
-    And def randomRollCall = organizer.createValidRollCall(randomLao)
+    Given def randomLao = organizer.generateValidLao()
+    And def randomRollCall = organizer.generateValidRollCall(randomLao)
     Given def validCreateRollCall =
       """
         {

tests/karate/src/test/java/be/features/rollCall/openRollCall.feature

@@ -8,8 +8,8 @@ Feature: Roll Call Open
     * call read(serverFeature)
     * call read(mockClientFeature)
     * def organizer = call createMockFrontend
-    * def lao = organizer.createValidLao()
-    * def rollCall = organizer.createValidRollCall(lao)
+    * def lao = organizer.generateValidLao()
+    * def rollCall = organizer.generateValidRollCall(lao)
 
     # This call executes all the steps to create a valid roll call on the server before every scenario
     # (lao creation, subscribe, catchup, roll call creation)
@@ -56,7 +56,7 @@ Feature: Roll Call Open
 
   @openRollCall3
   Scenario: Opening a Roll Call that was not created on the server returns an error
-    Given def newRollCall = organizer.createValidRollCall(lao)
+    Given def newRollCall = organizer.generateValidRollCall(lao)
     And def openNewRollCall = newRollCall.open()
     And def validOpenRollCall =
       """

tests/karate/src/test/java/be/features/utils/constants.feature

@@ -11,7 +11,7 @@ Feature: Constants
     * def ELECTION_RESULTS =        {"object": "election", "action": "result"}
 
     * def rootChannel = '/root'
-    * def random = Java.type('be.utils.RandomUtils')
+    * def random = Java.type('common.utils.RandomUtils')
 
     # Paths to util features
     * def utilsPath = 'classpath:be/features/utils/'

tests/karate/src/test/java/be/features/utils/mockClient.feature

@@ -6,7 +6,7 @@ Feature: Mock Client
     * def createMockFrontend =
       """
         function(){
-          var MockClient = Java.type('be.utils.MockClient')
+          var MockClient = Java.type('common.utils.MockClient')
           var mockFrontend = new MockClient(frontendWsURL)
           mockClients.push(mockFrontend)
           return mockFrontend
@@ -15,7 +15,7 @@ Feature: Mock Client
     * def createMockBackend =
       """
         function(){
-          var MockClient = Java.type('be.utils.MockClient')
+          var MockClient = Java.type('common.utils.MockClient')
           var mockBackend = new MockClient(backendWsURL)
           mockClients.push(mockBackend)
           return mockBackend

tests/karate/src/test/java/be/utils/JsonConverter.java

@@ -4,6 +4,7 @@
 import com.google.crypto.tink.subtle.Ed25519Sign;
 import com.intuit.karate.Json;
 import common.utils.Base64Utils;
+import common.utils.Hash;
 
 import java.nio.charset.StandardCharsets;
 import java.security.GeneralSecurityException;

tests/karate/src/test/java/be/utils/JsonConverterTest.java

@@ -2,6 +2,7 @@
 
 import com.intuit.karate.Json;
 import common.utils.Base64Utils;
+import common.utils.Hash;
 import org.junit.jupiter.api.Test;
 
 import java.security.GeneralSecurityException;

tests/karate/src/test/java/be/model/Election.java → tests/karate/src/test/java/common/model/Election.java

@@ -1,7 +1,7 @@
-package be.model;
+package common.model;
 
-import be.utils.Hash;
-import be.utils.RandomUtils;
+import common.utils.Hash;
+import common.utils.RandomUtils;
 
 import java.time.Instant;
 import java.util.*;

tests/karate/src/test/java/be/model/ElectionQuestion.java → tests/karate/src/test/java/common/model/ElectionQuestion.java

@@ -1,4 +1,4 @@
-package be.model;
+package common.model;
 
 import java.util.List;