Limits permitted commands in Runner (#1016)

- Limit permitted commands to those listed in runner.yaml config file. - Fixes API calls to send correct invalid status for both non-existing and not permitted commands
INTO-CPS-Association · Oct 28, 2024 · a42678a · a42678a
1 parent 1f2a070
commit a42678a
Show file tree

Hide file tree

Showing 24 changed files with 1,098 additions and 860 deletions.
diff --git a/servers/execution/runner/README.md b/servers/execution/runner/README.md
@@ -1,6 +1,6 @@
 # :runner: Runner
 
-A utility service to manage safe execution of remote commands.
+A utility service to manage safe execution of remote scripts / commands.
 User launches this from commandline and let the utility
 manage the commands to be executed.
 
@@ -10,8 +10,28 @@ Multiple runners can be active simultaneously on one computer.
 The commands are sent via the REST API and are executed on the computer
 with active runner.
 
+:warning: Thanks for trying out this software.
+This software is in early stages of development and is not
+recommended for production use. Each released package will have
+a working API and matching documentation in this README.
+However, there will be breaking changes in the API across each release
+until the package reaches version 1.0.0.
+
 ## :arrow_down: Install
 
+### NPM Registry
+
+The package is available on
+[npmjs](https://www.npmjs.com/package/@into-cps-association/runner).
+
+Install the package with the following command:
+
+```bash
+sudo npm install -g @into-cps-association/runner
+```
+
+### Github Registry
+
 The package is available in Github
 [packages registry](https://github.com/orgs/INTO-CPS-Association/packages).
 
@@ -30,34 +50,69 @@ needs to have _read:packages_ scope.
 
 ## :gear: Configure
 
-The microservices requires config specified in YAML format.
+The utility requires config specified in YAML format.
 The template configuration file is:
 
 ```ini
 port: 5000
-location: "script" #directory location of scripts
+location: 'script' #directory location of scripts
+commands: #list of permitted scripts
+  - create
+  - execute
 ```
 
-The file should be named as _runner.yaml_ and placed in the directory
-in which the _runner_ microservice is run.
+It is suggested that the configuration file be named as _runner.yaml_
+and placed in the directory in which the _runner_ microservice is run.
+
+The `location` refers to the relative location of the scripts directory
+with respect to the location of _runner.yaml_ file.
+
+However, there is no limitation on either the configuration filename or
+the `location`. The path to _runner.yaml_ can either be relative or
+absolute path. However, the `location` path is always relative path
+with respect to the path of _runner.yaml_ file.
+
+:warning: The commands must be executable. Please make sure that
+the commands have execute permission on Linux platforms.
 
 ## :pen: Create Commands
 
 The runner requires commands / scripts to be run.
 These need to be placed in the `location` specified in
-_runner.yaml_ file. The location must be relative to
-the directory in which the **runner** microservice is being
-run.
+_runner.yaml_ file.
+
+For example, the `location` directory might contain
+the two scripts: _create_ and _execute_. These two become
+valid command names that consumers of REST API can invoke.
+All other command execution requests result in invalid status.
 
 ## :rocket: Use
 
+Display help.
+
 ```bash
-runner # launch the digital twin runner
+$runner -h
+Usage: runner [options]
+
+Remote code execution for humans
+
+Options:
+  -v --version          package version
+  -c --config <string>  runner config file specified in yaml format (default: "runner.yaml")
+  -h --help             display help
 ```
 
+The config option is not mandatory. If it is not used, **runner** looks for
+_runner.yaml_ in the directory from which it is being run.
 Once launched, the utility runs at the port specified in
 _runner.yaml_ file.
 
+```bash
+runner  #use runner.yaml of the present working directory
+runner -c FILE-PATH       #absolute or relative path to config file
+runner --config FILE-PATH #absolute or relative path to config file
+```
+
 If launched on one computer,
 you can access the same at `http://localhost:<port>`.
 
@@ -70,9 +125,9 @@ for these two sources are:
 
 | REST API Route                 | HTTP Method | Return Value | Comment |
 | :----------------------------- |:--------|:----------- | :------ |
-| localhost:port | POST  | Returns the execution status of command | Executes the command provided. Each invocation appends to _array_ of commands executed so far. |
+| localhost:port | POST  | Returns the execution status of command | Executes the command provided. All the commands sent in the right JSON format gets stored in _history_. |
 | localhost:port | GET |  Returns the execution status of the last command sent via POST request. |  |
-| localhost:port/history | GET | Returns the array of POST requests received so far. |  |
+| localhost:port/history | GET | Returns the array of valid POST requests received so far. |  |
 
 #### POST Request to /
 
@@ -85,19 +140,21 @@ in _location_ directory.
 }
 ```
 
-If the command exists, a successful response will be
+If the command is in the permitted list of commands specified
+in _runner.yaml_ and the matching command / script exists in _location_,
+a successful execution takes place. The API response will be
 
 ```http
 {
   "status": "success"
 }
 ```
 
-If the does not exist, the response will be
+If the command is neither permitted nor available, the response will be
 
 ```http
 {
-  "status": "invalid command name"
+  "status": "invalid command"
 }
 ```
 
@@ -119,6 +176,20 @@ was successful, the status will be
 
 If the execution is unsuccessful, the status will be
 
+```http
+{
+  "name": "<command-name>",
+  "status": "invalid",
+  "logs": {
+    "stdout": "",
+    "stderr": ""
+  }
+}
+```
+
+If an incorrectly formatted JSON is sent via POST request,
+a validation error is returned.
+
 ```http
 {
   "message": "Validation Failed",
@@ -138,10 +209,10 @@ Both valid and invalid commands are recorded in the history.
     "name": "valid command"
   },
   {
-    "name": "valid command"
+    "name": "invalid command"
   },
   {
-    "name": "invalid command"
+    "name": "valid command"
   }
 ]
 ```
@@ -150,4 +221,5 @@ Both valid and invalid commands are recorded in the history.
 
 This software is owned by
 [The INTO-CPS Association](https://into-cps.org/)
-and is available under [the INTO-CPS License](./LICENSE.md).
+and is available under
+[the INTO-CPS License](https://odin.cps.digit.au.dk/into-cps/LICENSE.md).
diff --git a/servers/execution/runner/api/dev.api.http b/servers/execution/runner/api/dev.api.http
@@ -0,0 +1,71 @@
+// This file is to be used for testing runner during development
+
+@hostname = localhost
+@port = 5000
+@ContentType = application/json
+
+### 
+GET http://{{hostname}}:{{port}} HTTP/1.1
+content-type: {{ContentType}}
+
+###
+
+GET http://{{hostname}}:{{port}}/history HTTP/1.1
+content-type: {{ContentType}}
+
+###
+
+POST http://{{hostname}}:{{port}} HTTP/1.1
+content-type: {{ContentType}}
+
+{
+    "name": "create"
+}
+
+###
+
+GET http://{{hostname}}:{{port}} HTTP/1.1
+content-type: {{ContentType}}
+
+###
+
+GET http://{{hostname}}:{{port}}/history HTTP/1.1
+content-type: {{ContentType}}
+
+###
+
+POST http://{{hostname}}:{{port}} HTTP/1.1
+content-type: {{ContentType}}
+
+{
+    "name": "configure"
+}
+
+###
+
+GET http://{{hostname}}:{{port}} HTTP/1.1
+content-type: {{ContentType}}
+
+###
+
+GET http://{{hostname}}:{{port}}/history HTTP/1.1
+content-type: {{ContentType}}
+
+###
+
+POST http://{{hostname}}:{{port}} HTTP/1.1
+content-type: {{ContentType}}
+
+{
+    "command": "create"
+}
+
+###
+
+GET http://{{hostname}}:{{port}} HTTP/1.1
+content-type: {{ContentType}}
+
+###
+
+GET http://{{hostname}}:{{port}}/history HTTP/1.1
+content-type: {{ContentType}}
diff --git a/servers/execution/runner/api/workspace.api.http b/servers/execution/runner/api/workspace.api.http
@@ -0,0 +1,113 @@
+// This file is to be used for testing runner deployed
+// in the user workspace of DTaaS.
+// It is assumed that the Traefik gateway to DTaaS application
+// is using HTTP basic authentication for login
+
+@hostname = foo.com
+@port = 5000
+@User = alice
+@URLToken = 4b7d01e14269e12e38f3ed041043e58f877c6b63
+@AuthToken = VGVzdFazZXJZdGFhczpUZXN5VXNlckq0YWFzVGVxdFVzZGJEdGFhcw==
+@WorkspaceCookie = workspace-token-tool-5000="4b7d01e14269e45f56f3ed067129e58f877c6b63"
+@ContentType = application/json
+@_XSRFCookie = _xsrf:2|19tf64b5|2037f58cf647235b3a6c43k4l4v643fk|4349295348
+
+###
+
+GET http://{{hostname}}/{{User}}/shared/tools/{{port}}/?token={{URLToken}} HTTP/1.1
+Authorization: Basic {{AuthToken}}
+content-type: {{ContentType}}
+Cookie:{{WorkspaceCookie}}
+Cookie:{{_XSRFCookie}}
+
+###
+
+GET http://{{hostname}}/{{User}}/shared/tools/{{port}}/history?token={{URLToken}} HTTP/1.1
+Authorization: Basic {{AuthToken}}
+content-type: {{ContentType}}
+Cookie:{{WorkspaceCookie}}
+Cookie:{{_XSRFCookie}}
+
+###
+
+POST http://{{hostname}}/{{User}}/shared/tools/{{port}}/?token={{URLToken}} HTTP/1.1
+Authorization: Basic {{AuthToken}}
+content-type: {{ContentType}}
+Cookie:{{WorkspaceCookie}}
+Cookie:{{_XSRFCookie}}
+
+{
+    "name": "create"
+}
+
+###
+
+GET http://{{hostname}}/{{User}}/shared/tools/{{port}}/?token={{URLToken}} HTTP/1.1
+Authorization: Basic {{AuthToken}}
+content-type: {{ContentType}}
+Cookie:{{WorkspaceCookie}}
+Cookie:{{_XSRFCookie}}
+
+###
+
+GET http://{{hostname}}/{{User}}/shared/tools/{{port}}/history?token={{URLToken}} HTTP/1.1
+Authorization: Basic {{AuthToken}}
+content-type: {{ContentType}}
+Cookie:{{WorkspaceCookie}}
+Cookie:{{_XSRFCookie}}
+
+###
+
+POST http://{{hostname}}/{{User}}/shared/tools/{{port}}/?token={{URLToken}} HTTP/1.1
+Authorization: Basic {{AuthToken}}
+content-type: {{ContentType}}
+Cookie:{{WorkspaceCookie}}
+Cookie:{{_XSRFCookie}}
+
+{
+    "name": "configure"
+}
+
+###
+
+GET http://{{hostname}}/{{User}}/shared/tools/{{port}}/?token={{URLToken}} HTTP/1.1
+Authorization: Basic {{AuthToken}}
+content-type: {{ContentType}}
+Cookie:{{WorkspaceCookie}}
+Cookie:{{_XSRFCookie}}
+
+###
+
+GET http://{{hostname}}/{{User}}/shared/tools/{{port}}/history?token={{URLToken}} HTTP/1.1
+Authorization: Basic {{AuthToken}}
+content-type: {{ContentType}}
+Cookie:{{WorkspaceCookie}}
+Cookie:{{_XSRFCookie}}
+
+###
+
+POST http://{{hostname}}/{{User}}/shared/tools/{{port}}/?token={{URLToken}} HTTP/1.1
+Authorization: Basic {{AuthToken}}
+content-type: {{ContentType}}
+Cookie:{{WorkspaceCookie}}
+Cookie:{{_XSRFCookie}}
+
+{
+    "command": "create"
+}
+
+###
+
+GET http://{{hostname}}/{{User}}/shared/tools/{{port}}/?token={{URLToken}} HTTP/1.1
+Authorization: Basic {{AuthToken}}
+content-type: {{ContentType}}
+Cookie:{{WorkspaceCookie}}
+Cookie:{{_XSRFCookie}}
+
+###
+
+GET http://{{hostname}}/{{User}}/shared/tools/{{port}}/history?token={{URLToken}} HTTP/1.1
+Authorization: Basic {{AuthToken}}
+content-type: {{ContentType}}
+Cookie:{{WorkspaceCookie}}
+Cookie:{{_XSRFCookie}}