Skip to content

Commit

Permalink
Updated README.md - added framework configuration
Browse files Browse the repository at this point in the history
  • Loading branch information
codemental committed Jan 16, 2024
1 parent 00e9830 commit ee8fed4
Showing 1 changed file with 49 additions and 43 deletions.
92 changes: 49 additions & 43 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -23,6 +23,13 @@ The framework does not know which action runs in which test and also when an int
These features are required for the initial production version and they most probably will change how the test actions
will be used & configured.

## Configuration

| Key | Default | Description |
|------------------------|---------|------------------------------------------|
| Actions.TimeoutSeconds | 10 | Time after which an action will timeout. |
| Logging.Level | "info" | Logging level for the whole framework. |

## Roadmap

### v1.0
Expand All @@ -31,12 +38,12 @@ will be used & configured.

- standard go test runtime
- orchestration:
- run custom commands (to start apps)
- beforeSuite
- afterSuite
- run custom commands (to start apps)
- beforeSuite
- afterSuite
- execution context
- log what tests have been executed
- which were successful or failed
- log what tests have been executed
- which were successful or failed

#### Packages

Expand All @@ -47,27 +54,27 @@ will be used & configured.
#### Protocols

- HTTP1.1
- actions
- requests / responses
- endpoints:
- client
- server
- validation
- methods
- paths
- headers
- query params
- full body validation (JSON, plain, form)
- `@ignore@` in payload
- on values
- actions
- requests / responses
- endpoints:
- client
- server
- validation
- methods
- paths
- headers
- query params
- full body validation (JSON, plain, form)
- `@ignore@` in payload
- on values

#### Configuration

- read CLI flags (may not be possible with current setup):
- setup custom config location
- control active profile
- setup custom config location
- control active profile
- configuration file types:
- yaml
- yaml
- allow configuration based on profiles

#### Logging
Expand All @@ -81,7 +88,6 @@ will be used & configured.
- execution time per test
- per test suite


### v1.x

#### Runtime
Expand All @@ -91,44 +97,44 @@ will be used & configured.
#### HTTP

- actions:
- expect errors:
- timeout
- connection exception
- expect errors:
- timeout
- connection exception
- use variables in payloads
- validation:
- XML payload validation (with @ignore@ for values)
- specific path validation/ignore for XPath & JsonPath on receive actions
- schema validation (XML & JSON)
- OpenApi validation
- XML payload validation (with @ignore@ for values)
- specific path validation/ignore for XPath & JsonPath on receive actions
- schema validation (XML & JSON)
- OpenApi validation
- actions:
- retry until success
- retry until failure (simulate DDOS)
- send multiple requests
- n times configurable - w/o delay
- time range - w/o delay
- retry until success
- retry until failure (simulate DDOS)
- send multiple requests
- n times configurable - w/o delay
- time range - w/o delay

#### Metrics

- how fast the system under test sends a request/response
- test actions:
- fail test/test suite if metrics were/were not reached
- fail test/test suite if metrics were/were not reached
- test suite metrics report:
- with history of previous runs
- with delta compared to previous runs
- with history of previous runs
- with delta compared to previous runs

#### Features

- test reports:
- configurable
- configurable
- static response server:
- for different protocols
- fixed / random / round robin responses from configured data
- for different protocols
- fixed / random / round robin responses from configured data
- remote server & client endpoints:
- clarum http-endpoint instance deployed in k8s and controlled from localhost/pipeline
- clarum http-endpoint instance deployed in k8s and controlled from localhost/pipeline
- docker:
- API support and actions
- API support and actions
- k8s:
- API support and actions
- API support and actions

#### Communication

Expand Down

0 comments on commit ee8fed4

Please sign in to comment.