Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Set AWS steps to point to AWS services by default (Set AWSENV to LOCALSTACK to test on localstack) #58

Merged
merged 3 commits into from
Oct 18, 2023
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 1 addition & 0 deletions .github/workflows/node.js.yml
Original file line number Diff line number Diff line change
Expand Up @@ -46,6 +46,7 @@ jobs:
FAKE_SQL_USERNAME: MAF
FAKE_SQL_PASSWORD: MAF
LOCALSTACK_HOSTNAME: localhost
AWSENV: LOCALSTACK

- name: Status Create Action
if: ${{ always() }}
Expand Down
1,835 changes: 945 additions & 890 deletions package-lock.json

Large diffs are not rendered by default.

6 changes: 3 additions & 3 deletions packages/aws/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -29,16 +29,16 @@ require('@ln-maf/aws')

- Run `docker run --rm -it -p 4566:4566 -p 4510-4559:4510-4559 localstack/localstack:0.14.4` to spin up a localstack environment.
- Run `terraform apply -auto-approve` to prepare the localstack environment.
- Be sure environment variable `AWSENV` is set to false or does not exist
- Be sure environment variable `AWSENV` is set to `LOCALSTACK`
- Run individual tests, or run all tests using `npm t`

# Configurations

The AWS SDK V3 is used for communication to AWS / Localstack.
You need to provide credentials to AWS so that only your account and its resources are accessed by the SDK. For more information about obtaining your account credentials, see [Loading credentials in Node.js from the shared credentials file from AWS Documentation.](https://docs.aws.amazon.com/sdk-for-javascript/v3/developer-guide/loading-node-credentials-shared.html)
This can be point to a remote aws location, or point to a localstack configuration depending on the environment variable `AWSENV`. If `AWSENV` is set to `TRUE`, then the default configuration / credentials provided will be used. If `AWSENV` is set to false or does not exist, then the AWS MAF framework will use the default localstack configuration endpoint `http://localhost:4566`, or `http://${LOCALSTACK_HOSTNAME}:4566` if env variable `LOCALSTACK_HOSTNAME` is defined
This can be point to a remote aws location, or point to a localstack configuration depending on the environment variable `AWSENV`. If `AWSENV` is set to `LOCALSTACK`, then the AWS MAF framework will use the default localstack configuration endpoint `http://localhost:4566`, or `http://${LOCALSTACK_HOSTNAME}:4566` if env variable `LOCALSTACK_HOSTNAME` is defined

To run the example feature files, start the localstack and run the terraform script by running `terraform apply -auto-approve` to run initLocalstack.tf, then run `npm t`. `initLocalstack.tf` is needed for running the feature files in the features directory as this will set up the localstack environment for dev testing.
To run the example feature files, start the localstack and run the terraform script by running `terraform apply -auto-approve` to run initLocalstack.tf, then run `npm t` to start testing. `initLocalstack.tf` is needed for running the feature files in the features directory as this will set up the localstack environment for dev testing.

# Step Definitions

Expand Down
25 changes: 13 additions & 12 deletions packages/aws/package.json
Original file line number Diff line number Diff line change
Expand Up @@ -3,7 +3,7 @@
"publishConfig": {
"access": "public"
},
"version": "2.0.12",
"version": "2.2.0",
"description": "AWS steps for MAF. This contains S3, DynamoDB, SQS, ECS, Cloudwatch, and Lambda stepDefinitions",
"main": "index.js",
"scripts": {
Expand All @@ -12,14 +12,15 @@
"author": "Dominick Martelly",
"license": "Apache-2.0",
"dependencies": {
"@aws-sdk/client-cloudwatch-logs": "^3.121.0",
"@aws-sdk/client-dynamodb": "^3.121.0",
"@aws-sdk/client-ecs": "^3.121.0",
"@aws-sdk/client-lambda": "^3.121.0",
"@aws-sdk/client-s3": "^3.121.0",
"@aws-sdk/client-sqs": "^3.121.0",
"@ln-maf/core": "^2.0.1",
"chai": "^4.3.6",
"@aws-sdk/client-cloudwatch-logs": "^3.430.0",
"@aws-sdk/client-dynamodb": "^3.430.0",
"@aws-sdk/client-ecs": "^3.430.0",
"@aws-sdk/client-lambda": "^3.430.0",
"@aws-sdk/client-s3": "^3.430.0",
"@aws-sdk/client-sqs": "^3.430.0",
"@aws-sdk/client-ssm": "^3.430.0",
"@ln-maf/core": "^2.0.0",
"chai": "^4.0.0",
"luxon": "^2.4.0"
},
"peerDependencies": {
Expand All @@ -30,12 +31,12 @@
"@cucumber/cucumber": "^9.0.0",
"@ln-maf/core": "*",
"@ln-maf/validations": "^2.0.3",
"eslint": "^8.18.0",
"eslint": "^8.51.0",
"eslint-config-standard": "^17.0.0",
"eslint-plugin-import": "^2.26.0",
"eslint-plugin-import": "^2.28.0",
"eslint-plugin-node": "^11.1.0",
"eslint-plugin-promise": "^6.0.0",
"multiple-cucumber-html-reporter": "^3.4.0",
"multiple-cucumber-html-reporter": "^3.5.0",
"nyc": "^15.1.0"
},
"gitHead": "9bf55a2c86fa17707fabfb4320c3d722e270f9e6"
Expand Down
2 changes: 1 addition & 1 deletion packages/aws/runFeature.sh
Original file line number Diff line number Diff line change
@@ -1,4 +1,4 @@
AWSENV='FALSE'
AWSENV='LOCALSTACK'
mkdir -p test/report
if [[ "$ENVIRONMENT" == "COVERAGE" ]]; then
npx nyc --reporter=lcov --reporter=text cucumber-js $EXTRAS -f json:test/report/aws.json --require "stepDefinitions/*.js" features/$*
Expand Down
13 changes: 12 additions & 1 deletion packages/aws/stepDefinitions/cloudwatch-logs.js
Original file line number Diff line number Diff line change
@@ -1,16 +1,27 @@
const { setDefaultTimeout } = require('@cucumber/cucumber')
const { MAFWhen, filltemplate } = require('@ln-maf/core')
const { CloudWatchLogsClient, FilterLogEventsCommand } = require('@aws-sdk/client-cloudwatch-logs')
const { SSMClient, GetParameterCommand } = require('@aws-sdk/client-ssm')
const { DateTime } = require('luxon')

setDefaultTimeout(15 * 60 * 1000)

const cloudwatchLogsClientConfig = { maxAttempts: 3 }
if (process.env.AWSENV === undefined || process.env.AWSENV === '' || process.env.AWSENV.toUpperCase() === 'FALSE') {
if (process.env.AWSENV && process.env.AWSENV.toUpperCase() === 'LOCALSTACK') {
cloudwatchLogsClientConfig.endpoint = process.env.LOCALSTACK_HOSTNAME ? `http://${process.env.LOCALSTACK_HOSTNAME}:4566` : 'http://localhost:4566'
}
const cloudwatchLogsClient = new CloudWatchLogsClient(cloudwatchLogsClientConfig)

/**
* Returns the value of the parameter from the parameter store
*/
MAFWhen('parameter {string} value is retrieved from the parameter store', async function (parameterName) {
parameterName = filltemplate(parameterName, this.results)
const ssmClient = new SSMClient()
const res = await ssmClient.send(new GetParameterCommand({ Name: parameterName }))
return res.Parameter.Value
})

/**
* Returns at most 10 query calls of 1 MB / 10,000 logs from cloudwatch
*/
Expand Down
2 changes: 1 addition & 1 deletion packages/aws/stepDefinitions/dynamodb.js
Original file line number Diff line number Diff line change
Expand Up @@ -7,7 +7,7 @@ setDefaultTimeout(15 * 60 * 1000)
const DynamoDBClientConfig = {
maxAttempts: 3
}
if (process.env.AWSENV === undefined || process.env.AWSENV === '' || process.env.AWSENV.toUpperCase() === 'FALSE') {
if (process.env.AWSENV && process.env.AWSENV.toUpperCase() === 'LOCALSTACK') {
DynamoDBClientConfig.endpoint = process.env.LOCALSTACK_HOSTNAME ? `http://${process.env.LOCALSTACK_HOSTNAME}:4566` : 'http://localhost:4566'
}
const dbClient = new DynamoDBClient(DynamoDBClientConfig)
Expand Down
2 changes: 1 addition & 1 deletion packages/aws/stepDefinitions/ecs.js
Original file line number Diff line number Diff line change
Expand Up @@ -5,7 +5,7 @@ const { ECSClient, ListTaskDefinitionsCommand, ListClustersCommand, RunTaskComma
setDefaultTimeout(15 * 60 * 1000)

const ecsClientConfig = { maxAttempts: 3 }
if (process.env.AWSENV === undefined || process.env.AWSENV === '' || process.env.AWSENV.toUpperCase() === 'FALSE') {
if (process.env.AWSENV && process.env.AWSENV.toUpperCase() === 'LOCALSTACK') {
ecsClientConfig.endpoint = process.env.LOCALSTACK_HOSTNAME ? `http://${process.env.LOCALSTACK_HOSTNAME}:4566` : 'http://localhost:4566'
}
const ecsClient = new ECSClient(ecsClientConfig)
Expand Down
2 changes: 1 addition & 1 deletion packages/aws/stepDefinitions/lambda.js
Original file line number Diff line number Diff line change
Expand Up @@ -5,7 +5,7 @@ const { LambdaClient, InvokeCommand } = require('@aws-sdk/client-lambda')
setDefaultTimeout(15 * 60 * 1000)

const lambdaClientConfig = { maxAttempts: 3 }
if (process.env.AWSENV === undefined || process.env.AWSENV === '' || process.env.AWSENV.toUpperCase() === 'FALSE') {
if (process.env.AWSENV && process.env.AWSENV.toUpperCase() === 'LOCALSTACK') {
lambdaClientConfig.endpoint = process.env.LOCALSTACK_HOSTNAME ? `http://${process.env.LOCALSTACK_HOSTNAME}:4566` : 'http://localhost:4566'
}
const lambdaClient = new LambdaClient(lambdaClientConfig)
Expand Down
2 changes: 1 addition & 1 deletion packages/aws/stepDefinitions/s3.js
Original file line number Diff line number Diff line change
Expand Up @@ -6,7 +6,7 @@ const { getFilePath, MAFWhen, filltemplate, performJSONObjectTransform } = requi
setDefaultTimeout(15 * 60 * 1000)

const S3ClientConfig = { maxAttempts: 3, forcePathStyle: true }
if (process.env.AWSENV === undefined || process.env.AWSENV === '' || process.env.AWSENV.toUpperCase() === 'FALSE') {
if (process.env.AWSENV && process.env.AWSENV.toUpperCase() === 'LOCALSTACK') {
S3ClientConfig.endpoint = process.env.LOCALSTACK_HOSTNAME ? `http://${process.env.LOCALSTACK_HOSTNAME}:4566` : 'http://localhost:4566'
}
const s3Client = new S3Client(S3ClientConfig)
Expand Down
2 changes: 1 addition & 1 deletion packages/aws/stepDefinitions/sqs.js
Original file line number Diff line number Diff line change
Expand Up @@ -5,7 +5,7 @@ const { SQSClient, ListQueuesCommand, GetQueueAttributesCommand, SendMessageComm
setDefaultTimeout(15 * 60 * 1000)

const sqsClientConfig = { maxAttempts: 3 }
if (process.env.AWSENV === undefined || process.env.AWSENV === '' || process.env.AWSENV.toUpperCase() === 'FALSE') {
if (process.env.AWSENV && process.env.AWSENV.toUpperCase() === 'LOCALSTACK') {
sqsClientConfig.endpoint = process.env.LOCALSTACK_HOSTNAME ? `http://${process.env.LOCALSTACK_HOSTNAME}:4566` : 'http://localhost:4566'
}
const sqsClient = new SQSClient(sqsClientConfig)
Expand Down
Loading