Skip to content

Commit

Permalink
Update ARCHITECTURE.md
Browse files Browse the repository at this point in the history
  • Loading branch information
JordanLaserGit authored Nov 27, 2024
1 parent 8d8ba5a commit 5d585e6
Showing 1 changed file with 1 addition and 1 deletion.
2 changes: 1 addition & 1 deletion research_datastream/terraform/ARCHITECTURE.md
Original file line number Diff line number Diff line change
Expand Up @@ -10,7 +10,7 @@ Each green box represents a distinct AWS Lambda Function that is responsible for
Here's a rough idea of the workflow once a user submits an execution to the AWS statemachine.
1) User submits execution request at the cli via `aws stepfunctions start-execution`
2) AWS Lambda Function `EC2StarterFromAMI` spawns an instance from an AMI. This AMI must exist and be available within the region specified in the execution json. This function blocks until the instance is running and ready to accept commands.
* The instance spawned by `EC2StarterFromAMI` must do so starting from a template AMI. This is a user supplied field in the execution json, which allows the user to build from any AMI. This feature makes this state machine general purpose. The intended implementation is for NextGen executions (an example of that [here](https://github.com/CIROH-UA/ngen-datastream/edit/main/research_datastream/terraform/AWS_BASICS.md#1-aws-step-functions-state-machines)), however it is possible a user creates their own custom AMI for any sort of job.
* The instance spawned by `EC2StarterFromAMI` must do so starting from a template AMI. This is a user supplied field in the execution json, which allows the user to build from any AMI. This feature makes this state machine general purpose. The intended implementation is for NextGen executions (an example of that [here](https://github.com/CIROH-UA/ngen-datastream/blob/main/research_datastream/terraform/AWS_BASICS.md#1-aws-step-functions-state-machines)), however it is possible a user creates their own custom AMI for any sort of job.
3) AWS Lambda Function `Commander` issues the commands the user specifies in the execution json. If a command failure is encountered, the instance will be stopped and state machine execution will complete.
4) AWS Lambda Function `EC2Poller` will poll the instance for the status of the commands issued. If a command failure is encountered, the instance will be stopped and state machine execution will complete. This function blocks until the commands either fail or complete without error.
5) AWS Lambda Function `RunChecker` will check the existence of an s3 object. This ensures that the data written out by the instance actually exists.
Expand Down

0 comments on commit 5d585e6

Please sign in to comment.