The current version just support inputs from AWS SQS, in next versions will support also HTTP/S and Redis QUEUES.
In the next example, we will run a command after receiving a message from SQS, the message received will look like the JSON below
{
"Progress": 50,
"AccountId": "9999999999",
"Description": "Launching EC2 instance: i-00000009999",
"RequestId": "5f88cdf3-311c-4d61-bdec-0000000000",
"EndTime": "2018-04-18T14:17:22.779Z",
"AutoScalingGroupARN": "arn:aws:autoscaling:eu-west-1:9999999999:autoScalingGroup:0b35d38a-8270-45d0-a0d8-0000000000:autoScalingGroupName/SOMEGROUPNAME",
"ActivityId": "5f88cdf3-311c-4d61-bdec-0000000000",
"StartTime": "2018-04-18T14:16:26.804Z",
"Service": "AWS Auto Scaling",
"Time": "2018-04-18T14:17:22.779Z",
"EC2InstanceId": "i-00000009999", // <--- We read this
"StatusCode": "InProgress",
"StatusMessage": "",
"Details": {
"Subnet ID": "subnet-XXXXXXXX",
"Availability Zone": "eu-west-1a"
},
"AutoScalingGroupName": "SOMEGROUPNAME", // <--- We read this
"Cause": "At 2018-04-18T14:16:21Z a user request update of AutoScalingGroup constraints to min: 1, max: 5, desired: 3 changing the desired capacity from 2 to 3....",
"Event": "autoscaling:EC2_INSTANCE_LAUNCH" // <--- We read this, as condition
}
In the goreactor, we are ready to listen to messages and execute the command.
Please put attention in the "cond" parameter (is a condition, like an "if"). This means that the command will be executed JUST if we can find "Event = autoscaling:EC2_INSTANCE_LAUNCH" in the JSON. If this key is not there or the value is not what we defined, then goreactor will ignore the message
[logstream]
logstream = "stdout"
[[reactor]]
concurrent = 10
delay = "5s"
keepAliveInterval = "30s"
input = "sqs"
url = "https://sqs.eu-west-1.amazonaws.com/9999999999/testing"
profile = "default"
region = "eu-west-1"
maxNumberOfMessages = 10
output = "cmd"
cond = [
{ "$.Event" = "autoscaling:EC2_INSTANCE_LAUNCH" }
]
cmd = "/usr/local/bin/do-something-with-the-instance"
args = ["asg=$.AutoScalingGroupName", "instance_id=$.EC2InstanceId"]
workingDirectory = "/path/to/process/wd/"
/usr/local/bin/do-something-with-the instance asg=SOMEGROUPNAME instance_id=i-00000009999
- concurrent - Maximum of commands (cmd) that can run concurrently
- delay - Period of time that a new command runs without concurrency, after that period the next message will start. Format https://pkg.go.dev/time#ParseDuration
- keepAliveInterval - A signal will be sent to the underline input source to inform that the process is still running. Format https://pkg.go.dev/time#ParseDuration. In SQS input will trigger a request ChangeMessageVisibility (https://docs.aws.amazon.com/AWSSimpleQueueService/latest/APIReference/API_ChangeMessageVisibility.html)
- url - AWS SQS URL
- region - AWS Region
- profile - AWS Profile
- maxnumberofmessages - AWS SQS setting: The maximum number of messages to return. Amazon SQS never returns more messages than this value (however, fewer messages might be returned). Valid values: 1 to 10. Default: 1.
- noblocking is a boolean value that defines the parallelims of the reactors
- false (default): send the message to the reactors in sequence
- true: send the message in parallel to avoid blocking
The working directory of the process can be set with workingDirectory
.
It may be desired to execute some commands as a specific user.
NOTE: that this only works in unix or unix like systems and have only been tested in linux. In other OS, this will be ignored.
In order to do so, each [[reactor]] entry may have a user
value defined with the desired user.
The env will be empty (only containing the HOME - which can be overridden) unless you specify it in a env
section
that will only be used if user
is defined.
[[reactor]]
# (...) All the desired values
user = "john"
env = [
"PATH=/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin",
"AWS_PROFILE=stage"
]
This would run the reactor using john user and 2 environ: HOME set to the home defined for the john user, PATH defined to "PATH=/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin" and AWS_PROFILE set to "stage"
DO NOT USE SET THE SETUID BIT FOR GOREACTOR!!!!
-
$.path.with[0].jq.format
jq like format after
$.
See Usage case, message from SQS for an example -
$.path.to.array...
If the jq like expression (Starting with
$.
) is the only text in the argument and ends with...
See examples/ARRAY.md for an examplewill expand the array in the specified path in different arguments.
The use of this will replace the argument with N arguments where N is the number of elements in the array.
NOTE: Should be the only text in the given argument element on the array.
-
${variableName}
Variables will be substituted in the arguments if the variable is in the following list. (This can also come from an expanded array)
NOTE: Variables are case sensitive and curly brackets are mandatory
Currently, we have the following variables:
CreationTimestampMilliseconds
is the message creation time with in millisecondsCreationTimestampSeconds
is the message creation time in seconds See examples/ARRAY.md for an example
-
$..
If you want to select the full message you should use
$..
instead of$.
. -
$.Message
If the message has a field
"Message"
, the content of the original message will be replaced with the content of the field$.Message
. -
Indented json
The library used to parse jq like expressions sometimes has problems with new lines. It is better to use json messages without new lines or other indentations.
You can set logging to stdout with
[logstream]
logstream = "stdout"
You can set logging to firehose with
[logstream]
logstream = "firehose"
streamname = "example-goreactor-logs"
region = "eu-west-1"
You can disable logging if you leave logstream empty or set to none. Or not defining logstream at all
[logstream]
logstream = ""
[logstream]
logstream = "none"
The output will be a json per output line with the following format:
{"Host":"RUNNER_HOSTNAME","Pid":44274,"RID":1,"TID":1,"Line":0,"Output":"./print_some_lines_and_exit ","Status":"CMD","Timestamp":1635149955}
{"Host":"RUNNER_HOSTNAME","Pid":44274,"RID":1,"TID":1,"Line":1,"Output":"first line","Status":"RUN","Timestamp":1635149955}
{"Host":"RUNNER_HOSTNAME","Pid":44274,"RID":1,"TID":1,"Line":2,"Output":"second line","Status":"RUN","Timestamp":1635149955}
{"Host":"RUNNER_HOSTNAME","Pid":44274,"RID":1,"TID":1,"Line":3,"Output":"last line","Status":"RUN","Timestamp":1635149955}
{"Host":"RUNNER_HOSTNAME","Pid":44274,"RID":1,"TID":1,"Line":4,"Status":"END","Elapse":3.067383247,"Timestamp":1635149955}