Skip to content

Latest commit

 

History

History
189 lines (139 loc) · 6.82 KB

README.md

File metadata and controls

189 lines (139 loc) · 6.82 KB
Raw

Build Status Go Report Card Go

Goreactor, trigger a message and execute commands

The current version just support inputs from AWS SQS, in next versions will support also HTTP/S and Redis QUEUES.

Features

Substitutions in args entry in the config file

  • $.path.with[0].jq.format

    jq like format after $. See Usage case, message from SQS for an example

  • $.path.to.array...

    If the jq like expression (Starting with $.) is the only text in the argument and ends with ... See examples/ARRAY.md for an example

    will expand the array in the specified path in different arguments.

    The use of this will replace the argument with N arguments where N is the number of elements in the array.

    NOTE: Should be the only text in the given argument element on the array.

  • ${variableName}

    Variables will be substituted in the arguments if the variable is in the following list. (This can also come from an expanded array)

    NOTE: Variables are case sensitive and curly brackets are mandatory

    Currently, we have the following variables:

    • CreationTimestampMilliseconds is the message creation time with in milliseconds
    • CreationTimestampSeconds is the message creation time in seconds See examples/ARRAY.md for an example

  • $..

    If you want to select the full message you should use $.. instead of $..

  • $.Message

    If the message has a field "Message", the content of the original message will be replaced with the content of the field $.Message.

  • Indented json

    The library used to parse jq like expressions sometimes has problems with new lines. It is better to use json messages without new lines or other indentations.

Log outputs to stdout or firehose

You can set logging to stdout with

[logstream]
logstream = "stdout"

You can set logging to firehose with

[logstream]
logstream = "firehose"
streamname = "example-goreactor-logs"
region = "eu-west-1"

You can disable logging if you leave logstream empty or set to none. Or not defining logstream at all

[logstream]
logstream = ""
[logstream]
logstream = "none"

The output will be a json per output line with the following format:

{"Host":"RUNNER_HOSTNAME","Pid":44274,"RID":1,"TID":1,"Line":0,"Output":"./print_some_lines_and_exit ","Status":"CMD","Timestamp":1635149955}
{"Host":"RUNNER_HOSTNAME","Pid":44274,"RID":1,"TID":1,"Line":1,"Output":"first line","Status":"RUN","Timestamp":1635149955}
{"Host":"RUNNER_HOSTNAME","Pid":44274,"RID":1,"TID":1,"Line":2,"Output":"second line","Status":"RUN","Timestamp":1635149955}
{"Host":"RUNNER_HOSTNAME","Pid":44274,"RID":1,"TID":1,"Line":3,"Output":"last line","Status":"RUN","Timestamp":1635149955}
{"Host":"RUNNER_HOSTNAME","Pid":44274,"RID":1,"TID":1,"Line":4,"Status":"END","Elapse":3.067383247,"Timestamp":1635149955}

Set working directory

The working directory of the process can be set with workingDirectory.

Execute as a specific user

It may be desired to execute some commands as a specific user.

NOTE: that this only works in unix or unix like systems and have only been tested in linux. In other OS, this will be ignored.

In order to do so, each [[reactor]] entry may have a user value defined with the desired user. The env will be empty (only containing the HOME - which can be overridden) unless you specify it in a env section that will only be used if user is defined.

[[reactor]]
# (...) All the desired values
user = "john"
env = [
"PATH=/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin",
"AWS_PROFILE=stage"
]

This would run the reactor using john user and 2 environ: HOME set to the home defined for the john user, PATH defined to "PATH=/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin" and AWS_PROFILE set to "stage"

DO NOT USE SET THE SETUID BIT FOR GOREACTOR!!!!

Usage case, message from SQS

In the next example, we will run a command after receiving a message from SQS, the message received will look like the JSON below

Message received from SQS

{
    "Progress": 50,
    "AccountId": "9999999999",
    "Description": "Launching EC2 instance: i-00000009999",
    "RequestId": "5f88cdf3-311c-4d61-bdec-0000000000",
    "EndTime": "2018-04-18T14:17:22.779Z",
    "AutoScalingGroupARN": "arn:aws:autoscaling:eu-west-1:9999999999:autoScalingGroup:0b35d38a-8270-45d0-a0d8-0000000000:autoScalingGroupName/SOMEGROUPNAME",
    "ActivityId": "5f88cdf3-311c-4d61-bdec-0000000000",
    "StartTime": "2018-04-18T14:16:26.804Z",
    "Service": "AWS Auto Scaling",
    "Time": "2018-04-18T14:17:22.779Z",
    "EC2InstanceId": "i-00000009999", // <--- We read this
    "StatusCode": "InProgress",
    "StatusMessage": "",
    "Details": {
        "Subnet ID": "subnet-XXXXXXXX",
        "Availability Zone": "eu-west-1a"
    },
    "AutoScalingGroupName": "SOMEGROUPNAME", // <--- We read this
    "Cause": "At 2018-04-18T14:16:21Z a user request update of AutoScalingGroup constraints to min: 1, max: 5, desired: 3 changing the desired capacity from 2 to 3....",
    "Event": "autoscaling:EC2_INSTANCE_LAUNCH" // <--- We read this, as condition
}

Configuration in go reactor

In the goreactor, we are ready to listen to messages and execute the command.

Please put attention in the "cond" parameter (is a condition, like an "if"). This means that the command will be executed JUST if we can find "Event = autoscaling:EC2_INSTANCE_LAUNCH" in the JSON. If this key is not there or the value is not what we defined, then goreactor will ignore the message

[logstream]
logstream = "stdout"

[[reactor]]
concurrent = 10
delay = "5s"
input = "sqs"
url = "https://sqs.eu-west-1.amazonaws.com/9999999999/testing"
profile = "default"
region = "eu-west-1"
maxNumberOfMessages = 10
output = "cmd"
cond = [
    { "$.Event" = "autoscaling:EC2_INSTANCE_LAUNCH" }
]
cmd = "/usr/local/bin/do-something-with-the-instance"
args = ["asg=$.AutoScalingGroupName", "instance_id=$.EC2InstanceId"]
workingDirectory = "/path/to/process/wd/"

The daemon will execute a command like

/usr/local/bin/do-something-with-the instance asg=SOMEGROUPNAME instance_id=i-00000009999