Fog is a tool to manage your CloudFormation deployments and ensure you have all your config as code. Please note, fog is not a DSL. It only works with standard CloudFormation files and doesn't do anything you can't do using the AWS CLI (and some hacking around with jq). It just makes it easier by combining functionality and preventing you from needing to deal with unnecesary overhead like different commands for creating a new stack or updating an existing one. In addition, it has little helper functionalities like offering to remove an empty stack for you.
The main functionality for fog is to carry out deployments of CloudFormation templates. The logic is based on bash scripts I've written and used with multiple clients over a number of years. You can do a deployment using the fog deploy command. You can see the help functionality for this and other commands by adding the --help flag.
$ fog deploy --help
deploy allows you to deploy a CloudFormation stack
It does so by creating a ChangeSet and then asking you for approval before continuing. You can automatically approve or only create or deploy a changeset by using flags.
A name for the changeset will automatically be generated based on your preferred name, but can be overwritten as well.
When providing tag and/or parameter files, you can add multiple files for each. These are parsed in the order provided and later values will override earlier ones.
Examples:
fog deploy --stackname testvpc --template basicvpc --parameters vpc-private-only --tags "../globaltags/project,dev"
fog deploy --stackname fails3 --template fails3 --non-interactive
fog deploy --stackname myvpc --template basicvpc --parameters vpc-public --tags "../globaltags/project,dev" --config testconf/fog.yaml
Usage:
fog deploy [flags]
Flags:
-b, --bucket string The S3 bucket where the template should be uploaded to (optional)
-c, --changeset string The name of the changeset, when not provided it will be autogenerated
--create-changeset Only create a change set
--default-tags Add any default tags that are specified in your config file (default true)
--deploy-changeset Deploy a specific change set
-d, --deployment-file string The file to use for the deployment
--dry-run Do a dry run: create the changeset and immediately delete
-h, --help help for deploy
--non-interactive Run in non-interactive mode: automatically approve the changeset and deploy
-p, --parameters string The file(s) containing the parameter values, comma-separated for multiple
-n, --stackname string The name for the stack
-t, --tags string The file(s) containing the tags, comma-separated for multiple
-f, --template string The filename for the template
Global Flags:
--config string config file (default is fog.yaml in current directory, or $HOME/fog.yaml)
--debug Enable debug mode, mainly for development purposes
--file string Optional file to save the output to, in addition to stdout
--file-format string Optional format for the file, defaults to the same as output
--output string Format for the output, currently supported are table, csv, json, and dot (for certain functions) (default "table")
--profile string Use a specific AWS profile
--region string Use a specific AWS region
--timezone string Specify a timezone you want to use for any times shown in output. By default it uses your system's timezone
-v, --verbose Give verbose outputFog assumes that all values passed to it are stored in files. You can't pass parameters or tags as arguments, in an attempt to ensure that everything you do is stored in version control.
An example for running a deployment would be
$ fog deploy --stackname myvpc --template myvpc --parameters myvpc-dev --tags globaltags/dev,myvpcBy default this will look for the following files:
templates/myvpc.(yaml|yml|json|template|templ)
parameters/myvpc-dev.json
tags/globaltags/dev.json
tags/myvpc.json
All of these paths and extensions can be overwritten in the config file, as explained further on. But once these files are found, fog will attempt to create a change set for them. It will then show an overview of the change set and ask whether you wish to deploy it.
If you want to proceed, it will then show you real-time progress of the deployment, similar to how the Console does this and if successfull it will show you a table of the outputs.
In the case of a failure, it will instead show you an overview of all the steps that failed and the details for why that happened.
If it's a new stack, it will even offer to delete the stack for you as you can't retry the deployment until that is done.
At re:Invent 2023, AWS introduced the ability to automatically deploy CloudFormation stacks from your git repo, based on a stack deployment file. Fog supports using these same deployment-files as an alternative to the above configuration for parameter and tag files.
Example:
$ fog deploy --stackname myvpc --deployment-file vpc-private-onlyThis will then use the deployment file vpc-private.(yaml|yml|json) to get the template file (relative to the deployment file), defined parameters, and tags. You can't use parameter or tag files with a deployment file, but any tags defined in your config file will still be used.
As per the AWS documentation, a stack deployment file supports the following fields:
- template-file-path: this is relative to the deployment file
- parameters: key-value pairs of parameters
- tags: key-value pairs of tags
As you can see higher up, you can influence what is deployed using CLI arguments. For example, the --non-interactive flag will assume that you always say "yes" to questions like doing a deployment or deleting an empty stack on failure while --create-changeset will only create the change set so you can show it for review in your CI/CD tool before it is deployed after a manual approval.
Another way to influence what's run is to use a config file. This config file can be either a yaml, json, or toml file. Fog will by default check the current directory and your home directory (in that order), for a file named fog.yaml|json|toml but you can also provide a specific file using the --config file.
You can find an annotated example config in example-fog.yaml, or see its output by running fog demo settings but some highlights of the config file are:
- The ability to set the directories for templates/tags/parameters.
- Change the look of the table outputs (yes, that means you don't need to use the colours used in the screenshots)
- Set a standard name format for the change sets
- Set standard tags that need to be applied to every template you wish to deploy
- Set the root directory from which the
$TEMPLATEPATHplaceholder should be calculated
It is possible to set up optional prechecks in your configuration file. These are commands that will be run before your deployment and you can ensure that a negative result from these checks will prevent deployment. For example, you can use this to ensure your templates succeed on lint checks or follow the rules defined in your CloudFormation Guard setup.
You set this up in your config file in the templates section, and you can define as many as you like. As with other settings, it allows the use of the $TEMPLATEPATH placeholder to define the template.
templates:
prechecks:
- cfn-lint -t $TEMPLATEPATH #Use https://github.com/aws-cloudformation/cfn-lint
- cfn-guard validate -d $TEMPLATEPATH --rules myrules #Use https://github.com/aws-cloudformation/cloudformation-guard
stop-on-failed-prechecks: trueIf you don't define stop-on-failed-prechecks, or set it to false, fog will continue with the deployment even if issues are found.
For deployments you can only get the output in table format, but as said you have control over what they look like. If you wish to see what all the different options look like you can do so by running fog demo tables.
Other commands add various output formats like json, csv, or sometimes dotfiles for graphs. This works similar to the AWS CLI as in that you can set a default in your config file and override it with the --output flag.
The below diagram shows the flow that fog goes through when doing a deployment.
(If it's hard to read, open the linked SVG file in your graphics app of choice)
Another major feature added is the ability to run a report on your CloudFormation stacks. This report will show the create/update/delete events of your stack, grouped by event and resource.
Running this with either the markdown or html output formats set will also add a timeline of the events in the form of a mermaid graph and if multiple stacks are requested it will add a table of contents to the top of the output.
You can write the output to a file using the --file flag.
Example of full output written to file:
$ fog report --stackname demovpc43 --output markdown --file docs/fog-report-demo.mdSee docs/fog-report-demo.md for the output.
Example of inline output:
$ fog report --stackname demovpc43 --output markdown --latest| Stack | Account | Region | Type | Start time | Duration | Success |
|---|---|---|---|---|---|---|
| demovpc43 | ignoreme-demo (1234567890) | ap-southeast-2 | Update | 2022-05-26T22:34:13+10:00 | 1m1s | ✅ |
| Action | CfnName | Type | ID | Start time | Duration | Success |
|---|---|---|---|---|---|---|
| Add | InternetGateway | AWS::EC2::InternetGateway | 2022-05-26T22:34:17+10:00 | 19s | ✅ | |
| Modify | VPC | AWS::EC2::VPC | vpc-0582693d136c8d1bd | 2022-05-26T22:34:18+10:00 | 3s | ✅ |
| Add | RouteTablePublic | AWS::EC2::RouteTable | 2022-05-26T22:34:21+10:00 | 13s | ✅ | |
| Modify | RouteTablePrivate | AWS::EC2::RouteTable | rtb-02dda6c7d7dc07bdc | 2022-05-26T22:34:22+10:00 | 12s | ✅ |
| Add | SubnetAPub | AWS::EC2::Subnet | 2022-05-26T22:34:25+10:00 | 5s | ✅ | |
| Add | SubnetCPub | AWS::EC2::Subnet | 2022-05-26T22:34:25+10:00 | 5s | ✅ | |
| Modify | SubnetAPriv | AWS::EC2::Subnet | subnet-0d08715f3dc8719f0 | 2022-05-26T22:34:26+10:00 | 13s | ✅ |
| Add | SubnetBPub | AWS::EC2::Subnet | 2022-05-26T22:34:26+10:00 | 10s | ✅ | |
| Modify | SubnetCPriv | AWS::EC2::Subnet | subnet-0ba29189e259b3b29 | 2022-05-26T22:34:26+10:00 | 13s | ✅ |
| Modify | SubnetBPriv | AWS::EC2::Subnet | subnet-0d96f06ce27d5c388 | 2022-05-26T22:34:26+10:00 | 17s | ✅ |
| Add | AssignPublicRouteTableB | AWS::EC2::SubnetRouteTableAssociation | 2022-05-26T22:34:37+10:00 | 7s | ✅ | |
| Add | AssignPublicRouteTableC | AWS::EC2::SubnetRouteTableAssociation | 2022-05-26T22:34:37+10:00 | 3s | ✅ | |
| Add | AssignPublicRouteTableA | AWS::EC2::SubnetRouteTableAssociation | 2022-05-26T22:34:37+10:00 | 7s | ✅ | |
| Add | VPCGatewayAttachment | AWS::EC2::VPCGatewayAttachment | 2022-05-26T22:34:38+10:00 | 16s | ✅ | |
| Add | InternetRoutePublic | AWS::EC2::Route | 2022-05-26T22:34:55+10:00 | 16s | ✅ |
gantt
title Visual timeline of demovpc43 - Update event - Started 2022-05-26T22:34:13+10:00
dateFormat HH:mm:ss
axisFormat %H:%M:%S
Stack UPDATE_IN_PROGRESS :milestone, 22:34:13 , 0s
InternetGateway :22:34:17 , 19s
VPC :active, 22:34:18 , 3s
RouteTablePublic :22:34:21 , 13s
RouteTablePrivate :active, 22:34:22 , 12s
SubnetAPub :22:34:25 , 5s
SubnetCPub :22:34:25 , 5s
SubnetAPriv :active, 22:34:26 , 13s
SubnetBPub :22:34:26 , 10s
SubnetBPriv :active, 22:34:26 , 17s
SubnetCPriv :active, 22:34:26 , 13s
AssignPublicRouteTableC :22:34:37 , 3s
AssignPublicRouteTableA :22:34:37 , 7s
AssignPublicRouteTableB :22:34:37 , 7s
VPCGatewayAttachment :22:34:38 , 16s
InternetRoutePublic :22:34:55 , 16s
Stack UPDATE_COMPLETE_CLEANUP_IN_PROGRESS :milestone, 22:35:13 , 0s
Stack UPDATE_COMPLETE :milestone, 22:35:14 , 0s
While deployments and reports are the main features of fog, other commands have been added for convenience.
Running fog exports allows you to get an overview of all the exports in your current region. It will also show you whether they have been imported and by adding the --verbose flag it will show you by which stack. In addition, you can filter by either the stack name or export name, including the use of wildcards.
$ fog exports --stackname "*my*" --output csv
Exports for *my* in account <redacted> for region us-east-1
Export,Description,Stack,Value,Imported
myvpc-PRIVATE-SUBNET-IDS,IDs of the Private Subnets in the VPC,myvpc,"subnet-055c908d42ebe457e\,subnet-0efb4869fbb02ba97\,subnet-0d0def1b2fe23c603",No
myvpc-PUBLIC-SUBNET-IDS,IDs of the Public Subnets in the VPC,myvpc,"subnet-0ff8e5fa5476094d5\,subnet-0b500f7016c92a897\,subnet-06ad46345cd608711",No
myvpc-VPCID,The ID of the VPC,myvpc,vpc-08612404927cb7646,NoThis command let's you see all the resources managed by your CloudFormation templates.
The standard output shows the type, resource ID, and stack it's managed by. Verbose mode adds the logical ID in the CloudFormation stack and the status.
Using the stackname argument you can limit this to a specific stack using the stack's name or ID. If you provide a wildcard filter such as *dev* it will match all stacks that match that pattern.
fog resources --output json --verbose | jq .
[
{
"ID": "igw-04094534f89210d85",
"LogicalID": "InternetGateway",
"Stack": "myvpc",
"Status": "UPDATE_COMPLETE",
"Type": "AWS::EC2::InternetGateway"
},
{
"ID": "myvpc-Inter-RLQ8UX2ASBHV",
"LogicalID": "InternetRoutePublic",
"Stack": "myvpc",
"Status": "CREATE_COMPLETE",
"Type": "AWS::EC2::Route"
},
...<truncated>
]This will show your stacks and any dependencies that exist between them.
Dependencies can prevent updates from happening or prevent a stack from getting deleted. Right now dependencies being shown are export values that are imperted by other stacks. Upcoming is support for showing nested stacks.
This function supports the "dot" output format, which outputs the dependencies in a way that you can turn into a graphical format using a tool like graphviz.
fog dependencies --stackname "myvpc" --output dot | dot -T png -o docs/fog-dependencies-demo.png
Fog's drift detection builds upon the built-in drift detection from CloudFormation, but adds some nice to haves. This includes:
- Don't show a difference if the order of tags has changed
- Show differences for the routes in route tables
- Show differences for NACL rules
- Allow certain tags to be ignored for the drift result
There is a lot more planned for the application, and a roadmap etc. will soon show up on GitHub.
If you wish to contribute in any way (reporting bugs, requesting features, writing code), feel free to do so either by opening Issues or Pull Requests. For Pull Requests, just follow the standard pattern.
- Fork the repository
- Make your changes
- Make a pull request that explains what it does