GitHub Actions Bug: Workflows Cannot be Re-Run

[ScriptAutomate]

Two problems exist:

• I need to create a second, siloed off action related to the Documentation lookup errors. It does thousands of HTTP requests against AWS Documentation to find what documentation doesn't currently exist, or has links that are incorrect in the spec files. AWS will often rate limit this because of how many requests are happening, so this affects the rest of the workflow run steps.
• GitHub Actions has a bug, of which I has been told via support a few months ago, that was a known bug where the "Re-Run" action button doesn't appear to work. The GUI changes to show GitHub Actions is stuck in the launch step. This still seems to be a bug, and I still have an open ticket with support.

In order to reduce the blast radius of this step failing, I need to silo it out. In the meantime, I will have weird workflow modifications based on PUSH just so that I can force GitHub Action executions outside of cron.

[mattsb42-aws]

I need to create a second, siloed off action related to the Documentation lookup errors. It does thousands of HTTP requests against AWS Documentation to find what documentation doesn't currently exist, or has links that are incorrect in the spec files. AWS will often rate limit this because of how many requests are happening, so this affects the rest of the workflow run steps.

I haven't looked at specifically what your documentation lookups are, but most if not all of the AWS Docs should be mirrored in GitHub repos now[1], rendered in MarkDown. It might be simpler to pull those in as submodules and do whatever lookups you need locally.

[1] https://github.com/awsdocs

[ScriptAutomate]

@mattsb42-aws I originally relied on the GitHub source when I was doing some module code and help doc generation, but ran into errors.

Right now the repository uses both the GitHub repo documentation source and the live documentation of the AWS CloudFormation User Guide because of the problems between the two that I have run into in the past, and realized I could create audit tools around:

• The GitHub documentation mirror is used as a submodule for this repo for part of the lookups: https://github.com/awsdocs/aws-cloudformation-user-guide
• The live version of the CloudFormation User Guide actually differs from the GitHub repo, at times in weird ways, resulting in the following JSON generation: documentation-lookup-errors.json
  • Some follow-up work has resulted in some more detailed information of items seen here: documentation-broken-links-detailed.json

More in-depth conversation around this can be found here: GitHub AWS CFN User Guide Mirror: #428, which details what led up to the documentation auditing. Below is a snapshot of the defined problem types I ran into, helping describe the problems seen in generated data found in documentation-lookup-errors.json and documentation-broken-links-detailed.json.

---

BrokenDocLinks

• AWS Doc html page is broken, with the page unreachable and redirecting to UserGuide home page. Seems some can be an issue with the AWS CloudFormation User Guide website.
• URL in Documentation key of CloudFormationResourceSpecification.json file may simply be pointing to an incorrect html page enpoint, and needs to be updated. Seems some can be an issue with the CloudFormationResourceSpecification.json file itself.

Example I

AWS live documentation endpoint link is incorrect, redirecting to the Welcome page.

"AWS::AutoScaling::AutoScalingGroup.MixedInstancesPolicy": {
  "Broken": "http://docs.aws.amazon.com/AWSCloudFormation/latest/UserGuide/cfn-as-group-mixedinstancespolicy.html",
  "Fix": "http://docs.aws.amazon.com/AWSCloudFormation/latest/UserGuide/aws-properties-autoscaling-autoscalinggroup-mixedinstancespolicy.html",
  "Problem": "URL in CloudFormationResourceSpecification.json needs to update with the Fix key value.",
  "DiscoveredInVersion": "5.1.0",
  "DateOfDiscovery": "2019-08-09"
}

Example II

AWS live documentation endpoint doesn't exist, redirecting to the Welcome page.

"AWS::CodePipeline::CustomActionType.ArtifactDetails": {
  "Broken": "http://docs.aws.amazon.com/AWSCloudFormation/latest/UserGuide/aws-properties-codepipeline-customactiontype-artifactdetails.html",
  "Fix": null,
  "Problem": "AWS Doc html page is broken, with this page unreachable and redirecting to UserGuide home.",
  "DiscoveredInVersion": "5.3.0",
  "DateOfDiscovery": "2019-08-23"
}

MissingDocLinks

• Documentation key is not present in the CloudFormationResourceSpecification.json file, and should be added. Seems to be an issue with the CloudFormationResourceSpecification.json file.

Example

Exists in the live documentation, but a documentation link is missing in the spec file.

"AWS::Transfer::User.SshPublicKey": {
  "Documentation": "http://docs.aws.amazon.com/AWSCloudFormation/latest/UserGuide/aws-properties-transfer-user-sshpublickey.html",
  "DiscoveredInVersion": "5.3.0",
  "DateOfDiscovery": "2019-08-23"
}

MissingFromAWSGitHubSourceRepo

• Source repo pages are missing, even though they are present in AWS CFN UserGuide docs. Both causes seem to point to the possibility of the GitHub repo being updated via a manual instead of automated process, allowing for unclear periods of documentation lag and the potential of missing content to go unsynced. The causes look to be either:
  • The GitHub mirror repo has yet to catch up to the latest release(s). Source repo expected to lag behind in content by maybe 1 - 2 weeks?
  • The GitHub mirror never had certain documentation pushed to it, not merely because it is lagging behind, but because some kind of periodic update was missed, leaving the mirror as missing particular pieces of documenation. This seems to currently be the case for documentation or types such as AWS::DLM::LifecyclePolicy.Parameters, AWS::EFS::FileSystem.LifecyclePolicy, and AWS::Glue::Trigger.NotificationProperty. These are not available in the mirror, but are in the live docs. The first two types are referenced in release history from a June 13, 2019 release, with the AWS::Glue::Trigger.NotificationProperty being mentioned in the September 26, 2019 release.

Example

Exists, in live documentation but doesn't in the GitHub mirror.

"AWS::EFS::FileSystem.LifecyclePolicy": {
  "Broken": "https://github.com/awsdocs/aws-cloudformation-user-guide/tree/master/doc_source/aws-properties-elasticfilesystem-filesystem-lifecyclepolicy.md",
  "Problem": "Missing from GitHub repo.",
  "DateOfDiscovery": "2019-06-13"
}

NOTE: With many of these issues being discovered due to the Documentation properties in the CloudFormationResourceSpecification.json files, this repo was made to download and track changes/errors within each release, as the spec files currently don't exist in an AWS docs repo (referenced in GitHub AWS CFN User Guide Mirror: #4)