cnspec scan aws --discover all -f policies/aws-operational-best-practices.mql.yaml against Mondoo Development account runs reproducibily into a time-out error

[mm-weber]

Describe the bug
cnspec scan aws --discover all -f policies/aws-operational-best-practices.mql.yaml against Mondoo Development account runs reproducibily into a time-out error

Redacted output

x unable to create runtime for asset error="rpc error: code = Unknown desc = dial tcp xxx.xxx.xx.xxx:22: connect: connection timed out" asset=amazonlinux2-for-ebs-volume-scan
→ reported panic to Mondoo platform
panic: runtime error: invalid memory address or nil pointer dereference [recovered]
	panic: runtime error: invalid memory address or nil pointer dereference
[signal SIGSEGV: segmentation violation code=0x1 addr=0x0 pc=0x149bb6d]

goroutine 1 [running]:
go.mondoo.com/cnquery/v10/providers-sdk/v1/upstream/health.ReportPanic({0x1e5adcf, 0x6}, {0x23eb648, 0x7}, {0x23eb650, 0x7}, {0x0, 0x0, 0x0?})
	/home/manuel/.gvm/pkgsets/go1.21.3/global/pkg/mod/go.mondoo.com/cnquery/[email protected]/providers-sdk/v1/upstream/health/errors.go:30 +0x16c
panic({0x1b4c020?, 0x3352950?})
	/home/manuel/.gvm/gos/go1.21.3/src/runtime/panic.go:914 +0x21f
go.mondoo.com/cnquery/v10/explorer/scan.DiscoverAssets({0xc00012e150?, 0x1eadfd1?}, 0xc000750a00, 0xc0019f3678?, {0x2412848, 0x33e4520})
	/home/manuel/.gvm/pkgsets/go1.21.3/global/pkg/mod/go.mondoo.com/cnquery/[email protected]/explorer/scan/discovery.go:119 +0x6ad
go.mondoo.com/cnspec/v10/policy/scan.(*LocalScanner).distributeJob(0xc000129ce0, 0xc000129d50, {0x2410028?, 0xc001768630}, 0xc0005aec00)
	/home/manuel/go/src/go.mondoo.io/cnspec/policy/scan/local_scanner.go:256 +0x1a9
go.mondoo.com/cnspec/v10/policy/scan.(*LocalScanner).RunIncognito(0x240fff0?, {0x2410028, 0xc001768630}, 0xc000129d50)
	/home/manuel/go/src/go.mondoo.io/cnspec/policy/scan/local_scanner.go:182 +0x7b
go.mondoo.com/cnspec/v10/apps/cnspec/cmd.RunScan(0xc001244e40, {0x0?, 0x0?, 0x0?})
	/home/manuel/go/src/go.mondoo.io/cnspec/apps/cnspec/cmd/scan.go:362 +0x2f2
go.mondoo.com/cnspec/v10/apps/cnspec/cmd.glob..func24(0x33a9fe0?, 0xc001110000?, 0x33e4520?)
	/home/manuel/go/src/go.mondoo.io/cnspec/apps/cnspec/cmd/scan.go:137 +0xbf
go.mondoo.com/cnquery/v10/cli/providers.setConnector.func2(0xc000cd3000?, {0xc0007514c0?, 0x0, 0x4})
	/home/manuel/.gvm/pkgsets/go1.21.3/global/pkg/mod/go.mondoo.com/cnquery/[email protected]/cli/providers/providers.go:496 +0xa42
github.com/spf13/cobra.(*Command).execute(0xc000cd5500, {0xc000751480, 0x4, 0x4})
	/home/manuel/.gvm/pkgsets/go1.21.3/global/pkg/mod/github.com/spf13/[email protected]/command.go:987 +0xaa3
github.com/spf13/cobra.(*Command).ExecuteC(0x33797e0)
	/home/manuel/.gvm/pkgsets/go1.21.3/global/pkg/mod/github.com/spf13/[email protected]/command.go:1115 +0x3ff
github.com/spf13/cobra.(*Command).Execute(...)
	/home/manuel/.gvm/pkgsets/go1.21.3/global/pkg/mod/github.com/spf13/[email protected]/command.go:1039
go.mondoo.com/cnspec/v10/apps/cnspec/cmd.Execute()
	/home/manuel/go/src/go.mondoo.io/cnspec/apps/cnspec/cmd/root.go:120 +0x7d
main.main()
	/home/manuel/go/src/go.mondoo.io/cnspec/apps/cnspec/cnspec.go:14 +0x85

To Reproduce
Steps to reproduce the behavior:

1. Go to '...'
2. Select '....'
3. Scroll down to '....'
4. Note the error

Expected behavior
A clear and concise description of what you expected to happen.

Screenshots or CLI Output
If applicable, add screenshots or the CLI output to help explain your problem.

Desktop (please complete the following information):

• OS: [e.g. macOS]
• OS Version: [e.g. 13.0]
• Browser if applicable: [e.g. Chrome, Firefox]
• Browser Version: [e.g. 106]

Additional context
Add any other context about the problem here.

[mm-weber]

This seems solved:

/home/manuel/projects/cloud-security-testing (manuel/improve-aws-cloud*)$ cnspec scan aws --discover all -f ~/projects/cnspec-enterprise-policies/policies/aws-operational-best-practices.mql.yaml
→ loaded configuration from /home/manuel/.config/mondoo/mondoo.yml using source $MONDOO_CONFIG_PATH
→ using service account credentials
! Scanning with local bundles will switch into --incognito mode by default. Your results will not be sent upstream.
→ discover related assets for 1 asset(s)
→ no AWS region found, using us-east-1
! no public ip address found asset=ivan-scanner
! no public ip address found asset=k8s-operator01
! no public ip address found asset=k8s-operator02
! no public ip address found asset=k8s-operator03
! no public ip address found asset=preslav-linux
! cannot use ssm session credentials for connection asset=mi-054b97883ba2375c2 id=mi-054b97883ba2375c2
→ no AWS region found, using us-east-1
→ no AWS region found, using us-east-1
→ no AWS region found, using us-east-1
→ no AWS region found, using us-east-1
→ no AWS region found, using us-east-1
x unable to create runtime for asset error="rpc error: code = Unknown desc = dial tcp 174.129.98.225:22: connect: connection timed out" asset=amazonlinux2-for-ebs-volume-scan
x unable to create runtime for asset error="rpc error: code = Unknown desc = dial tcp 3.15.200.34:22: connect: connection timed out" asset=ivan-test
x unable to create runtime for asset error="rpc error: code = Unknown desc = dial tcp 18.224.37.254:22: connect: connection timed out" asset="My Amazon Linux"
→ no AWS region found, using us-east-1
→ no AWS region found, using us-east-1

 AWS Account mondoo-dev (921877552404)                                                         ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 100% score: F
 ivan-scanner                                                                                  ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 100% score: B
 k8s-operator01                                                                                ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 100% score: C
 k8s-operator02                                                                                ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 100% score: C
 k8s-operator03                                                                                ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 100% score: C
 preslav-linux                                                                                 ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 100% score: C
 mi-054b97883ba2375c2                                                                          ──────────────────────────────────────────────────────────────────────    X score: X
 vjtest@sha256:ce0e28e908e5b400b97d4001b7f1c60d39e91a3aa0cd005370a7672f138f0bc8                ──────────────────────────────────────────────────────────────────────    X score: X
 dvwa-container-escape@sha256:008a768bd699c208aa515bc5ddf518599c66ff63a1f56dde60497f8b2fae809a ──────────────────────────────────────────────────────────────────────    X score: X

 6/9 scanned 3/9 errored                                                                       ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 100%

Asset: AWS Account mondoo-dev (921877552404)
--------------------------------------------

Checks:
✓ Pass:  A 100  Checks that both AWS VPN tunnels provided by AWS site-to-site are in UP status
✓ Pass:  A 100  Ensures that all instances belong to a VPC
✕ Fail:  B  70  Ensure IAM password policy require at least one symbol
✕ Fail:  C  30  Ensure EBS volume encryption is enabled
✓ Pass:  A 100  Checks that S3 buckets do not allow public write access
✓ Pass:  A 100  Checks that all CMKs are not scheduled for deletion
✕ Fail:  B  70  Checks if logging is enabled on all S3 buckets
✕ Fail:  B  70  Checks that EBS optimization is enabled for all instances that support EBS optimization
✓ Pass:  A 100  Checks that all autoscaling groups assoc with a load balancer use healthchecks
✓ Pass:  A 100  Checks that at least one cloud trail is a multi-region trail
✓ Pass:  A 100  Checks that all Redshift clusters have encryption at rest, logging and node type.
✕ Fail:  B  70  Checks that all CloudTrail trails are configured to use the server side encryption KMS
✕ Fail:  C  50  Checks that all log groups in Amazon CloudWatch Logs are encrypted with KMS
✕ Fail:  C  50  Checks that all SageMaker notebook instances are configured to use KMS
✓ Pass:  A 100  Checks that all Redshift clusters have allowVersionUpgrade enabled and preferredMaintenanceWindow and automatedSnapshotRetentionPeriod set
✕ Fail:  C  50  Ensure S3 bucket access logging is enabled on the CloudTrail S3 bucket
✓ Pass:  A 100  Checks if the ElastiCache Redis clusters have automatic backup turned on
✓ Pass:  A 100  Checks the status of AWS Systems Manager patch compliance
✕ Fail:  B  70  Checks whether HTTP to HTTPS redirection is configured on all application load balancer http listeners
✕ Fail:  B  70  Checks whether detailed monitoring is enabled on the instance
✕ Fail:  C  50  Checks whether classic and application load balancers have logging enabled
✓ Pass:  A 100  Checks that all Elastic IP addresses are attached to ec2 instances or in-use ENIs
✕ Fail:  B  70  Checks if exported logs are enabled for all RDS DB instances
✕ Fail:  D  15  Ensure hardware MFA is enabled for the "root user" account
✕ Fail:  D  20  Ensures no instances have a public IP
✓ Pass:  A 100  Checks whether IAM groups have at least one IAM user
✓ Pass:  A 100  Checks whether Lambda functions are configured with function-level concurrent execution limit
✕ Fail:  C  50  Checks that all SNS topics are encrypted with KMS
✓ Pass:  A 100  Checks whether Lambda functions are configured with a dead letter queue
✕ Fail:  D  25  Ensure multi-factor authentication (MFA) is enabled for all IAM users that have a console password
✓ Pass:  A 100  Elasticsearch domains should be in a VPC
✕ Fail:  F   0  Amazon EBS snapshots should not be publicly restorable
✕ Fail:  B  70  Checks whether instances are managed by Amazon Systems Manager
✕ Fail:  B  70  Checks if all RDS instances have deletion protection enabled
✕ Fail:  B  70  Checks whether S3 buckets have cross-region replication enabled
✕ Fail:  B  70  Checks that all CloudTrail trails are configured to send logs to AWS CloudWatch
✓ Pass:  A 100  Ensure no IAM policies allow Administrative access over all resources
✕ Fail:  C  50  Attached Amazon EBS volumes should be encrypted at-rest
✓ Pass:  A 100  Checks that all SageMaker endpoint configurations are configured to use KMS
✓ Pass:  A 100  Checks whether ACM Certificates in your account are marked for expiration within 30 days
✕ Fail:  B  70  Checks that EC2 instances use security groups that are attached to an Elastic Network Interface
✕ Fail:  D  20  Checks whether IAM users are members of at least one IAM group
✕ Fail:  A  80  Checks whether the Lambda functions exists within a VPC
✕ Fail:  B  70  Checks if AWS GuardDuty has findings that are non archived
✕ Fail:  B  70  Checks whether high availability is enabled for all RDS DB instances
✓ Pass:  A 100  Ensure all S3 buckets employ encryption-at-rest
✓ Pass:  A 100  Checks that all Classic Load Balancers use SSL certificates provided by AWS Certificate Manager
✓ Pass:  A 100  Ensure no root user account access key exists
✓ Pass:  A 100  Checks the compliance status AWS systems manager association
✓ Pass:  A 100  Security groups should not allow ingress from 0.0.0.0/0 to port 22
✕ Fail:  B  70  Checks whether the account password policy for IAM users meets the specified requirements
✓ Pass:  A 100  Security groups should not allow unrestricted access to ports with high risk
✓ Pass:  A 100  Ensure routing tables for VPC peering are "least access"
✓ Pass:  A 100  Checks that GuardDuty is enabled in all regions
✓ Pass:  A 100  Checks that all Classic Load Balancer are configured with SSL or HTTPS listeners
✕ Fail:  D  20  Checks that versioning is enabled for all S3 buckets
✕ Fail:  B  70  Checks whether EBS volumes are attached to EC2 instances and configured for deletion on instance termination
✕ Fail:  D  20  Checks that any security group with 0.0.0.0/0 of any VPC allows only specific inbound TCP/UDP traffic
✓ Pass:  A 100  Checks that all methods in Amazon API Gateway have caching enabled and encrypted
✕ Fail:  B  70  Checks whether EFS filesystems are included in AWS Backup plans
✕ Fail:  C  50  VPC flow logging should be enabled in all VPCs
✕ Fail:  B  70  Checks if each S3 bucket has default lock enabled
✓ Pass:  A 100  Checks that all RDS DB instances have encryption enabled for snapshots.
✕ Fail:  B  70  Ensure IAM password policy requires at least one uppercase letter
✕ Fail:  C  50  Checks that all AWS Secrets Manager secrets have rotation enabled
✓ Pass:  A 100  Elasticsearch domains should have encryption at-rest enabled
✕ Fail:  B  70  Ensure IAM password policy require at least one number
✕ Fail:  B  70  Checks that AWS Security Hub is enabled for the account
✓ Pass:  A 100  Amazon Elastic MapReduce cluster master nodes should not have public IP addresses
✕ Fail:  F   5  Checks that all RDS instances are not publicly accessible
✕ Fail:  B  70  Ensure IAM password policy require at least one lowercase letter
✓ Pass:  A 100  Ensure MFA is enabled for the "root user" account
✓ Pass:  A 100  Checks that all RDS snapshots are not publicly accessible
✓ Pass:  A 100  DynamoDB tables should automatically scale capacity with demand
✓ Pass:  A 100  Checks whether enhanced monitoring is enabled for all RDS DB instances
✕ Fail:  D  20  Checks whether the instance metadata version is configured with IMDSv2 (HTTP tokens required)
✕ Fail:  F   5  Checks that S3 buckets do not allow public read access
✕ Fail:  F   5  Checks whether the AWS IAM users have multi-factor authentication (MFA) enabled
✓ Pass:  A 100  Ensure that encryption is enabled for RDS Instances
✕ Fail:  B  70  Checks that all Redshift clusters require TLS/SSL encryption to connect to SQL clients.
✓ Pass:  A 100  Checks whether Redshift clusters are publicly accessible
✕ Fail:  B  70  Checks whether the active access keys are rotated within the number of days specified in maxAccessKeyAge (default 90)
✕ Fail:  D  20  Ensure the default security group of every VPC restricts all traffic
✕ Fail:  C  50  DynamoDB tables should be covered by a backup plan
✕ Fail:  B  70  Ensure IAM Users Receive Permissions Only Through Groups
✕ Fail:  C  40  Ensure rotation for customer created CMKs is enabled
✕ Fail:  D  20  Checks whether direct internet access is disabled for all Amazon SageMaker notebook instance
✓ Pass:  A 100  Checks that all buckets are encrypted with KMS
✕ Fail:  D  20  Checks if Amazon Simple Storage Service (S3) has bucket level public access restrictions at the bucket level.
✕ Fail:  B  70  Checks that all inline IAM policies that belong to users, roles, and groups do not allow blocked actions on arbitrary KMS keys
✓ Pass:  A 100  Checks that all projects containing env variables AWS_ACCESS_KEY_ID and AWS_SECRET_ACCESS_KEY are not in plaintext
✓ Pass:  A 100  Checks whether the policy attached to the Lambda function prohibits public access
✓ Pass:  A 100  CloudTrail should be enabled
✕ Fail:  B  70  CloudTrail log file validation should be enabled
✓ Pass:  A 100  Checks that all DynamoDB tables are encrypted with AWS Key Management Service (KMS)
! Error:        Checks if the required S3 public access block settings are configured from account level
✕ Fail:  C  50  Application Load Balancer deletion protection should be enabled
✓ Pass:  A 100  Elasticsearch domains should encrypt data sent between nodes
✓ Pass:  A 100  Checks whether AWS Database Migration Service replication instances are public
✕ Fail:  B  70  Checks whether CloudWatch alarms have at least one alarm action, one INSUFFICIENT_DATA action, or one OK action enabled.
✓ Pass:  A 100  RDS DB instances should be covered by a backup plan
✕ Fail:  B  70  Checks that all AWS IAM users have passwords or active access keys that have not been used in maxCredentialUsageAge days (default 90)
✕ Fail:  B  70  Checks that all IAM policies do not allow blocked actions on KMS keys
✓ Pass:  A 100  Checks that all methods in Amazon API Gateway have logging enabled
✕ Fail:  B  70  Ensure IAM password policy expires passwords within 90 days or less
✓ Pass:  A 100  Checks that all projects using github or bitbucket as the source use oauth
✕ Fail:  B  70  Checks that point in time recovery (PITR) is enabled for all AWS DynamoDB tables

Asset: My Amazon Linux
----------------------

error: rpc error: code = Unknown desc = dial tcp 18.224.37.254:22: connect: connection timed out

Asset: dvwa-container-escape@sha256:008a768bd699c208aa515bc5ddf518599c66ff63a1f56dde60497f8b2fae809a
----------------------------------------------------------------------------------------------------

error: rpc error: code = InvalidArgument desc = asset doesn't support any policies

Asset: ivan-scanner
-------------------

Checks:
✓ Pass:  A 100  Checks whether the instance metadata version is configured with IMDSv2 (HTTP tokens required)
✓ Pass:  A 100  Ensures no instances have a public IP
✕ Fail:  B  70  Checks whether detailed monitoring is enabled on the instance
✕ Fail:  B  70  Checks that EBS optimization is enabled for all instances that support EBS optimization
✓ Pass:  A 100  Ensures that all instances belong to a VPC
✕ Fail:  B  70  Checks whether EBS volumes are attached to EC2 instances and configured for deletion on instance termination
✕ Fail:  B  70  Checks whether instances are managed by Amazon Systems Manager

Asset: k8s-operator01
---------------------

Checks:
✕ Fail:  D  20  Checks whether the instance metadata version is configured with IMDSv2 (HTTP tokens required)
✓ Pass:  A 100  Checks whether EBS volumes are attached to EC2 instances and configured for deletion on instance termination
✓ Pass:  A 100  Ensures that all instances belong to a VPC
✓ Pass:  A 100  Ensures no instances have a public IP
✕ Fail:  B  70  Checks whether detailed monitoring is enabled on the instance
✕ Fail:  B  70  Checks whether instances are managed by Amazon Systems Manager
✕ Fail:  B  70  Checks that EBS optimization is enabled for all instances that support EBS optimization

Asset: k8s-operator02
---------------------

Checks:
✓ Pass:  A 100  Checks whether EBS volumes are attached to EC2 instances and configured for deletion on instance termination
✕ Fail:  D  20  Checks whether the instance metadata version is configured with IMDSv2 (HTTP tokens required)
✓ Pass:  A 100  Ensures that all instances belong to a VPC
✓ Pass:  A 100  Ensures no instances have a public IP
✕ Fail:  B  70  Checks whether detailed monitoring is enabled on the instance
✕ Fail:  B  70  Checks whether instances are managed by Amazon Systems Manager
✕ Fail:  B  70  Checks that EBS optimization is enabled for all instances that support EBS optimization

Asset: k8s-operator03
---------------------

Checks:
✕ Fail:  B  70  Checks whether detailed monitoring is enabled on the instance
✓ Pass:  A 100  Ensures that all instances belong to a VPC
✕ Fail:  B  70  Checks whether instances are managed by Amazon Systems Manager
✕ Fail:  B  70  Checks that EBS optimization is enabled for all instances that support EBS optimization
✓ Pass:  A 100  Ensures no instances have a public IP
✓ Pass:  A 100  Checks whether EBS volumes are attached to EC2 instances and configured for deletion on instance termination
✕ Fail:  D  20  Checks whether the instance metadata version is configured with IMDSv2 (HTTP tokens required)

Asset: mi-054b97883ba2375c2
---------------------------

error: rpc error: code = InvalidArgument desc = asset doesn't support any policies

Asset: preslav-linux
--------------------

Checks:
✓ Pass:  A 100  Ensures no instances have a public IP
✕ Fail:  B  70  Checks that EBS optimization is enabled for all instances that support EBS optimization
✓ Pass:  A 100  Checks whether EBS volumes are attached to EC2 instances and configured for deletion on instance termination
✕ Fail:  B  70  Checks whether detailed monitoring is enabled on the instance
✕ Fail:  B  70  Checks whether instances are managed by Amazon Systems Manager
✓ Pass:  A 100  Ensures that all instances belong to a VPC
✕ Fail:  D  20  Checks whether the instance metadata version is configured with IMDSv2 (HTTP tokens required)

Asset: vjtest@sha256:ce0e28e908e5b400b97d4001b7f1c60d39e91a3aa0cd005370a7672f138f0bc8
-------------------------------------------------------------------------------------

error: rpc error: code = InvalidArgument desc = asset doesn't support any policies


Scanned 10 assets

AWS Account
    F AWS Account mondoo-dev (921877552404)

AWS EC2 Instance
    B ivan-scanner
    C k8s-operator01
    C k8s-operator02
    C k8s-operator03
    C preslav-linux

AWS SSM Instance
    X mi-054b97883ba2375c2

Alpine Linux v3.11
    X vjtest@sha256:ce0e28e908e5b400b97d4001b7f1c60d39e91a3aa0cd005370a7672f138f0bc8

Debian GNU/Linux 9 (stretch)
    X dvwa-container-escape@sha256:008a768bd699c208aa515bc5ddf518599c66ff63a1f56dde60497f8b2fae809a

Summary
=======

Score Distribution		Asset Distribution
------------------		------------------
A   0 assets      		Alpine Linux v3.11             1
B   1 assets      		AWS Account                    1
C   4 assets      		Debian GNU/Linux 9 (stretch)   1
D   0 assets      		AWS EC2 Instance               5
F   1 assets      		AWS SSM Instance               1
X   4 assets