Merge pull request #211 from SumoLogic/sumo_247402

Updated log-group-connector sam version to 1.0.14 and added support for AutoSubscribeLogGroupByTags

aws-observability-terraform/source-module/README.md

@@ -71,7 +71,7 @@
 | <a name="input_auto_enable_access_logs"></a> [auto\_enable\_access\_logs](#input\_auto\_enable\_access\_logs) | Enable Application Load Balancer (ALB) Access logging.<br>            You have the following options:<br>            New - Automatically enables access logging for newly created ALB resources to collect logs for ALB resources. This does not affect ALB resources already collecting logs.<br>            Existing - Automatically enables access logging for existing ALB resources to collect logs for ALB resources.<br>            Both - Automatically enables access logging for new and existing ALB resources.<br>            None - Skips Automatic access Logging enable for ALB resources. | `string` | `"Both"` | no |
 | <a name="input_auto_enable_classic_lb_access_logs"></a> [auto\_enable\_classic\_lb\_access\_logs](#input\_auto\_enable\_classic\_lb\_access\_logs) | Enable Classic Load Balancer (CLB) Access logging.<br>            You have the following options:<br>            New - Automatically enables access logging for newly created CLB resources to collect logs for CLB resources. This does not affect CLB resources already collecting logs.<br>            Existing - Automatically enables access logging for existing CLB resources to collect logs for CLB resources.<br>            Both - Automatically enables access logging for new and existing CLB resources.<br>            None - Skips Automatic access Logging enable for CLB resources. | `string` | `"Both"` | no |
 | <a name="input_auto_enable_logs_subscription"></a> [auto\_enable\_logs\_subscription](#input\_auto\_enable\_logs\_subscription) | Subscribe log groups to Sumo Logic Lambda Forwarder.<br>            You have the following options:<br>            New - Automatically subscribes new log groups to send logs to Sumo Logic.<br>            Existing - Automatically subscribes existing log groups to send logs to Sumo Logic.<br>            Both - Automatically subscribes new and existing log groups.<br>            None - Skips Automatic subscription. | `string` | `"Both"` | no |
-| <a name="input_auto_enable_logs_subscription_options"></a> [auto\_enable\_logs\_subscription\_options](#input\_auto\_enable\_logs\_subscription\_options) | filter - Enter regex for matching CloudWatch Log groups name. Regex will check for the Log groups name. Visit https://help.sumologic.com/03Send-Data/Collect-from-Other-Data-Sources/Auto-Subscribe_AWS_Log_Groups_to_a_Lambda_Function#Configuring_parameters | <pre>object({<br>    filter = string<br>  })</pre> | <pre>{<br>  "filter": "apigateway|lambda|rds"<br>}</pre> | no |
+| <a name="input_auto_enable_logs_subscription_options"></a> [auto\_enable\_logs\_subscription\_options](#input\_auto\_enable\_logs\_subscription\_options) | filter - Enter regex for matching logGroups. Regex will check for the name. <br> tags_filter - Enter comma separated key value pairs for filtering logGroups using tags. Ex KeyName1=string,KeyName2=string. This is optional leave it blank if tag based filtering is not needed. <br> Visit https://help.sumologic.com/docs/send-data/collect-from-other-data-sources/autosubscribe-arn-destination/#configuringparameters | <pre>object({<br>    filter = string <br>    tags_filter = string <br> })</pre>                                                                                                                  | <pre>{<br>  "filter": "lambda" <br>  "tags_filter" = "" <br> }</pre>                                                                                                                                                                                                                                                   |    no    |
 | <a name="input_aws_account_alias"></a> [aws\_account\_alias](#input\_aws\_account\_alias) | Provide the Name/Alias for the AWS environment from which you are collecting data. This name will appear in the Sumo Logic Explorer View, metrics, and logs.<br>            Please leave this blank if you are going to deploy the solution in multiple AWS accounts.<br>            Do not include special characters in the alias. | `string` | n/a | yes |
 | <a name="input_classic_lb_log_source_url"></a> [classic\_lb\_log\_source\_url](#input\_classic\_lb\_log\_source\_url) | Required if you are already collecting Classic LB logs. Provide the existing Sumo Logic Classic LB Source API URL. The account, accountid, and region fields will be added to the Source. For information on how to determine the URL, see [View or Download Source JSON Configuration](https://help.sumologic.com/03Send-Data/Sources/03Use-JSON-to-Configure-Sources/Local-Configuration-File-Management/View-or-Download-Source-JSON-Configuration). | `string` | `""` | no |
 | <a name="input_classic_lb_source_details"></a> [classic\_lb\_source\_details](#input\_classic\_lb\_source\_details) | Provide details for the Sumo Logic Classic Load Balancer source. If not provided, then defaults will be used.<br>            To enable collection of classic load balancer logs, set collect\_classic\_lb\_logs to true and provide configuration information for the bucket.<br>            If create\_bucket is false, provide a name of an existing S3 bucket where you would like to store loadbalancer logs. If this is empty, a new bucket will be created in the region.<br>            If create\_bucket is true, the script creates a bucket, the name of the bucket has to be unique; this is achieved internally by generating a random-id and then post-fixing it to the “aws-observability-” string.<br>            path\_expression - This is required in case the above existing bucket is already configured to receive Classic LB access logs. If this is blank, Sumo Logic will store logs in the path expression: *classicloadbalancing/AWSLogs/*/elasticloadbalancing/*/* | <pre>object({<br>    source_name     = string<br>    source_category = string<br>    description     = string<br>    bucket_details = object({<br>      create_bucket        = bool<br>      bucket_name          = string<br>      path_expression      = string<br>      force_destroy_bucket = bool<br>    })<br>    fields = map(string)<br>  })</pre> | <pre>{<br>  "bucket_details": {<br>    "bucket_name": "aws-observability-random-id",<br>    "create_bucket": true,<br>    "force_destroy_bucket": true,<br>    "path_expression": "*classicloadbalancing/AWSLogs/<ACCOUNT-ID>/elasticloadbalancing/<REGION-NAME>/*"<br>  },<br>  "description": "This source is created using Sumo Logic terraform AWS Observability module to collect AWS Classic LoadBalancer logs.",<br>  "fields": {},<br>  "source_category": "aws/observability/clb/logs",<br>  "source_name": "Classic lb Logs (Region)"<br>}</pre> | no |

aws-observability-terraform/source-module/main.tf

@@ -230,9 +230,10 @@ module "cloudwatch_logs_lambda_log_forwarder_module" {
   }
 
   auto_enable_logs_subscription = var.auto_enable_logs_subscription
-  app_semantic_version = "1.0.12"
+  app_semantic_version = "1.0.14"
   auto_enable_logs_subscription_options = {
     filter = var.auto_enable_logs_subscription_options.filter
+    tags_filter = var.auto_enable_logs_subscription_options.tags_filter
   }
 }
 
@@ -259,9 +260,10 @@ module "kinesis_firehose_for_logs_module" {
   }
 
   auto_enable_logs_subscription = var.auto_enable_logs_subscription
-  app_semantic_version = "1.0.12"
+  app_semantic_version = "1.0.14"
   auto_enable_logs_subscription_options = {
     filter = var.auto_enable_logs_subscription_options.filter
+    tags_filter = var.auto_enable_logs_subscription_options.tags_filter
   }
 }
 

aws-observability-terraform/source-module/variables.tf

@@ -499,12 +499,18 @@ variable "auto_enable_logs_subscription" {
 variable "auto_enable_logs_subscription_options" {
   type = object({
     filter = string
+    tags_filter = string
   })
+
   description = <<EOT
-		    filter - Enter regex for matching CloudWatch Log groups name. Regex will check for the Log groups name. Visit https://help.sumologic.com/03Send-Data/Collect-from-Other-Data-Sources/Auto-Subscribe_AWS_Log_Groups_to_a_Lambda_Function#Configuring_parameters
-	    EOT
+		filter - Enter regex for matching logGroups. Regex will check for the name.
+        tags_filter - Enter comma separated key value pairs for filtering logGroups using tags. Ex KeyName1=string,KeyName2=string. This is optional leave it blank if tag based filtering is not needed.
+        Visit https://help.sumologic.com/docs/send-data/collect-from-other-data-sources/autosubscribe-arn-destination/#configuringparameters
+	EOT
+
   default = {
     filter = "apigateway|lambda|rds"
+    tags_filter = ""
   }
 }
 

aws-observability/apps/autoenable/auto_enable.template.yaml

@@ -70,6 +70,10 @@ Parameters:
     Type: String
     Default: '\/aws\/(lambda|apigateway|rds)'
     Description: "Enter regex for matching logGroups. Regex will check for the name. Visit https://help.sumologic.com/docs/send-data/collect-from-other-data-sources/autosubscribe-arn-destination/#configuringparameters"
+  AutoSubscribeLogGroupByTags:
+    Type: String
+    Default: ""
+    Description: "Enter comma separated key value pairs for filtering logGroups using tags. Ex KeyName1=string,KeyName2=string. This is optional leave it blank if tag based filtering is not needed. Visit https://help.sumologic.com/docs/send-data/collect-from-other-data-sources/autosubscribe-arn-destination/#configuringparameters"
   AutoSubscribeRoleArn:
     Type: String
     Default: ""
@@ -154,10 +158,11 @@ Resources:
     Properties:
       Location:
         ApplicationId: arn:aws:serverlessrepo:us-east-1:956882708938:applications/sumologic-loggroup-connector
-        SemanticVersion: 1.0.12
+        SemanticVersion: 1.0.14
       Parameters:
         DestinationArnType: !Ref AutoSubscribeDestinationARNType
         DestinationArnValue: !Ref AutoSubscribeDestinationARN
         LogGroupPattern: !Ref AutoSubscribeLambdaLogGroupPattern
+        LogGroupTags: !Ref AutoSubscribeLogGroupByTags
         UseExistingLogs: !If [auto_subscribe_exisitng_log_groups, "true", "false"]
         RoleArn: !Ref AutoSubscribeRoleArn

aws-observability/apps/permissionchecker/permissioncheck.nested.template.test.yaml

@@ -663,7 +663,7 @@ Resources:
     Properties:
       Location:
         ApplicationId: arn:aws:serverlessrepo:us-east-1:956882708938:applications/sumologic-loggroup-connector
-        SemanticVersion: 1.0.12
+        SemanticVersion: 1.0.14
       Parameters:
         DestinationArnType: "Lambda"
         DestinationArnValue: !GetAtt CloudWatchEventFunction.Outputs.SumoCWLogsLambdaArn

aws-observability/apps/permissionchecker/permissioncheck.nested.template.yaml

@@ -671,7 +671,7 @@ Resources:
     Properties:
       Location:
         ApplicationId: arn:aws:serverlessrepo:us-east-1:956882708938:applications/sumologic-loggroup-connector
-        SemanticVersion: 1.0.12
+        SemanticVersion: 1.0.14
       Parameters:
         DestinationArnType: "Lambda"
         DestinationArnValue: !GetAtt CloudWatchEventFunction.Outputs.SumoCWLogsLambdaArn

aws-observability/templates/sumologic_observability.master.template.yaml

@@ -56,6 +56,7 @@ Metadata:
           - Section7bLambdaCloudWatchLogsSourceUrl
           - Section7cAutoSubscribeLogGroupsOptions
           - Section7dAutoSubscribeLogGroupPattern
+          - Section7eAutoSubscribeLogGroupByTags
 
       - Label:
           default: "8. Sumo Logic Root Cause Explorer Sources"
@@ -134,6 +135,9 @@ Metadata:
         default: "Subscribe log groups to Destination (Lambda or kinesis firehose delivery stream)"
       Section7dAutoSubscribeLogGroupPattern:
         default: "Regex for filtering CloudWatch Log Groups. If you want to collect from all CloudWatch Log Groups use .*"
+      Section7eAutoSubscribeLogGroupByTags:
+        default: "Tags for filtering CloudWatch Log Groups."
+
 
       Section8aRootCauseExplorerOptions:
         default: "Select the Sumo Logic Root Cause Explorer Sources"
@@ -324,6 +328,10 @@ Parameters:
     Type: String
     Default: '\/aws\/(lambda|apigateway|rds)'
     Description: "Enter regex for matching CloudWatch Log groups name. Regex will check for the Log group name. Visit https://help.sumologic.com/03Send-Data/Collect-from-Other-Data-Sources/Auto-Subscribe_AWS_Log_Groups_to_a_Lambda_Function#Configuring_parameters"
+  Section7eAutoSubscribeLogGroupByTags:
+    Type: String
+    Default: ""
+    Description: "Enter comma separated key value pairs for filtering logGroups using tags. Ex KeyName1=string,KeyName2=string. This is optional leave it blank if tag based filtering is not needed. Visit https://help.sumologic.com/docs/send-data/collect-from-other-data-sources/autosubscribe-arn-destination/#configuringparameters"
 
   Section8aRootCauseExplorerOptions:
     Type: String
@@ -475,7 +483,7 @@ Mappings:
   CommonData:
     NestedTemplate:
       BucketName: "sumologic-appdev-aws-sam-apps"
-      Version: "v2.10.0"
+      Version: "v2.11.0"
     CollectorDetails:
       SumoLogicAccountID: 926226587429
       CollectorNamePrefix: "aws-observability"
@@ -627,7 +635,7 @@ Resources:
       scanInterval: 60
       ToUpdate: 
         Fn::Base64: !Sub
-          - "${a}-${b}-${c}-${d}-${e}-${f}-${g}-${h}-${i}-${j}-${k}-${l}-${m}-${n}-${o}-${p}-${q}-${r}-${s}-${t}-${u}-${v}"
+          - "${a}-${b}-${c}-${d}-${e}-${f}-${g}-${h}-${i}-${j}-${k}-${l}-${m}-${n}-${o}-${p}-${q}-${r}-${s}-${t}-${u}-${v}-${w}"
           - a: !Ref Section2aAccountAlias
             b: !Ref Section3aInstallObservabilityApps
             c: !Ref Section4aCreateMetricsSourceOptions
@@ -644,12 +652,13 @@ Resources:
             n: !Ref Section7bLambdaCloudWatchLogsSourceUrl
             o: !Ref Section7cAutoSubscribeLogGroupsOptions
             p: !Ref Section7dAutoSubscribeLogGroupPattern
-            q: !Ref Section9aAutoEnableS3LogsELBResourcesOptions
-            r: !Ref Section9bELBCreateLogSource
-            s: !Ref Section9cELBLogsSourceUrl
-            t: !Ref Section9dELBS3LogsBucketName
-            u: !Ref Section10aAppInstallLocation
-            v: !FindInMap [CommonData, NestedTemplate, Version]
+            q: !Ref Section7eAutoSubscribeLogGroupByTags
+            r: !Ref Section9aAutoEnableS3LogsELBResourcesOptions
+            s: !Ref Section9bELBCreateLogSource
+            t: !Ref Section9cELBLogsSourceUrl
+            u: !Ref Section9dELBS3LogsBucketName
+            v: !Ref Section10aAppInstallLocation
+            w: !FindInMap [CommonData, NestedTemplate, Version]
 
   CreateCommonResources:
     Type: AWS::CloudFormation::Stack
@@ -729,6 +738,7 @@ Resources:
         AutoSubscribeDestinationARNType: !If [create_kf_logs_source, "Kinesis",  "Lambda" ]
         AutoSubscribeDestinationARN: !If [create_kf_logs_source, !GetAtt CreateCommonResources.Outputs.KinesisLogsDeliveryStreamARN, !If [ create_cw_logs_source, !GetAtt CreateCommonResources.Outputs.CloudWatchLambdaARN, "" ] ]
         AutoSubscribeLambdaLogGroupPattern: !Ref Section7dAutoSubscribeLogGroupPattern
+        AutoSubscribeLogGroupByTags: !Ref Section7eAutoSubscribeLogGroupByTags
         AutoSubscribeRoleArn: !If [create_kf_logs_source, !GetAtt CreateCommonResources.Outputs.KinesisLogsRoleARN,  "" ]
         ELBAutoEnableS3Logs: !If [auto_enable_s3_logs_elb, "Yes", "No"]
         ELBAutoEnableS3LogsOptions: !Ref Section9aAutoEnableS3LogsELBResourcesOptions