AWS Shipper Terraform Module for CloudWatch Integration

Overview

Our latest AWS integration provides the easiest way to connect with Coralogix. By using a predefined Lambda function, you can seamlessly send AWS logs and events to your Coralogix subscription for detailed analysis, monitoring, and troubleshooting.

CloudWatch configuration

Name	Description	Type	Default	Required
log_groups	A comma-separated list of CloudWatch log group names to monitor. For example, (log-group1, log-group2, log-group3).	`list(string)`	n/a	yes
log_group_prefix	list of strings of log group prefixes. The code will use these prefixes to create permissions for the Lambda instead of creating for each log group permission it will use the prefix with a wild card to give the Lambda access for all of the log groups that start with these prefix. This parameter doesn't replace the `log_groups` parameter. For more information, refer to the Note below.	`list(string)`	n/a	no

!!! note\n The log_group variable will get a list of log groups and then add them to the Lambda as triggers, each log group will also add permission to the Lambda, in some cases when there are a lot of log groups this will cause an error because the code tries to create too many permissions for the Lambda (AWS have a limitation for the number of permission that you can have for a Lambda), and this is why we have the log_group_prefix parameter, this parameter will add only permission to the Lambda using a wildcard( * ). for example, in case I have the log groups: log1,log2,log3 instead that the code will create for each of the log group permission to trigger the shipper Lambda then you can set log_group_prefix = ["log"], and then it will create only 1 permission for all of the log groups to trigger the shipper Lambda, but you will still need to set log_groups = ["log1","log2","log3"]. When using this parameter, you will not be able to see the log groups as triggers for the Lambda. If you need to add multiple log groups to the Lambda function using regex, refer to our Lambda manager

Configuration examples

CloudWatch (default)

module "coralogix-shipper-cloudwatch" {
  source = "coralogix/aws/coralogix//modules/coralogix-aws-shipper"

  coralogix_region   = "EU1"
  integration_type   = "CloudWatch"
  api_key            = "XXXXXXXX-XXXX-XXXX-XXXX-XXXXXXX"
  application_name   = "cloudwatch-application"
  subsystem_name     = "cloudwatch-subsystem"
  log_groups         = ["log_gruop"]
}

CloudWatch with lambda-manager

In some cases, you will have a large number of log groups that you would like to monitor. In this case, instead of adding the log groups manually, you can use the lambda-manager to add a subscription to your coralogix-shipper lambda using a regex. Pay attention that the lambda-manager will also add new log groups to the integration automatically. For more information, please refer to the lambda-manager

module "coralogix-shipper-cloudwatch" {
  source = "coralogix/aws/coralogix//modules/coralogix-aws-shipper"

  coralogix_region   = "EU1"
  integration_type   = "CloudWatch"
  api_key            = "XXXXXXXX-XXXX-XXXX-XXXX-XXXXXXX"
  application_name   = "cloudwatch-application"
  subsystem_name     = "cloudwatch-subsystem"
  log_groups         = ["log_gruop"]
}

module "coralogix-lambda-manager" {
  source = "coralogix/aws/coralogix//modules/lambda-manager"

  regex_pattern                = "log_groups_name*"
  destination_arn              = module.coralogix-shipper-cloudwatch.lambda_function_arn[0]
  destination_type             = "lambda"
  scan_old_loggroups           = true
  log_group_permissions_prefix = ["log_groups_name"]
}

Important note: the log_group_permissions_prefix is optional, and will ONLY add permissions to the lambda and will not add the subscription. For more information about the variables, please refer to the lambda-manager README

Name	Description	Type	Default	Required
coralogix_region	The Coralogix location region, available options: [`EU1`, `EU2`, `AP1`, `AP2`, `AP3`, `US1`, `US2`, `Custom`]	`string`	n/a	yes
custom_domain	If using a custom domain name for your private cluster, Coralogix will send telemetry from the specified address (e.g. custom.coralogix.com). There is no need to add `ingress.` to the domain.	`string`	n/a	no
integration_type	The AWS service to integrate with Coralogix. Possible values: S3, CloudTrail, VpcFlow, CloudWatch, S3Csv, SNS, SQS, Kinesis, CloudFront, MSK, Kafka, EcrScan.	`string`	`S3`	yes
api_key	The Coralogix Send Your Data - API key validates your authenticity. This value can be a direct Coralogix API key or an AWS secret manager ARN containing the API key.	`string`	n/a	yes
store_api_key_in_secrets_manager	Enable this to store your API key securely. Otherwise, it will remain exposed in plain text as an environment variable in the Lambda function console.	bool	true	no
application_name	The name of your application. For a dynamic value, use `$.my_log.field`. This option is not supported since version `1.1.0` for the source code	string	n\a	yes
subsystem_name	The name of your subsystem. For a dynamic value, use `$.my_log.field` for CloudWatch log group leave empty. This option is not supported since version `1.1.0` for the source code	string	n\a	yes

For additional configuration options, please refer to our main Terraform documentation.

Wiz

Next Kafka