Skip to content

README.md

Latest commit

 

History

History
164 lines (141 loc) · 8.1 KB

README.md

File metadata and controls

164 lines (141 loc) · 8.1 KB
Raw

Logging Destination: Datadog

This addon configures AWS Kinesis Firehose to send Fleet's osquery logs to Datadog. It creates:

  1. Kinesis Firehose delivery streams for each log type (results, status, and audit)
  2. A single S3 bucket for storing all failed delivery attempts
  3. IAM roles and policies for the Firehose streams to access the S3 bucket
  4. An IAM policy for Fleet to access the Firehose streams

How to use

module "datadog-logging" {
  source = "github.com/fleetdm/fleet-terraform//addons/logging-destination-datadog?ref=tf-mod-addon-datadog-logging-v1.1.2"

  datadog_api_key = "your-datadog-api-key"

  # Optional: customize other settings
  # datadog_url = "https://custom-datadog-endpoint.com"
  # s3_bucket_config = {
  #   name_prefix = "custom-bucket-prefix"
  #   expires_days = 7
  # }
  # log_destinations = {
  #   results = {
  #     name = "custom-results-stream-name"
  #     buffering_size = 1
  #     buffering_interval = 60
  #     s3_buffering_size = 10
  #     s3_buffering_interval = 400
  #     common_attributes = [
  #       {
  #         name  = "service"
  #         value = "fleet-osquery-results"
  #       },
  #       {
  #         name  = "environment"
  #         value = "production"
  #       }
  #     ]
  #   },
  #   status = {
  #     name = "custom-status-stream-name"
  #     buffering_size = 1
  #     buffering_interval = 60
  #     s3_buffering_size = 10
  #     s3_buffering_interval = 400
  #     common_attributes = [
  #       {
  #         name  = "service"
  #         value = "fleet-osquery-status"
  #       },
  #       {
  #         name  = "environment"
  #         value = "production"
  #       }
  #     ]
  #   },
  #   audit = {
  #     name = "custom-audit-stream-name"
  #     buffering_size = 1
  #     buffering_interval = 60
  #     s3_buffering_size = 10
  #     s3_buffering_interval = 400
  #     common_attributes = [
  #       {
  #         name  = "service"
  #         value = "fleet-audit"
  #       },
  #       {
  #         name  = "environment"
  #         value = "production"
  #       }
  #     ]
  #   }
  # }
  # compression_format = "GZIP"
}

Then you can use the module's outputs in your Fleet configuration:

module "fleet" {
  source = "github.com/fleetdm/fleet-terraform?depth=1&ref=tf-mod-root-v1.26.1"
  certificate_arn = module.acm.acm_certificate_arn

  vpc = {
    name = local.vpc_name
    # azs = ["us-east-2a", "us-east-2b", "us-east-2c"]
  }

  fleet_config = {
    image = "fleetdm/fleet:v4.70.1"
    autoscaling = {
      min_capacity = 2
      max_capacity = 5
    }
    mem = 4096
    cpu = 512
    extra_environment_variables = merge(
      local.fleet_environment_variables,
      # Uncomment to enable Datadog logging
      module.datadog-logging.fleet_extra_environment_variables
    )
    extra_iam_policies = concat(
      # Uncomment to enable Datadog logging
      module.datadog-logging.fleet_extra_iam_policies,
    )
  }

  # ... other Fleet configuration ...
}

Requirements

No requirements.

Providers

Name Version
aws 6.41.0

Modules

No modules.

Resources

Name Type
aws_iam_policy.firehose resource
aws_iam_policy.firehose-logging resource
aws_iam_role.firehose resource
aws_iam_role_policy_attachment.firehose resource
aws_kinesis_firehose_delivery_stream.datadog resource
aws_s3_bucket.datadog-failure resource
aws_s3_bucket_lifecycle_configuration.datadog-failure resource
aws_s3_bucket_public_access_block.datadog-failure resource
aws_s3_bucket_server_side_encryption_configuration.datadog-failure resource
aws_iam_policy_document.firehose-logging data source
aws_iam_policy_document.firehose_policy data source
aws_iam_policy_document.osquery_firehose_assume_role data source
aws_region.current data source

Inputs

Name Description Type Default Required
compression_format Compression format for the Firehose delivery stream string "UNCOMPRESSED" no
datadog_api_key Datadog API key for authentication string n/a yes
datadog_url Datadog HTTP API endpoint URL string n/a yes
log_destinations A map of configurations for Datadog Firehose delivery streams. 
map(object({
    name                  = string
    buffering_size        = number
    buffering_interval    = number
    s3_buffering_size     = number
    s3_buffering_interval = number
    content_encoding      = string
    common_attributes = optional(list(object({
      name  = string
      value = string
    })), [])
  }))
 
{
  "audit": {
    "buffering_interval": 60,
    "buffering_size": 2,
    "common_attributes": [],
    "content_encoding": "NONE",
    "name": "fleet-audit-datadog",
    "s3_buffering_interval": 400,
    "s3_buffering_size": 10
  },
  "results": {
    "buffering_interval": 60,
    "buffering_size": 2,
    "common_attributes": [],
    "content_encoding": "NONE",
    "name": "fleet-osquery-results-datadog",
    "s3_buffering_interval": 400,
    "s3_buffering_size": 10
  },
  "status": {
    "buffering_interval": 60,
    "buffering_size": 2,
    "common_attributes": [],
    "content_encoding": "NONE",
    "name": "fleet-osquery-status-datadog",
    "s3_buffering_interval": 400,
    "s3_buffering_size": 10
  }
}
 no
s3_bucket_config Configuration for the S3 bucket used to store failed Datadog delivery attempts 
object({
    name_prefix  = optional(string, "fleet-datadog-failure")
    expires_days = optional(number, 1)
  })
 
{
  "expires_days": 1,
  "name_prefix": "fleet-datadog-failure"
}
 no

Outputs

Name Description
fleet_extra_environment_variables Environment variables to configure Fleet to use Datadog logging via Firehose
fleet_extra_iam_policies IAM policies required for Fleet to log to Datadog via Firehose
fleet_s3_datadog_failure_config S3 bucket details - datadog-failure