Try Catch Finally

How to send a notification when a CodePipeline Blue/Green ECS Deployment has switched

Problem #

Amazon's ECS service is an excellent way to run containerised applications 24/7 in production. CodeDeploy makes it simple to configure a Blue/Green deployment strategy for your ECS cluster to enable zero-downtime deployments with the added security of an easy roll back mechanism that can be called into action at any point within a user configurable grace period.

The issue with this is that if you choose to orchestrate everything with CodePipeline, it will only notify you when CodeDeploy starts to initiate the Blue/Green deployment process, and then if successful, when it has completely finished. Finished in this case being after the grace period has expired and the previous task set has been terminated (meaning you can no longer easily roll back). This might be all well and good, but I wanted to receive a Slack notification at the exact moment my load balancer had cut over to the new target group and the latest version of my application was serving traffic. This was important for me to know so that I could automate the process of invalidating the CloudFront Distribution's cache at the right moment and make the most of the available roll-back window.

How to solve it #

The CodeDeploy Deployment Group resource has a trigger_configuration block which is documented here.

See below for a Terraform snippet of the relevant IaC for how to implement this. I've commented out the events which I find less useful (mostly because I'm already notified of Start/Ready/Success style events which are emitted by the parent CodePipeline).

resource "aws_codedeploy_deployment_group" "this" {
	...
	trigger_configuration {
	  // (Only trigger useful event notifications)
    trigger_events = [
	    #"DeploymentStart",
	    #"DeploymentSuccess",
	    "DeploymentFailure",
	    "DeploymentStop",
	    "DeploymentRollback",
	    #"DeploymentReady",
	    #"InstanceStart",
	    "InstanceSuccess",
	    "InstanceFailure",
	    #"InstanceReady"
	  ]
	  trigger_name       = "${var.service_name}--DG-triggers"
	  trigger_target_arn = var.notifications--sns_topic_arn
	}
}

As you can see above, the target of the CodeDeploy Deployment Group "Trigger Events" is an SNS Topic. The definition of which should be something like this:

resource "aws_sns_topic" "ecs" {
  name         = "ecs"
  display_name = "SlackECSDeploymentNotifications"
}

data "aws_iam_policy_document" "ecs" {
  statement {
    effect = "Allow"
    actions = [
      "sns:Publish"
    ]
    principals {
      type        = "AWS"
      identifiers = ["*"]
    }
    resources = [
      aws_sns_topic.ecs.arn
    ]
    condition {
      test     = "StringEquals"
      variable = "AWS:SourceOwner"
      values   = [data.aws_caller_identity.current.account_id]
    }
  }
}

resource "aws_sns_topic_policy" "ecs" {
  arn    = aws_sns_topic.ecs.arn
  policy = data.aws_iam_policy_document.ecs.json
}

With the CodeDeploy Deployment Group now configured to send the specific, granular ECS deployment events we are interested in to an SNS Topic, we can now subscribe to them however we choose. I mentioned sending notifications to Slack, and invalidating a CloudFront Distribution's cache, so have included two example Golang Lambda Functions which I've written to achieve these two goals.

Lambda: Send ECS Trigger Events To Slack

Lambda: On ECS Deploy Success Invalidate CloudFront Cache