Better Utilization of Terraform Resource Keys

When managing multiple dissimilar resources in Terraform, best practices dictate that you use the for_each meta-argument. Furthermore, when needing to convert a list to a set or map for chaining, the advice given is usually of the form:

for_each = {
  for k, v in aws_subnet.all : k => v
}

This will work but unfortunately it becomes unwieldy the more you need to rely on these generated resources.

A motivating example

In one of Adrian Cantrill’s excellent courses on AWS, we are tasked with setting up a multi-tier subnet design:

(Courtesy of https://learn.cantrill.io)

We can set this up in Terraform like so:

locals {
  subnet_tiers = ["reserved", "db", "app", "web"]
  subnet_azs   = ["A", "B", "C"]
  subnet_tiers_azs = [
    for i, s in setproduct(local.subnet_tiers, local.subnet_azs) : {
      name   = s[0]
      az     = s[1]
      cidr   = cidrsubnet(var.vpc_cidr, 4, i)
    }
  ]
}

# It's not possible to directly use `subnet_tier_azs` here since it only
# accepts maps or a set of strings
resource "aws_subnet" "all" {
  for_each = {
    for k, v in local.subnet_tiers_azs : k => v
  }

  tags = {
    Name = "sn-${each.value.name}-${each.value.az}"
  }
  # ... etc.
}

This is serviceable¹ and the Name tag allows us to pick a single subnet. Let’s create a bastion in the web-A subnet:

resource "aws_instance" "bastion" {
  # ...
  # Get the subnet ID from the tag `sn-web-A`
  subnet_id = one([
    for v in aws_subnet.all : v.id if length(regexall(".*-web-A", v["tags"]["Name"])) > 0
  ])
}

Having to rely on one, length and regex functions to pick out the web-A subnet becomes error-prone and unergonomic.

A better way

There is a better way to name our resources upon creation. Instead of k => v in the for_each block, we can explicitly name the key of each subnets:

resource "aws_subnet" "all" {
  for_each = {
    for k, v in local.subnet_tiers_azs : "${v.name}-${v.az}" => v
  }
  # ...
}

Now, we can refer to any subnet we’d like by using the key we’d named. Let’s replace the subnet_id value above during creation of our bastion host:

subnet_id = aws_subnet.all["web-A"].id

The other upside of this method is that resource chaining forward will also use these keys as default. This helped give me a better understanding of how Terraform creates the final structure for resources made from for_each blocks.