How to use for_each in Terraform

Ever found yourself copying and pasting the same Terraform resource block over and over again, just changing a few values? If you’re nodding your head right now, you’re about to discover your new favorite Terraform feature. The for_each meta-argument is like having a smart assistant that creates multiple resources for you based on a collection of items – and trust me, once you start using it, there’s no going back.

What is for_each in Terraform?

Think of for_each as your resource multiplication machine. Instead of writing ten separate EC2 instance blocks, you can write one and let Terraform do the heavy lifting. It’s a meta-argument that allows you to create multiple instances of a resource based on a map or set of strings. Each instance gets its own unique identifier, making them independent and manageable.

The beauty of for_each lies in its simplicity and power. You provide it with a collection, and it iterates through each item, creating a separate resource instance for each one. It’s like telling Terraform, “Hey, take this blueprint and create one of these for every item in my list, but make each one unique.”

Why use for_each instead of count?

You might be wondering, “Wait, doesn’t count do something similar?” Well, yes and no. While both can create multiple resources, they work fundamentally differently, and choosing the right one can save you from future headaches.

The count meta-argument creates resources based on a number, giving them array-like indices (0, 1, 2, etc.). This sounds great until you need to remove the second item from your list. Suddenly, what was index 2 becomes index 1, and Terraform wants to destroy and recreate resources unnecessarily. It’s like removing a book from the middle of a numbered bookshelf – everything after it needs to be renumbered.

With for_each, each resource gets a unique key instead of a numeric index. Remove an item? No problem – only that specific resource gets deleted. Add one in the middle? It slides right in without affecting the others. This makes your infrastructure more predictable and reduces those nerve-wracking moments during terraform apply.

Basic syntax and structure

Let’s dive into how for_each actually looks in practice. The syntax is refreshingly straightforward:

resource "aws_instance" "example" {
  for_each = var.instance_configs

  ami           = each.value.ami
  instance_type = each.value.type
  tags = {
    Name = each.key
  }
}

The magic happens with the each object. When Terraform processes the for_each loop, it provides two special variables:

• each.key: The map key or set member from your collection
• each.value: The corresponding value (for maps) or the same as the key (for sets)

You can use for_each with two types of collections: maps and sets. Maps give you key-value pairs, perfect when you need to associate configurations with names. Sets are simpler – just a collection of unique strings where the key and value are the same.

Real-world examples with AWS resources

Let’s get our hands dirty with some practical examples that you can actually use in your projects.

Creating multiple S3 buckets with different configurations

Here’s a scenario: you need to create several S3 buckets for different environments, each with its own settings:

variable "s3_buckets" {
  default = {
    dev = {
      versioning = true
      lifecycle_days = 30
    }
    staging = {
      versioning = true
      lifecycle_days = 60
    }
    prod = {
      versioning = true
      lifecycle_days = 90
    }
  }
}

resource "aws_s3_bucket" "environment_buckets" {
  for_each = var.s3_buckets

  bucket = "my-app-${each.key}-bucket"

  tags = {
    Environment = each.key
    ManagedBy   = "Terraform"
  }
}

resource "aws_s3_bucket_versioning" "versioning" {
  for_each = var.s3_buckets

  bucket = aws_s3_bucket.environment_buckets[each.key].id

  versioning_configuration {
    status = each.value.versioning ? "Enabled" : "Disabled"
  }
}

Managing IAM users and policies

Need to create multiple IAM users with specific policies? for_each makes it a breeze:

locals {
  developers = {
    "alice" = "frontend"
    "bob"   = "backend"
    "carol" = "devops"
  }
}

resource "aws_iam_user" "developers" {
  for_each = local.developers

  name = each.key
  path = "/developers/"

  tags = {
    Team = each.value
  }
}

resource "aws_iam_user_login_profile" "developer_logins" {
  for_each = local.developers
  user                    = aws_iam_user.developers[each.key].name
  password_reset_required = true
}

Working with maps and sets

The choice between maps and sets depends on your use case. Sets are perfect when you just need a list of items without associated configurations:

variable "availability_zones" {
  type    = set(string)
  default = ["us-east-1a", "us-east-1b", "us-east-1c"]
}

resource "aws_subnet" "public" {
  for_each = var.availability_zones

  vpc_id            = aws_vpc.main.id
  availability_zone = each.value
  cidr_block        = cidrsubnet(aws_vpc.main.cidr_block, 8, index(tolist(var.availability_zones), each.value))

  tags = {
    Name = "Public-${each.value}"
  }
}

Maps shine when you need to associate configurations with identifiers:

variable "ec2_instances" {
  type = map(object({
    instance_type = string
    volume_size   = number
  }))
  default = {
    web_server = {
      instance_type = "t3.medium"
      volume_size   = 50
    }
    api_server = {
      instance_type = "t3.large"
      volume_size   = 100
    }
  }
}

resource "aws_instance" "servers" {
  for_each = var.ec2_instances

  ami           = data.aws_ami.latest.id
  instance_type = each.value.instance_type

  root_block_device {
    volume_size = each.value.volume_size
  }

  tags = {
    Name = each.key
    Type = "Application Server"
  }
}

Advanced techniques and best practices

Ready to level up your for_each game? Let’s explore some advanced patterns that’ll make your configurations even more powerful.

Conditional resource creation

Sometimes you want to create resources only when certain conditions are met. You can combine for_each with filtering:

locals {
  production_buckets = {
    for name, config in var.all_buckets :
    name => config if config.environment == "production"
  }
}

resource "aws_s3_bucket" "production_only" {
  for_each = local.production_buckets
  bucket = each.key
  # ... other configurations
}

Nested loops and complex data structures

When dealing with complex relationships, you might need to flatten nested structures:

locals {
  # Flatten user-role combinations
  user_roles = flatten([
    for user, roles in var.user_permissions : [
      for role in roles : {
        user = user
        role = role
      }
    ]
  ])
  # Convert to map for for_each
  user_role_map = {
    for item in local.user_roles :
    "${item.user}-${item.role}" => item
  }
}

resource "aws_iam_user_policy_attachment" "user_policies" {
  for_each = local.user_role_map
  user       = each.value.user
  policy_arn = "arn:aws:iam::aws:policy/${each.value.role}"
}

Dynamic block generation within resources

You can even use for_each inside resource blocks with dynamic blocks:

resource "aws_security_group" "example" {
  name = "web-sg"

  dynamic "ingress" {
    for_each = var.ingress_rules

    content {
      from_port   = ingress.value.from_port
      to_port     = ingress.value.to_port
      protocol    = ingress.value.protocol
      cidr_blocks = ingress.value.cidr_blocks
    }
  }
}

Common pitfalls and how to avoid them

Let me save you from some headaches I’ve encountered (and seen others encounter) when working with for_each.

The computed value trap

One of the most frustrating errors you’ll see is: “The for_each value depends on resource attributes that cannot be determined until apply.” This happens when you try to use values that Terraform doesn’t know until it actually creates the resources:

# This will fail!
resource "aws_instance" "bad_example" {
  for_each = aws_instance.other.tags  # Can't use this!
  # ...
}

# Solution: Use data sources or separate the configuration
data "aws_instances" "existing" {
  # ... filters
}

resource "aws_instance" "good_example" {
  for_each = data.aws_instances.existing.tags
  # ...
}

Type conversion issues

Terraform is strict about types. If you’re getting type errors, make sure you’re converting your data correctly:

# Convert list to set
for_each = toset(var.my_list)

# Convert set to list for indexing
cidr_block = cidrsubnet(var.vpc_cidr, 8, index(tolist(var.availability_zones), each.value))

Resource dependencies

When resources created with for_each depend on each other, reference them correctly:

resource "aws_eip" "example" {
  for_each = aws_instance.servers
  instance = each.value.id  # Reference the specific instance
  domain   = "vpc"
}

Migrating from count to for_each

If you’ve got existing infrastructure using count, migrating to for_each requires careful planning. Here’s a strategic approach:

1. First, identify resources using count that would benefit from for_each
2. Create the new resources with for_each alongside the existing ones
3. Import the existing resources into the new configuration
4. Remove the old count-based configuration

Here’s an example migration:

# Old configuration with count
# resource "aws_instance" "old_servers" {
#   count = length(var.server_names)
#   name  = var.server_names[count.index]
# }

# New configuration with for_each
resource "aws_instance" "new_servers" {
  for_each = toset(var.server_names)
  name     = each.value
}

# After importing: terraform import aws_instance.new_servers["web-1"] i-1234567890abcdef0

Performance considerations

While for_each is powerful, it’s not always the best choice for every situation. When you’re creating hundreds or thousands of similar resources, consider these points:

• State file size: Each resource created with for_each adds to your state file. Large state files can slow down operations.
• Plan/Apply time: More resources mean longer plan and apply times.
• Alternative approaches: For large-scale deployments, consider using modules or even separate Terraform configurations.

If you’re creating more than 100 similar resources, ask yourself if there’s a better architectural approach. Could you use auto-scaling groups? Would a module pattern work better?

Debugging for_each configurations

When things go wrong (and they will), here are your best debugging tools:

# Use outputs to inspect your data
output "debug_keys" {
  value = [for k, v in var.my_map : k]
}

output "debug_values" {
  value = [for k, v in var.my_map : v]
}

# Use local values to test transformations
locals {
  debug_transformation = {
    for name, config in var.complex_input :
    name => config if can(config.some_field)
  }
}

# The terraform console is your friend
# Run: terraform console
# Then: var.my_map
#       local.debug_transformation

Conclusion

The for_each meta-argument is more than just a convenience feature – it’s a fundamental tool that transforms how you write and maintain Terraform configurations. By treating resources as unique entities with meaningful identifiers rather than numbered items in a list, you create infrastructure code that’s more maintainable, predictable, and aligned with how we think about cloud resources.

Whether you’re managing a handful of S3 buckets or orchestrating complex multi-tier applications, mastering for_each will make your Terraform code cleaner, more efficient, and easier to understand. Start small, experiment with the examples provided, and gradually incorporate these patterns into your infrastructure as code. Remember, the goal isn’t just to make Terraform work – it’s to make it work elegantly and maintainably for you and your team.

FAQs

Can I use for_each with modules in Terraform?

Absolutely! Using for_each with modules is one of the most powerful patterns in Terraform. You can create multiple module instances just like with resources. Simply add for_each to your module block and reference values using each.key and each.value. This is particularly useful when deploying similar infrastructure across multiple environments or regions.

What’s the difference between for_each and dynamic blocks?

While both iterate over collections, they serve different purposes. The for_each meta-argument creates multiple resource or module instances, each with its own address in the state file. Dynamic blocks, on the other hand, create multiple configuration blocks within a single resource. Use for_each when you need separate resources and dynamic blocks when you need repeated configuration within one resource.

How do I reference resources created with for_each in other parts of my configuration?

Resources created with for_each are referenced using bracket notation with the key: resource_type.resource_name[each_key]. For example, aws_instance.servers["web-1"]. If you need to reference all instances, you can use values() or keys() functions: values(aws_instance.servers)[*].id gets all instance IDs.

Can I use both count and for_each on the same resource?

No, Terraform doesn’t allow using both count and for_each on the same resource or module block. You must choose one or the other. If you need complex iteration patterns, consider using for_each with transformed data structures using Terraform’s expression language to achieve the desired result.

What happens to the Terraform state when I remove an item from my for_each collection?

When you remove an item from your for_each collection, Terraform will only destroy the resource associated with that specific key. Other resources remain untouched, and their state addresses don’t change. This is one of the main advantages of for_each over count – it provides stable resource addressing that doesn’t shift when items are added or removed from your collection.