Improve for_each flexibility using optional()

Introduction

In my previous blog post about Decluttering Terraform code using for_each, I discussed how using the for_each argument can lead to cleaner, dynamic, and more maintainable terraform code. Now, let’s explore how the optional() modifier can complement this approach and further enhance the flexibility of our terraform code.

The optional() modifier in Terraform is a powerful tool that allows you to mark attributes as optional in an object() type constraint. It’s particularly useful when you want to provide default values for certain attributes but also want to make those attributes optional and allow them to be overridden when needed. In conjunction with the map(object()) type constraint and for_each meta-argument, the optional() modifier will allow you to make certain attributes within the object optional for each resource instance created.

Benefits

Using optional() in conjunction with for_each can provide several benefits:

Flexibility in Configuration

It allows you to make certain attributes or resource instances optional in your configuration. This flexibility is particularly useful when you need to create or manage resources conditionally based on specific circumstances or configurations.

Reduced Boilerplate Code and Abstraction

When you have optional configurations that might not be required in every scenario, using optional() can help you avoid duplicating resource blocks, and writing complex conditionals to handle whether an argument should be set or not. This can lead to cleaner and more maintainable code.

In most cases, you end up having to resort to using abstraction layers which can increase the complexity of projects because now you have more than 1 tool to learn, troubleshoot, manage, lifecycle, and teach other people. The more you abstract, the more you have to understand the underlying system to make sense of it.

Improved Code Reusability

When you need to create similar resources with slight configuration variations, you can use the for_each along with the optional() modifier to manage these variations without duplicating resource blocks. This promotes code reuse and reduces the likelihood of errors caused by copy-pasting similar resource blocks.

Simplified Logic and Improved Readability

The use of optional() can simplify your Terraform code by reducing the need for complex conditional statements to control resource arguments. This can lead to cleaner and more readable configurations. Coupled with for_each, you can keep your terraform code files more concise.

Prepwork

For the sake of continuity let’s start by reusing the final code we had from Decluttering Terraform code using for_each as this post is meant to build on top of the previous post as part of the Terraform for_each Series. Here are the main.tf file contents.

# in main.tf
variable "vpc_name" {
  type    = string
  default = null
}

variable "vpc_cidr_block" {
  type    = string
  default = null
}

variable "snets_configuration" {
  type    = map(object({
    cidr_block = string
  }))
  default = {}
}

resource "aws_vpc" "this" {
  cidr_block = var.vpc_cidr_block

  tags = {
    Name = var.vpc_name
  }
}

resource "aws_subnet" "this" {
  for_each   = var.snets_configuration
  vpc_id     = aws_vpc.this.id
  cidr_block = each.value.cidr_block

  tags = {
    Name = each.key
  }
}

And here are the terraform.tfvars configuration file contents.

# in terraform.tfvars
vpc_name       = "my-demo-vpc"
vpc_cidr_block = "10.0.0.0/16"

snets_configuration = {
  my-demo1-snet = {
    cidr_block = "10.0.1.0/24"
  }
  my-demo2-snet = {
    cidr_block = "10.0.2.0/24"
  }
  my-demo3-snet = {
    cidr_block = "10.0.3.0/24"
  }
}

You should be able to just apply this and expect the same result as below. Now go ahead and proceed by applying the changes.

Scenario 1: Creating Resources With Optional Attributes

Sweet! Now we’ve got something to work and iterate against! Let’s start by adding the ability to OPTIONALLY declare which availability zones our subnets will go into because by default subnets get assigned to availability zones at random. We can achieve this by simply adding the availability_zone_id argument into our main.tf file’s aws_subnet.this resource block and we will also have to add another string attribute to our snets_configuration variable block that will represent the ID of the availability zone in which the subnet will be created.

# in main.tf
variable "vpc_name" {
  type    = string
  default = null
}

variable "vpc_cidr_block" {
  type    = string
  default = null
}

variable "snets_configuration" {
  type     = map(object({
    cidr_block           = string
    availability_zone_id = optional(string)
  }))
  default = {}
}

resource "aws_vpc" "this" {
  cidr_block = var.vpc_cidr_block

  tags = {
    Name = var.vpc_name
  }
}

resource "aws_subnet" "this" {
  for_each             = var.snets_configuration
  vpc_id               = aws_vpc.this.id
  cidr_block           = each.value.cidr_block
  availability_zone_id = each.value.availability_zone_id

  tags = {
    Name = each.key
  }
}

Now that we’ve modified our code to support the ability to put our subnets inside availability zones, we will then have to update our configuration/data in our terraform.tfvars so we can start using this new feature. Let’s start by assigning our my-demo1-snet to the apse2-az1 zone.

# in terraform.tfvars
vpc_name        = "my-demo-vpc"
vpc_cidr_block  = "10.0.0.0/16"

snets_configuration = {
  my-demo1-snet = {
    cidr_block           = "10.0.1.0/24"
    availability_zone_id = "apse2-az1"
  }
  my-demo2-snet = {
    cidr_block = "10.0.2.0/24"
  }
  my-demo3-snet = {
    cidr_block = "10.0.3.0/24"
  }
}

Let’s test this new code by running terraform apply.

With the new code changes we have given our subnet resource the ability to optionally declare an availability_zone_id. Our subnet resource block can now dynamically create multiple subnets based on a flexible configuration approach.

Scenario 2: Handling Conflicting Optional Arguments

Sometimes you might have to transition to using new arguments due to deprecations or simply to make your Terraform code flexible enough to cater to various scenarios. Using optional() together with for_each can be helpful in such situations. Continuing from our example above, let’s attempt to add support for optionally setting the availability_zone argument. We will do this to simulate scenarios in which we want to accommodate multiple ways of declaring availability zones within our code. Let’s begin by making modifications to our main.tf.

# in main.tf
variable "vpc_name" {
  type    = string
  default = null
}

variable "vpc_cidr_block" {
  type    = string
  default = null
}

variable "snets_configuration" {
  type     = map(object({
    cidr_block           = string
    availability_zone    = optional(string)
    availability_zone_id = optional(string)
  }))
  default = {}
}

resource "aws_vpc" "this" {
  cidr_block = var.vpc_cidr_block

  tags = {
    Name = var.vpc_name
  }
}

resource "aws_subnet" "this" {
  for_each             = var.snets_configuration
  vpc_id               = aws_vpc.this.id
  cidr_block           = each.value.cidr_block
  availability_zone    = each.value.availability_zone
  availability_zone_id = each.value.availability_zone_id

  tags = {
    Name = each.key
  }
}

To summarize, we’ve just made the following changes to our code;

1. We’ve added the availability_zone attribute to the snets_configuration variable block as an optional string representing the name of the availability zone where the subnet will be placed.

2. We’ve added the availability_zone argument to the aws_subnet.this resource block and is set to source the value from each.value.availability_zone, allowing for the option to specify the name of the availability zone.

Now let’s make changes to our input variables by modifying our terraform.tfvars. First, let’s update our my-demo1-snet subnet to use availability_zone = "ap-southeast-2b" instead of availability_zone_id = "apse2-az1". Let’s also, make our my-demo2-snet subnet use availability_zone = "ap-southeast-2a" and my-demo3-snet to use availability_zone_id = "apse2-az3"

# in terraform.tfvars
vpc_name        = "my-demo-vpc"
vpc_cidr_block  = "10.0.0.0/16"

snets_configuration = {
  my-demo1-snet = {
    cidr_block        = "10.0.1.0/24"
    availability_zone = "ap-southeast-2b"
  }
  my-demo2-snet = {
    cidr_block        = "10.0.2.0/24"
    availability_zone = "ap-southeast-2a"
  }
  my-demo3-snet = {
    cidr_block           = "10.0.3.0/24"
    availability_zone_id = "apse2-az3"
  }
}

Let’s run terraform apply.

Great! We’ve implemented our changes, and only my-demo2-snet was recreated. This recreation occurred because our input value effectively shifted it from ap-southeast-2b to ap-southeast-2a. Our code has now achieved the level of flexibility necessary to enable users to adopt their preferred methods of defining availability zones. Moreover, this flexibility extends to facilitating smooth migration between different availability zone arguments.

Scenario 3: Simplified Optional Dynamic Blocks

By default, an optional attribute with no default value set will always implicitly default to null. The ability to set a default value in an optional attribute can significantly simplify the generation of dynamic blocks within our resources, adding another layer of flexibility to our code.

In many cases, when orchestrating resource provisioning, it makes sense for certain resources to be provisioned together, either due to technical or functional requirements or because of your infrastructure’s logical or hierarchical layout. Embedding these resource relationships in your Terraform code or module is sensible. With the use of for_each and optional(), you can not only establish these resource relationships but also greatly simplify your code’s logic while maintaining a high level of flexibility.

With these considerations in mind, let’s modify our main.tf so that every time we create a subnet, we also create and associate a Network ACL with it. This approach aligns with the concept of baking in relationships. Moreover, we’ll add the option to optionally include ingress rules for the Network ACL created alongside the subnet. To demonstrate the use of default values with optional(), we’ll make some of the required attributes optional.

# in main.tf
variable "vpc_name" {
  type    = string
  default = null
}

variable "vpc_cidr_block" {
  type    = string
  default = null
}

variable "snets_configuration" {
  type = map(object({
    cidr_block           = string
    availability_zone    = optional(string)
    availability_zone_id = optional(string)
    ingress_rules = optional(map(object({
      action      = optional(string, "allow")
      protocol    = string
      cidr_block  = string
      from_port   = number
      to_port     = number
    })), {})
  }))
  default = {}
}

resource "aws_vpc" "this" {
  cidr_block = var.vpc_cidr_block

  tags = {
    Name = var.vpc_name
  }
}

resource "aws_subnet" "this" {
  for_each             = var.snets_configuration
  vpc_id               = aws_vpc.this.id
  cidr_block           = each.value.cidr_block
  availability_zone    = each.value.availability_zone
  availability_zone_id = each.value.availability_zone_id

  tags = {
    Name = each.key
  }
}

resource "aws_network_acl" "this" {
  for_each   = var.snets_configuration
  vpc_id     = aws_vpc.this.id
  subnet_ids = [aws_subnet.this[each.key].id]

  dynamic "ingress" {
    for_each = each.value.ingress_rules
    content {
      rule_no    = ingress.key
      action     = ingress.value.action
      protocol   = ingress.value.protocol
      cidr_block = ingress.value.cidr_block
      from_port  = ingress.value.from_port
      to_port    = ingress.value.to_port
    }
  }

  tags = {
    Name = format("%s-acl", each.key)
  }
}

First, let’s have a look at our snets_configuration variable block, this is used to configure subnets along with optional ingress rules. Let’s go and break down the changes we made here:

1. The ingress_rules attribute is optional and allows us to define a map of ingress rules for the subnet’s network ACL. Each key in the map represents a rule identifier (which in this case we will use the rule identifier as the rule_no later on), and each rule follows the structure specified in the inner map(object(...)) type.

2. All attributes defined in the map(object(...)) are required attributes, except for the action attribute, which we have designated as optional with a default value of "allow". This means that if you provide input to the aws_network_acl resource without defining an action attribute, even though the action argument is required when defining an ingress block in the aws_network_acl resource, the resource will automatically set the action to its default value of "allow". This capability allows you to incorporate assumptions into your Terraform code, making it more flexible and reducing the verbosity of your input variables.

3. The second {} in the ingress_rules type specifies the default value for the map of ingress rules. It means that if no ingress rules are provided, an empty map will be used. Because we are setting up an empty map as the default value here we will be able to simplify the way we iterate against this map later on when we reference this inside our dynamic block’s for_each.

Next, let’s have a look at the aws_network_acl.this resource block we just added, this resource block defines the creation of an AWS network ACLs (Access Control Lists) based on the provided configuration in the var.snets_configuration variable and associate it to the respective subnets. Let’s get a breakdown of this newly created resource block:

1. The for_each = var.snets_configuration argument iterates over each element in the var.snets_configuration map. It creates a separate instance of the resource for each element in the map.

2. subnet_ids = [aws_subnet.this[each.key].id] is an array of subnet IDs where this network ACL will be applied. In this case, it’s set to the subnet ID corresponding to the current element’s key in the map. This ensures that the created network ACL will always be associated/assigned to the respective subnet.

3. The ingress dynamic block is used to define ingress rules for the created network ACL. It iterates over each defined ingress rule within the ingress_rules attribute of the current element in the map.

   • The for_each = each.value.ingress_rules argument inside the ingress dynamic block iterates over each element inside the var.snets_configuration.ingress_rules map. Because the ingress_rules’ default value is {} - thanks to optional(), we were able to simplify this line instead of using for_each = each.value.ingress_rules != null ? each.value.ingress_rules : {}.

     

   • rule_no represents the key (rule number) of the ingress rule in the map.

   • Other attributes like action, protocol, cidr_block, from_port, and to_port are extracted from the values of the ingress rule.

To summarize, this code dynamically creates AWS network ACLs based on the provided configuration in var.snets_configuration. It associates the ACLs with the created subnet with the same key, and for each ACL, it dynamically adds ingress rules.

Now let’s try out our new code by modifying input values in our terraform.tfvars.

# in terraform.tfvars
vpc_name       = "my-demo-vpc"
vpc_cidr_block = "10.0.0.0/16"

snets_configuration = {
  my-demo1-snet = {
    cidr_block        = "10.0.1.0/24"
    availability_zone = "ap-southeast-2b"
    ingress_rules = {
      100 = {
        action      = "allow"
        protocol    = "tcp"
        cidr_block  = "10.3.0.0/18"
        from_port   = 80
        to_port     = 80
      },
      200 = {
        protocol    = "tcp"
        cidr_block  = "10.3.0.0/18"
        from_port   = 443
        to_port     = 443
      },
    }
  }
  my-demo2-snet = {
    cidr_block        = "10.0.2.0/24"
    availability_zone = "ap-southeast-2a"
  }
  my-demo3-snet = {
    cidr_block           = "10.0.3.0/24"
    availability_zone_id = "apse2-az3"
  }
}

Given our code changes before, this should automatically create network ACLs per subnet however the first subnet (my-demo1-snet) has additional ingress rules defined in its ingress_rules, the expectation is that it should just create those rules for the first ACL (my-demo1-snet-acl) associated with the first subnet (my-demo1-snet). Now go ahead and apply this change but before hitting yes review the plan and verify if it is doing things as expected.

Notice how my-demo1-snet is created with rule_no = 200 and the action argument defaults to "allow"? This behaviour is due to our default value setting. It allows us to simplify assumptions about our infrastructure and reduce verbosity in input configuration declarations.

Now go ahead and try making changes to our terraform.tfvars and experience a more data-driven and flexible way of making changes to your infrastructure:

1. Try adding more ingress_rules to my-demo1-snet.

2. Try adding a rule without the action attribute defined to my-demo1-snet.

3. Try adding ingress_rules to my-demo2-snet and my-demo3-snet.

4. Try removing my-demo3-snet.

5. Try removing one ingress_rules from my-demo1-snet.

6. Consider removing all ingress_rules or the attribute itself from my-demo1-snet. Did nothing happen? This issue arises from how providers have implemented nested blocks for certain resources like aws_network_acl. A relevant example can be found here.

   In most implementations of nested blocks within a dynamic block expression, encountering an empty list results in the dynamic block generating no instances of that block. Consequently, the resource ignores previously defined managed objects within the dynamic block. To address this, you can use attribute-as-block mode. This is necessary because some nested blocks are implemented as attribute-as-block instead of normal nested blocks, which work seamlessly with dynamic block expressions.

Scenario 4: Simplified Optional Attribute As Blocks

Given the limitation we observed in Scenario 5, where a dynamic block expression fed with an empty collection starts ignoring pre-existing managed objects instead of removing them, we have identified that a workaround for this is to use attribute-as-block mode. In cases where a resource’s nested block argument is implemented using attribute-as-block mode, it is better to use Arbitrary Expressions with Argument Syntax. This will effectively allow us to pass an empty list and explicitly assign an empty list value, rather than assigning no value at all and thus retaining and ignoring any existing objects, bypassing the issue we encountered when using dynamic blocks in Scenario #3. Let’s go and update our main.tf to use attribute-as-block mode.

# in main.tf
variable "vpc_name" {
  type    = string
  default = null
}

variable "vpc_cidr_block" {
  type    = string
  default = null
}

variable "snets_configuration" {
  type = map(object({
    cidr_block           = string
    availability_zone    = optional(string)
    availability_zone_id = optional(string)
    ingress_rules        = optional(map(object({
      from_port       = number
      to_port         = number
      action          = optional(string, "allow")
      protocol        = string
      cidr_block      = optional(string)
      ipv6_cidr_block = optional(string)
      icmp_type       = optional(number)
      icmp_code       = optional(number)
    })), {})
  }))
  default = {}
}

resource "aws_vpc" "this" {
  cidr_block = var.vpc_cidr_block

  tags = {
    Name = var.vpc_name
  }
}

resource "aws_subnet" "this" {
  for_each             = var.snets_configuration
  vpc_id               = aws_vpc.this.id
  cidr_block           = each.value.cidr_block
  availability_zone    = each.value.availability_zone
  availability_zone_id = each.value.availability_zone_id

  tags = {
    Name = each.key
  }
}

resource "aws_network_acl" "this" {
  for_each   = var.snets_configuration
  vpc_id     = aws_vpc.this.id
  subnet_ids = [aws_subnet.this[each.key].id]

  /*
    This is using attribute-as-blocks instead of dynamic blocks.
      See:
        - https: //developer.hashicorp.com/terraform/language/attr-as-blocks
        - https: //github.com/hashicorp/terraform-provider-aws/issues/29501
  */
  ingress = [
    for key, value in each.value.ingress_rules : {
      rule_no         = tonumber(key)
      from_port       = value.from_port
      to_port         = value.to_port
      action          = value.action
      protocol        = value.protocol
      cidr_block      = value.cidr_block
      ipv6_cidr_block = value.ipv6_cidr_block
      icmp_type       = value.icmp_type
      icmp_code       = value.icmp_code
    }
  ]

  tags = {
    Name = format("%s-acl", each.key)
  }
}

Let’s have a look at the changes we made to our snets_configuration variable block:

1. The ingress_rules object’s cidr_block attribute has been changed to be an optional attribute.

2. We have added additional optional attributes to the ingress_rules object’s cidr_block attribute such as ipv6_cidr_block, icmp_type, and icmp_code.

Because the ingress_rules object will be consumed by the ingress argument, which will use attribute-as-block mode, we have to declare all possible arguments that we can set for the ingress nested block. This is necessary regardless of whether the arguments are required or optional. This is because we will use the argument syntax to replace the dynamic block expression, and the argument syntax will raise an error if you pass it a collection with a missing argument, even if the missing argument is optional.

Next, let’s take a look at the changes we made to the aws_network_acl resource block:

1. Previously, we used dynamic blocks to iterate over each ingress_rules element and configure individual ingress rules within the Network ACL. With the new change, we used attribute-as-blocks to create a list of ingress rules by iterating over the ingress_rules elements.

2. The ingress = [...] section uses a list comprehension to iterate over the ingress_rules defined in var.snets_configuration for each subnet. For each ingress rule, it creates a block with attributes. These attributes are populated with values from the corresponding ingress_rules in var.snets_configuration.

3. We have added additional arguments inside the ingress = [...] section such as ipv6_cidr_block, icmp_type, and icmp_code, this is because it will raise an error as previously stated.

Next, let’s take a look at the changes we made to the aws_network_acl resource block:

1. Previously, we used dynamic blocks to iterate over each ingress_rules element and configure individual ingress rules within the Network ACL. With the new change, we used attribute-as-blocks to create a list of ingress rules by iterating over the ingress_rules elements.

2. The ingress = [...] section uses list comprehension to iterate over the ingress_rules defined in var.snets_configuration for each subnet. For each ingress rule, it creates a block with attributes. These attributes are populated with values from the corresponding ingress_rules in var.snets_configuration.

3. We have added additional arguments inside the ingress = [...] section, such as ipv6_cidr_block, icmp_type, and icmp_code, as previously mentioned. This is necessary to prevent errors, as explained earlier.

Now, let’s validate if this new code solves the issue we had earlier, First, let’s edit our terraform.tfvars and add some ingress_rules to my-demo2-snet to set things up.

# in terraform.tfvars
vpc_name       = "my-demo-vpc"
vpc_cidr_block = "10.0.0.0/16"

snets_configuration = {
  my-demo1-snet = {
    cidr_block        = "10.0.1.0/24"
    availability_zone = "ap-southeast-2b"
    ingress_rules = {
      100 = {
        action      = "allow"
        protocol    = "tcp"
        cidr_block  = "10.3.0.0/18"
        from_port   = 80
        to_port     = 80
      },
      200 = {
        protocol    = "tcp"
        cidr_block  = "10.3.0.0/18"
        from_port   = 443
        to_port     = 443
      },
    }
  }
  my-demo2-snet = {
    cidr_block        = "10.0.2.0/24"
    availability_zone = "ap-southeast-2a"
    ingress_rules = {
      100 = {
        action      = "allow"
        protocol    = "tcp"
        cidr_block  = "10.3.0.0/18"
        from_port   = 80
        to_port     = 80
      },
      200 = {
        protocol    = "tcp"
        cidr_block  = "10.3.0.0/18"
        from_port   = 443
        to_port     = 443
      },
    }
  }
  my-demo3-snet = {
    cidr_block           = "10.0.3.0/24"
    availability_zone_id = "apse2-az3"
  }
}

Let’s apply this for now.

Now, let’s remove the ingress_rules block from my-demo2-snet. This action should effectively pass an empty collection to our underlying resource and consequently delete all pre-existing objects created previously.

# in terraform.tfvars
vpc_name       = "my-demo-vpc"
vpc_cidr_block = "10.0.0.0/16"

snets_configuration = {
  my-demo1-snet = {
    cidr_block        = "10.0.1.0/24"
    availability_zone = "ap-southeast-2b"
    ingress_rules = {
      100 = {
        action      = "allow"
        protocol    = "tcp"
        cidr_block  = "10.3.0.0/18"
        from_port   = 80
        to_port     = 80
      },
      200 = {
        protocol    = "tcp"
        cidr_block  = "10.3.0.0/18"
        from_port   = 443
        to_port     = 443
      },
    }
  }
  my-demo2-snet = {
    cidr_block        = "10.0.2.0/24"
    availability_zone = "ap-southeast-2a"
  }
  my-demo3-snet = {
    cidr_block           = "10.0.3.0/24"
    availability_zone_id = "apse2-az3"
  }
}

Sweet, now our nested block is truly dynamic. But notice how we’ve just added a bunch of code, and there’s so much attribute-to-argument mapping going on. This makes things clear as to how attributes are mapped to arguments, especially when programmatically iterating through collections.

We have opportunities to simplify our code and reduce clutter in this section. When examining the ingress = [...] section, we can further streamline it if we ensure that the attributes we are passing have the same names as the arguments that ingress expects. This would entail making changes to our snets_configuration as well. Let’s update our main.tf code.

# in main.tf
variable "vpc_name" {
  type    = string
  default = null
}

variable "vpc_cidr_block" {
  type    = string
  default = null
}

variable "snets_configuration" {
  type = map(object({
    cidr_block           = string
    availability_zone    = optional(string)
    availability_zone_id = optional(string)
    ingress_rules        = optional(map(object({
      rule_no         = number
      from_port       = number
      to_port         = number
      action          = optional(string, "allow")
      protocol        = string
      cidr_block      = optional(string)
      ipv6_cidr_block = optional(string)
      icmp_type       = optional(number)
      icmp_code       = optional(number)
    })), {})
  }))
  default = {}
}

resource "aws_vpc" "this" {
  cidr_block = var.vpc_cidr_block

  tags = {
    Name = var.vpc_name
  }
}

resource "aws_subnet" "this" {
  for_each             = var.snets_configuration
  vpc_id               = aws_vpc.this.id
  cidr_block           = each.value.cidr_block
  availability_zone    = each.value.availability_zone
  availability_zone_id = each.value.availability_zone_id

  tags = {
    Name = each.key
  }
}

resource "aws_network_acl" "this" {
  for_each   = var.snets_configuration
  vpc_id     = aws_vpc.this.id
  subnet_ids = [aws_subnet.this[each.key].id]

  /*
    This is using attribute-as-blocks instead of dynamic blocks.
      See:
        - https: //developer.hashicorp.com/terraform/language/attr-as-blocks
        - https: //github.com/hashicorp/terraform-provider-aws/issues/29501
  */
  ingress = [
    for rules in each.value.ingress_rules : rules
  ]

  tags = {
    Name = format("%s-acl", each.key)
  }
}

Let’s look at the changes:

1. Notice now that we’ve added the rule_no attribute as a required attribute for the ingress_rule object. This is to ensure that we are always passing attributes with the same names as the ingress argument expects.

2. We have further simplified the ingress = [...] section, wherein we just tell it to iterate through all the rules inside ingress_rules as-is without having to map each ingress_rules attribute to each ingress argument since the attributes are named similarly to the expected arguments.

Now lets update our terraform.tfvars.

# in terraform.tfvars
vpc_name       = "my-demo-vpc"
vpc_cidr_block = "10.0.0.0/16"

snets_configuration = {
  my-demo1-snet = {
    cidr_block        = "10.0.1.0/24"
    availability_zone = "ap-southeast-2b"
    ingress_rules = {
      foo = {
        rule_no = 100, action = "allow", protocol = "tcp", cidr_block = "10.3.0.0/18",
        from_port = 80, to_port = 80
      },
      bar = {
        rule_no = 200, protocol = "tcp", cidr_block = "10.3.0.0/18", from_port = 443, to_port = 443
      },
    }
  }
  my-demo2-snet = {
    cidr_block        = "10.0.2.0/24"
    availability_zone = "ap-southeast-2a"
  }
  my-demo3-snet = {
    cidr_block           = "10.0.3.0/24"
    availability_zone_id = "apse2-az3"
  }
}

Notice that the key is now foo and bar instead of 100 and 200? The key can now be anything as long as it is unique since the code now expects an attribute called rule_no to be present, and the key is now used solely for unique identification.

Additionally, we’ve reduced clutter in our terraform.tfvars in two ways:

1. For the object foo, we compressed the attributes into two lines. Notice that we’ve added commas to separate the attributes.

2. For the object bar, we compressed the attributes as well, similar to what we did for foo, but this time in a single line.

There is a slight issue with nested blocks that were implemented using attribute-as-blocks. When you add new item(s) to the list, the resource removes pre-existing item(s) and re-adds them along with the new item(s). This behavior occurs whenever a nested block is implemented as an attribute-as-block, regardless of whether you use a dynamic block expression or argument syntax.

However, there’s no need to worry because, despite how it may seem, this process doesn’t actually remove and re-add pre-existing configurations on the cloud provider side. Instead, during an apply, the updated list is sent as a single API call to the cloud provider.

You can simulate this behavior by using the Terraform code from Scenario 3 and Scenario 4 and adding a new item to the ingress_rules object-type attribute.

We do have a solution to this behavior, and that is using the resource equivalent of most nested blocks. For example, we can leverage the aws_network_acl_rule resource instead of aws_network_acl resource’s ingress and egress nested block arguments. We will explore these options in our next segment when we start talking about the flatten() function.

Conclusion

In this blog post, we’ve explored how the optional() modifier and the for_each meta-argument in Terraform can collaborate to enhance the flexibility and clarity of our configurations.

By making attributes inside collection-type variables optional using optional(), we simplify code, reduce duplication, and create a more readable structure. Combining this with for_each allows us to generate multiple instances of resources with slight variations, promoting code reuse and efficient management.

Through examples, we’ve demonstrated how this approach simplifies logic, empowers data-driven configuration, and fosters improved coupling between resources.

In the next segment, we will explore the implementation of the flatten() function to further enhance the flexibility of our Terraform code and simplify the logic involved in establishing relationships between resources. Additionally, we will investigate the possibility of combining flatten() with for_each and optional() to make resources optional. We will also investigate how we can leverage these techniques to reduce our reliance on dynamic blocks and attribute-as-blocks, opting for their resource equivalents, all while maintaining a data-driven approach.