Part of the 2023 Cornell AWS Direct Connect Architecture Migration process creates new tags on Cornell AWS VPCs that use Direct Connect. Those tags, prefixed by "cit:", can cause Terraform to hiccup if you use Terraform to manage AWS network resources.
Please don't allow Terraform to delete the tags prefixed by "cit:", or the "Cost Center" tag! They are important for the migration to the v2 Direct Connect architecture. If you (or Terraform) delete those tags, they will be recreated before the migration proceeds. Deleting the "Cost Center" tag on TGW attachments will result in customers paying for TGW attachment costs instead of CIT. |
This is what it looks like when Terraform finds those tags, and makes a plan to delete them:
# terraform plan aws_vpc.blank-vpc: Refreshing state... [id=vpc-cde7e0a8] ... aws_route_table_association.v2-private-1: Refreshing state... [id=rtbassoc-08f9e7ea923cc8454] Terraform used the selected providers to generate the following execution plan. Resource actions are indicated with the following symbols: ~ update in-place Terraform will perform the following actions: # aws_subnet.example will be updated in-place ~ resource "aws_subnet" "example" { id = "subnet-0d705338215b4d08b" ~ tags = { - "cit:dc-arch-migration-description" = "No change." -> null - "cit:dc-arch-migration-target" = "no" -> null - "cit:dc-arch-version" = "v1" -> null - "cit:subnet-type" = "public" -> null # (1 unchanged element hidden) } ~ tags_all = { - "cit:dc-arch-migration-description" = "No change." -> null - "cit:dc-arch-migration-target" = "no" -> null - "cit:dc-arch-version" = "v1" -> null - "cit:subnet-type" = "public" -> null # (1 unchanged element hidden) } # (14 unchanged attributes hidden) } Plan: 0 to add, 1 to change, 0 to destroy. |
There are two options to tell Terraform to ignore the "cit:" tags, depending on the AWS provider version you are using:
ignore_tags
setting in the provider configuration. This is by far the simplest approach.lifecycle
stanza to the all the affected resources and setting the ignore_changes
attribute.Since the "Cost Center" tag is added only to a new Transit Gateway Attachment that will be added to VPCs, you probably won't need to include it in configurations shown below, unless you plan to import the new TGW Attachment resources into your own Terraform configuration. |
ignore_tags
in AWS Provider ConfigurationThis option can be used for any AWS provider version >= 2.60.0
provider "aws" { # ... potentially other configuration ... ignore_tags { key_prefixes = ["cit:"] } } |
ignore_changes
in lifecycle
stanza for Each Resourceresource "aws_subnet" "example" { cidr_block = "10.92.117.128/25" vpc_id = aws_vpc.example.id ... tags = { Name = "example-subnet" } lifecycle { ignore_changes = [ tags["cit:dc-arch-migration-description"], tags["cit:dc-arch-migration-target"], tags["cit:dc-arch-version"], tags["cit:dc-vgw"], tags["cit:subnet-type"], tags["cit:tgw-attachment-target"], tags["cit:tgw-attachment-guidance"], tags["Cost Center"], ] } } |
You will need to upgrade Terraform to at least version 0.12.3 and then use the configuration above.
resource "aws_subnet" "example" { cidr_block = "10.92.117.128/25" vpc_id = aws_vpc.example.id ... tags = { Name = "example-subnet" } lifecycle { ignore_changes = [ "tags.%", "tags.cit:dc-arch-migration-description", "tags.cit:dc-arch-migration-target", "tags.cit:dc-arch-version", "tags.cit:dc-vgw", "tags.cit:subnet-type", "tags.cit:tgw-attachment-target", "tags.cit:tgw-attachment-guidance", "tags.Cost Center", ] } } |
If your Terraform version or AWS provider version doesn't support (or behave as expected) with the options above, you should be able to, at least, tell Terraform to ignore all changes to tags, as shown below:
lifecycle { ignore_changes = [ tags ] } |
Or...
lifecycle { ignore_changes = [ "tags" ] } |
This guide does not describe how to import pre-existing resources into Terraform. See Terraform documentation/tutorial for how-to and concepts: Import Terraform Configuration |
Generally, you will take these steps:
This Terraform configuration will need minor edits to correspond to the resources in your VPC.
Terraform configuration | utility-resources.tf |
---|
This Terraform configuration is a super basic template of the v2 Route Table resources added to customer VPCs. You will generally want to take one of two approaches:
Terraform template | route-tables.tf |
---|
Each customer VPC had secondary a CIDR block added to it. The utility subnets were created from this CIDR. We do not recommend adding a aws_vpc_ipv4_cidr_block_association resource to your Terraform configuration in order to manage this secondary CIDR block into your Terraform configuration. The reason behind this is that these secondary CIDRs were allocated using Amazon VPC IP Address Manager (IPAM) and customers do not have privileges to these IPAM resources.