Introduction

This document provides details about the Direct Connect architecture migration we will be executing in early 2023.

Scope

As of 29 Nov 2022, 65 Cornell AWS accounts were configured to use Direct Connect. During this migration, all those AWS accounts will have their existing Direct Connect connectivity updated to use new pathways and AWS resources to connect the Cornell campus network to AWS. 

Within those 65 Cornell AWS accounts, only the network resources within VPCs using Direct Connect will be affected. Other VPCs in those Cornell AWS accounts will not be affected.

Nomenclature

We use the following terminology:

• Version 1 (v1) architecture – This is the network architecture used by Cornell AWS Direct Connect networking prior to the 2023 migration.
• Version 2 (v2) architecture – This is the network architecture used by Cornell AWS Direct Connect networking after the 2023 migration.
• VPC – Virtual Private Cloud
• DC – Direct Connect
• TGW – Transit Gateway
• VGW – Virtual Private Gateway

FAQs

• How do I tell if my AWS account will be affected by this change?
• How will this change affect my AWS account costs?
• Does this change affect VPC peering?
• When, specifically, will this migration occur?
• What if I use Terraform or a similar tool to manage the network resources in my AWS account?

Architectures

These diagrams show the network resources within Cornell AWS accounts that connect a VPC to the Cornell campus network via Direct Connect.

Version 1 (pre-2023)

draw.io source: dc-arch-legacy.customer.v2.drawio

Version 2 (2023)

draw.io source: dc-arch-2023.customer.v2.drawio

What Is Changing?

Before the migration is executed, a set of resources in Cornell AWS accounts will be tagged with details about the migration. In addition, a small set of new resources that support the v2 architecture will be created in Cornell AWS accounts. After the migration is complete, a few resources not used in the v1 architecture will be deleted.

Cornell AWS customers will have the opportunity to provide feedback before migration and any resource deletion that affects their AWS accounts.

New Resources

Resource Groups

New AWS resource groups collect references to relevant AWS account resources in one place (per Cornell AWS account) for easy reference and review:

Resources can and will appear in multiple resource groups!

• cit-dc-arch-migration-affected-resources – These resources will be directly affected by this migration. These resources include:
  • new resources that support the v2 architecture
  • resources that support the v1 architecture and will no longer be needed for the v2 architecture
  • resources that will remain, but will have their configuration changed to support the v2 architecture
  • Transit Gateway Attachments should appear in this list, but a limitation of AWS Resource Groups appears to exclude these resources.
• cit-dc-arch-version-1-resources – All network resources that support or utilize the v1 architecture
• cit-dc-arch-version-2-resources – All newly-created resources that support the v2 architecture
  • Transit Gateway Attachments should appear in this list, but a limitation of AWS Resource Groups appears to exclude these resources.

After the v1 → v2 migration is complete, v1 resources will either be deleted (if they are not used in the v2 architecture) or relabeled as v2 resources (if they continue to be used in the v2 architecture).

Route Tables

The AWS Transit Gateways used in the v2 architecture require different routing rules than the Virtual Private Gateways (VGW) used in the v1 architecture. Each VPC Route Table that references a Virtual Private Gateway will be duplicated and, in the new Route Table, rules referencing a VGW will be replaced  with rules referencing a TGW Attachment.

These new Route Tables will be created prior to the migration, but will not actually be utilized until the migration is executed. When migration is executed, subnets associated with the v1 Route Tables will be re-associated to the corresponding v2 Route Tables. Similarly, if the "main" Route Table for the VPC references a VGW, the corresponding v2 Route Table will be set as the "main" Route Table for the VPC.

Transit Gateway Attachments

Transit Gateway Attachments are the mechanism that VPCs connect to Transit Gateways. The Transit Gateways we use in the v2 architecture reside in a central AWS account, and a TGW Attachment is what links the VPC in a Cornell AWS account to those central TGWs.

Unlike Virtual Private Gateways, TGW Attachments connect to specific subnets in a VPC. We will be making these TGW Attachments to multiple private subnets in your VPCs. For best resiliency, we will select private subnets in multiple Availability Zones (AZs) for the TGW Attachments. In most Cornell AWS accounts, each private subnet resides in a unique AZ. If your VPC contains more than one private subnet in a given AZ, we will consult with AWS account owners to determine the best private subnet to select for the TGW Attachments. This is because each AZ can accommodate exactly one TGW Attachment.

Tagging

For this migration, we are tagging AWS resources to provide information about how the each resource is involved in the migration itself, the v1 architecture, and the v2 architecture.

Tag Key	Tag Values	Description	VPC	Subnets	Route Tables	Transit Gateway Attachments	Virtual Private Gateways	Direct Connect Virtual Interfaces
cit:dc-arch-migration-target	yes/no	Will this resource itself be affected as part of the migration?
cit:dc-arch-migration-description	prose	Description of planned changes to this resource
cit:dc-arch-version	v1/v2	Is this a v1 or v2 architecture resource? After migration, v1 resources utilized in the v2 architecture will be relabeled as v2 resources.
cit:dc-arch-migration-new-resource	yes/no	Is this a new resource specifically created for the v2 architecture?	n/a	n/a			n/a	n/a
cit:dc-arch-migration-replaces	resource ID	If this v2 resource will be replacing a v1 resource, this ID references the resource that will be replaced.	n/a	n/a		n/a	n/a	n/a
cit:subnet-type	public/private	Is this a private or public subnet? Public subnets are those with a route to an Internet Gateway. Private subnets are all subnets that are not public.	n/a		n/a	n/a	n/a	n/a
cit:tgw-attachment-target	yes/no/guidance-required	Will a Transit Gateway be attached to this subnet? If "guidance-required" then account owners will be consulted about the TGW Attachments.	n/a		n/a	n/a	n/a	n/a

Resource Deletion

After migration is complete, a few resources will be deleted. These are:

• Virtual Private Gateways (VGWs)
• Direct Connect Virtual Private Interfaces (DCVIFs)

Neither VGWs nor DCVIFs have a role in the v2 architecture.

Migration Process

Timeline

References

• 2022-11-02 Upcoming AWS Direct Connect Changes