...

Tag Key	Tag Values	Description	VPC	Subnets	Route Tables	NACLs ‡	Transit Gateway Attachments	Virtual Private Gateways	Direct Connect Virtual Interfaces
cit:dc-arch-migration-target	yes/no	Will this resource itself be affected as part of the migration?
cit:dc-arch-migration-description	prose	Description of planned changes to this resource
cit:dc-arch-version	v1/v2	Is this a v1 or v2 architecture resource? After migration, v1 resources utilized in the v2 architecture will be relabeled as v2 resources.
cit:dc-arch-migration-new-resource	yes/no	Is this a new resource specifically created for the v2 architecture?	n/a	n/a				n/a	n/a
cit:dc-arch-migration-replaces	resource ID	If this v2 resource will be replacing a v1 resource, this ID references the resource that will be replaced.	n/a	n/a		n/a	n/a	n/a	n/a
cit:subnet-type	public/private/utility	Is this a private or public subnet? Public subnets are those with a route to an Internet Gateway. Utility subnets will be created specifically to use for TGW Attachments. Private subnets are all subnets that are not public and are not utility subnets.	n/a		n/a	n/a	n/a	n/a	n/a
cit:tgw-attachment-target	yes/no	Will a Transit Gateway be attached to this subnet?	n/a		n/a	n/a	n/a	n/a	n/a
cit:dc-arch-exclude-tgw-attachment-guidanceroutes	yestbd/attach/no-attach	This tag provides a place for a human reviewer to provide guidance about whether a TGW attachment should be made to the tagged subnet. tbd → guidance has not yet been provided attach → human guidance says to attach the subnet to the TGW no-attach → human guidance says not to attach the subnet to the TGW Of all these tags, this is the only tag whose value should be updated by customers.	n/a		is applied only to the new Route Table that is created for use by the TGW Attachment utility subnets	n/a	n/a		n/a	n/a	n/a	n/a	n/a
cit:dc-vgw	yes/no	Does this Route Table contain rules referencing a VGW?	n/a	n/a		n/a	n/a	n/a	n/a
Cost Center	R524755	This tag added to TGW Attachments will result in CIT paying for the $0.05/hr cost of attaching a VPC to a TGW.	n/a	n/a	n/a	n/a		n/a	n/a

‡ Only the NACL created for use by utility subnets will be tagged.

Direct Connect Gateways are also involved in the migration but cannot be tagged.

...

The AWS Transit Gateways used in the v2 architecture require different routing rules than the Virtual Private Gateways (VGW) used in the v1 architecture. Each VPC Route Table that references a Virtual Private Gateway will be duplicated and, in the new Route Table, rules referencing a VGW will be replaced  with rules referencing a TGW Attachment. The new Route Tables will not include "blackhole" routes (i.e. routes to resources, like old peering connections, that no longer exist) from the original Route Tables.

These new Route Tables will be created prior to the migration, but will not actually be utilized until the migration is executed. When migration is executed, subnets associated with the v1 Route Tables will be re-associated to the corresponding v2 Route Tables. Similarly, if the "main" Route Table for the VPC references a VGW, the corresponding v2 Route Table will be set as the "main" Route Table for the VPC.

...

Unlike Virtual Private Gateways, TGW Attachments connect to specific subnets in a VPC. We will be making these TGW Attachments to multiple private the utility subnets in your VPCs. For best resiliency, we will select private subnets in multiple Availability Zones (AZs) for the TGW Attachments. In most Cornell AWS accounts, each private subnet resides in a unique AZ. If your VPC contains more than one private subnet in a given AZ, we will consult with AWS account owners to determine the best private subnet to select for the TGW Attachments. This is because each AZ can accommodate exactly one TGW Attachment.

TGW Attachments will be created during the Migration phase of the migration. See Migration Process.

Resource Deletion

After migration is complete, a few resources will be deleted during the Cleanup phase of the migration. These are:

• Virtual Private Gateways (VGWs)
• Direct Connect Virtual Private Interfaces (DCVIFs)
• Direct Connect Gateways that supported the v1 architecture.

Neither VGWs nor DCVIFs have a role in the v2 architecture.

...

VPCs which were specifically created for this purpose. 

TGW Attachments will be created during the Migration phase of the migration. See Migration Process.

Resource Deletion

After migration is complete, a few resources will be deleted during the Cleanup phase of the migration. These are:

• Virtual Private Gateways (VGWs)
• Direct Connect Virtual Private Interfaces (DCVIFs)
• Direct Connect Gateways that supported the v1 architecture.

Neither VGWs nor DCVIFs have a role in the v2 architecture.

Anchor
costs costs

Costs

The management and routing simplification offered by the v2 architecture comes with a shift in costs seen by Cornell AWS accounts using Direct Connect, but the overall impact to Cornell AWS account costs should be negligible.

V1 Architecture

Cornell AWS

The management and routing simplification offered by the v2 architecture comes with a shift in costs seen by Cornell AWS accounts using Direct Connect, but the overall impact to Cornell AWS account costs should be negligible.

V1 Architecture

Cornell AWS accounts using Direct Connect saw these Direct Connect-related charges:

...

draw.io source: direct-connect-migration-process.v2.drawio

none

Phase	Stage	Timeframe	Status	Activity	Impact on Cornell AWS Account VPC Networks
Preparation	Data Collection	November 2022		Gather information about Direct Connect resources and connected VPCs in Cornell AWS accounts	none
	Resource Tagging	14 Dec 2022		Add tags to existing resources in customer accounts to assist with targeting, identification, status, intended disposition	none
	Resource Groups			Create Transit Gateway in CIT AWS account Create Resource Groups for resources involved in the migration in customer accounts	none
	Customer Input #1	15 Dec 2022 - 02 Jan 2023		Cornell AWS account owner/admin review Cornell AWS account owner/admin feedback solicited	none
Migration	Transit Gateway Attachments	03 Jan 2023 - 06 Jan 2023		Utility Subnets Transit Gateway Attachments created in customer accounts v2 Route Tables created in customer accounts NACLs for Utility Subnets	none
	Customer Input #2	09 Jan 2023 - 13 Jan 2023		Cornell AWS account owner/admin review Cornell AWS account owner/admin feedback solicited Route Table and/or TGW Attachments adjusted according to customer input	none	VPC Routing Updated	16 Jan 2023	v2 Route Tables activated v1 Route Tables deactivated	VPC-to-campus traffic will be routed through the v2 architecture	Campus Direct Connect Routes Updated	17 Jan 2023	Campus-side routing updated to begin using the v2 architecture for campus-to-AWS traffic	campus-to-VPC traffic will be routed through the V2 architecture	Cleanup	Customer Account Cleanup	23 Jan 2023 - 31 Jan 2023	VGWs and DC VIFs in customer accounts deleted	none	Campus Direct Connect Cleanup	Campus Direct Connect resources deleted or decommissioned	Route Table and/or TGW Attachments adjusted according to customer input	none
	v2 BGP Updated	17 Jan 2023 7am		v2 Direct Connect infrastructure will have BGP configuration changed to begin advertising new routes via I2CC	Azure-to-AWS-VPC traffic may begin to use the v2 architecture (in just the one direction). This is limited only to Azure-to-AWS-VPC traffic due to Cornell's network architecture.
	VPC Routing Updated	17 Jan 2023 9am		v2 Route Tables activated v1 Route Tables deactivated	VPC-to-campus traffic will be routed through the v2 architecture Azure-to-AWS-VPC traffic may use the v2 architecture.
	Campus Direct Connect Routes Updated	18 Jan 2023 9am		Direct Connect Virtual Interfaces in customer accounts will be disabled. This causes  DC traffic to begin using the v2 architecture for campus-to-AWS traffic	campus-to-VPC traffic will be routed through the V2 architecture all Azure-to-AWS-VPC traffic will be routed through the v2 architecture
Cleanup	Customer Account Cleanup	23 Jan 2023 - 31 Jan 2023		VGWs and DC VIFs in customer accounts deleted	none
	Campus Direct Connect Cleanup			Campus Direct Connect resources deleted or decommissioned	none

Anchor
custom-schedul custom-schedul

Alternate Migration Days

Customers have the option to request that migration for their VPC(s) occur during the week of Jan 9-13 instead of the default migration dates of January 17 and 18. This is especially encouraged for customers that have a separate sandbox or development VPC that needs to be migrated. We can also support taking both migration steps on the same alternate day, but we'd like to leave a 1-4 hour gap between migration steps to confirm that the "VPC Routing Updated" step was successful before continuing to the "Campus Direct Connect Routes Updates" step.

Anchor
rollback rollback

Rollback

Both the "VPC Routing Updated" and the "Campus Direct Connect Routes Updated" steps have simple rollback mechanism. If you discover problems with networking in your VPC after either step and think the change needs to be rolled back, send an email to cloud-incident@cornell.edu and ping Paul Allen (pea1) on Teams.

• The rollback for the "VPC Routing Updated" step is to reassign the original Route Tables to the public and private subnets. This will rollback takes effect immediately.
• The rollback for the "Campus Direct Connect Routes Updated" step is to the cancel the failover of the Direct Connect Virtual Interfaces that we triggered to initial the campus routing updates. This rollback takes 5-20 minutes to complete. 

FAQs

How do I tell if my AWS account will be affected by this change?

...

The exact pathways that Direct Connect traffic takes will change between the v1 and v2 architectures. But, the starting point (e.g., your VPC) and endpoint (e.g. a campus VLAN) of this traffic will constant.

When, specifically, will this migration occur?

...

VLAN) of this traffic will constant.

When, specifically, will this migration occur?

See detailed schedule above.

Is there any flexibility in migration dates?

Yes. See Alternate Migration Days above.

Can the migration be rolled back?

Yes. Each of the two active migration steps ("VPC Routing Updated" and "Campus Direct Connect Routes Updated") can be individually rolled back for each migrating AWS VPC. See Rollback above

...

.

What if I use Terraform or a similar tool to manage the network resources in my AWS account?

...

• Cornell Documentation
  • Cornell AWS Accounts Affected by 2023 Direct Connect Architecture Migration
  • Terraform Configuration Guidance for 2023 Direct Connect Architecture Migration
  • Cornell AWS Direct Connect Routing Diagrams
  • Announcements
    • 2023-01-16 AWS Direct Connect Architecture Migration Execution
    • 2023-01-10 AWS Direct Connect Architecture Migration Customer Review and Feedback #2 
    • 2022-12-20 AWS Direct Connect Architecture Design Update
    • 2022-12-15 AWS Direct Connect Architecture Migration Customer Review and Feedback 
    • 2022-12-09 AWS Direct Connect Architecture Migration Preparation Continues
    • 2022-11-02 Upcoming AWS Direct Connect Changes
• External Documentation
  • AWS Direct Connect
  • AWS Transit Gateway
  • Internet 2 Cloud Connect

...