Team based privacy using IAM, VPC, Transit Gateway and DNS

Objectives

• Learn how to isolate infrastructure using IAM and Resource groups
• Create the VPCs and associated resources such as subnets, network ACLs, security groups, instances.
• Address microservices by DNS name resolution using DNS Services.
• Connect VPCs via Transit Gateway.
• Transparently configure a Load Balancer for an application.

Abstract Architecture:

In the diagram above the end user is accessing the applications. The applications are leveraging shared microservices. The company has separate DevOps teams that own application1, application2 and shared. A networking team focuses on connectivity and network security. The DevOps teams manage Virtual Service Instances, VSIs, used to implement the services they create and support.

Concrete Architecture

The following architecture implements the isolation and connectivity requirements set by the company. Notice that application1, shared, and application2 are VPCs. The single zone and subnet in each VPC can be expanded to a more detailed multi zone implementation over time.

Before you begin

This tutorial requires:

• IBM Cloud CLI with these plugins:
  • Transit Gateway (tg)
  • IBM Cloud VPC (vpc-infrastructure)
  • DNS Services (dns)
• git to clone source code repository.
• terraform CLI to run the Terraform commands.

You will find instructions to download and install these tools for your operating environment in the Getting started with solution tutorials guide.

In addition:

• Check for user permissions. Be sure that your user account has sufficient permissions to create and manage VPC resources, create a IBM Cloud® Transit Gateway and create a IBM Cloud® Transit Gateway services. See the list of required permissions for VPC. You will also need the ability to create resource groups and IAM resources like access groups, policies, service IDs, ...
• You need an SSH key to connect to the virtual servers. If you don't have an SSH key, see the instructions for creating a key for VPC.

Plan the Identity and Access Management Environment

The admin team will enable the other teams to administer their resources as much as possible. The admin team will manage users and control access but will not create and destroy the resources shown in the architecture diagram.

Editor, Operator, Viewer and Manager are IAM access roles. Each service defines the exact meaning of the roles and the associated actions.

Teams:

• Admin - define the account structure such as resource groups, access groups, users, roles.
• Network - create network resources such as DNS Services, Transit Gateway service, VPC and subnets.
• Shared - create VSI and block devices in the shared VPC. Create DNS records for shared services.
• Application1 - create VSI and block devices in the application1 VPC
• Application2 - create VSI and block devices in the application2 VPC.

IAM Conceptual

Network Team

A conceptual team ownership model was implemented. The network team administers all the network resources and hence most of what is shown in the diagram.

Shared Team

The shared team creates the VSI in its isolated VPC. In addition, the team needs to write records into the DNS service since the IP addresses of the VSIs are determined at creation time. Operator access to the VPC, subnets and security groups are required to create a VSI.

The application teams needs the same access as the shared team except manager access to the DNS Services.

Application team access:

IAM Architecture

If you have a good understanding of resource groups and IAM Access Groups you can quickly skim this section and start the steps to create the resources.

Access Groups

An access group will be created for each team. Access policies are added to access groups, and then users (team members) are added to the access group to grant access.

The Transit Gateway service instance is managed exclusively by the network team. Editor access is required for creation. Manager access to the created instance allows VPCs to be connected to the Transit Gateway.

In this example, a single DNS zone, widgets.example.com will be created and access to the DNS zone will be permitted to all the VPCs. The DNS service instance is created by the network team (Editor role) and permitting the zones requires the Manager role. DNS resolution at run time in an instance on a VPC does not require IAM access. The shared team needs to list the DNS instances (Viewer role) and add an A or CNAME record (Manager role).

VPC Infrastructure Service (IS) consists of about 15 different service types. Some are only of concern to the network team, like network ACLs (access control lists). Others are only of concern to the microservice teams, like VSI instances. But some are edited by the network team and operated by the microservice team, like subnets. The network team will create the subnet and a microservice team will create an instance in a subnet. For the purpose of this tutorial the VPC IS service types, Transit Gateway and DNS are summarized for each access group in the table below. The contents of the table are the required roles.

Required roles assigned to teams based on their responsibilities

Service	network	shared	application
Transit Gateway	Editor, Manager
DNS	Editor, Manager	Viewer, Manager
IS: Network ACL	Editor
IS: Instance, Volume, Floating IP, SSH Key, Image, Load Balancer		Editor	Editor
IS: VPC, Subnet, Security Group	Editor	Operator	Operator

Resource Groups

The shared team and the network team are now nicely separated. But how is Application1 isolated from Shared and Application2? They are Editor for the same types of services.

This is where resource groups can help out. Each service instance (i.e. resource) has a resource group attribute that is initialized upon creation and cannot be changed. In other words, each resource is in one resource group.

Resource Group diagram:

Each microservice team will be allowed the access in the corresponding resource group. The network team will have access to all of these resource groups.

The network resource group contains Transit Gateway and the DNS service. The network team has access to these resources. The shared team will have Manager access to the DNS service. The shared team needs to write the DNS entries for the shared services.

Later in the tutorial, after all resources have been created, it can be informative to open the Resources list in the IBM Cloud console. It is possible to filter on resource group.

Create a local working environment

All the operations will be performed in a bash shell and making use of terraform and ibmcloud command. You will find instructions to download and install these tools for your operating environment in the Getting started with solution tutorials guide.

1. Git clone the following repository:

   git clone https://github.com/IBM-Cloud/vpc-tg-dns-iam
   cd vpc-tg-dns-iam
   

   Verify there is a directory for each team (skipping application2 for now):

   • admin/
   • network/
   • shared/
   • application1/

2. Create the terraform.tfvars file:

   cp terraform.tfvars.template terraform.tfvars
   

   Next, edit terraform.tfvars to set these variables:

   • ssh_key_name - it is required to specify an existing SSH key in the ibm_region as specified in the Before you begin section above.
   • ibm_region - replace the default value, us-south, if required. The CLI command ibmcloud regions will display all possible regions.
   • basename - replace the default value, widget0, with a name that is 7 characters or less, if required. Most resources created will use this as a name prefix.
   • Do not uncomment the transit_gateway or shared_lb at this time.

3. Windows users only: If git does not create the symbolic link on your Windows computer it is required to copy the file contents to the team folders:

   cp variables.tf terraform.tfvars admin
   cp variables.tf terraform.tfvars network
   cp variables.tf terraform.tfvars shared
   cp variables.tf terraform.tfvars application1
   

export TF_VAR_ibmcloud_api_key=0thisIsNotARealKeyALX0vkLNSUFC7rMLEWYpVtyZaS9

provider ibm {
 ibmcloud_api_key = var.ibmcloud_api_key # initialized with the TF_VAR_ibmcloud_api_key
}

ibmcloud login --apikey $TF_VAR_ibmcloud_api_key

Create the IAM-enabled resources (Admin Team)

The admin team will need to have Admin access to the IAM-enabled resources in the account used in this tutorial. See How do I assign a user full access as an account administrator?. The admin team will be responsible for creating the IAM resources. The instructions below use the ibmcloud iam api-key-create command to create an API key for the admin. The API key will be used by Terraform to perform tasks on your behalf.

The PI keys are the same as a password to your account. Keep the API keys safe.

1. Initialize and verify the basename shell variable. Verify it matches the basename in the terraform.tfvars file:

   eval $(grep basename terraform.tfvars | sed -e 's/  *//g' -e 's/#.*//')
   echo basename=$basename
   

2. Change directory, generate and source your personal API key in local.env. When Terraform is invoked it will become you. Terraform will be the administrator:

   cd admin
   echo export TF_VAR_ibmcloud_api_key=$(ibmcloud iam api-key-create $basename-admin --output json | jq .apikey) > local.env
   cat local.env
   source local.env
   

3. Apply of the main.tf Terraform configuration file creates the following resources:

   • Resource groups for each team
   • Access groups for each team and a service ID in each access group
   • Access group policies for the resource groups
   • Access group policies for the resources

   terraform init
   terraform apply
   

4. Verify the resources were created:

   ibmcloud resource groups | grep $basename
   

   ibmcloud iam access-groups | grep $basename
   

   ibmcloud iam service-ids | grep $basename
   

   ibmcloud iam access-group-policies $basename-network
   

   Output will resemble something like this:

   $ ibmcloud resource groups | grep $basename
   widget0-application2   36b06a303f224f28ad42aebbb491cc44   false           ACTIVE
   widget0-shared         91518c45e47a427fa4f63edb58e4f227   false           ACTIVE
   widget0-network        bf6e75cd71854576a31056abced2abf0   false           ACTIVE
   widget0-application1   db2f3dc8aacf4a6aa2d2fa07e795cb57   false           ACTIVE
   $ ibmcloud iam access-groups | grep $basename
   widget0-application1   AccessGroupId-26b7ef37-78db-4a2c-a2af-7f6591e73c15   application1 administrators
   widget0-application2   AccessGroupId-8afdec20-f760-4a15-8f3e-296d81818028   application2 administrators
   widget0-network        AccessGroupId-d50b129d-9adc-4fc4-b297-487b3f938ec5   network administrators
   widget0-shared         AccessGroupId-30d73f44-5602-47a7-846e-e6480c9dceff   shared administrators
   $ ibmcloud iam service-ids | grep $basename
   ServiceId-5e919b97-380c-4343-a337-3901cafbd956   widget0-application2                                                                                                       2020-07-15T21:25+0000   2020-07-15T22:03+0000   application 2 service id                                                                                                                                              false
   ServiceId-307df062-f4b7-45f8-8ec8-94ad1ed61730   widget0-network                                                                                                            2020-07-15T21:49+0000   2020-07-15T22:03+0000   network service id                                                                                                                                                    false
   $ ibmcloud iam access-group-policies $basename-network
   Retrieving all policies of access group widget0-network under account 8675309 as jenny@example.com
   OK
   
   Policy ID:   00ceb354-7360-4ad5-9fda-5c03e462c5c0
   Roles:       Editor
   Resources:
                Resource Group ID     91518c45e47a427fa4f63edb58e4f227
                Resource Group Name   widget0-shared
                Service Name          is
                flowLogCollectorId    *
                Memo                  Policy applies to the resource(s) within the resource group
   
   Policy ID:   115ebe9f-eea0-4308-9e7f-bb887d64426b
   Roles:       Editor
   Resources:
                Resource Group ID     db2f3dc8aacf4a6aa2d2fa07e795cb57
                Resource Group Name   widget0-application1
                Service Name          is
                vpnGatewayId          *
                Memo                  Policy applies to the resource(s) within the resource group
   ...
   

5. Optionally navigate to the account Resource groups and find the resource groups.

6. Optionally navigate to Access groups to see the access groups, click an access group, then click the Service IDs panel at the top to see the service ID created.

Create VPCs and DNS (Network Team)

The network team will create the network resources to match the architecture ensuring that the connectivity goals are satisfied and the teams are isolated in their VPC. They do not want to control the details of the VPC Instances. It is likely that the number of applications, size of computers, DNS records for microservices etc. will be in constant flux and not a concern of the network team.

The Admin team has provided them just the right amount of permissions to create the VPC is resources, the DNS Services and the Transit Gateway service.

1. Change directory, generate an API key in the local.env and become a member of the network access group:

   team=network
   cd ../$team
   echo export TF_VAR_ibmcloud_api_key=$(ibmcloud iam service-api-key-create $team $basename-$team --output json | jq .apikey) > local.env
   cat local.env
   source local.env
   

2. Optionally, open the variables_network.tf file and notice the CIDR block specification and the zone layout. In the snippet below notice that shared and application1 are specified without overlapping IP addresses:

   variable network_architecture {
     default = {
       shared = {
         cidr        = "10.0.0.0/16"
         cidr_remote = "10.0.0.0/8"
         zones = {
           z1 = {
             zone_id = "1",
             cidr    = "10.0.0.0/24",
           }
           z2 = {
             zone_id = "2",
             cidr    = "10.0.1.0/24",
       ...
       application1 = {
         cidr        = "10.1.0.0/16"
         cidr_remote = "0.0.0.0"
         zones = {
           z1 = {
             zone_id = "1",
             cidr    = "10.1.0.0/24",
           }
           z2 = {
             zone_id = "2",
             cidr    = "10.1.1.0/24",
        ...
   

   The Transit Gateway will have a connection to each VPC and routes traffic based on the CIDR ranges. So, avoiding overlaps will ensure success.

3. Create the resources:

   terraform init
   terraform apply
   

4. List the VPC resources

   The VPC resources created is summarized by the output of the subnets command, shown below, edited for brevity. Notice the three VPCs, the non-overlapping CIDR blocks, and the resource groups membership:

   ibmcloud target -r $(grep ibm_region terraform.tfvars | sed -e 's/  *//g' -e 's/#.*//' -e 's/.*=//' -e 's/"//g')
   ibmcloud is subnets --all-resource-groups | grep $basename
   

   The output will look like this:

   $ ibmcloud is subnets --all-resource-groups | grep $basename
   widget0-shared-z1           available   10.0.0.0/24    widget0-shared           us-south-1   widget0-shared
   widget0-shared-z2           available   10.0.1.0/24    widget0-shared           us-south-2   widget0-shared
   widget0-shared-z3           available   10.0.2.0/24    widget0-shared           us-south-3   widget0-shared
   widget0-application1-z1     available   10.1.0.0/24    widget0-application1     us-south-1   widget0-application1
   widget0-application1-z2     available   10.1.1.0/24    widget0-application1     us-south-2   widget0-application1
   widget0-application1-z3     available   10.1.2.0/24    widget0-application1     us-south-3   widget0-application1
   widget0-application2-z1     available   10.2.0.0/24    widget0-application2     us-south-1   widget0-application2
   widget0-application2-z2     available   10.2.1.0/24    widget0-application2     us-south-2   widget0-application2
   widget0-application2-z3     available   10.2.2.0/24    widget0-application2     us-south-3   widget0-application2
   

5. Optionally, navigate to the Virtual Private Clouds and find the VPCs, subnets and all the other resources created above.

6. Optionally, investigate the Terraform configuration in main.tf to understand the DNS Services initialization. The DNS Services instance and zone were created with the Terraform snippet:

   resource "ibm_resource_instance" "dns" {
     name              = "${var.basename}-dns"
     resource_group_id = data.ibm_resource_group.shared.id
     location          = "global"
     service           = "dns-svcs"
     plan              = "standard-dns"
   }
   
   resource "ibm_dns_zone" "widgets_example_com" {
     name        = "widgets.example.com"
     instance_id = ibm_resource_instance.dns.guid
     description = "this is a description"
     label       = "this-is-a-label"
   }
   

   The zone is then added to a VPC as a permitted network:

   resource "ibm_dns_permitted_network" "shared" {
     instance_id = ibm_resource_instance.dns.guid
     zone_id     = ibm_dns_zone.widgets_example_com.zone_id
     vpc_crn     = module.vpc_shared.vpc.crn
     type        = "vpc"
   }
   
   

7. List the DNS configuration. A DNS Services instance was created. The widgets.example.com zone was created. Finally, the zone was added to all the VPCs.

   ibmcloud dns instances
   

   ibmcloud dns zones -i $basename-dns
   

   zone_id=$(ibmcloud dns zones -i $basename-dns --output json | jq -r '.[] | .id')
   ibmcloud dns permitted-networks $zone_id -i $basename-dns
   

   The output will resemble this:

   $ ibmcloud dns instances
   Retrieving service instances for service 'dns-svcs' ...
   OK
   Name          ID                                     Location   State    Service Name   
   widget0-dns   3b0d5546-5999-4c7e-a757-f5fd21dd44ed   global     active   dns-svcs   
   $ ibmcloud dns zones -i $basename-dns
   Listing zones for service instance 'widget0-dns' ...
   OK
   ID                                               Name                  Status   
   5a1a2295-1c38-49dd-9809-aaaf5a127e79c1b          widgets.example.com   ACTIVE   
   $ zone_id=$(ibmcloud dns zones -i $basename-dns --output json | jq -r '.[] | .id')
   $ ibmcloud dns permitted-networks $zone_id -i $basename-dns
   Listing permitted networks for zone '5a1a2295-1c38-49dd-9809-f5a127e79c1b' ...
   OK
   Name                   ID                                          Type   VPC_CRN                                                                                                               State   
   widget0-shared         r006-353208ab-4e95-46fb-934b-b5566cde8975   vpc    crn:v1:bluemix:public:is:us-south:a/713c783d9a507a53135fe6793c37cc74::vpc:r006-353208ab-4e95-46fb-934b-b5566cde8975   ACTIVE   
   widget0-application1   r006-287258c6-2eb3-4908-b326-6f03c3e47aa6   vpc    crn:v1:bluemix:public:is:us-south:a/713c783d9a507a53135fe6793c37cc74::vpc:r006-287258c6-2eb3-4908-b326-6f03c3e47aa6   ACTIVE   
   widget0-application2   r006-fa51e99e-bd93-4451-a4eb-76eed9939d3c   vpc    crn:v1:bluemix:public:is:us-south:a/713c783d9a507a53135fe6793c37cc74::vpc:r006-fa51e99e-bd93-4451-a4eb-76eed9939d3c   ACTIVE   
   

8. Optionally, navigate to the resource list and find the DNS Services, click on it and investigate.

Create the shared microservice and associated DNS record (Shared Team)

1. Change directory, generate an API key in the local.env and become a member of the shared access group:

   team=shared
   cd ../$team
   echo export TF_VAR_ibmcloud_api_key=$(ibmcloud iam service-api-key-create $team $basename-$team --output json | jq .apikey) > local.env
   cat local.env
   source local.env
   

2. Optionally, dig deeper at this point into the Terraform source code. The shared team is going to provide microservices. Although the network team has already created the shared VPC and some network resources the shared team will create the instance and choose the instance profile. A Linux configuration script and simple demo application is provided in the user_data attribute and discussed in the Application Team section below.

   In main.tf notice these two resources:

   locals {
     network_context = data.terraform_remote_state.network.outputs.shared
   }
   
   resource ibm_is_instance "vsishared" {
     name           = "${var.basename}-shared-vsi"
     vpc            = local.network_context.vpc.id
     resource_group = data.ibm_resource_group.shared.id
     zone           = local.network_context.subnets["z1"].zone
     keys           = [data.ibm_is_ssh_key.ssh_key.id]
     image          = data.ibm_is_image.image.id
     profile        = var.profile[var.generation]
   
     primary_network_interface {
       subnet = local.network_context.subnets["z1"].id
       security_groups = [
         local.network_context.security_group_outbound_all.id, # nodejs is not available on an IBM mirror
         local.network_context.security_group_ibm_dns.id,
         local.network_context.security_group_data_inbound.id,
       ]
     }
     user_data = module.user_data_app.user_data_centos
   }
   
   resource ibm_dns_resource_record "shared" {
     count = var.shared_lb ? 0 : 1 # shared load balancer?
     instance_id = local.network_context.dns.guid
     zone_id     = local.network_context.dns.zone_id
     type        = "A"
     name        = "shared"
     rdata       = ibm_is_instance.vsishared.primary_network_interface[0].primary_ip.address
     ttl         = 3600
   }
   

   The local.network_context above is the output generated by the network team specifically for the shared team.

3. Create the shared resources:

   terraform init
   terraform apply
   

4. Optionally, navigate to the Virtual server instances for VPC and find the shared instance. Click on it and verify the following:

   • The instance has no incoming connectivity from the public internet (check the Security Groups)
   • Locate the private IP address

5. Optionally, navigate to the resource list and find the DNS Services, click on it and find the DNS record with the name shared. Notice the Value is the private IP address of the instance.

Create a publicly facing microservice for an application (Application1 Team)

1. Change directory, generate an API key in the local.env and become a member of the application1 access group:

   team=application1
   cd ../$team
   echo export TF_VAR_ibmcloud_api_key=$(ibmcloud iam service-api-key-create $team $basename-$team --output json | jq .apikey) > local.env
   cat local.env
   source local.env
   

   The application1 team resources are very similar to the shared team's. In fact, they are a little simpler since - it is not required to put records into the DNS Services. The application uses the address http://shared.widgets.example.com to access the shared microservice.

2. Optionally, investigate the source code that initializes the CentOS instance. It has been captured in a Terraform module shared by all the teams during this exploratory stage.

   ../common/user_data_app/main.tf:

   locals {
     shared_app_user_data_centos = <<EOS
   #!/bin/sh
   curl -sL https://rpm.nodesource.com/setup_20.x | sudo bash -
   yum install nodejs -y
   cat > /app.js << 'EOF'
   ${file("${path.module}/app.js")}
   EOF
   cat > /lib/systemd/system/a-app.service << 'EOF'
   ${file("${path.module}/a-app.service")}
   EOF
   systemctl daemon-reload
   systemctl start a-app
   EOS
   }
   
   output user_data_centos {
     value = replace(local.shared_app_user_data_centos, "REMOTE_IP", var.remote_ip)
   }
   

   Detailed explanation:

   • Nodejs is installed with the curl and yum commands
   • The nodejs application is put into /app.js.
   • A systemctl service is created for app.js
   • The service is started

3. Optionally investigate the app.js contents. It has two particularly interesting sections. First, there is a /info link that returns a description of the instance running the app. ../common/user_data_app/app.js:

   const server = http.createServer((req, res) => {
     switch(req.url) {
     case '/info':
       res.statusCode = 200;
       res.setHeader('Content-Type', 'application/json');
       res.end(JSON.stringify({
         req_url:  req.url,
         os_hostname:  os.hostname(),
         ipArrays: ips()
       }, null, 3));
       break
     case '/remote':
       getRemote(req, res)
       break
   

   Second, the /remote link calls to a remote server IP and returns the description of that remote along with the remote_url and remote_ip addresses used to access the remote.

   const IP='REMOTE_IP'
   
   function getRemote(req, res) {
     path = '/info'
     remote_url = 'http://' + IP + ':3000' + path
     http.get(remote_url, (resp) => {
       let rawData = '';
       resp.on('data', (chunk) => { rawData += chunk; });
       resp.on('end', () => {
         try {
           console.log(rawData)
           rawObj = JSON.parse(rawData)
           res.statusCode = 200;
           res.end(JSON.stringify({remote_url: remote_url, remote_ip: resp.connection.remoteAddress, remote_info: rawObj}, null, 3))
   

   In our case the REMOTE_IP will be shared.widgets.example.com because of the following in common/user_data_app/main.tf:

   output user_data_centos {
     value = replace(local.shared_app_user_data_centos, "REMOTE_IP", var.remote_ip)
   }
   

   And back in application1/main.tf:

   module user_data_app {
     source    = "../common/user_data_app"
     remote_ip = "shared.widgets.example.com"
   }
   

4. Create the resources:

   terraform init
   terraform apply
   

   Results look something like this:

   $ terraform apply
   ...
   Apply complete! Resources: 2 added, 0 changed, 0 destroyed.
   
   Outputs:
   
   ibm1_private_ip = "10.1.0.4"
   ibm1_public_ip = "52.116.140.202"
   test_info = "curl 52.116.140.202:3000/info"
   test_remote = "curl 52.116.140.202:3000/remote"
   

   Try the two curl commands (test_info, test_remote) that were suggested above. Copy the statements from your output.

   curl 52.116.140.202:3000/info
   

   The first one results in something like what was captured below:

   {
      "req_url": "/info",
      "os_hostname": "widget0-application1-vsi",
      "ipArrays": [
         [
            "10.1.0.4"
         ]
      ]
   }
   

   Then try the second command (from your output):

   curl 52.116.140.202:3000/remote
   

   Wait, that second curl command did not work! Fix it in the next step. Remember these curl commands, you will use them again shortly.

Create Transit Gateway

IBM Cloud Transit Gateway is a network service used to interconnect IBM Cloud VPC resources providing dynamic scalability, high availability and peace of mind that data isn’t traversing the public internet. Earlier the CIDR blocks for each of the VPCs were chosen without overlap to allow Transit Gateway to route packets by IP address.

1. Change directory and become a member of the network group (use the existing API key):

   cd ../network
   source local.env
   

2. Optionally investigate the Terraform files. Open the main.tf file and you will notice the Transit Gateway resources (ibm_tg). Each has a count = var.transit_gateway ? 1 : 0. This is a Terraform construct that creates an array of resources of length 1 or 0 based on the value of transit_gateway. An array of length 0 will result in no resource. For example:

   resource "ibm_tg_gateway" "tgw"{
     count = var.transit_gateway ? 1 : 0
     name              = "${var.basename}-tgw"
     location          = var.ibm_region
     global            = false
     resource_group    = data.ibm_resource_group.network.id
   } 
   

3. Edit the terraform.tfvars file and uncomment the line transit_gateway = true to enable the provisioning of Transit Gateway.

4. Apply the change

   terraform apply
   

5. Print the Transit Gateway using the commands below.

   ibmcloud tg gateways
   GATEWAY_ID=$(ibmcloud tg gateways --output json | sed -e '/^OK$/d' | jq -r '.[]|select(.name=="'$basename-tgw'")|.id')
   ibmcloud tg connections $GATEWAY_ID
   

   Notice the three VPC connections in the output. It will resemble this:

   $ ibmcloud tg gateways
   Listing gateways under account 
   OK
   
   GatewayID           e2801c16-1a6d-4d47-9c58-1a3b3c1d9b1b
   CRN                 crn:v1:bluemix:public:transit:us-south:a/86785309::gateway:e2801c16-1a6d-4d47-9c58-1a3b3c1d9b1b
   Name                widget0-tgw
   Routing             local
   Location            us-south
   Created             2020-07-16T09:09:38.048-07:00
   Resource group ID   bf6e75cd71854576a31056abced2abf0
   Status              available
   
   $ GATEWAY_ID=$(ibmcloud tg gateways --output json | sed -e '/^OK$/d' | jq -r '.[]|select(.name=="'$basename-tgw'")|.id')
   $ ibmcloud tg connections $GATEWAY_ID
   Listing connections for gateway e2801c16-1a6d-4d47-9c58-1a3b3c1d9b1b under account 
   OK
   
   Name                    widget0-shared
   Network Type            vpc
   Connection ID           dff6ecfd-388d-471a-908a-98880426fbee
   Status                  attached
   Default Prefix Filter   permit
   NetworkID               crn:v1:bluemix:public:is:us-south:a/86785309::vpc:r006-b08a7c2c-c0ea-4908-b0ab-b96cd8ba221a
   
   Name                    widget0-application1
   Network Type            vpc
   Connection ID           bbce29f9-9ce4-47d4-911d-5341601cea07
   Status                  attached
   Default Prefix Filter   permit
   NetworkID               crn:v1:bluemix:public:is:us-south:a/86785309::vpc:r006-8fdc0e7e-3a98-4f6b-93e0-505c61e3faac
   
   Name                    widget0-application2
   Network Type            vpc
   Connection ID           208c00cc-aee2-498e-8b1c-37ddc276f200
   Status                  attached
   Default Prefix Filter   permit
   NetworkID               crn:v1:bluemix:public:is:us-south:a/86785309::vpc:r006-fa80afa7-b16b-4db7-95dd-69a558db4285
   

6. Optionally navigate the Transit Gateway and find the gateway created above.

7. Execute the curl command from above that failed earlier to verify there is a path from the application1 VPC to the shared VPC. It will look something like this:

   $ curl 169.48.152.220:3000/remote
   
   {
      "remote_url": "http://shared.widgets.example.com:3000/info",
      "remote_ip": "10.0.0.4",
      "remote_info": {
         "req_url": "/info",
         "os_hostname": "widget0-shared-vsi",
         "ipArrays": [
            [
               "10.0.0.4"
            ]
         ]
      }
   }
   

Insert a Load Balancer and replace the DNS record

1. Change directory and become a member of the shared access group (use the existing API key):

   cd ../shared
   source local.env
   

2. Optionally investigate the Terraform configuration files. The Load Balancer for VPC service distributes traffic among multiple server instances within the same region of your VPC. The shared team can balance load between multiple instances. For now the load balancer pool will only have the single instance created earlier. See the lb.tf for the implementation. In the ibm_is_lb resource use the dns clause to identify the private DNS zone to place the load balancer DNS address:

   resource "ibm_is_lb" "shared_lb" {
     ...
     dns {
       instance_crn = local.network_context.dns.crn
       zone_id      = local.network_context.dns.zone_id
     }
   }
   

3. Provide a DNS CNAME record shared.widgets.example.com to identify the load balancer, so the applications continue to work without source code changes:

   # shared.widgets.example.com
   resource ibm_dns_resource_record "shared_lb" {
     count = var.shared_lb ? 1 : 0 # shared load balancer?
     instance_id = local.network_context.dns.guid
     zone_id     = local.network_context.dns.zone_id
     type        = "CNAME"
     name        = "shared"
     rdata       = ibm_is_lb.shared_lb[0].hostname
     ttl         = 3600
   }
   

   The same count = var.shared_lb ? 1 : 0 is used. The load balancer hostname will be something like b7911a41-us-south.widgets.example.com. The CNAME record will be used by the applications.

4. Edit the terraform.tfvars file and uncomment shared_lb = true. Then apply the changes:

   terraform apply
   

5. Execute the curl .../remote command from the previous application1 section (ignore the output just generated for the shared microservice). Notice that the remote_ip is 10.0.1.4, the load balancer, and the remote_info is 10.0.0.4, the instance. Curl a few more times and notice the remote_ip for the load balancer may change.

   $ curl 169.48.152.220:3000/remote
   
   {
      "remote_url": "http://shared.widgets.example.com:3000/info",
      "remote_ip": "10.0.1.4",
      "remote_info": {
         "req_url": "/info",
         "os_hostname": "widget0-shared-vsi",
         "ipArrays": [
            [
               "10.0.0.4"
            ]
         ]
      }
   }
   

Create a publicly facing microservice for an application (Application2 Team)

The second application team environment is identical to the first. Optionally, create application2 by modifying application1.

1. Enter the ./application1 directory and create the application2 directory

   cd ../application1
   mkdir ../application2
   sed -e 's/application1/application2/g' main.tf > ../application2/main.tf
   cp terraform.tfvars variables.tf versions.tf ../application2
   

2. Change directory, generate an API key in the local.env, and become a member of the application2 access group:

   team=application2
   cd ../$team
   echo export TF_VAR_ibmcloud_api_key=$(ibmcloud iam service-api-key-create $team $basename-$team --output json | jq .apikey) > local.env
   cat local.env
   source local.env
   

3. Create the resources:

   terraform init
   terraform apply
   

4. Test the curl commands

Remove resources

1. Destroy the resources. You can change (cd) to the team directories in order, and execute source local.env; terraform destroy. The order is application2, application1, shared, network, admin. There is also a script that will do this for you:

   cd ..
   ./bin/destroy.sh
   

Expand the tutorial

Conclusions

The architecture of a system is influenced by the containment and ownership of cloud resources. It is important for architects from all aspects of the system contribute their concerns to the architecture. Each team needs the ability to control the resources they produce and release. Isolation will reduce the likelihood of problems and contain the blast radius when problems occur.

Related content

• Tutorial: Public frontend and private backend in a Virtual Private Cloud,
• Tutorial: Deploy a LAMP stack using Terraform