Terraform

Overview:

Terraform automates cloud infrastructure provisioning using declarative code. To ensure security and compliance, Terraform code is stored in version-controlled repositories with restricted access, and state files are encrypted in secure backends. Credentials are managed through environment variables or secret management systems, and only authorized personnel can perform Terraform operations. Configurations adhere to organizational and regulatory standards, are validated through automated testing, and comply with policy-as-code tools. Clear documentation is maintained for all configurations and changes. Resource provisioning is automated with enforced tagging policies, cost management is monitored, and resources are regularly optimized. Change management processes are in place for reviewing and approving infrastructure changes. State files are regularly backed up, and recovery procedures are tested. All configuration changes are tracked with version control, and an incident response plan is maintained. Terraform operations are logged and reviewed, and compliance reports are generated to ensure adherence to policies and regulations. These practices ensure secure, compliant, and efficient use of Terraform.

Terraform usage in GCP

TL;DR

gcloud config configurations list
Does the desired configuration exist?
1. If yes:
  1. gcloud config configurations activate tf-bla-bla
2. If no:
  1. gcloud config configurations create tf-gcp-project-123
  2. gcloud config set project gcp-project-123
  3. gcloud config set auth/impersonate_service_account terraform@gcp-project-123.iam.gserviceaccount.com
    1. Service accounts can be listed with gcloud iam service accounts list
    2. If no Terraform service account exists for the project, create a new Terraform service account
Authenticate yourself to GCP:
1. gcloud auth application-default login
2. gcloud auth login
Quick checkup to see if you are impersonating or NOT
1. gcloud auth print-access-token | cut -d. -f1-2
2. gcloud auth list
Choose the correct Terraform workspace, or deployment environment:
1. terraform workspace select <nightly|stable>
  1. It is vitally important that the workspace is correct for the deployment environment you are working on. as the workspace name is often used as a variable in the Terraform configurations, for example: product-cc-${terraform.workspace}
After writing your configuration, run the plan to see if there are any errors:
1. terraform plan
Apply changes after everything looks good again:
1. terraform apply
Push changes to Github!

gcloud

In order to avoid accidents with terraform, we should create gcloud configurations that are limited to run on a specific project by impersonating a service account attached to the project. Here's how you might divide between service account configurations and non-service account configurations:

NAME                        IS_ACTIVE  ACCOUNT                     PROJECT            
att-nightly                 False      miro.salo@aholadigital.com  att-nightly
attracs-devops              False      miro.salo@aholadigital.com  attracs-devops
attracs-services            False      miro.salo@aholadigital.com  attracs-services-195009
default                     False      miro.salo@aholadigital.com
testing-miro                False      miro.salo@aholadigital.com  testing-miro-4950712
tf-att-nightly              True       miro.salo@aholadigital.com  att-nightly
tf-attracs-devops           False      miro.salo@aholadigital.com  attracs-devops
tf-attracs-services-195009  False      miro.salo@aholadigital.com  attracs-services-195009
tf-testing-miro-4950712     False      miro.salo@aholadigital.com  testing-miro-4950712

Simply add a tf- prefix to the configuration that uses service account impersonation, like so: gcloud config configurations create tf-my-project

Find out whether a terraform service account exists in your project

In order to complete the service account impersonation, we'll need to figure out if we need to create the service account first. Run the following command: gcloud iam service-accounts list

DISPLAY NAME                            EMAIL                                                          
App Engine default service account      testing-miro-4950712@appspot.gserviceaccount.com
Confidential computing service account  confidential@testing-miro-4950712.iam.gserviceaccount.com
Compute Engine default service account  265113577898-compute@developer.gserviceaccount.com
test-sa-bigquery                        test-sa-bigquery@testing-miro-4950712.iam.gserviceaccount.com
terraform                               terraform@testing-miro-4950712.iam.gserviceaccount.com

If you do have a terraform service account here, great, you can skip the next part and continue to adding the service account impersonation to your gcloud configurations. Otherwise continue to the next step.

Create a terraform service account and give it proper permissions

You can use the following terraform configurations to create the service account:

[

github.com

https://github.com/Attracs/adi-infra/tree/main/terraform/gcp/templates/create-terraform-sa

](https://github.com/Attracs/adi-infra/tree/main/terraform/gcp/templates/create-terraform-sa)

When running these configurations, use your own account to run them, instead of service account impersonation.

Adding service account impersonation to your gcloud configurations

Now that a terraform service account exists in the project, we can proceed with adding it to your gcloud config: gcloud config set auth/impersonate_service_account terraform@my-project.iam.gserviceaccount.com

Finally, authenticate yourself and set the service account to be the default account that gcloud commands will use: gcloud auth application-default login Complete the authentication and you should now see the following on your terminal when running gcloud commands:

WARNING: This command is using service account impersonation. All API calls will be executed as [terraform@my-project.iam.gserviceaccount.com].

Terraform workspaces

You can use Terraform workspaces to manage dev, nightly, stable and other environments without having to copy and paste your Terraform configs across each environment. Simply create a new workspace with terraform workspace new workspace-name to get you started. You can switch between existing workspaces with terraform workspace select workspace-name. You can then use your workspaces as variables in for example the names of your Terraform configurations. You could have a resource like this:

resource "google_service_account" "service_account" {
  account_id   = "serviceaccount-${terraform.workspace}"
  display_name = "${terraform.workspace} Service Account"
}

This would generate a service account with account id serviceaccount-workspace-name and display name workspace-name Service Account. You can also make workspace-specific configurations by naming configuration blocks in the config.tf file with the name of your workspace. Further documentation below under config.tf -specific documentation

Terraform files

The most important files that you should have on any GCP Terraform project are these: * api.tf * config.tf * main.tf * terraform.tfvars * variables.tf Below is detailed documentation of the function for each of these files:

api.tf

Used to loop through each API to make sure it is enabled so our Terraform configurations can run properly.

variable "gcp_service_list" {
  description = "The list of apis necessary for the project"
  type = list(string)
  default = [
    "iam.googleapis.com",
    "cloudkms.googleapis.com",
    "cloudresourcemanager.googleapis.com"
  ]
}

#enable apis for project    
resource "google_project_service" "gcp_services" {
  for_each = toset(var.gcp_service_list)
  project = var.project
  service = each.key
}

config.tf

Used to reference variables from a single place and define workspace-specific configurations

locals {
  configs = {
    # This area applies it's configs to all environments
    region = "europe-north-1"

    nightly = {
      # This area applies it's configs to nightly environment
      service_account = {
        account_id  = "serviceaccount-${terraform.workspace}"
        name        = "${terraform.workspace} Service Account"
      }
    }
  }
}

main.tf

Used to define providers (GCP, AWS, Azure etc.) and basic variables needed to help in configurations. You should probably copy the provider part from the Terraform providers web-page as the version will update frequently. We will likely stop using Terraform's own code soon anyways due to the licensing issues and switch to the open-source Terraform OpenTofu.

terraform {
  required_providers {
    google = {
      source = "hashicorp/google"
      version = "4.60.1"
    }
  }
}

provider "google" {
  # Configuration options
  project = var.project
  region  = var.region
  zone    = var.zone
}

# Use cloud storage as backend, change PREFIX!
terraform {     
  backend "gcs" {       
    bucket = "adi-terraform-state"      
    prefix = "terraform/state/tms-confidential-compute"
  }     
}

We save our Terraform states in the cloud storage bucket so everyone always has the same states, otherwise states would be out-of-sync and everyone would be creating and deleting each other's resources.

terraform.tfvars

Here we define project so we can use main.tf to define project as var.project that we can then use in our Terraform configurations.

project = "my-project"

variables.tf

Here we store defaults for commonly used variables.

variable "project" { }

variable "region" {
  default = "europe-north1"
}

variable "zone" {
  default = "europe-north1-a"
}

variable "billing_account" {
  type = string
  default = "01228C-D05D29-D4D6B0"  
}

#attracs.net id: 440817507889
variable "org_id" {
  type = string
  default = "440817507889"
}