infrastructure-notes/Terraform Notes.md

Terraform Notes.md

Hashicorp Terraform is used to deploy AWS resources by writing code.

Folder Structure

00-cis-hardening - CIS Hardening for MDR root - (Ryan D'Amour, how does this go to other accounts)
00-organizations-and-iam - IAM Roles and Policies across accounts (NOTE: No workspaces, applies everywhere)
00-state-mgmt - S3 buckets for state management (may be prerequisite for others)
01-eips - Elastic IPs and Associated DNS Record (protection from accidentally deletion)
02-msoc_vpc - Managed SOC VPC (msoc is old name) - Meat and potatoes of command and control
03-mgmt - ? Maybe Unused ? - Most appears to be junk, tread carefully.
04-ghe - GitHub Enterprise - May be junk, GHE may be created elsewehere. Tread carefully.
05-customer_portal - Web App for Customers in Docker using ECR, in its own vpc, running on ec2 running docker, not in fargate)
10-custpod1 - Splunk Monitoring Console + junk (Could probably burn and update)
11-codebuild - Code Build to make RPMs
12-fargate - Fargate for syslog-ng that gets ghe logs into moose
100-moose - Our splunk environment (watch for modules of modules of modules)
101-afs - AFS Customer Environment
102-saf - SAF ("Smart and Final") - Powered Down through console - DO NOT TOUCH THE TF
103-nga - FEDRAMP SPONSOR NGA ("National Gallery of Art"), sometimes referred to as Gallery.
104-coalfire - Our FedRAMP Auditors (Standard customer with kali box)
105-cf2 - Our FedRAMP Auditors 2nd Environment
106-ma-c19 - Massachusetts Covid-19 (Internal AFS customer)
107-la-c19 - Louisiana Covid-19 (Internal AFS customer)
common - Common files that are symbolicly linked into other folders
modules - Reusable code - Do not run terraform here! A mix of homebrewed and third party modules.

TFswitcher

• Introduction to tfswitch - Main site
• tfswitch Github

• Installation

  brew install warrensbox/tap/tfswitch
  brew install warrensbox/tap/tgswitch
  

If there is a file that has a terraform version specified, running tfswitch will automatically switch to that version.

Debug

06/2020

Enable debug
export TF_LOG=DEBUG
export TF_LOG_PATH=./terraform.log

Disable debug
export TF_LOG=

Workspaces

05/2020

---

#workspaces are being used to break up environments. 
terraform workspace list
terraform workspace select test


#Strange errors? Unexpected results? try this
rm .terraform
terraform init

#State issues
terraform state show aws_ami.msoc_base
terraform refresh -target=data.aws_ami.msoc_base

Terraform also has a DynamoDB State lock (msoc-terraform-lock). This will prevent terraform state breakage.

To manually remove the lock: Terraform CLI - Command: force-unlock

---

View TF code Terraform AWS modules Github

---

Modules

We are using the aws ec2-instance module

• Terrform Registry Modules

• Terraform module which creates EC2 instance(s) on AWS - v2.13.0

• Terraform AWS Module Github

var.something means this is a module that needs the variable to run. Your code will fill the variable. data is a read-only terrafom object that queries provider or generates something on the localhost locals are variables that can refer to variables or other locals variables - expecting data from somewhere else. provider instance of the API

Some files are symlinks.

ln -s ../common/variables.tf variables.tf
ln -s ../amis.tf amis.tf
ln -s ../../../../prod/aws-us-gov/mdr-prod-c2/090-instance-vault/README.md README.md

---

IAM Role

get this error?

aws_iam_policy.nga_instance_policy: Error creating IAM policy nga_instance_tag_read: AccessDenied:

add this

  provider = "aws.iam_admin"

---

In terraform .tf files when the self = true. That is for putting the security group into itself. e.g. add the security group to the security groups rules.

the Terraform is setup in folders. each folder is a project and apply should be run in the folder. Common is the execption as some of the projects are dependent on that folder.

Role and Policy have to be done in the IAM terraform

iam_data.tf

02-msoc_vpc/lambda.tf with security groups

terraform plan -target=
terraform plan -target=module.sensu_go_server.aws_instance.this -target=module.sensu_go_server.aws_route53_record.private

terraform apply -target=module.sensu_server.aws_route53_record.private -target=module.sensu_server.aws_instance.this

terraform apply -target=aws_security_group_rule.outbound_to_sensu -target=module.sensu_servers_sg.aws_security_group_rule.ingress_with_cidr_blocks[0] -target=module.sensu_servers_sg.aws_security_group_rule.ingress_with_cidr_blocks[1]

terraform apply -target=module.vpc_default_security_groups.aws_security_group_rule.typical_host_outbound_to_sensu_8081 -target=aws_security_group_rule.vault_server_to_sensu -target=module.vpc_default_security_groups.aws_security_group_rule.typical_host_outbound_to_sensu_5672


terraform apply -target=module.afs_cluster.module.vpc_default_security_groups.aws_security_group_rule.typical_host_outbound_to_sensu_5672 -target=module.afs_cluster.module.vpc_default_security_groups.aws_security_group_rule.typical_host_outbound_to_sensu_8081

Updating to TF13 and AWS 3

We were generally using AWS 3 as soon as it came out. Now, the aws provider version is specified in xdr-terraform-live/terragrunt.hcl.

Updating from TF12 to TF13 had no major issues. If you run into a module that isn't functioning first try:

terragrunt init --reconfigure
terragrunt apply

If it's an issue with a provider, you may need to add a required_providers tag to one of your files (e.g. providers-okta.tf)that looks like this:

terraform {
  required_providers {
    okta = {
      source  = "oktadeveloper/okta"
    }
  }
}