# AWS Notes.md

For the latest AWS account information see [Cloud Accounts Wiki](https://github.xdr.accenturefederalcyber.com/mdr-engineering/msoc-infrastructure/wiki/cloud-accounts)


### LEGACY AWS ACCOUNTS
Root Account Alias (in AWS): defpoint-mdr-root     
Root Account Alias (for terraform/in `~/.aws/{config,credentials}`): mdr-root     
Root AWS Account ID:   350838957895     
GovCloud Root Account: 701290387780     

Test Account ID: 527700175026     
Prod Account ID: 477548533976     
Org  Account ID: 228011623757     

`assumeRole` to the test and prod accounts

---
terraform has been setup to handle the cis checks for AWS. they are found in terraform/00-cis-hardening.

Get a encoded error message from AWS?

```
AWS_PROFILE=mdr-test aws sts decode-authorization-message --encoded-message Q7h4sTOW_n_znBB7ojNotL
```

For prettier output:
```
aws --profile=mdr-test-c2-gov sts decode-authorization-message --encoded-message Q7h4sTOW_n_znBB7ojNotL... | \
        jq '.["DecodedMessage"]' | \
        sed 's/\\"/"/g' | \
        sed 's/^"//' | \
        sed 's/"$//' | \
        jq
```


-------------------------------------------
Cloudtrail metric Alarms


so .. cloudtrail writes a trail

that trail is written into a cloudwatch logs log group

in the log group, there are a number of metric filters

the metric filters create metrics, upon which a metric alarm is set

when events matching the metric filter arrive, the metric goes up, triggerting the alarm
new messages

the alarm has an SNS topic it writes to that emails me that the "metric was exceeded"



----------------------------------------------
AWS Systems Manager agent

systemctl start amazon-ssm-agent

----------------------------------------------

## AWS User Password Distribution ( legecy )

```
https://gpgtools.org/ download and install
use gpg keychain to generate pub/private keys
something something terraform
echo "wcFMA2sXDKYLpzaU<redacted>bf6clQ043oDkHIrcWK509UIy5GUpEqBV/WLmuCMHkXUgnxy12HY8qBErF58vB7/VXs5pCKp4SDYWEtK73fKmYZ5wJDW6j6OHkpYI4USZXjVYb+Utt56Qprk4KiT6VlFNNPo00r2YDABDdtxPJS3N9REzHqp+7oR2SQkiyEhcF3ZwILk2fH4mc1VQUiFu68RCqbt+QfmDt3OHIRZVPvrS4AHkCbj2fdgkbAaRMJ/21TBn8OE8WuDR4NHh5w/gWeK5m6754DzkjVLxDpsvPG2UR9ErwANEo+BI4upil2vgT+S63PIVsAmTew/7QpPavttP4rUBM47h5cMA"|base64 -D  | gpg -d
```

```
#Export in one line and base64
gpg --export | base64
```
## saml2aws
2020-05-07

on OSX, install `saml2aws` via:
```
# 2021-04-21 - you should not need to install the tap any more
#brew tap versent/homebrew-taps
brew install saml2aws
```
otherwise follow instructions on [Github](https://github.com/Versent/saml2aws).

Create `~/.saml2aws` with the following:
```
[commercial]
app_id               =
url                  = https://mdr-multipass.okta.com/home/amazon_aws/0oa3cn8m68DzMXNbc297/272
username             = your.okta.username
provider             = Okta
mfa                  = PUSH
skip_verify          = false
timeout              = 0
aws_urn              = https://signin.aws.amazon.com/saml
aws_session_duration = 28800
aws_profile          = commercial
resource_id          =
subdomain            =
role_arn             = arn:aws:iam::471284459109:role/user/mdr_engineer_readonly
http_attempts_count  =
http_retry_delay     =

[govcloud]
app_id               =
url                  = https://mdr-multipass.okta.com/home/amazon_aws/0oa3e1gtozzfDFAel297/272
username             = your.okta.username
provider             = Okta
mfa                  = PUSH
skip_verify          = false
timeout              = 0
aws_urn              = https://signin.amazonaws-us-gov.com/saml
aws_session_duration = 28800
aws_profile          = govcloud
resource_id          =
subdomain            =
role_arn             = arn:aws-us-gov:iam::701290387780:role/user/mdr_engineer_readonly
http_attempts_count  =
http_retry_delay     =
region               = us-gov-east-1

[cyber-range]
app_id               =
url                  = https://mdr-multipass.okta.com/home/amazon_aws/0oa3cn8m68DzMXNbc297/272
username             = your.okta.username
provider             = Okta
mfa                  = PUSH
skip_verify          = false
timeout              = 0
aws_urn              = https://signin.aws.amazon.com/saml
aws_session_duration = 28800
aws_profile          = cyber-range
resource_id          =
subdomain            =
role_arn             = arn:aws:iam::952430311316:role/user/mdr_engineer_readonly
http_attempts_count  =
http_retry_delay     =
```

Then you can sign in with:
```
saml2aws login --idp-account=govcloud
```

And use the profile via:
```
aws --profile govcloud ec2 describe-instances
```

Authentication not working? `saml2aws` uses the MAC keychain to store your password. When you update your password you need to update the password in keychain as well. Or resetup saml2aws. 

## AssumeRole Extension and ~/.aws/config
08/03/2020

Install the plugin for chrome/firefox:     
Firefox [aws-extend-switch-roles](https://addons.mozilla.org/en-US/firefox/addon/aws-extend-switch-roles3/#&gid=1&pid=3) 

Chrome [aws-extend-switch-roles](https://chrome.google.com/webstore/detail/aws-extend-switch-roles/jpmkfafbacpgapdghgdpembnojdlgkdl/related?hl=en)

Copy this into `~/.aws/credentials`
```
[govcloud]
region = us-gov-east-1

[commercial]
```

Then, create a link from `files/config` to `~/.aws/config`: (your path may vary)
```
ln -s ~/infrastructure-notes/files/config ~/.aws/config
```

## CloudWatch 

bit of code to search cloudwatch
```
fields @timestamp, @message 
| filter errorCode="AccessDenied" 
| fields coalesce(userIdentity.invokedBy,userIdentity.principalId) as whoo,  coalesce(requestParameters.bucketName,errorMessage) as target
| stats count() as count by bin(1d) as time, whoo, eventName, target
| sort count desc
```


### Using jq with EC2 API

relevant links

* [Reshaping JSON with jq](https://programminghistorian.org/en/lessons/json-and-jq)
* [AWS, JQ and bash command cheat sheet](https://gist.github.com/lukeplausin/b64c10f8b524bb310e0083756c42caf6)

#### dump a list of ec2 instances and some of their relevant fields into a CSV

```
AWS_PROFILE=dps_prod aws ec2 describe-instances | 
	jq -r '.Reservations[].Instances[] | [ (.Tags[]?|select(.Key=="Name")|.Value?), .InstanceId, .LaunchTime, .StateTransitionReason, .State.Name  ] | @csv'
```

### AWS Instance Scheduler in TEST

Test Ec2 instances get shutdown at night. This is accomplished via a lambda application. To prevent an EC2 instance from getting shutdown, just remove the Schedule tag.

## Fixing bad volume tags

The `volume_tags` field in `aws_instance` doesn't handle updates (due to a bug where it was updating other attached volumes).

To add volume tags:

1. Generate a list of bad volumes:
```
aws --profile legacy-mdr-test ec2 describe-volumes --filters "Name=attachment.status,Values=attached" --output yaml | grep VolumeId | awk '{print $2}' | sort -u > badvolumes.legacy-mdr-test.txt
```

2. Add the tag to all of them:
```
for i in `cat badvolumes.legacy-mdr-test.txt`
do
  echo $i
  aws --profile legacy-mdr-test ec2 create-tags --resources $i --tags "Key=Snapshot,Value=Daily"
done
```

### AWS Automatic Snapshots

Snapshot Daily

In both test and prod the EC2 EBS Lifecycle Manager handles the automatic backups for EBS Volumes. The Lifecycle Manager has a policy that looks for the tag, Snapshot:Daily. If the volume has that tag, then a snapshot is taken. 

### Cleaning Old AMIs

The open source `amicleaner` is very useful for cleaning our old AMIs, but needs a patched version. To install:
```
# Get a fork of amicleaner with the asg fix:
git clone git@github.com:ronaldosaheki/aws-amicleaner.git
cd aws-amicleaner
git checkout origin/fix-asg
pip3 install --user -e .
ln -s ~/Library/Python/3.9/bin/amicleaner /usr/local/bin/amicleaner
```

You now have amicleaner in your path, and can run `~/xdr-terraform-live/bin/clean_old_amis.sh`.