# Vault Notes.md Vault is setup with dynamoDB as the backend. Vault has 3 nodes in a cluster and an AWS ALB as the frontend. The Vault is unsealed with AWS KMS instead of the usual master key. the vault binary is located at `/usr/local/bin/vault` Additional Notes are located here: [msoc-infrastructure - Vault README.md](https://github.xdr.accenturefederalcyber.com/mdr-engineering/msoc-infrastructure/blob/master/salt/fileroots/vault/README.md) ## How to log into CLI on the Vault server. - login to the web interface - copy token - run this on vault-1 `vault login` - paste token and login Auth Error? Try populating the Bash variables. `export VAULT_ADDR=https://vault.pvt.xdrtest.accenturefederalcyber.com` Connectivity Issue? Try hitting the health endpoint via curl. Unset the proxy variables to avoid using the proxy. `env -u http_proxy -u https_proxy -u HTTP_PROXY -u HTTPS_PROXY -u no_proxy -u NO_PROXY curl --insecure https://127.0.0.1/v1/sys/health` 1. change made to the service file Unknown lvalue 'StartLimitIntervalSec' in section 'Service' Failed to parse capability in bounding/ambient set, ignoring: CAP_IPC_LOCK,CAP_NET_BIND_SERVICE Oct 30 13:31:32 vault-1 systemd: [/etc/systemd/system/vault.service:16] Failed to parse capability in bounding/ambient set, ignoring: CAP_IPC_LOCK,CAP_NET_BIND_SERVICE ## TEST VAULT Notes [msoc-infrastructure - Vault README.md](https://github.xdr.accenturefederalcyber.com/mdr-engineering/msoc-infrastructure/blob/master/salt/fileroots/vault/README.md) 1. stop vault service from salt on all vault instances - 1.1 `salt vault* cmd.run 'systemctl stop vault'` 2. wipe dynamoDB (select items-> actions -> delete) until there are no more items (BE SURE to BACKUP FIRST!) 3. start vault - 3.1 run salt state to ensure it is in the correct state with all policies on disk. - 3.2 `salt vault* state.sls vault` 4. On `vault-1`, init vault RUN on the server not salt (avoid the recovery keys from getting into logs) - 4.1 `VAULT_ADDR=https://vault.pvt.xdrtest.accenturefederalcyber.com vault operator init -tls-skip-verify=true -recovery-shares=5 -recovery-threshold=2` 5. login - 5.1 `VAULT_ADDR=https://vault.pvt.xdrtest.accenturefederalcyber.com vault login -tls-skip-verify=true -method=token` - 5.2 Do yourself a favor and setup some Bash Variables or run commands from salt ``` export VAULT_ADDR=https://vault.pvt.xdrtest.accenturefederalcyber.com export VAULT_ADDR=https://127.0.0.1 export VAULT_ADDR=https://vault.pvt.xdrtest.accenturefederalcyber.com export VAULT_SKIP_VERIFY=1 ``` 6. setup okta auth - 6.1 `VAULT_ADDR=https://vault.pvt.xdrtest.accenturefederalcyber.com vault auth enable okta` - 6.2 `VAULT_ADDR=https://vault.pvt.xdrtest.accenturefederalcyber.com vault write -tls-skip-verify=true auth/okta/config base_url="okta.com" organization="mdr-multipass" token="api_token_here"` - 6.2 `VAULT_ADDR=https://vault.pvt.xdrtest.accenturefederalcyber.com vault write -tls-skip-verify=true auth/okta/config base_url="okta.com" organization="mdr-multipass" token="$( cat ~/.okta-token )"` - 6.3 `VAULT_ADDR=https://vault.pvt.xdrtest.accenturefederalcyber.com vault auth list` - 6.4 `set the TTL for the okta auth method` - 6.4.1 `VAULT_ADDR=https://vault.pvt.xdrtest.accenturefederalcyber.com vault auth tune -default-lease-ttl=3h -max-lease-ttl=3h okta/` 7. Enable/add Policies - 7.1 vault policy write -tls-skip-verify=true admins /etc/vault/admins.hcl - 7.3 vault policy write -tls-skip-verify=true engineers /etc/vault/engineers.hcl - 7.4 vault policy write -tls-skip-verify=true clu /etc/vault/clu.hcl - 7.5 vault policy write -tls-skip-verify=true onboarding /etc/vault/onboarding.hcl - 7.6 vault policy write -tls-skip-verify=true portal /etc/vault/portal.hcl - 7.7 vault policy write -tls-skip-verify=true soc /etc/vault/soc.hcl - 7.8 vault policy write salt-master /etc/vault/salt-master.hcl - 7.9 vault policy write saltstack/minions /etc/vault/salt-minions.hcl 8. Add external groups - 8.1 vault write identity/group name="admins" policies="admins" type="external" - 8.2 vault write identity/group name="mdr-engineers" policies="engineers" type="external" - 8.3 vault write identity/group name="vault-admins" policies="admins" type="external" - 8.4 vault write identity/group name="soc-lead" policies="soc" type="external" - 8.5 vault write identity/group name="soc-tier-3" policies="soc" type="external" 9. add alias through the GUI. (use the root token to login or a temp root token (better)) - 9.1 Access -> Groups -> admins -> Aliases -> Create alias -> mdr-admins - 9.2 Access -> Groups -> mdr-engineers -> Aliases -> Create alias -> mdr-engineers - 9.3 Access -> Groups -> vault-admins -> Aliases -> Create alias -> vault-admin - 9.4 Access -> Groups -> soc-lead -> Aliases -> Create alias -> Analyst-Shift-Lead - 9.5 Access -> Groups -> soc-tier-3 -> Aliases -> Create alias -> Analyst-Tier-3 | groups | alias | policy | | ------ | ----- | ------ | | admins | mdr-admins | admins | | mdr-engineers | mdr-engineers | engineers | | vault-admins | vault-admin | admins | | soc-lead | Analyst-Shift-Lead | soc | | soc-tier-3 | Analyst-Tier-3 | soc | 10. enable the file audit - 10.1 `VAULT_ADDR=https://vault.pvt.xdrtest.accenturefederalcyber.com vault audit enable -tls-skip-verify=true file file_path=/var/log/vault.log` 11. enable the aws & approle auth - 11.1 `VAULT_ADDR=https://vault.pvt.xdrtest.accenturefederalcyber.com vault auth enable -tls-skip-verify=true aws` - 11.2 `setup approle auth using the salt-master policy` - 11.2.1 `vault auth enable approle` - 11.2.2 `vault write auth/approle/role/salt-master token_max_ttl=3h token_policies=salt-master` 12. configure the aws policies on the role (clu and portal) UPDATE THE AWS ACCOUNT!!! - 12.1 VAULT_ADDR=https://vault.pvt.xdrtest.accenturefederalcyber.com vault write auth/aws/role/portal auth_type=iam bound_iam_principal_arn=arn:aws:iam::527700175026:role/portal-instance-role policies=portal max_ttl=24h - 12.2 VAULT_ADDR=https://vault.pvt.xdrtest.accenturefederalcyber.com vault write auth/aws/role/clu auth_type=iam bound_iam_principal_arn=arn:aws:iam::527700175026:role/clu-instance-role policies=clu max_ttl=24h 13. Create the kv V2 secret engines VAULT_ADDR=https://vault.pvt.xdrtest.accenturefederalcyber.com ~/Documents/MDR/Vault/vault secrets enable -path=engineering kv-v2 vault secrets enable -path=engineering kv-v2 vault secrets enable -path=ghe-deploy-keys kv-v2 vault secrets enable -path=jenkins kv-v2 vault secrets enable -path=onboarding kv-v2 vault secrets enable -path=onboarding-afs kv-v2 vault secrets enable -path=onboarding-gallery kv-v2 vault secrets enable -path=onboarding-saf kv-v2 vault secrets enable -path=portal kv-v2 vault secrets enable -path=soc kv-v2 vault secrets enable -version=1 -path=salt kv vault write salt/pillar_data auth="abc123" 14. Export the secrets (be sure to export your bash variable for VAULT_TOKEN DON'T Use ROOT TOKEN!) #export ``` #!/bin/bash while read p; do /Users/brad.poulton/.go/src/vault-backend-migrator/vault-backend-migrator -export $p/data/ -metadata $p/metadata/ -file $p-secrets.json -ver 2 done