Home Explore Help
Register Sign In
AFS
/
infrastructure-notes
2
0
Fork 0
Files Issues 0 Pull Requests 0 Wiki
Tree: 883a468866
Branches Tags
infrastructure-...
/
Vault Notes.md

Vault Notes.md 12 KB
History Raw

Vault Notes.md

Vault is setup with dynamoDB as the backend. Vault has 3 nodes in a cluster and an AWS ALB as the frontend. The Vault is unsealed with AWS KMS instead of the usual master key.

the vault binary is located at /usr/local/bin/vault

Additional Notes are located here: msoc-infrastructure - Vault README.md

How to log into CLI on the Vault server.

  • login to the web interface
  • copy token
  • run this on vault-1 vault login
  • paste token and login

Auth Error? Try populating the Bash variables.
export VAULT_ADDR=https://vault.pvt.xdrtest.accenturefederalcyber.com

Connectivity Issue? Try hitting the health endpoint via curl. Unset the proxy variables to avoid using the proxy. env -u http_proxy -u https_proxy -u HTTP_PROXY -u HTTPS_PROXY -u no_proxy -u NO_PROXY curl --insecure https://127.0.0.1/v1/sys/health

  1. change made to the service file Unknown lvalue 'StartLimitIntervalSec' in section 'Service'

Failed to parse capability in bounding/ambient set, ignoring: CAP_IPC_LOCK,CAP_NET_BIND_SERVICE

Oct 30 13:31:32 vault-1 systemd: [/etc/systemd/system/vault.service:16] Failed to parse capability in bounding/ambient set, ignoring: CAP_IPC_LOCK,CAP_NET_BIND_SERVICE

TEST VAULT Notes

msoc-infrastructure - Vault README.md

  1. stop vault service from salt on all vault instances
    • 1.1 salt vault* cmd.run 'systemctl stop vault'
  2. wipe dynamoDB (select items-> actions -> delete) until there are no more items (BE SURE to BACKUP FIRST!)
  3. start vault
    • 3.1 run salt state to ensure it is in the correct state with all policies on disk.
    • 3.2 salt vault* state.sls vault
  4. On vault-1, init vault RUN on the server not salt (avoid the recovery keys from getting into logs)
    • 4.1 VAULT_ADDR=https://vault.pvt.xdrtest.accenturefederalcyber.com vault operator init -tls-skip-verify=true -recovery-shares=5 -recovery-threshold=2

  5. login

    • 5.1 VAULT_ADDR=https://vault.pvt.xdrtest.accenturefederalcyber.com vault login -tls-skip-verify=true -method=token

    • 5.2 Do yourself a favor and setup some Bash Variables or run commands from salt

      export VAULT_ADDR=https://vault.pvt.xdrtest.accenturefederalcyber.com
export VAULT_ADDR=https://127.0.0.1
export VAULT_ADDR=https://vault.pvt.xdrtest.accenturefederalcyber.com
export VAULT_SKIP_VERIFY=1

  6. setup okta auth

    • 6.1 VAULT_ADDR=https://vault.pvt.xdrtest.accenturefederalcyber.com vault auth enable okta
    • 6.2 VAULT_ADDR=https://vault.pvt.xdrtest.accenturefederalcyber.com vault write -tls-skip-verify=true auth/okta/config base_url="okta.com" organization="mdr-multipass" token="api_token_here"
    • 6.2 VAULT_ADDR=https://vault.pvt.xdrtest.accenturefederalcyber.com vault write -tls-skip-verify=true auth/okta/config base_url="okta.com" organization="mdr-multipass" token="$( cat ~/.okta-token )"
    • 6.3 VAULT_ADDR=https://vault.pvt.xdrtest.accenturefederalcyber.com vault auth list
    • 6.4 set the TTL for the okta auth method
    • 6.4.1 VAULT_ADDR=https://vault.pvt.xdrtest.accenturefederalcyber.com vault auth tune -default-lease-ttl=3h -max-lease-ttl=3h okta/

  7. Enable/add Policies

    • 7.1 vault policy write -tls-skip-verify=true admins /etc/vault/admins.hcl
    • 7.3 vault policy write -tls-skip-verify=true engineers /etc/vault/engineers.hcl
    • 7.4 vault policy write -tls-skip-verify=true clu /etc/vault/clu.hcl
    • 7.5 vault policy write -tls-skip-verify=true onboarding /etc/vault/onboarding.hcl
    • 7.6 vault policy write -tls-skip-verify=true portal /etc/vault/portal.hcl
    • 7.7 vault policy write -tls-skip-verify=true soc /etc/vault/soc.hcl
    • 7.8 vault policy write salt-master /etc/vault/salt-master.hcl
    • 7.9 vault policy write saltstack/minions /etc/vault/salt-minions.hcl

  8. Add external groups

    • 8.1 vault write identity/group name="admins" policies="admins" type="external"
    • 8.2 vault write identity/group name="mdr-engineers" policies="engineers" type="external"
    • 8.3 vault write identity/group name="vault-admins" policies="admins" type="external"
    • 8.4 vault write identity/group name="soc-lead" policies="soc" type="external"
    • 8.5 vault write identity/group name="soc-tier-3" policies="soc" type="external"

  9. add alias through the GUI. (use the root token to login or a temp root token (better))

    • 9.1 Access -> Groups -> admins -> Aliases -> Create alias -> mdr-admins
    • 9.2 Access -> Groups -> mdr-engineers -> Aliases -> Create alias -> mdr-engineers
    • 9.3 Access -> Groups -> vault-admins -> Aliases -> Create alias -> vault-admin
    • 9.4 Access -> Groups -> soc-lead -> Aliases -> Create alias -> Analyst-Shift-Lead
    • 9.5 Access -> Groups -> soc-tier-3 -> Aliases -> Create alias -> Analyst-Tier-3
groups alias policy
admins mdr-admins admins
mdr-engineers mdr-engineers engineers
vault-admins vault-admin admins
soc-lead Analyst-Shift-Lead soc
soc-tier-3 Analyst-Tier-3 soc

  1. enable the file audit

    • 10.1 VAULT_ADDR=https://vault.pvt.xdrtest.accenturefederalcyber.com vault audit enable -tls-skip-verify=true file file_path=/var/log/vault.log

  2. enable the aws & approle auth

    • 11.1 VAULT_ADDR=https://vault.pvt.xdrtest.accenturefederalcyber.com vault auth enable -tls-skip-verify=true aws
    • 11.2 setup approle auth using the salt-master policy
    • 11.2.1 vault auth enable approle
    • 11.2.2 vault write auth/approle/role/salt-master token_max_ttl=3h token_policies=salt-master

  3. configure the aws policies on the role (clu and portal) UPDATE THE AWS ACCOUNT!!!

  4. Create the kv V2 secret engines VAULT_ADDR=https://vault.pvt.xdrtest.accenturefederalcyber.com ~/Documents/MDR/Vault/vault secrets enable -path=engineering kv-v2 vault secrets enable -path=engineering kv-v2 vault secrets enable -path=ghe-deploy-keys kv-v2 vault secrets enable -path=jenkins kv-v2 vault secrets enable -path=onboarding kv-v2 vault secrets enable -path=onboarding-afs kv-v2 vault secrets enable -path=onboarding-gallery kv-v2 vault secrets enable -path=onboarding-saf kv-v2 vault secrets enable -path=portal kv-v2 vault secrets enable -path=soc kv-v2 vault secrets enable -version=1 -path=salt kv

vault write salt/pillar_data auth="abc123"

  1. Export the secrets (be sure to export your bash variable for VAULT_TOKEN DON'T Use ROOT TOKEN!)

#export

#!/bin/bash

while read p; do
  /Users/brad.poulton/.go/src/vault-backend-migrator/vault-backend-migrator -export $p/data/ -metadata $p/metadata/ -file $p-secrets.json -ver 2
done <engines.txt

#import

#!/bin/bash

while read p; do
  /Users/brad.poulton/.go/src/vault-backend-migrator/vault-backend-migrator -import $p/ -file $p-secrets.json -ver 2
done <engines.txt

/Users/brad.poulton/.go/src/vault-backend-migrator/vault-backend-migrator -export engineering/data/ -metadata engineering/metadata/ -file engineering-secrets.json -ver 2

/Users/brad.poulton/.go/src/vault-backend-migrator/vault-backend-migrator -export ghe-deploy-keys/data/ -metadata ghe-deploy-keys/metadata/ -file ghe-deploy-keys-secrets.json -ver 2

/Users/brad.poulton/.go/src/vault-backend-migrator/vault-backend-migrator -export jenkins/data/ -metadata jenkins/metadata/ -file jenkins-secrets.json -ver 2

/Users/brad.poulton/.go/src/vault-backend-migrator/vault-backend-migrator -export onboarding/data/ -metadata onboarding/metadata/ -file onboarding-secrets.json -ver 2

/Users/brad.poulton/.go/src/vault-backend-migrator/vault-backend-migrator -export onboarding-afs/data/ -metadata onboarding-afs/metadata/ -file onboarding-afs-secrets.json -ver 2

/Users/brad.poulton/.go/src/vault-backend-migrator/vault-backend-migrator -export onboarding-gallery/data/ -metadata onboarding-gallery/metadata/ -file onboarding-gallery-secrets.json -ver 2

/Users/brad.poulton/.go/src/vault-backend-migrator/vault-backend-migrator -export onboarding-saf/data/ -metadata onboarding-saf/metadata/ -file onboarding-saf-secrets.json -ver 2

/Users/brad.poulton/.go/src/vault-backend-migrator/vault-backend-migrator -export portal/data/ -metadata portal/metadata/ -file portal-secrets.json -ver 2

/Users/brad.poulton/.go/src/vault-backend-migrator/vault-backend-migrator -export salt/ -metadata salt/metadata/ -file salt-secrets.json

  1. Import the json secret files back into vault

    /Users/brad.poulton/.go/src/vault-backend-migrator/vault-backend-migrator -import engineering/ -file engineering-secrets.json -ver 2

/Users/brad.poulton/.go/src/vault-backend-migrator/vault-backend-migrator -import ghe-deploy-keys/ -file ghe-deploy-keys-secrets.json -ver 2

/Users/brad.poulton/.go/src/vault-backend-migrator/vault-backend-migrator -import jenkins/ -file jenkins-secrets.json -ver 2

/Users/brad.poulton/.go/src/vault-backend-migrator/vault-backend-migrator -import onboarding/ -file onboarding-secrets.json -ver 2

/Users/brad.poulton/.go/src/vault-backend-migrator/vault-backend-migrator -import onboarding-afs/ -file onboarding-afs-secrets.json -ver 2

/Users/brad.poulton/.go/src/vault-backend-migrator/vault-backend-migrator -import onboarding-gallery/ -file onboarding-gallery-secrets.json -ver 2

/Users/brad.poulton/.go/src/vault-backend-migrator/vault-backend-migrator -import onboarding-saf/ -file onboarding-saf-secrets.json -ver 2

/Users/brad.poulton/.go/src/vault-backend-migrator/vault-backend-migrator -import portal/ -file portal-secrets.json -ver 2

/Users/brad.poulton/.go/src/vault-backend-migrator/vault-backend-migrator -import salt/ -file salt-secrets.json

AWS Auth

the Vault instances have access to AWS IAM Read.

curl -v --header "X-Vault-Token:$VAULT_TOKEN" --request LIST 

https://vault.mdr.defpoint.com:443/v1/auth/aws/roles --insecure
  1. map okta to policies ( not needed ) 8.1 VAULT_ADDR=https://vault.pvt.xdrtest.accenturefederalcyber.com vault policy write -tls-skip-verify=true auth/okta/groups/mdr-admins policies=admins

Vault Logs

cat 0c86fda6-1139-7914-fef5-6b7532e9fb5a | grep -v -F '"operation":"list"' | grep -v -F '"operation":"read"'
cat c3c0b50b-9429-355d-8c8f-038e093c3e4b | grep -v -F '"operation":"list"' | grep -v -F '"operation":"read"'

entity_34d6c410 -< nothing in logs    
"entity_id":"c3c0b50b-9429-355d-8c8f-038e093c3e4b
entity_ba27bb07 < - nothing in logs
0c86fda6-1139-7914-fef5-6b7532e9fb5a

Vault with Terraform

This simple tf will grab a secret from the vault. Note that the password is printed in plaintext and there is currently no way to avoid that short of putting the values in environment variables.

And for some reason, this prompts 3 times.

variable login_username {}
variable login_password {}

provider "vault" {
  address = "https://vault.mdr.defpoint.com"
  auth_login {
    path = "auth/okta/login/${var.login_username}"

    parameters = {
      password = var.login_password
    }
  }
}

data "vault_generic_secret" "palo_auth" {
  path = "engineering/palo_alto/firewall_notes"
}

output "secret" {
  value = data.vault_generic_secret.palo_auth.data
}

Vault Timeouts for FedRAMP

Tune the auth method to reduce the life of the token provided for this auth method. AC-2(5) Sort of but not really!

vault auth tune -default-lease-ttl=3h -max-lease-ttl=3h okta/
vault auth tune -default-lease-ttl=15m -max-lease-ttl=15m okta/