infrastructure-notes/Portal Notes.md

Portal Notes

XDR Portal Notes https://github.mdr.defpoint.com/MDR-Content/customer_portal/wiki

Portal is a custom application running on Django app in docker.

Help! Portal is down!

Start with a docker container restart salt 'ip-10' cmd.run 'docker restart nginx' salt 'ip-10' cmd.run 'docker restart portal'

Deploy Process

salt 'ip-10*' test.ping
salt 'ip-10*' cmd.run 'docker images'
salt 'ip-10*' cmd.run 'docker container ls'
salt 'ip-10*' cmd.run 'docker stop portal'
salt 'ip-10*' cmd.run 'docker stop nginx'
salt 'ip-10*' cmd.run 'docker rm portal'
salt 'ip-10*' cmd.run 'docker rm nginx'
salt 'ip-10*' cmd.run 'docker images'
salt 'ip-10*' cmd.run 'docker images --digests'
salt 'ip-10*' cmd.run 'docker rmi <image-id>'
salt 'ip-10*' state.sls docker
salt 'ip-10*' state.sls docker.portal --output-diff

(from the wiki page https://github.mdr.defpoint.com/MDR-Content/customer_portal/wiki) Last time i tried the ec2_tags grain targeting did not work.

salt -G 'ec2_tags:Name:customer-portal' cmd.run "docker images" – You will need to grab the docker image ID for the container that needs to be updated
salt -G 'ec2_tags:Name:customer-portal' cmd.run "docker stop portal"
salt -G 'ec2_tags:Name:customer-portal' cmd.run "docker rm portal"
salt -G 'ec2_tags:Name:customer-portal' cmd.run "docker rmi ${image id from above}
salt -G 'ec2_tags:Name:customer-portal' state.sls docker
salt -G 'ec2_tags:Name:customer-portal' state.sls docker.portal

Command line access

docker exec -ti nginx bash docker exec -ti portal bash

salt 'ip-10*' cmd.run 'docker restart portal'
salt 'ip-10*' cmd.run 'docker rm -f portal'
salt 'ip-10*' cmd.run 'docker rm -f nginx'
salt 'ip-10*' cmd.run 'docker pull 350838957895.dkr.ecr.us-east-1.amazonaws.com/portal_server'
salt 'ip-10*' cmd.run 'docker pull 350838957895.dkr.ecr.us-east-1.amazonaws.com/django_nginx'
salt 'ip-10*' cmd.run 'docker image ls'

salt 'ip-10*' state.sls docker.portal

Docker entrypoint.sh & Varibles

If the app.env is empty or has null values in it, then the docker did not pull the vault variables correctly.

cat /opt/portal/docker-entrypoint.sh
cat /opt/portal/app.env

Vault Connectivity

The setting of the ENVIRONMENT variable is in Salt code in the docker.portal.sls. The vault URL changes based on the ENVIRONMENT URL in the docker-entrypoint.sh file:

if [[ $ENVIRONMENT == "production" ]]; then
  export vault_addr="https://vault.mdr.defpoint.com"
  export token=$(/usr/local/src/vault_auth.sh production | jq .auth.client_token | sed -e 's/"//g')
elif [[ $ENVIRONMENT == "test" ]]; then
  export vault_addr="https://vault.mdr-test.defpoint.com"
  export token=$(/usr/local/src/vault_auth.sh test | jq .auth.client_token | sed -e 's/"//g')
fi

Did the container pull the variables correctly? salt customer-portal* cmd.run 'docker exec portal cat /opt/portal/app.env | grep POSTGRES'

#get the token from AWS Auth
/usr/local/src/vault_auth.sh test
export token=xxx

#use the token to access the portal secrets
curl --header "X-Vault-Token: $token" https://vault.pvt.xdrtest.accenturefederalcyber.com/v1/portal/data/env

Admin access

After attempting to login to the site, run this command to get admin access. python3 manage.py promote richard.t.page@accenturefederal.com python3 manage.py promote brad.poulton@accenturefederal.com

Your IP must be whitelisted! https://portal.xdrtest.accenturefederalcyber.com/admin

Troubleshooting the docker image

Logs

Nginx /var/log/nginx/portal.access.log docker logs -f nginx

Portal

salt 'ip-10*' cmd.run 'docker container ls'
salt 'ip-10*' cmd.run 'docker exec portal ls'
salt 'ip-10*' cmd.run 'docker exec portal cat /opt/portal/saml/idps.json'
salt 'ip-10*' cmd.run 'docker exec portal cat /opt/portal/saml/sp.json'
This will init the portal variables by pulling them from vault. SHOULD NOT NEED TO RUN IT
salt 'ip-10*' cmd.run 'docker exec portal sh /opt/portal/init.sh'
salt 'ip-10*' cmd.run 'docker exec portal cat /opt/portal/init.sh'
Portal auths to Vault then pulls the creds
salt 'ip-10*' cmd.run 'docker exec portal cat /usr/local/src/vault_auth.sh'

docker exec -ti portal /usr/local/src/vault_auth.sh test

ALL THE ERRORS: nginx: [emerg] host not found in upstream "portal:8000" in /etc/nginx/nginx.conf:27

{"errors":["error making upstream request: error making request: Post https://sts.amazonaws.com//: dial tcp 52.94.241.129:443: i/o timeout"]}

[WARNING ] The following arguments were ignored because they are not recognized by docker-py: [u'dns-search', u'network-alias'] [WARNING ] The following arguments were ignored because they are not recognized by docker-py: [u'dns-search']

SOULTION: NOT SURE! try stopping docker containers and service and starting back up with salt state. seems to be proxy issue working server... [dev]root@ip-10-97-10-248:~:# docker exec portal wget portal --2020-04-30 17:44:37-- http://portal/ Resolving proxy.msoc.defpoint.local (proxy.msoc.defpoint.local)... 10.96.101.188 Connecting to proxy.msoc.defpoint.local (proxy.msoc.defpoint.local)|10.96.101.188|:80... connected. Proxy request sent, awaiting response... 503 Service Unavailable 2020-04-30 17:44:38 ERROR 503: Service Unavailable. broken server... [dev]root@ip-10-97-9-59:~:# docker exec portal wget portal --2020-04-30 17:27:45-- http://portal/ Resolving proxy.msoc.defpoint.local (proxy.msoc.defpoint.local)... failed: Name or service not known. wget: unable to resolve host address 'proxy.msoc.defpoint.local'

docker exec portal wget portal

sha256:598168ec922e79106fa3f8af35dd33313aa32ae859e77673b65d52ce93852810

Updating Email Settings

A bunch of settings (email, postgres, etc) are pulled dynamically from Vault as if Vault is a parameter store.

https://vault.mdr.defpoint.com/ui/vault/secrets/portal/show/env https://vault.mdr-test.defpoint.com/ui/vault/secrets/portal/show/env

Changes to the parameters require restarting the container.