# Portal Notes XDR Portal [Notes link in our Github](https://github.xdr.accenturefederalcyber.com/MDR-Content/mdr-content/wiki/Customer-Portal) Portal is a custom application running on Django app in docker. ## Help! Portal is down! Start with a docker container restart ``` salt 'customer-portal*' cmd.run 'docker container ls' salt 'customer-portal*' cmd.run 'docker restart portal nginx' ``` Verify Vault service is running. Logs are sent to Splunk. See Splunk search below in this document. ## Command line access ``` docker exec -ti nginx bash docker exec -ti portal bash # root user careful! docker exec -u root -ti portal bash salt 'customer-portal*' cmd.run 'docker restart portal' salt 'customer-portal*' cmd.run 'docker rm -f portal' salt 'customer-portal*' cmd.run 'docker rm -f nginx' salt 'customer-portal*' cmd.run 'docker pull 350838957895.dkr.ecr.us-east-1.amazonaws.com/portal_server' salt 'customer-portal*' cmd.run 'docker pull 350838957895.dkr.ecr.us-east-1.amazonaws.com/django_nginx' salt 'customer-portal*' cmd.run 'docker image ls' salt 'customer-portal*' state.sls docker.portal ``` ## Docker entrypoint.sh & Varibles If the app.env is empty or has null values in it, then the docker did not pull the vault variables correctly. ``` cat /opt/portal/docker-entrypoint.sh cat /opt/portal/app.env ``` ## Vault Connectivity The setting of the ENVIRONMENT variable is in Salt code in the docker.portal.sls. The vault URL changes based on the ENVIRONMENT URL in the docker-entrypoint.sh file: ``` if [[ $ENVIRONMENT == "production" ]]; then export vault_addr="https://vault.mdr.defpoint.com" export token=$(/usr/local/src/vault_auth.sh production | jq .auth.client_token | sed -e 's/"//g') elif [[ $ENVIRONMENT == "test" ]]; then export vault_addr="https://vault.mdr-test.defpoint.com" export token=$(/usr/local/src/vault_auth.sh test | jq .auth.client_token | sed -e 's/"//g') fi ``` Did the container pull the variables correctly? `salt customer-portal* cmd.run 'docker exec portal cat /opt/portal/app.env | grep POSTGRES'` ``` #get the token from AWS Auth /usr/local/src/vault_auth.sh test export token=xxx #use the token to access the portal secrets curl --header "X-Vault-Token: $token" https://vault.pvt.xdrtest.accenturefederalcyber.com/v1/portal/data/env ``` ## Admin access After attempting to login to the site, run this command to get admin access. `python3 manage.py promote richard.t.page@accenturefederal.com` `python3 manage.py promote brad.poulton@accenturefederal.com` Your IP must be whitelisted to access the [Admin Site](https://portal.xdrtest.accenturefederalcyber.com/admin) TEST LB ACCESS 6b6cdd72cdec ERROR 2021-04-19 23:10:46,130 - Invalid HTTP_HOST header: 'portal-alb-test-617285333.us-gov-east-1.elb.amazonaws.com'. You may need to add 'portal-alb-test-617285333.us-gov-east-1.elb.amazonaws.com' to ALLOWED_HOSTS. [django.security.DisallowedHost:88] 54 ALLOWED_HOSTS is located.... `/opt/portal/dps_customer_portal/settings.py` You can allow all subdomains like this, but OKTA doesn't support it: ``` ALLOWED_HOSTS = [ PORTAL_DOMAIN, 'localhost', '127.0.0.1', 'portal.xdr.accenturefederalcyber.com', # This will make prod transition smoother 'portal.xdrtest.accenturefederalcyber.com', # This will make testing easier '.us-gov-east-1.elb.amazonaws.com', # Allow LB ] ``` python3 manage.py shell from user_portal.models import User User.objects.filter(email="richard.t.page@accenturefederal.com").delete() User.objects.filter(email="wesley.a.leonard@accenturefederal.com").delete() ## Troubleshooting the docker image ### Logs logs are being sent to Splunk. This logging is configured in Salt. `index=os source="journald://docker" host="customer-portal-*"` Nginx ``` /var/log/nginx/portal.access.log docker logs -f nginx ``` Portal `startup.log` I assume the logs going to splunk are the `stdout`. ``` salt 'customer-portal*' cmd.run 'docker container ls' salt 'customer-portal*' cmd.run 'docker exec portal ls' salt 'customer-portal*' cmd.run 'docker exec portal cat /opt/portal/saml/idps.json' salt 'customer-portal*' cmd.run 'docker exec portal cat /opt/portal/saml/sp.json' #This will init the portal variables by pulling them from vault. SHOULD NOT NEED TO RUN IT salt 'customer-portal*' cmd.run 'docker exec portal sh /opt/portal/init.sh' salt 'customer-portal*' cmd.run 'docker exec portal cat /opt/portal/init.sh' #Portal auths to Vault then pulls the creds salt 'customer-portal*' cmd.run 'docker exec portal cat /usr/local/src/vault_auth.sh' docker exec -ti portal /usr/local/src/vault_auth.sh test ``` ALL THE ERRORS: nginx: [emerg] host not found in upstream "portal:8000" in /etc/nginx/nginx.conf:27 {"errors":["error making upstream request: error making request: Post https://sts.amazonaws.com//: dial tcp 52.94.241.129:443: i/o timeout"]} [WARNING ] The following arguments were ignored because they are not recognized by docker-py: [u'dns-search', u'network-alias'] [WARNING ] The following arguments were ignored because they are not recognized by docker-py: [u'dns-search'] SOULTION: NOT SURE! try stopping docker containers and service and starting back up with salt state. seems to be proxy issue working server... [dev]root@ip-10-97-10-248:~:# docker exec portal wget portal --2020-04-30 17:44:37-- http://portal/ Resolving proxy.msoc.defpoint.local (proxy.msoc.defpoint.local)... 10.96.101.188 Connecting to proxy.msoc.defpoint.local (proxy.msoc.defpoint.local)|10.96.101.188|:80... connected. Proxy request sent, awaiting response... 503 Service Unavailable 2020-04-30 17:44:38 ERROR 503: Service Unavailable. broken server... [dev]root@ip-10-97-9-59:~:# docker exec portal wget portal --2020-04-30 17:27:45-- http://portal/ Resolving proxy.msoc.defpoint.local (proxy.msoc.defpoint.local)... failed: Name or service not known. wget: unable to resolve host address 'proxy.msoc.defpoint.local' docker exec portal wget portal sha256:598168ec922e79106fa3f8af35dd33313aa32ae859e77673b65d52ce93852810 # Updating Email Settings A bunch of settings (email, postgres, etc) are pulled dynamically from Vault as if Vault is a parameter store. [Vault Prod](https://vault.pvt.xdr.accenturefederalcyber.com/ui/vault/secrets/portal/show/env) [Vault Dev](https://vault.pvt.xdrtest.accenturefederalcyber.com/ui/vault/secrets/portal/show/env) Changes to the parameters require restarting the container. # Migrating to GovCloud ## 2. Database Dump on legacy portal server ip-10-81-8-205 ``` curl https://s3.amazonaws.com/rds-downloads/rds-ca-2019-root.pem -o root.crt mkdir .postgresql mv root.crt .postgresql # Have to install pg10 from the official postgres repositories, but we'll remove it when we're done sudo yum install https://download.postgresql.org/pub/repos/yum/10/redhat/rhel-7-x86_64/pgdg-redhat-repo-latest.noarch.rpm sudo yum install postgresql11 # yes, yes, yes pg_dump postgresql://root-e26bbf10-ee24-2d05-97b7-47c54d9224a1@customerportal.csskgjc1suov.us-east-1.rds.amazonaws.com:5432/customerportal?sslmode=verify-full | gzip > backup.psql.gz # Enter password which you got from Vault - portal/env/POSTGRES_PASSWORD ``` ## Restore the database NOTE: New DB URL: customerportal.csqclvntmsrg.us-gov-east-1.rds.amazonaws.com ``` scp prod-ip-10-81-8-205:backup.psql.gz ./restore.psql.gz scp restore.psql.gz gc-prod-customer-portal-26a: ssh gc-prod-customer-portal-26a # NOTE: This url is different from above, as there's a differnet cert needed curl https://s3.us-gov-west-1.amazonaws.com/rds-downloads/rds-ca-us-gov-east-1-2017-root.pem -o root.crt mkdir .postgresql mv root.crt .postgresql # Have to install pg10 from the official postgres repositories, but we'll remove it when we're done sudo yum install https://download.postgresql.org/pub/repos/yum/10/redhat/rhel-7-x86_64/pgdg-redhat-repo-latest.noarch.rpm sudo yum install postgresql10 # create user CREATE USER root-e26bbf10-ee24-2d05-97b7-47c54d9224a1 WITH PASSWORD ''; GRANT ALL PRIVILEGES ON DATABASE customerportal TO root-e26bbf10-ee24-2d05-97b7-47c54d9224a1; zcat restore.psql.gz | psql --set ON_ERROR_STOP=on --single-transaction postgresql://portal@customerportal.csqclvntmsrg.us-gov-east-1.rds.amazonaws.com:5432/customerportal?sslmode=verify-full # No errors the first time, wtf? ``` To restore a second time: ``` echo 'DROP DATABASE customerportal;' | psql postgresql://portal@customerportal.csqclvntmsrg.us-gov-east-1.rds.amazonaws.com:5432/postgres?sslmode=verify-full echo 'CREATE DATABASE customerportal;' | psql postgresql://portal@customerportal.csqclvntmsrg.us-gov-east-1.rds.amazonaws.com:5432/postgres?sslmode=verify-full psql postgresql://portal@customerportal.csqclvntmsrg.us-gov-east-1.rds.amazonaws.com:5432/postgres?sslmode=verify-full CREATE ROLE "root-e26bbf10-ee24-2d05-97b7-47c54d9224a1"; DROP ROLE "root-e26bbf10-ee24-2d05-97b7-47c54d9224a1"; \du \q # the root-e26bbf10-ee24-2d05-97b7-47c54d9224a1 user is not really needed lets switch it to the portal user. find and replace root-e26bbf10-ee24-2d05-97b7-47c54d9224a1 to portal echo 'DROP DATABASE jira' | psql postgresql://jira@jira2.csqclvntmsrg.us-gov-east-1.rds.amazonaws.com:5432/postgres?sslmode=verify-full echo 'CREATE DATABASE jira' | psql postgresql://jira@jira2.csqclvntmsrg.us-gov-east-1.rds.amazonaws.com:5432/postgres?sslmode=verify-full zcat restore.psql.gz | psql --set ON_ERROR_STOP=on --single-transaction postgresql://jira@jira2.csqclvntmsrg.us-gov-east-1.rds.amazonaws.com:5432/jira?sslmode=verify-full ``` ## Vuln Scan 08/16/2021 ### Create user ``` [gc-dev]root@customer-portal-982:~:# docker exec portal python3 manage.py shell --command="from user_portal.models import User,Company; u=User.objects.create_user(username='svc-scan', password='Uj4eY45cu59]eVL'); u.userprofileinfo.company=Company.objects.first(); u.save();" INFO 2021-08-16 16:12:26,022 - TokenAdmin already registered (TokenProxy) [user_portal.admin:87] 6787 INFO 2021-08-16 16:12:26,022 - TokenAdmin already registered (TokenProxy) [user_portal.admin:87] 6787 ``` ### Verify user login into portal and go to /admin side. then users. Ensure svc-scan shows up. ### View login form go to https://portal.xdrtest.accenturefederalcyber.com/login/ and open developer tools in browser. Find the below code with the style="display: none" ```