|
|
@@ -1,5 +1,38 @@
|
|
|
# ThreatQ Notes
|
|
|
|
|
|
+## TQ HelpCenter access
|
|
|
+
|
|
|
+https://helpcenter.threatq.com/welcome.htm
|
|
|
+
|
|
|
+## TQ Passwords:
|
|
|
+
|
|
|
+TQ HelpCenter: https://vault.pvt.xdr.accenturefederalcyber.com/ui/vault/secrets/engineering/show/threatq/helpcenter
|
|
|
+Admin Account: https://vault.pvt.xdr.accenturefederalcyber.com/ui/vault/secrets/engineering/show/threatq/admin
|
|
|
+MySQL root account: https://vault.pvt.xdr.accenturefederalcyber.com/ui/vault/secrets/engineering/show/threatq/mysql_admin
|
|
|
+
|
|
|
+
|
|
|
+## Operational things
|
|
|
+
|
|
|
+### Maintenance mode
|
|
|
+
|
|
|
+
|
|
|
+TQ has "maintenance mode" where app won't let anyone log in.
|
|
|
+
|
|
|
+To enable maintenance mode:
|
|
|
+
|
|
|
+```
|
|
|
+php /var/www/api/artisan down
|
|
|
+```
|
|
|
+
|
|
|
+Hardcore maintenance is to also stop apache httpd. Use systemctl.
|
|
|
+
|
|
|
+To go out of maint:
|
|
|
+
|
|
|
+```
|
|
|
+php /var/www/api/artisan up
|
|
|
+```
|
|
|
+
|
|
|
+
|
|
|
## Compilation Notes
|
|
|
|
|
|
### Packer things
|
|
|
@@ -10,6 +43,12 @@ We have a specific packer build for the TQ AMI. It's (mostly) unlike any other
|
|
|
|
|
|
### Firstboot
|
|
|
|
|
|
+First thing - make sure all the filesystems are grown. Somehow we miss this?
|
|
|
+
|
|
|
+```
|
|
|
+mount | awk '$5 == "xfs" { print $3 }' | xargs -n 1 xfs_growfs
|
|
|
+```
|
|
|
+
|
|
|
When doing firstboot on our TQ AMI, there are some un-hardenings in place.
|
|
|
TQ expects during firstboot that the root password is a known value, that
|
|
|
apache has the rights to `su` to root using that password, and that
|
|
|
@@ -17,23 +56,27 @@ selinux is permissive.
|
|
|
|
|
|
After the GUI panel during firstboot is filled out (setting a new root
|
|
|
password, a mysql root password, and an initial admin account), then we can
|
|
|
-undo the un-hardenings.
|
|
|
+undo the un-hardenings. The firstboot GUI handles selinux.
|
|
|
|
|
|
-If you're doing a firstboot as part of restoring a backup from another TQ
|
|
|
-system, then please skip this part and come back!
|
|
|
+IF YOU'RE DOING A FIRSTBOOT AS PART OF RESTORING A BACKUP FROM ANOTHER TQ
|
|
|
+SYSTEM, THEN PLEASE SKIP THIS PART AND COME BACK!
|
|
|
|
|
|
```
|
|
|
gpasswd -d apache sugroup
|
|
|
echo "root:*" | chpasswd -e
|
|
|
systemctl mask cockpit.service
|
|
|
systemctl mask cockpit.socket
|
|
|
-shutdown -r now
|
|
|
+systemctl stop cockpit
|
|
|
```
|
|
|
|
|
|
-Now hopefully you can enable a salt minion without breaking anything
|
|
|
+You might have to reboot after this, depending on what you're doing. If you're
|
|
|
+restoring TQ from backup, probably not.
|
|
|
|
|
|
### Restoring a backup
|
|
|
|
|
|
+GO THROUGH THE FIRSTBOOT STEPS VIA THE WEBUI, SET ADMIN AND MYSQL PASSWORDS
|
|
|
+AND INSTALL THE LICENSE
|
|
|
+
|
|
|
You have to go through firstboot first, but DO NOT DO the unhardening
|
|
|
above yet!
|
|
|
|
|
|
@@ -48,6 +91,15 @@ APP_INSTANCE_ID=18112b11-a13a-414b-93e8-b6b87adf27a5
|
|
|
Now to do the restore ... docs are at https://helpcenter.threatq.com/index.htm#t=ThreatQ_Platform%2FBackup_and_Restore%2FBackup_and_Restore.htm
|
|
|
but there are couple of things not there.
|
|
|
|
|
|
+Check to see if your mysql root password is right
|
|
|
+
|
|
|
+```
|
|
|
+mysql -u root -p
|
|
|
+```
|
|
|
+
|
|
|
+If you can connect to the DB good. If not reset the pw. There's some tomfoolery here I don't get yet.
|
|
|
+See bottom of these notes for how to reset the mysql PW.
|
|
|
+
|
|
|
```
|
|
|
$ sudo -i
|
|
|
# umask 0022
|
|
|
@@ -62,6 +114,74 @@ Now go back up and look at the un-hardening above and do that. You probably don
|
|
|
have to reboot twice.
|
|
|
|
|
|
|
|
|
+## Making a backup
|
|
|
+
|
|
|
+We have a script /usr/local/bin/tqbackup that does the needful-ish. But if it's not
|
|
|
+there or if you need to specify some weird options (like --exclude-solr),
|
|
|
+you can run a backup directly. Why would you want to exclude solr? We've had problems
|
|
|
+with the backup working. If you restore a backup that lacks solr, then you'll need to
|
|
|
+reindex everything.
|
|
|
+
|
|
|
+```
|
|
|
+cd /var/www/api
|
|
|
+php artisan threatq:backup --exclude-solr
|
|
|
+```
|
|
|
+
|
|
|
+The backup of solr goes to /tmp first so you'll need sufficient space there. If the backup
|
|
|
+seems to be hanging / taking forever, it might be because of insufficient space.
|
|
|
+
|
|
|
+## Reindexing
|
|
|
+
|
|
|
+TQ uses Solr like elasticsearch. Most of your UI interactions use Solr, but the
|
|
|
+real data is persisted in the DB. Sometimes you'll need to reindex. What you'd
|
|
|
+think of as an index in elasticsearch is a "core" in Solr.
|
|
|
+
|
|
|
+From TQ support to do a full reindex of everything:
|
|
|
+
|
|
|
+```
|
|
|
+php /var/www/api/artisan threatq:solr-import --full --all
|
|
|
+```
|
|
|
+
|
|
|
+This may take a few hours. The biggest cores by size are "indicators" and
|
|
|
+"objectlinks", and they will take the longest to reindex. Last reindex took
|
|
|
+4ish hours. Folks can still use TQ while it's going, but a lot of things
|
|
|
+"won't show up"
|
|
|
+
|
|
|
+### Checking current reindex status
|
|
|
+
|
|
|
+```
|
|
|
+for i in indicators events objectlinks signatures attachments adversaries tasks campaign course_of_action exploit_target incident ttp attack_pattern identity intrusion_set malware report tool vulnerability; do
|
|
|
+ echo "============ $i ============"
|
|
|
+ php /var/www/api/artisan threatq:solr-import-status $i
|
|
|
+done
|
|
|
+```
|
|
|
+
|
|
|
+Each "core" (table/index?) will dump something like:
|
|
|
+
|
|
|
+```
|
|
|
+Solr Dataimport
|
|
|
+
|
|
|
+Status: busy
|
|
|
+ Type: delta
|
|
|
+
|
|
|
+Started At: 2022-08-19 22:28:02
|
|
|
+
|
|
|
+Stats:
|
|
|
++-----------------------------------+---------------------+
|
|
|
+| Time Elapsed | 0:23:23.485 |
|
|
|
+| Total Requests made to DataSource | 2165017 |
|
|
|
+| Total Rows Fetched | 4806984 |
|
|
|
+| Total Documents Processed | 203189 |
|
|
|
+| Total Documents Skipped | 0 |
|
|
|
+| Delta Dump started | 2022-08-19 22:28:02 |
|
|
|
+| Identifying Delta | 2022-08-19 22:28:02 |
|
|
|
+| Deltas Obtained | 2022-08-19 22:28:08 |
|
|
|
+| Building documents | 2022-08-19 22:28:08 |
|
|
|
+| Total Changed Documents | 1574988 |
|
|
|
++-----------------------------------+---------------------+
|
|
|
+```
|
|
|
+
|
|
|
+
|
|
|
## Patching Notes
|
|
|
|
|
|
TQ patching is a little different (of course). You have to be very careful about
|
|
|
@@ -80,3 +200,45 @@ package
|
|
|
|
|
|
ALWAYS do a `yum check-update` and make sure everything looks reasonable and that
|
|
|
Centos packages aren't replacing their RHEL equivalents.
|
|
|
+
|
|
|
+
|
|
|
+## Some Common Problems
|
|
|
+
|
|
|
+### Mysql root password
|
|
|
+
|
|
|
+During firstboot, our password we set doesn't "take". Connecting to the database becomes hard.
|
|
|
+Or any other situation where you need to reset the mysql password. Adapted from a Digital Ocean
|
|
|
+help article https://www.digitalocean.com/community/tutorials/how-to-reset-your-mysql-or-mariadb-root-password
|
|
|
+
|
|
|
+```
|
|
|
+cat <<EOF > /etc/systemd/system/mariadb.service.d/damnit.conf
|
|
|
+[Service]
|
|
|
+Environment="MYSQLD_OPTS=--skip-grant-tables --skip-networking"
|
|
|
+EOF
|
|
|
+
|
|
|
+systemctl daemon-reload
|
|
|
+systemctl restart mariadb
|
|
|
+
|
|
|
+ps -ef | grep mysql # looking for the skip settings to exist
|
|
|
+
|
|
|
+mysql -u root
|
|
|
+# These are mysql commands
|
|
|
+FLUSH PRIVILEGES;
|
|
|
+ALTER USER 'root'@'localhost' IDENTIFIED BY '<SOMEPASSWORD>';
|
|
|
+exit
|
|
|
+
|
|
|
+# Back to shell
|
|
|
+rm -f /etc/systemd/system/mariadb.service.d/damnit.conf
|
|
|
+systemctl daemon-reload
|
|
|
+systemctl restart mariadb
|
|
|
+ps -ef | grep mysql # looking for the skip settings to NOT exist
|
|
|
+
|
|
|
+```
|
|
|
+
|
|
|
+### Weird things in sensu
|
|
|
+
|
|
|
+I had to comment out the IPv6 "::1" /etc/hosts entry to shut up sensu and its https check.
|
|
|
+
|
|
|
+I also had to `chmod 755 /var/log/audit` to shut up the disk space check for it.
|
|
|
+
|
|
|
+I thought both of these were in salt but maybe not
|
Brad Poulton