# Sensu Notes.md

See (Sensu Go Migration Notes.md) file for more details

## Sensu License
5/6/2021

Congrats, we are over 100 servers in Prod. We now need a license. 

These are the steps to apply the license. Manually write the license file to the Sensu server. Manually apply it with sensuctl. It is not handled by TF or salt at the moment. Sorry I was busy!

```
sudo -i
vim /etc/sensu/conf.d/sensu_license.json
chown sensu: /etc/sensu/conf.d/sensu_license.json
chmod 600 /etc/sensu/conf.d/sensu_license.json
vim /root/sensu-password  # see vault engineering/sensu/sensu-prod
sensuctl configure -n --username 'admin' --password $( cat /root/sensu-password ) --namespace default --url 'https://127.0.0.1:8080' --insecure-skip-tls-verify
sensuctl license info
sensuctl create --file /etc/sensu/conf.d/sensu_license.json
sensuctl license info --format tabular
sensuctl logout
echo "" > /root/sensu-password
```


## Sensu Upgrade
08/03/2020

https://docs.sensu.io/sensu-go/latest/operations/maintain-sensu/upgrade/

1. download latest packages for master and agents to repo server and run `yum clean all` on sensu master 
2. if needed, update salt states to ensure they are up-to-date
2.1. Stop sensu services on sensu-master `systemctl stop sensu-agent` `systemctl stop sensu-backend`
3. update sensu software on the sensu-master
4. `yum update sensu-go-backend`
   `yum update sensu-go-cli`
   `yum update sensu-go-agent`
   `systemctl daemon-reload`
5. restart the sensu services
   `systemctl start sensu-backend`
   `systemctl start sensu-agent`
6. run `yum clean all` on salt minions
7. Stop agent on minion `systemctl stop sensu-agent`
8. Upgrade agent on minion `yum update sensu-go-agent -y`
9. Run this `systemctl daemon-reload`
10. start agent `systemctl start sensu-agent`
11. verify with this `salt '*' cmd.run 'sensu-agent version'`
`salt -C '* not salt* not sensu* not jira*' cmd.run 'sensu-agent version'


In version 5.16 the default password was removed in favor of a sensu-backend init with bash variables. 

Sen$uP@ssw0rd!

systemctl start sensu-backend
export SENSU_BACKEND_CLUSTER_ADMIN_USERNAME=YOUR_USERNAME
export SENSU_BACKEND_CLUSTER_ADMIN_PASSWORD=YOUR_PASSWORD
sensu-backend init


sensuctl create --file filename.json


## OIDC Okta Auth
---
type: oidc
api_version: authentication/v2
metadata:
  name: oidc_okta
spec:
  additional_scopes:
  - groups
  - email
  client_id: 0oa2qxyf3q0DC6Dj2297
  client_secret: bpKGPQ9-ILxZI_UrcpGgT5jsmT7JT6X6LLMa3kF_
  redirect_uri: https://sensu.msoc.defpoint.local:8000/api/enterprise/authentication/v2/oidc/callback
  server: https://mdr-multipass.okta.com
  groups_claim: groups
  groups_prefix: 'okta:'
  username_claim: email
  username_prefix: 'okta:'
  
#cluster role binding for okta
sensuctl cluster-role-binding create okta --cluster-role=cluster-admin --group=okta:mdr-admins
sensuctl cluster-role-binding create mdr-admin --cluster-role=mdr-admin --group=ldap:mdr-admins
  
sensuctl cluster-role-binding list


type: ClusterRoleBinding
api_version: core/v2
metadata:
  name: cluster-admin
spec:
  role_ref:
    name: cluster-admin
    type: ClusterRole
  subjects:
  - name: okta:group
    type: Group
    
Look for the JWT in the tokens URL parameter. 
```
{
  "exp": 1596583324,
  "jti": "4d048b63688296361fd96b1ac7d77ef1",
  "sub": "okta:brad.poulton@accenturefederal.com",
  "groups": [
    "okta:splunk-role-admin",
    "okta:Everyone",
    "okta:vault-admin",
    "okta:phantom-role-administrator",
    "okta:VPN",
    "okta:aws#afs-mdr-common-services#mdr_engineer_readonly#471284459109",
    "okta:aws-us-gov#afs-mdr-common-services-gov#mdr_engineer_readonly#701290387780",
    "okta:AWS - MDR_Engineer-Readonly Role",
    "okta:mdr-admins",
    "okta:VictorOps"
  ],
  "provider": {
    "provider_id": "oidc_okta",
    "provider_type": "oidc",
    "user_id": "00u22ymdgdKPTDyR5297"
  },
  "api_key": false
}
```    

### LDAP Auth    
```
running ldap search with basedn \"ou=groups, dc=mdr-multipass, dc=okta, dc=com\" and filter \"(\u0026(objectclass=groupOfNames)(uniqueMember=uid=brad.poulton,ou=users,dc=mdr-multipass,dc=okta,dc=com))\"
ldapsearch -x -H ldaps://mdr-multipass.ldap.okta.com -b dc=mdr-multipass,dc=okta,dc=com -D "uid=ldap.read@defpoint.com,dc=mdr-multipass,dc=okta,dc=com" -W
```
```
# brad.poulton, users, mdr-multipass.okta.com
dn: uid=brad.poulton,ou=users,dc=mdr-multipass,dc=okta,dc=com
objectClass: top
objectClass: person
objectClass: organizationalPerson
objectClass: inetOrgPerson
uid: brad.poulton
uniqueIdentifier: 00u22ymdgdKPTDyR5297
organizationalStatus: ACTIVE
givenName: Brad
sn: Poulton
cn: Brad Poulton
mail: brad.poulton@accenturefederal.com
mobile: 4355126342
```
```
# mdr-admins, groups, mdr-multipass.okta.com
dn: cn=mdr-admins,ou=groups,dc=mdr-multipass,dc=okta,dc=com
objectClass: top
objectClass: groupofUniqueNames
cn: mdr-admins
uniqueIdentifier: 00g1m5jakrmiDwISV297
uniqueMember: uid=chris.lynch,ou=users,dc=mdr-multipass,dc=okta,dc=com
uniqueMember: uid=ryan.damour,ou=users,dc=mdr-multipass,dc=okta,dc=com
uniqueMember: uid=duane.waddle,ou=users,dc=mdr-multipass,dc=okta,dc=com
uniqueMember: uid=brad.poulton,ou=users,dc=mdr-multipass,dc=okta,dc=com
```
```
Jan 14 23:48:51 sensu sensu-backend: {"component":"authentication/v2","level":"debug","msg":"running ldap search with basedn \"ou=groups, dc=mdr-multipass, dc=okta, dc=com\" and filter \"(\u0026(objectclass=groupOfNames)(uniqueMember=uid=brad.poulton,ou=users,dc=mdr-multipass,dc=okta,dc=com))\"","time":"2020-01-14T23:48:51Z"}
```
brad-test
SensuA123

# Defragging the Database

If /var starts filling up, a likely candidate is the etcd database. This can be compacted and defragged to free up space, but the tool to do so isn't installed by default.

To defrag: (based off [this document](https://docs.sensu.io/sensu-go/latest/operations/maintain-sensu/troubleshoot/))
```
sudo yum install -y etcd3
sudo bash
export ETCDCTL_API=3
export ETCDCTL_CACERT=/etc/sensu/conf.d/ca.pem
export ETCDCTL_ENDPOINTS="http://localhost:2379"
etcdctl endpoint status
# SHould outptu something like: http://localhost:2379, 8927110dc66458af, 3.3.17, 1.2 GB, true, 1122, 531838152
etcdctl defrag
exit
sudo yum remove etcd3
```

If you get the error `No help topic for 'endpoint'`, then your environment variables aren't set (did you set them before you sudo'd?).
If you get the error `Error:  context deadline exceeded`, then your endpoint is wrong, probably https instead of http.

