infrastructure-notes/Sensu Go Notes.md

Sensu Go Notes

See Sensu Go Migration Notes file for more details

Sensu License

5/6/2021

Congrats, we are over 100 servers in Prod. We now need a license for in GC PROD.

These are the steps to apply the license. Download the license from Sensu Go Website; Use the creds in Vault to access the XDR Sensu Account. Manually write the license file to the Sensu server. Manually apply it with sensuctl. It is not handled by TF or Salt at the moment. Sorry I was busy!

sudo -i
vim /etc/sensu/conf.d/sensu_license.json
chown sensu: /etc/sensu/conf.d/sensu_license.json
chmod 600 /etc/sensu/conf.d/sensu_license.json

# see vault engineering/sensu/sensu-prod
vim /root/sensu-password 

sensuctl configure -n --username 'admin' --password $( cat /root/sensu-password ) --namespace default --insecure-skip-tls-verify --url 'https://127.0.0.1:8080'
sensuctl license info
sensuctl create --file /etc/sensu/conf.d/sensu_license.json
sensuctl license info --format tabular
sensuctl logout
echo "" > /root/sensu-password

Sensu Upgrade

• Upgrade Notes moved to Sensu Upgrade Notes

OIDC Okta Auth

---

type: oidc
api_version: authentication/v2
metadata:
name: oidc_okta
spec:
additional_scopes:

• groups
  

• email
  client_id: 0oa2qxyf3q0DC6Dj2297
  client_secret: 'PUT CLIENT SECRET HERE'
  redirect_uri: https://sensu.pvt.xdrtest.accenturefederalcyber.com:8080/api/enterprise/authentication/v2/oidc/callback
  server: https://mdr-multipass.okta.com
  groups_claim: groups
  groups_prefix: 'okta:'
  username_claim: email
  username_prefix: 'okta:'

  # Cluster role binding for okta
  sensuctl cluster-role-binding create okta --cluster-role=cluster-admin --group=okta:mdr-admins
  sensuctl cluster-role-binding create mdr-admin --cluster-role=mdr-admin --group=ldap:mdr-admins
    
  sensuctl cluster-role-binding list
  

type: ClusterRoleBinding
api_version: core/v2 metadata:
name: cluster-admin
spec:
role_ref:

• name: cluster-admin
  
• type: ClusterRole
  
  

subjects:

• name: okta:group
  
• type: Group
  
  

Look for the JWT in the tokens URL parameter.

{
  "exp": 1596583324,
  "jti": "4d048b63688296361fd96b1ac7d77ef1",
  "sub": "okta:brad.poulton@accenturefederal.com",
  "groups": [
    "okta:splunk-role-admin",
    "okta:Everyone",
    "okta:vault-admin",
    "okta:phantom-role-administrator",
    "okta:VPN",
    "okta:aws#afs-mdr-common-services#mdr_engineer_readonly#471284459109",
    "okta:aws-us-gov#afs-mdr-common-services-gov#mdr_engineer_readonly#701290387780",
    "okta:AWS - MDR_Engineer-Readonly Role",
    "okta:mdr-admins",
    "okta:VictorOps"
  ],
  "provider": {
    "provider_id": "oidc_okta",
    "provider_type": "oidc",
    "user_id": "00u22ymdgdKPTDyR5297"
  },
  "api_key": false
}

LDAP Auth

running ldap search with basedn \"ou=groups, dc=mdr-multipass, dc=okta, dc=com\" and filter \"(\u0026(objectclass=groupOfNames)(uniqueMember=uid=brad.poulton,ou=users,dc=mdr-multipass,dc=okta,dc=com))\"
ldapsearch -x -H ldaps://mdr-multipass.ldap.okta.com -b dc=mdr-multipass,dc=okta,dc=com -D "uid=ldap.read@defpoint.com,dc=mdr-multipass,dc=okta,dc=com" -W

# brad.poulton, users, mdr-multipass.okta.com
dn: uid=brad.poulton,ou=users,dc=mdr-multipass,dc=okta,dc=com
objectClass: top
objectClass: person
objectClass: organizationalPerson
objectClass: inetOrgPerson
uid: brad.poulton
uniqueIdentifier: 00u22ymdgdKPTDyR5297
organizationalStatus: ACTIVE
givenName: Brad
sn: Poulton
cn: Brad Poulton
mail: brad.poulton@accenturefederal.com
mobile: 4355126342

# mdr-admins, groups, mdr-multipass.okta.com
dn: cn=mdr-admins,ou=groups,dc=mdr-multipass,dc=okta,dc=com
objectClass: top
objectClass: groupofUniqueNames
cn: mdr-admins
uniqueIdentifier: 00g1m5jakrmiDwISV297
uniqueMember: uid=chris.lynch,ou=users,dc=mdr-multipass,dc=okta,dc=com
uniqueMember: uid=ryan.damour,ou=users,dc=mdr-multipass,dc=okta,dc=com
uniqueMember: uid=duane.waddle,ou=users,dc=mdr-multipass,dc=okta,dc=com
uniqueMember: uid=brad.poulton,ou=users,dc=mdr-multipass,dc=okta,dc=com

Jan 14 23:48:51 sensu sensu-backend: {"component":"authentication/v2","level":"debug","msg":"running ldap search with basedn \"ou=groups, dc=mdr-multipass, dc=okta, dc=com\" and filter \"(\u0026(objectclass=groupOfNames)(uniqueMember=uid=brad.poulton,ou=users,dc=mdr-multipass,dc=okta,dc=com))\"","time":"2020-01-14T23:48:51Z"}

brad-test SensuA123

Defragging the Database

If /var starts filling up, a likely candidate is the etcd database. This can be compacted and defragged to free up space, but the tool to do so isn't installed by default.

To defrag: (based off Troubleshoot Sensu document)

sudo yum install -y etcd3
sudo bash
export ETCDCTL_API=3
export ETCDCTL_CACERT=/etc/sensu/conf.d/ca.pem
export ETCDCTL_ENDPOINTS="http://localhost:2379"
etcdctl endpoint status
# Should output something like: http://localhost:2379, 8927110dc66458af, 3.3.17, 1.2 GB, true, 1122, 531838152
etcdctl defrag
exit
sudo yum remove etcd3

If you get the error: No help topic for 'endpoint', then your environment variables aren't set (did you set them before you sudo'd?).
If you get the error: Error: context deadline exceeded, then your endpoint is wrong, probably https instead of http.