# Sensu Upgrade Notes

### Places where Official Sensu Go code and Documentation is located
- Official [Sensu Go Repo Github](https://github.com/sensu/sensu-go/releases)
- Official [Sensu Go Website](https://sensu.io/)
- Official [Sensu Go Upgrade Documentation](https://docs.sensu.io/sensu-go/latest/operations/maintain-sensu/upgrade/).  
- Official Sensu Hosted Package Repo Service [Packagecloud](https://packagecloud.io/sensu/stable/)

> :warning: We will use our XDR Internal `Reposerver` for all upgrade methods - See [How to add a new package to the Reposerver](Reposerver%20Notes.md)


### Sensu Go Upgrade History
 - [Jira MSOCI-1565 ticket - Upgrade Sensu to 6.2.X](https://jira.xdr.accenturefederalcyber.com/browse/MSOCI-1565)
 - [Jira MSOCI-1908 ticket - Upgrade Sensu to 6.4.3](https://jira.xdr.accenturefederalcyber.com/browse/MSOCI-1908)
 - [Jira MSOCI-1969 ticket - Upgrade Sensu to 6.6.1](https://jira.xdr.accenturefederalcyber.com/browse/MSOCI-1969)
 - [Pending MSOCI-2027 ticket - Upgrade Sensu to 6.7](https://jira.xdr.accenturefederalcyber.com/browse/MSOCI-2027)

We want to deploy the new code in iterations so that we can quickly abort deployment if we run in to any issues.  Start with `GC Test` XDR Infrastructure first.

Starting with Moose and Internal infra within `GC TEST`.  After deployment is verfied and functional, let it bake for 24-48 hrs before `GC Prod` deployment.

1. Download latest packages for `Sensu backend`, `Sensu agents`, `Sensuctl` (Sensu CLI) to `Repo server` and run `yum clean all` on `Sensu Backend` server - See [Reposerver](Reposerver%20Notes.md) notes.

2. If needed, update Salt states to ensure they are up-to-date

3. Stop Sensu services on `Sensu Backend` server
    ```
    systemctl stop sensu-agent 
    systemctl stop sensu-backend
    ```

4. Update Sensu Go software on the `Sensu Backend` server
    ```
   yum update sensu-go-backend
   yum update sensu-go-cli
   yum update sensu-go-agent
   systemctl daemon-reload
    ```
5. Restart the Sensu services 
    ```
    systemctl start sensu-backend
    systemctl start sensu-agent

    systemctl status sensu-backend
    systemctl status sensu-agent
    ```
6. `XDR Infrastructure` in `GC Test` first; Run `yum clean all` on Salt minions; then LCPs and customers
    ```
    salt -C '* not ( afs* or nga* or ma-* or dc-c19* or la-c19* or bas-* or ca-c19* or frtib* or dgi* or threatq* or qcompliance* or vmray* )' cmd.run 'yum clean all'
    
    salt -C '* not *.local not *.pvt.xdr.accenturefederalcyber.com' cmd.run 'sensu-agent version'
    ```

7. Verify and then Stop agent on minions `systemctl stop sensu-agent`
    ```
    #XDR Infrastructure
    salt -C '* not ( afs* or nga* or ma-* or dc-c19* or la-c19* or bas-* or ca-c19* or frtib* or dgi* or threatq* or qcompliance* or vmray* )' cmd.run 'sensu-agent version'
    date; salt -C '* not ( afs* or nga* or ma-* or dc-c19* or la-c19* or bas-* or ca-c19* or frtib* or dgi* or threatq* )' cmd.run 'systemctl stop sensu-agent'

    #LCPs
    salt -C '* not *.local not *.pvt.xdr.accenturefederalcyber.com' cmd.run 'sensu-agent version'
    date; salt -C '* not *.local not *.pvt.xdr.accenturefederalcyber.com' cmd.run 'systemctl stop sensu-agent'

    #Customer Slices
    salt -C 'afs*local or afs*com or ma-*com or la-*com or nga*com or nga*local or dc*com or bas-*com or frtib*com or ca-c19*com or dgi*com' cmd.run 'sensu-agent version'

    date; salt -C 'afs*local or afs*com or ma-*com or la-*com or nga*com or nga*local or dc*com or bas-*com or frtib*com or ca-c19*com or dgi*com' cmd.run 'systemctl stop sensu-agent'

    ```
8. Update the agent on minion `yum update sensu-go-agent -y`
    ```
    #XDR Infrastructure
    salt -C '* not ( afs* or nga* or ma-* or dc-c19* or la-c19* or bas-* or ca-c19* or frtib* or dgi* or threatq* )' cmd.run 'yum update sensu-go-agent -y'

    #LCPs
    salt -C '* not *.local not *.pvt.xdr.accenturefederalcyber.com' cmd.run 'yum update sensu-go-agent -y'

    #Customer Slices
    salt -C 'afs*local or afs*com or ma-*com or la-*com or nga*com or nga*local or dc*com or bas-*com or frtib*com or ca-c19*com or dgi*com' cmd.run 'yum update sensu-go-agent -y'
    ```

9. Reload the daemon `systemctl daemon-reload`
    ```
    #XDR Infrastructure
    date; salt -C '* not ( afs* or nga* or ma-* or dc-c19* or la-c19* or bas-* or ca-c19* or frtib* or dgi* or threatq* )' cmd.run 'systemctl daemon-reload'

    #LCPs
    date; salt -C '* not *.local not *.pvt.xdr.accenturefederalcyber.com' cmd.run 'systemctl daemon-reload'

    #Customer Slices
    date; salt -C 'afs*local or afs*com or ma-*com or la-*com or nga*com or nga*local or dc*com or bas-*com or frtib*com or ca-c19*com or dgi*com' cmd.run 'systemctl daemon-reload'
    ```

10. Start agent `systemctl start sensu-agent`
    ```
    #XDR Infrastructure
    date; salt -C '* not ( afs* or nga* or ma-* or dc-c19* or la-c19* or bas-* or ca-c19* or frtib* or dgi* or threatq* )' cmd.run 'systemctl start sensu-agent'

    #LCPs
    date; salt -C '* not *.local not *.pvt.xdr.accenturefederalcyber.com' cmd.run 'systemctl start sensu-agent'

    #Customer Slices
    date; salt -C 'afs*local or afs*com or ma-*com or la-*com or nga*com or nga*local or dc*com or bas-*com or frtib*com or ca-c19*com or dgi*com' cmd.run 'systemctl start sensu-agent'
    ```

11. Verify with this: 
    ```
    salt '*' cmd.run 'sensu-agent version'
    salt -C '* not salt* not sensu* not jira*' cmd.run 'sensu-agent version'
    ```

In `version 5.16` the default password was removed in favor of a sensu-backend init with bash variables. 

Sen$uP@ssw0rd!

```
systemctl start sensu-backend
export SENSU_BACKEND_CLUSTER_ADMIN_USERNAME=YOUR_USERNAME
export SENSU_BACKEND_CLUSTER_ADMIN_PASSWORD=YOUR_PASSWORD
sensu-backend init
sensuctl create --file filename.json
```