# Asset Inventory

The XDR asset inventory is a set of scripts to gather data about XDR assets and store them in the MOOSE KV Store. From there, the data is generated into a report for the compliance team and converted to a CSV (via a saved search) for ES purposes.

## Code

At present, the code is stored as part of the [msoc_infrastructure](https://github.mdr.defpoint.com/mdr-engineering/msoc-infrastructure/tree/master/salt/fileroots/salt_master/files/xdr_asset_inventory) git project.

Code is written in Python 3 and distributed to the salt-master servers via a salt state.

This app is supported via the [SA-Moose spunk app](https://github.mdr.defpoint.com/MDR-Content/SA-moose). See [`collections.conf`](https://github.mdr.defpoint.com/MDR-Content/SA-moose/blob/master/default/collections.conf) for all fields, and [`FIELDS.md`](https://github.mdr.defpoint.com/mdr-engineering/msoc-infrastructure/blob/master/salt/fileroots/salt_master/files/xdr_asset_inventory/FIELDS.md) for field descriptions.

## Overview

There are two scripts `gather_aws.py` and `gather_salt.py`. Each runs separately, and gathers data from the respective source. It is assumed that future scripts will be added for additional sources of data.

Each script operates independently. It:
  1) Gathers the information from its data source.
  2) Grabs the existing record, if any, from the Splunk KV Store
  3) Combines the information together. In the case of AWS, prefers the information in AWS. In the case of Salt, prefers the information already present in the kv store (see [Bugs and Known Issues]).

## Accessing in Splunk

The data is may be accessed in moose by using `| inputlookup xdr_assets_lookup`.

The following searches may be useful:
### Nicely Formatted Assets
```
| inputlookup xdr_assets_lookup 
| where lastseen>relative_time(now(), "-30d")
| fieldformat firstseen=strftime(firstseen, "%+") 
| fieldformat lastseen=strftime(lastseen, "%+") 
| table resource name fqdn ip mac owner priority category state firstseen lastseen
```

### Assets not seen in 30 days
```
| inputlookup xdr_assets_lookup 
| where lastseen<relative_time(now(), "-30d") 
| fieldformat firstseen=strftime(firstseen, "%+") 
| fieldformat lastseen=strftime(lastseen, "%+") 
| table age resource name fqdn ip mac owner priority category state firstseen lastseen
```

### Assets Detected by Salt or AWS but not both
```
| inputlookup xdr_assets_lookup 
| search NOT(category=salt category=aws) 
| table name ip resource category firstseen lastseen 
| fieldformat firstseen=strftime(firstseen, "%+") 
| fieldformat lastseen=strftime(lastseen, "%+")
```

## The KV Store Key

Every unique resource is stored. To calculate a unique key, the sha256 of a unique "resource id" is generated. This resource id is:
* For instances in aws, the full arn of the resource (whether detected via `gather_aws.py` or `gather_salt.py`).
* For instances in salt, a unique id of the format `salt://{salt-master}/lifecycle:{lifecycle}/{minionid}/{serialnumber}`

## The "Category" Field

The categories are always merged together with existing data. This means that categories can be added but never removed (if they need to be removed, you will need to do so manually). This allows categories to be added via salt or terraform (AWS tags), whichever is more appropriate.

## AWS Permissions

The `gather_aws.py` must be able to `assumeRole` into the `service/salt-master-inventory-role` role in the account to be inventoried. This is managed by terraform.

## Bugs and Known Issues
* `gather_salt.py` always prefers the existing data. This means that salt information won't override information gathered via AWS, which is desired. But it also means that data gathered from salt will never override existing data. This is not ideal.
* Category tags are never removed, even if they are removed from the source data.