Asset Inventory

The XDR asset inventory is a set of scripts to gather data about XDR assets and store them in the MOOSE KV Store. From there, the data is generated into a report for the compliance team and converted to a CSV (via a saved search) for ES purposes.

Code

At present, the code is stored as part of the msoc_infrastructure git project.

Code is written in Python 3 and distributed to the salt-master servers via a salt state.

This app is supported via the SA-Moose splunk app. See collections.conf for all fields, and FIELDS.md for field descriptions.

Overview

There are two scripts gather_aws.py and gather_salt.py. Each runs separately, and gathers data from the respective source. It is assumed that future scripts will be added for additional sources of data.

Each script operates independently. It: 1) Gathers the information from its data source. 2) Grabs the existing record, if any, from the Splunk KV Store 3) Combines the information together. In the case of AWS, prefers the information in AWS. In the case of Salt, prefers the information already present in the kv store (see [Bugs and Known Issues]).

Accessing in Splunk

The data is may be accessed in moose by using | inputlookup xdr_assets_lookup.

The following searches may be useful:

Nicely Formatted Assets

| inputlookup xdr_assets_lookup 
| where lastseen>relative_time(now(), "-30d")
| fieldformat firstseen=strftime(firstseen, "%+") 
| fieldformat lastseen=strftime(lastseen, "%+") 
| table resource name fqdn ip mac owner role priority category state firstseen lastseen

Assets not seen in 30 days

| inputlookup xdr_assets_lookup 
| where lastseen<relative_time(now(), "-30d") 
| fieldformat firstseen=strftime(firstseen, "%+") 
| fieldformat lastseen=strftime(lastseen, "%+") 
| table age resource name fqdn ip mac owner role priority category state firstseen lastseen

Assets Detected by Salt or AWS but not both

| inputlookup xdr_assets_lookup 
| search NOT(category=salt category=aws) 
| table name ip resource category firstseen lastseen 
| fieldformat firstseen=strftime(firstseen, "%+") 
| fieldformat lastseen=strftime(lastseen, "%+")

Lookup by instance ID

| inputlookup xdr_assets_lookup 
| fieldformat lastseen=strftime(lastseen, "%+") 
| search resource=*i-0d5072669fb00c2fb* OR resource=*i-0afe11fb13db84d2c* 
| table name fqdn resource ip lastseen

The KV Store Key

Every unique resource is stored. To calculate a unique key, the sha256 of a unique "resource id" is generated. This resource id is:

For instances in aws, the full arn of the resource (whether detected via gather_aws.py or gather_salt.py).
For instances in salt, a unique id of the format salt://{salt-master}/lifecycle:{lifecycle}/{minionid}/{serialnumber}

The "Category" Field

The categories are always merged together with existing data. This means that categories can be added but never removed (if they need to be removed, you will need to do so manually). This allows categories to be added via salt or terraform (AWS tags), whichever is more appropriate.

AWS Permissions

The gather_aws.py must be able to assumeRole into the service/salt-master-inventory-role role in the account to be inventoried. This is managed by terraform.

Bugs and Known Issues

gather_salt.py always prefers the existing data. This means that salt information won't override information gathered via AWS, which is desired. But it also means that data gathered from salt will never override existing data. This is not ideal.
Category tags are never removed, even if they are removed from the source data.

Adding Additional Information for the FedRAMP Inventory Report

The easiest way to add information to the report is to use splunk after the data is gathered. Open the dashboard and open the first panel in search to see examples of this being done.

Asset Inventory Notes.md 4.4 KB Lien permanent Historique Raw