infrastructure-notes/salt_splunk_HEC Notes.md

on HF had to uninstall requests and reinstall it to get a new version to get rid of the error about the certs import error ImportError: cannot import name certs pip list | grep requests yum list installed | grep requests sudo pip uninstall requests sudo pip uninstall urllib3 sudo yum install python-urllib3 sudo yum install python-requests pip install boto3

yum list installed | grep ssl_match ssl_match_hostname is not getting picked up my requests and crashing the HF returner to splunk next step deploy to all minions and see if HF is unquie or not. ***ERROR: python import requests ***

salt salt* pillar.item splunk_http_forwarder

index=salt | spath fun | search fun="grains.items"

salt saf*local state.sls salt_minion.salt_minion_configs network.connect iratemoses.mdr.defpoint.com 8088 cmd.run 'tail -50 /var/log/salt/minion' pillar.item splunk_http_forwarder

moose-splunk-indexer-3 | 10.80.103.198 | vpc-0b676c4efd

not working ip-10-81-9-10.msoc.defpoint.local | customer-portal | vpc-0f45bf3132 | subnet-0de23b03ea ip-10-81-8-205.msoc.defpoint.local | customer-portal | vpc-0f45bf3132 | subnet-0c173d841b Connection timed out need to update terraform git PR submitted waiting on review/approval

---

saf-splunk-ds-1 10.1.10.161 | ddps01.corp.smartandfinal.com saf-splunk-syslog-1 10.1.10.163 | ddps03.corp.smartandfinal.com saf-splunk-syslog-2 10.1.10.164 | ddps04.corp.smartandfinal.com saf-splunk-dcn-1 10.1.10.162 | ddps02.corp.smartandfinal.com ERRORs: ConnectionError: ('Connection aborted.', error(104, 'Connection reset by peer'))

Name mismatch afssplds100.us.accenturefederal.com | afs-splunk-ds-2 afssplds102.us.accenturefederal.com | ... afssplhf101.us.accenturefederal.com afssplhf102.us.accenturefederal.com afssplhf103.us.accenturefederal.com afssplhf104.us.accenturefederal.com

aws-splnks1-tts.nga.gov aws-syslog1-tts.nga.gov aws-syslog2-tts.nga.gov

[prod]brad_poulton@salt-master:~:$ salt '*' network.get_hostname moose-splunk-indexer-2.msoc.defpoint.local:

moose-splunk-indexer-2.msoc.defpoint.local

saf-splunk-indexer-3.msoc.defpoint.local:

saf-splunk-indexer-3.msoc.defpoint.local

afs-splunk-indexer-1.msoc.defpoint.local:

afs-splunk-indexer-1.msoc.defpoint.local

phantom.msoc.defpoint.local:

phantom.msoc.defpoint.local

moose-splunk-hf.msoc.defpoint.local:

moose-splunk-hf.msoc.defpoint.local

nga-splunk-sh.msoc.defpoint.local:

nga-splunk-sh.msoc.defpoint.local

saf-splunk-cm.msoc.defpoint.local:

saf-splunk-cm.msoc.defpoint.local

openvpn.msoc.defpoint.local:

openvpn.msoc.defpoint.local

ip-10-81-8-205.msoc.defpoint.local:

ip-10-81-8-205.msoc.defpoint.local

afs-splunk-indexer-2.msoc.defpoint.local:

afs-splunk-indexer-2.msoc.defpoint.local

saf-splunk-indexer-2.msoc.defpoint.local:

saf-splunk-indexer-2.msoc.defpoint.local

moose-splunk-indexer-3.msoc.defpoint.local:

moose-splunk-indexer-3.msoc.defpoint.local

reposerver.msoc.defpoint.local:

reposerver.msoc.defpoint.local

nga-splunk-indexer-2.msoc.defpoint.local:

nga-splunk-indexer-2.msoc.defpoint.local

afs-splunk-hf.msoc.defpoint.local:

afs-splunk-hf.msoc.defpoint.local

nga-splunk-indexer-3.msoc.defpoint.local:

nga-splunk-indexer-3.msoc.defpoint.local

afs-splunk-syslog-2:

afssplhf104.us.accenturefederal.com

moose-splunk-indexer-1.msoc.defpoint.local:

moose-splunk-indexer-1.msoc.defpoint.local

moose-splunk-sh.msoc.defpoint.local:

moose-splunk-sh.msoc.defpoint.local

sensu.msoc.defpoint.local:

sensu.msoc.defpoint.local

moose-splunk-cm.msoc.defpoint.local:

moose-splunk-cm.msoc.defpoint.local

saf-splunk-indexer-1.msoc.defpoint.local:

saf-splunk-indexer-1.msoc.defpoint.local

ip-10-81-9-10.msoc.defpoint.local:

ip-10-81-9-10.msoc.defpoint.local

afs-splunk-sh.msoc.defpoint.local:

afs-splunk-sh.msoc.defpoint.local

nga-splunk-indexer-1.msoc.defpoint.local:

nga-splunk-indexer-1.msoc.defpoint.local

clu.msoc.defpoint.local:

clu.msoc.defpoint.local

dps-idm-1.msoc.defpoint.local:

dps-idm-1.msoc.defpoint.local

vault-3.msoc.defpoint.local:

vault-3.msoc.defpoint.local

nga-splunk-cm.msoc.defpoint.local:

nga-splunk-cm.msoc.defpoint.local

afs-splunk-ds-1:

afssplds102.us.accenturefederal.com

afs-splunk-syslog-1:

afssplhf103.us.accenturefederal.com

afs-splunk-syslog-3:

afssplhf101.us.accenturefederal.com

bastion.msoc.defpoint.local:

bastion.msoc.defpoint.local

saf-splunk-dcn-1:

ddps02.corp.smartandfinal.com

saf-splunk-sh.msoc.defpoint.local:

saf-splunk-sh.msoc.defpoint.local

saf-splunk-hf.msoc.defpoint.local:

saf-splunk-hf.msoc.defpoint.local

salt-master.msoc.defpoint.local:

salt-master.msoc.defpoint.local

splunk-mc.msoc.defpoint.local:

splunk-mc.msoc.defpoint.local

nga-splunk-hf.msoc.defpoint.local:

nga-splunk-hf.msoc.defpoint.local

jira-server.msoc.defpoint.local:

jira-server.msoc.defpoint.local

mailrelay.msoc.defpoint.local:

mailrelay.msoc.defpoint.local

proxy.msoc.defpoint.local:

proxy.msoc.defpoint.local

afs-splunk-syslog-4:

afssplhf102.us.accenturefederal.com

nga-splunk-syslog-1:

aws-syslog1-tts.nga.gov

vault-1.msoc.defpoint.local:

vault-1.msoc.defpoint.local

saf-splunk-syslog-2:

ddps04.corp.smartandfinal.com

afs-splunk-ds-2:

afssplds100.us.accenturefederal.com

afs-splunk-cm.msoc.defpoint.local:

afs-splunk-cm.msoc.defpoint.local

afs-splunk-indexer-3.msoc.defpoint.local:

afs-splunk-indexer-3.msoc.defpoint.local

nga-splunk-ds-1:

aws-splnks1-tts.nga.gov

nga-splunk-syslog-2:

aws-syslog2-tts.nga.gov

vault-2.msoc.defpoint.local:

vault-2.msoc.defpoint.local

saf-splunk-syslog-1:

ddps03.corp.smartandfinal.com

saf-splunk-ds-1:

ddps01.corp.smartandfinal.com