MDR Splunk NGA Data Pull Request.txt 2.5 KB

1234567891011121314151617181920212223242526272829303132333435363738394041424344454647484950515253545556575859606162636465666768697071727374757677787980818283848586878889909192939495
  1. stand up a new "search head" that just has splunk installed on it, no need to configure the splunk instance. the splunk instance will query the actual search head and pull the data out. See hurricane labs python script.
  2. https://jira.mdr.defpoint.com/browse/MSOCI-1013
  3. vpc-05e0cf38982e048db
  4. subnet-0a2384bce743cf303
  5. MSOC_RedHat_Minion_201807250350 (ami-01c2c25dc719d3546) USED CENTOS 7 AWS AMI
  6. m4.large
  7. generated SSH key pair bradp.pem
  8. nga-splunk-searches
  9. username is centos
  10. delete key pair when done from AWS and the bastion host! bradp
  11. delete svc-searches from nga splunk SH when done
  12. delete 1TB EBS volume when done
  13. search "index=network sourcetype=qos_syslog CA98C333-F830-0B45-A543-4450CDFDA84A 1571414560 Accept 47048" -output rawdata -maxout 0 -max_time 0 -uri https://10.2.2.122:8089
  14. start fail
  15. 1019_1020export.raw
  16. 1018_1019 times:
  17. head - 2019-09-15T09:14:59
  18. tail - 2019-09-15T09:09:31
  19. end fail
  20. 1091_1092export.raw
  21. 1093_1094 times:
  22. head - 2019-09-14T14:14:59
  23. tail - 2019-09-14T14:00:00
  24. i=5000
  25. start time 2019-09-15T09:14:59
  26. stop time 2019-09-14T14:00:00
  27. start fail
  28. 784_785export.raw
  29. 783_784 times:
  30. head - 2019-09-17T19:59:59
  31. tail 2019-09-17T19:46:54
  32. end fail
  33. 857_858export.raw
  34. 859_860 times:
  35. head 2019-09-17T00:29:59
  36. tail 2019-09-17T00:15:00
  37. i=6000
  38. start time 2019-09-17T20:00:00
  39. stop time 2019-09-17T00:15:00
  40. start fail
  41. 909_910export.raw
  42. 907_908 times:
  43. head - 2019-09-16T12:59:59
  44. tail - 2019-09-16T12:45:00
  45. end fail
  46. 982_983export.raw
  47. 985_986 times:
  48. head - 2019-09-15T17:29:59
  49. tail - 2019-09-15T17:15:00
  50. i=7000
  51. start time 2019-09-15T17:30:00
  52. stop time 2019-09-16T12:45:00
  53. #from my mac
  54. aws s3 ls s3://nga-mdr-data-pull
  55. aws s3 cp nga-splunk-pull.zip s3://nga-mdr-data-pull
  56. aws --profile=mdr-prod s3 presign s3://nga-mdr-data-pull/nga-splunk-pull.zip --expires-in 86400
  57. aws --profile=mdr-prod s3 presign s3://nga-mdr-data-pull/nga-splunk-pull.zip --expires-in 604800
  58. https://nga-mdr-data-pull.s3.amazonaws.com/nga-splunk-pull.zip?AWSAccessKeyId=ASIAW6MA4LDMBGUOE7Q6&Signature=6WZ9KdHfH4rj28Ey5hrTib8HcHM%3D&x-amz-security-token=FQoGZXIvYXdzEFIaDCbQsc24x7kkQnhLQSL%2FAV4UBSVowGvhyMyS41rQtbtnmznvrbIu5Y9CCrxJ65RP%2BMeHz7Jkwu8BFEzNeeIT5M6Dfcd1NdFkqXBjE54y6G6HujSSLPk8gp2UqGDKkqMDE3qzrXfHRKaIlMInkACQi6VPpRDjFYGnnILS8vO5gjzqr9HUAsIgfVwpEuVf%2FPBbEcuUH87kZS6FqyQHTBc%2BcPk8KetsX2IuLmpOVAysip3IGgx2duVETNqKH0uXOM%2FUBygyJ7gD3DLoQWqCHQvxG0AfO0vEkRAZxgLKSDm6E2c8d9mJ5I6yXl2xBK7ii5bKWmhWtnPGYrErVFTxhfqeI6SHwzJOsLlNdkAC6nSKRyi1wMztBQ%3D%3D&Expires=1572625186
  59. tail -1 1018_1019export.raw