소스 검색

Adds ports script

and current script output for future reference
Brad Poulton 4 년 전
부모
커밋
6f8a488aec
3개의 변경된 파일285개의 추가작업 그리고 22개의 파일을 삭제
  1. 205 0
      scripts/all_xdr_port_references.csv
  2. 42 22
      scripts/ports_scrapper.py
  3. 38 0
      scripts/xdr_port_references.csv

+ 205 - 0
scripts/all_xdr_port_references.csv

@@ -0,0 +1,205 @@
+/Users/brad.poulton/Documents/GitHub/xdr-terraform-modules/base/splunk_servers/indexer_cluster/security-group-indexers.tf,splunk-web-in,from_port,8000,tcp
+/Users/brad.poulton/Documents/GitHub/xdr-terraform-modules/base/splunk_servers/indexer_cluster/security-group-indexers.tf,splunk-hec-in,from_port,8088,tcp
+/Users/brad.poulton/Documents/GitHub/xdr-terraform-modules/base/splunk_servers/indexer_cluster/security-group-indexers.tf,splunk-hec-in-moose,from_port,8088,tcp
+/Users/brad.poulton/Documents/GitHub/xdr-terraform-modules/base/splunk_servers/indexer_cluster/security-group-indexers.tf,splunk-api-in-access,from_port,8089,tcp
+/Users/brad.poulton/Documents/GitHub/xdr-terraform-modules/base/splunk_servers/indexer_cluster/security-group-indexers.tf,splunk-api-in-moose,from_port,8089,tcp
+/Users/brad.poulton/Documents/GitHub/xdr-terraform-modules/base/splunk_servers/indexer_cluster/security-group-indexers.tf,splunk-idx-replication,from_port,9887,tcp
+/Users/brad.poulton/Documents/GitHub/xdr-terraform-modules/base/splunk_servers/indexer_cluster/security-group-indexers.tf,splunk-data-in,from_port,9997,tcp
+/Users/brad.poulton/Documents/GitHub/xdr-terraform-modules/base/splunk_servers/indexer_cluster/security-group-indexers.tf,splunk-data-in,to_port,9998,tcp
+/Users/brad.poulton/Documents/GitHub/xdr-terraform-modules/base/splunk_servers/indexer_cluster/security-group-indexers.tf,splunk-data-in-moose,from_port,9997,tcp
+/Users/brad.poulton/Documents/GitHub/xdr-terraform-modules/base/splunk_servers/indexer_cluster/security-group-indexers.tf,splunk-data-in-moose,to_port,9998,tcp
+/Users/brad.poulton/Documents/GitHub/xdr-terraform-modules/base/splunk_servers/indexer_cluster/security-group-indexers.tf,splunk-idx-replication-out,from_port,9887,tcp
+/Users/brad.poulton/Documents/GitHub/xdr-terraform-modules/base/splunk_servers/indexer_cluster/security-group-indexers.tf,splunk-api-out,from_port,8089,tcp
+/Users/brad.poulton/Documents/GitHub/xdr-terraform-modules/base/splunk_servers/indexer_cluster/security-group-elbs.tf,hec-https-in,from_port,443,tcp
+/Users/brad.poulton/Documents/GitHub/xdr-terraform-modules/base/splunk_servers/indexer_cluster/security-group-elbs.tf,hec-in,from_port,8088,tcp
+/Users/brad.poulton/Documents/GitHub/xdr-terraform-modules/base/splunk_servers/indexer_cluster/security-group-elbs.tf,hec-out,from_port,8088,tcp
+/Users/brad.poulton/Documents/GitHub/xdr-terraform-modules/base/splunk_servers/indexer_cluster/security-group-elb-pvt.tf,hec-pvt-https-in-moose,from_port,443,tcp
+/Users/brad.poulton/Documents/GitHub/xdr-terraform-modules/base/splunk_servers/indexer_cluster/security-group-elb-pvt.tf,hec-pvt-https-in-customer,from_port,443,tcp
+/Users/brad.poulton/Documents/GitHub/xdr-terraform-modules/base/splunk_servers/indexer_cluster/security-group-elb-pvt.tf,hec-pvt-in-moose,from_port,8088,tcp
+/Users/brad.poulton/Documents/GitHub/xdr-terraform-modules/base/splunk_servers/indexer_cluster/security-group-elb-pvt.tf,hec-pvt-in-customer,from_port,8088,tcp
+/Users/brad.poulton/Documents/GitHub/xdr-terraform-modules/base/splunk_servers/indexer_cluster/security-group-elb-pvt.tf,hec-pvt-out,from_port,8088,tcp
+/Users/brad.poulton/Documents/GitHub/xdr-terraform-modules/base/splunk_servers/cluster_master/main.tf,splunk-web-in,from_port,8000,tcp
+/Users/brad.poulton/Documents/GitHub/xdr-terraform-modules/base/splunk_servers/cluster_master/main.tf,splunk-api-in,from_port,8089,tcp
+/Users/brad.poulton/Documents/GitHub/xdr-terraform-modules/base/splunk_servers/cluster_master/main.tf,splunk-api-in-moose,from_port,8089,tcp
+/Users/brad.poulton/Documents/GitHub/xdr-terraform-modules/base/splunk_servers/cluster_master/main.tf,splunk-api-out,from_port,8089,tcp
+/Users/brad.poulton/Documents/GitHub/xdr-terraform-modules/base/splunk_servers/cluster_master/main.tf,splunk-data-out,from_port,9997,tcp
+/Users/brad.poulton/Documents/GitHub/xdr-terraform-modules/base/splunk_servers/cluster_master/main.tf,splunk-data-out,to_port,9998,tcp
+/Users/brad.poulton/Documents/GitHub/xdr-terraform-modules/base/splunk_servers/searchhead/main.tf,splunk-web-in,from_port,8000,tcp
+/Users/brad.poulton/Documents/GitHub/xdr-terraform-modules/base/splunk_servers/searchhead/main.tf,splunk-api-in,from_port,8089,tcp
+/Users/brad.poulton/Documents/GitHub/xdr-terraform-modules/base/splunk_servers/searchhead/main.tf,splunk-api-out,from_port,8089,tcp
+/Users/brad.poulton/Documents/GitHub/xdr-terraform-modules/base/splunk_servers/searchhead/main.tf,splunk-api-out-to-all,from_port,8089,tcp
+/Users/brad.poulton/Documents/GitHub/xdr-terraform-modules/base/splunk_servers/searchhead/main.tf,splunk-data-out,from_port,9997,tcp
+/Users/brad.poulton/Documents/GitHub/xdr-terraform-modules/base/splunk_servers/searchhead/main.tf,splunk-data-out,to_port,9998,tcp
+/Users/brad.poulton/Documents/GitHub/xdr-terraform-modules/base/splunk_servers/searchhead/elb.tf,searchhead-alb-api-in,from_port,8089,tcp
+/Users/brad.poulton/Documents/GitHub/xdr-terraform-modules/base/splunk_servers/searchhead/elb.tf,searchhead-alb-https-in,from_port,443,tcp
+/Users/brad.poulton/Documents/GitHub/xdr-terraform-modules/base/splunk_servers/searchhead/elb.tf,searchhead-alb-8000-in,from_port,8000,tcp
+/Users/brad.poulton/Documents/GitHub/xdr-terraform-modules/base/splunk_servers/searchhead/elb.tf,searchhead-http-in,from_port,80,tcp
+/Users/brad.poulton/Documents/GitHub/xdr-terraform-modules/base/splunk_servers/searchhead/elb.tf,searchhead-alb-8000-out,from_port,8000,tcp
+/Users/brad.poulton/Documents/GitHub/xdr-terraform-modules/base/splunk_servers/searchhead/elb.tf,searchhead-alb-api-out,from_port,8089,tcp
+/Users/brad.poulton/Documents/GitHub/xdr-terraform-modules/base/splunk_servers/heavy_forwarder/main.tf,splunk-web-in,from_port,8000,tcp
+/Users/brad.poulton/Documents/GitHub/xdr-terraform-modules/base/splunk_servers/heavy_forwarder/main.tf,splunk-api-in,from_port,8089,tcp
+/Users/brad.poulton/Documents/GitHub/xdr-terraform-modules/base/splunk_servers/heavy_forwarder/main.tf,splunk-api-out,from_port,8089,tcp
+/Users/brad.poulton/Documents/GitHub/xdr-terraform-modules/base/splunk_servers/heavy_forwarder/main.tf,splunk-data-out,from_port,9997,tcp
+/Users/brad.poulton/Documents/GitHub/xdr-terraform-modules/base/splunk_servers/heavy_forwarder/main.tf,splunk-data-out,to_port,9998,tcp
+/Users/brad.poulton/Documents/GitHub/xdr-terraform-modules/base/splunk_servers/alsi/elb-elastic.tf,alsi-alb-elastic-https-in,from_port,443,tcp
+/Users/brad.poulton/Documents/GitHub/xdr-terraform-modules/base/splunk_servers/alsi/elb-elastic.tf,alsi-hec-http-in,from_port,80,tcp
+/Users/brad.poulton/Documents/GitHub/xdr-terraform-modules/base/splunk_servers/alsi/elb-elastic.tf,alsi-alb-elastic-9200-out,from_port,9200,tcp
+/Users/brad.poulton/Documents/GitHub/xdr-terraform-modules/base/splunk_servers/alsi/elb-master.tf,alsi-master-alb-https-in,from_port,443,tcp
+/Users/brad.poulton/Documents/GitHub/xdr-terraform-modules/base/splunk_servers/alsi/elb-master.tf,alsi-master-http-in,from_port,80,tcp
+/Users/brad.poulton/Documents/GitHub/xdr-terraform-modules/base/splunk_servers/alsi/elb-master.tf,alsi-master-alb-9000-out,from_port,9000,tcp
+/Users/brad.poulton/Documents/GitHub/xdr-terraform-modules/base/splunk_servers/alsi/workers.tf,alsi_worker_alb_elastic1,from_port,9000,tcp
+/Users/brad.poulton/Documents/GitHub/xdr-terraform-modules/base/splunk_servers/alsi/workers.tf,alsi_worker_alb_elastic2,from_port,9200,tcp
+/Users/brad.poulton/Documents/GitHub/xdr-terraform-modules/base/splunk_servers/alsi/workers.tf,alsi_worker_vpn_in1,from_port,9000,tcp
+/Users/brad.poulton/Documents/GitHub/xdr-terraform-modules/base/splunk_servers/alsi/workers.tf,alsi_worker_vpn_in2,from_port,9200,tcp
+/Users/brad.poulton/Documents/GitHub/xdr-terraform-modules/base/splunk_servers/alsi/workers.tf,alsi_worker_vpn_in3,from_port,9997,tcp
+/Users/brad.poulton/Documents/GitHub/xdr-terraform-modules/base/splunk_servers/alsi/workers.tf,alsi_worker_vpn_in3,to_port,9998,tcp
+/Users/brad.poulton/Documents/GitHub/xdr-terraform-modules/base/splunk_servers/alsi/workers.tf,alsi_worker_vpn_in4,from_port,8088,tcp
+/Users/brad.poulton/Documents/GitHub/xdr-terraform-modules/base/splunk_servers/alsi/workers.tf,alsi_worker_external_in,from_port,9997,tcp
+/Users/brad.poulton/Documents/GitHub/xdr-terraform-modules/base/splunk_servers/alsi/workers.tf,alsi_worker_external_in,to_port,9998,tcp
+/Users/brad.poulton/Documents/GitHub/xdr-terraform-modules/base/splunk_servers/alsi/workers.tf,alsi-interconnections,from_port,4200,tcp
+/Users/brad.poulton/Documents/GitHub/xdr-terraform-modules/base/splunk_servers/alsi/workers.tf,alsi-worker-splunk-mgmt,from_port,8089,tcp
+/Users/brad.poulton/Documents/GitHub/xdr-terraform-modules/base/splunk_servers/alsi/workers.tf,alsi-worker-splunk-data,from_port,9997,tcp
+/Users/brad.poulton/Documents/GitHub/xdr-terraform-modules/base/splunk_servers/alsi/workers.tf,alsi-worker-splunk-data,to_port,9998,tcp
+/Users/brad.poulton/Documents/GitHub/xdr-terraform-modules/base/splunk_servers/alsi/master.tf,alsi-master-alb-web-in,from_port,9000,tcp
+/Users/brad.poulton/Documents/GitHub/xdr-terraform-modules/base/splunk_servers/alsi/master.tf,alsi-master-vpn-web-in,from_port,9000,tcp
+/Users/brad.poulton/Documents/GitHub/xdr-terraform-modules/base/splunk_servers/alsi/master.tf,alsi-master-interconnections,from_port,4200,tcp
+/Users/brad.poulton/Documents/GitHub/xdr-terraform-modules/base/splunk_servers/alsi/master.tf,alsi-master-splunk-mgmt,from_port,8089,tcp
+/Users/brad.poulton/Documents/GitHub/xdr-terraform-modules/base/splunk_servers/alsi/master.tf,alsi-master-splunk-data,from_port,9997,tcp
+/Users/brad.poulton/Documents/GitHub/xdr-terraform-modules/base/splunk_servers/alsi/master.tf,alsi-master-splunk-data,to_port,9998,tcp
+/Users/brad.poulton/Documents/GitHub/xdr-terraform-modules/base/splunk_servers/alsi/elb-hec.tf,alsi-alb-hec-https-in,from_port,443,tcp
+/Users/brad.poulton/Documents/GitHub/xdr-terraform-modules/base/splunk_servers/alsi/elb-hec.tf,alsi-elastic-http-in,from_port,80,tcp
+/Users/brad.poulton/Documents/GitHub/xdr-terraform-modules/base/splunk_servers/alsi/elb-hec.tf,alsi-alb-hec-8088-out,from_port,8088,tcp
+/Users/brad.poulton/Documents/GitHub/xdr-terraform-modules/base/salt_master/main.tf,http-out,from_port,80,tcp
+/Users/brad.poulton/Documents/GitHub/xdr-terraform-modules/base/salt_master/main.tf,https-out,from_port,443,tcp
+/Users/brad.poulton/Documents/GitHub/xdr-terraform-modules/base/customer_portal_lambda/main.tf,portal_lambda_splunk_out,from_port,8089,tcp
+/Users/brad.poulton/Documents/GitHub/xdr-terraform-modules/base/customer_portal_lambda/main.tf,portal_lambda_splunk_in,from_port,8089,tcp
+/Users/brad.poulton/Documents/GitHub/xdr-terraform-modules/base/bastion/main.tf,ssh-in,from_port,22,tcp
+/Users/brad.poulton/Documents/GitHub/xdr-terraform-modules/base/bastion/main.tf,ssh-out,from_port,22,tcp
+/Users/brad.poulton/Documents/GitHub/xdr-terraform-modules/base/bastion/main.tf,http-out,from_port,80,tcp
+/Users/brad.poulton/Documents/GitHub/xdr-terraform-modules/base/bastion/main.tf,https-out,from_port,443,tcp
+/Users/brad.poulton/Documents/GitHub/xdr-terraform-modules/base/customer_portal/main.tf,customer_portal,from_port,443,tcp
+/Users/brad.poulton/Documents/GitHub/xdr-terraform-modules/base/customer_portal/main.tf,customer_portal_postgres_outbound,from_port,5432,tcp
+/Users/brad.poulton/Documents/GitHub/xdr-terraform-modules/base/customer_portal/main.tf,customer_portal_http_outbound,from_port,80,tcp
+/Users/brad.poulton/Documents/GitHub/xdr-terraform-modules/base/customer_portal/main.tf,customer_portal_https_outbound,from_port,443,tcp
+/Users/brad.poulton/Documents/GitHub/xdr-terraform-modules/base/customer_portal/main.tf,customer_portal_smtps_outbound,from_port,465,tcp
+/Users/brad.poulton/Documents/GitHub/xdr-terraform-modules/base/customer_portal/elb.tf,customer_portal_alb_https,from_port,443,tcp
+/Users/brad.poulton/Documents/GitHub/xdr-terraform-modules/base/customer_portal/elb.tf,customer_portal_alb_https_test,from_port,443,tcp
+/Users/brad.poulton/Documents/GitHub/xdr-terraform-modules/base/customer_portal/elb.tf,customer_portal_alb_http,from_port,80,tcp
+/Users/brad.poulton/Documents/GitHub/xdr-terraform-modules/base/customer_portal/elb.tf,customer_portal_sensu_check,from_port,443,tcp
+/Users/brad.poulton/Documents/GitHub/xdr-terraform-modules/base/customer_portal/elb.tf,customer_portal_alb,from_port,443,tcp
+/Users/brad.poulton/Documents/GitHub/xdr-terraform-modules/base/customer_portal/rds.tf,customer_portal_postgres_inbound,from_port,5432,tcp
+/Users/brad.poulton/Documents/GitHub/xdr-terraform-modules/base/interconnects/security-groups.tf,trusted_ssh,from_port,22,tcp
+/Users/brad.poulton/Documents/GitHub/xdr-terraform-modules/base/interconnects/security-groups.tf,bgp_ingress,from_port,179,tcp
+/Users/brad.poulton/Documents/GitHub/xdr-terraform-modules/base/interconnects/security-groups.tf,ipsec_l2tp_ingress,from_port,1701,udp
+/Users/brad.poulton/Documents/GitHub/xdr-terraform-modules/base/interconnects/security-groups.tf,ipsec_ike_ingress,from_port,500,udp
+/Users/brad.poulton/Documents/GitHub/xdr-terraform-modules/base/interconnects/security-groups.tf,ipsec_ike_nat_t_ingress,from_port,4500,udp
+/Users/brad.poulton/Documents/GitHub/xdr-terraform-modules/base/interconnects/security-groups.tf,ipsec_egress,from_port,0,all
+/Users/brad.poulton/Documents/GitHub/xdr-terraform-modules/base/openvpn/main.tf,openvpn-in,from_port,1194,udp
+/Users/brad.poulton/Documents/GitHub/xdr-terraform-modules/base/openvpn/main.tf,openvpn-https-in,from_port,443,tcp
+/Users/brad.poulton/Documents/GitHub/xdr-terraform-modules/base/openvpn/main.tf,openvpn-splunk-out,from_port,8000,tcp
+/Users/brad.poulton/Documents/GitHub/xdr-terraform-modules/base/openvpn/main.tf,openvpn-https-out,from_port,443,tcp
+/Users/brad.poulton/Documents/GitHub/xdr-terraform-modules/base/openvpn/main.tf,openvpn-https-alt-out,from_port,8443,tcp
+/Users/brad.poulton/Documents/GitHub/xdr-terraform-modules/base/openvpn/main.tf,openvpn-phantom-out,from_port,8888,tcp
+/Users/brad.poulton/Documents/GitHub/xdr-terraform-modules/base/openvpn/main.tf,openvpn-github-ssh-out,from_port,122,tcp
+/Users/brad.poulton/Documents/GitHub/xdr-terraform-modules/base/openvpn/main.tf,openvpn-ssh-out,from_port,22,tcp
+/Users/brad.poulton/Documents/GitHub/xdr-terraform-modules/base/openvpn/main.tf,openvpn-nessus-out,from_port,8834,tcp
+/Users/brad.poulton/Documents/GitHub/xdr-terraform-modules/base/openvpn/main.tf,openvpn-nessus-out,to_port,8835,tcp
+/Users/brad.poulton/Documents/GitHub/xdr-terraform-modules/base/openvpn/main.tf,openvpn-license-server-out,from_port,443,tcp
+/Users/brad.poulton/Documents/GitHub/xdr-terraform-modules/base/openvpn/main.tf,openvpn-ldap-out,from_port,636,tcp
+/Users/brad.poulton/Documents/GitHub/xdr-terraform-modules/base/jira/instance_jira/securitygroup-server.tf,jira_server_inbound_8080,from_port,8080,tcp
+/Users/brad.poulton/Documents/GitHub/xdr-terraform-modules/base/jira/instance_jira/securitygroup-server.tf,jira_server_inbound_alb_8080,from_port,8080,tcp
+/Users/brad.poulton/Documents/GitHub/xdr-terraform-modules/base/jira/instance_jira/securitygroup-server.tf,jira_server_outbound_postgres,from_port,5432,tcp
+/Users/brad.poulton/Documents/GitHub/xdr-terraform-modules/base/jira/rds_jira/security-groups.tf,jira_rds_in,from_port,5432,tcp
+/Users/brad.poulton/Documents/GitHub/xdr-terraform-modules/base/jira/rds_jira/security-groups.tf,jira_security_in,from_port,0,-1
+/Users/brad.poulton/Documents/GitHub/xdr-terraform-modules/base/jira/rds_jira/security-groups.tf,jira_security_in,to_port,65535,-1
+/Users/brad.poulton/Documents/GitHub/xdr-terraform-modules/base/github/securitygroup-backupserver.tf,ghe_backup_server_122_to_github,from_port,122,tcp
+/Users/brad.poulton/Documents/GitHub/xdr-terraform-modules/base/github/securitygroup-backupserver.tf,ghe_backup_server_122_to_legacy,from_port,122,tcp
+/Users/brad.poulton/Documents/GitHub/xdr-terraform-modules/base/github/securitygroup-backupserver.tf,ghe_backup_server_egress_nfs,from_port,2049,tcp
+/Users/brad.poulton/Documents/GitHub/xdr-terraform-modules/base/github/securitygroup-backupserver.tf,ghe_backup_server_ingress_nfs,from_port,2049,tcp
+/Users/brad.poulton/Documents/GitHub/xdr-terraform-modules/base/github/securitygroups-load-balancers.tf,ghe_elb_external_inbound_https_22_cidr,from_port,22,tcp
+/Users/brad.poulton/Documents/GitHub/xdr-terraform-modules/base/github/securitygroups-load-balancers.tf,ghe_elb_external_inbound_http_cidr,from_port,80,tcp
+/Users/brad.poulton/Documents/GitHub/xdr-terraform-modules/base/github/securitygroups-load-balancers.tf,ghe_elb_external_inbound_https_cidr,from_port,443,tcp
+/Users/brad.poulton/Documents/GitHub/xdr-terraform-modules/base/github/securitygroups-load-balancers.tf,ghe_elb_external_inbound_https_cidr,to_port,444,tcp
+/Users/brad.poulton/Documents/GitHub/xdr-terraform-modules/base/github/securitygroups-load-balancers.tf,ghe_elb_external_outbound_ssh,from_port,23,tcp
+/Users/brad.poulton/Documents/GitHub/xdr-terraform-modules/base/github/securitygroups-load-balancers.tf,ghe_elb_external_outbound_http,from_port,80,tcp
+/Users/brad.poulton/Documents/GitHub/xdr-terraform-modules/base/github/securitygroups-load-balancers.tf,ghe_elb_external_outbound_https,from_port,443,tcp
+/Users/brad.poulton/Documents/GitHub/xdr-terraform-modules/base/github/securitygroups-load-balancers.tf,ghe_elb_internal_inbound_https_cidr,from_port,443,tcp
+/Users/brad.poulton/Documents/GitHub/xdr-terraform-modules/base/github/securitygroups-load-balancers.tf,ghe_elb_internal_inbound_https_8443_cidr,from_port,8443,tcp
+/Users/brad.poulton/Documents/GitHub/xdr-terraform-modules/base/github/securitygroups-load-balancers.tf,ghe_elb_internal_inbound_https_22_cidr,from_port,22,tcp
+/Users/brad.poulton/Documents/GitHub/xdr-terraform-modules/base/github/securitygroups-load-balancers.tf,ghe_elb_internal_outbound_https,from_port,443,tcp
+/Users/brad.poulton/Documents/GitHub/xdr-terraform-modules/base/github/securitygroups-load-balancers.tf,ghe_elb_internal_outbound_8444_https,from_port,8443,tcp
+/Users/brad.poulton/Documents/GitHub/xdr-terraform-modules/base/github/securitygroups-load-balancers.tf,ghe_elb_internal_outbound_8444_https,to_port,8444,tcp
+/Users/brad.poulton/Documents/GitHub/xdr-terraform-modules/base/github/securitygroups-load-balancers.tf,ghe_elb_internal_outbound_23_https,from_port,23,tcp
+/Users/brad.poulton/Documents/GitHub/xdr-terraform-modules/base/github/securitygroup-server.tf,ghe_server_inbound_ssh_cidr,from_port,22,tcp
+/Users/brad.poulton/Documents/GitHub/xdr-terraform-modules/base/github/securitygroup-server.tf,ghe_server_inbound_https_external_elb_23,from_port,23,tcp
+/Users/brad.poulton/Documents/GitHub/xdr-terraform-modules/base/github/securitygroup-server.tf,ghe_server_inbound_https_internal_elb_23,from_port,23,tcp
+/Users/brad.poulton/Documents/GitHub/xdr-terraform-modules/base/github/securitygroup-server.tf,ghe_server_inbound_external_elb_80,from_port,80,tcp
+/Users/brad.poulton/Documents/GitHub/xdr-terraform-modules/base/github/securitygroup-server.tf,ghe_server_inbound_mgmt_ssh_cidr,from_port,122,tcp
+/Users/brad.poulton/Documents/GitHub/xdr-terraform-modules/base/github/securitygroup-server.tf,ghe_server_inbound_mgmt_ssh_sgs,from_port,122,tcp
+/Users/brad.poulton/Documents/GitHub/xdr-terraform-modules/base/github/securitygroup-server.tf,ghe_server_inbound_mgmt_ssh_backup_sgs,from_port,122,tcp
+/Users/brad.poulton/Documents/GitHub/xdr-terraform-modules/base/github/securitygroup-server.tf,ghe_server_inbound_https_cidr,from_port,443,tcp
+/Users/brad.poulton/Documents/GitHub/xdr-terraform-modules/base/github/securitygroup-server.tf,ghe_server_inbound_https_cidr,to_port,444,tcp
+/Users/brad.poulton/Documents/GitHub/xdr-terraform-modules/base/github/securitygroup-server.tf,ghe_server_inbound_https_external_elb,from_port,443,tcp
+/Users/brad.poulton/Documents/GitHub/xdr-terraform-modules/base/github/securitygroup-server.tf,ghe_server_inbound_https_external_elb,to_port,444,tcp
+/Users/brad.poulton/Documents/GitHub/xdr-terraform-modules/base/github/securitygroup-server.tf,ghe_server_inbound_https_internal_elb,from_port,443,tcp
+/Users/brad.poulton/Documents/GitHub/xdr-terraform-modules/base/github/securitygroup-server.tf,ghe_server_inbound_https_internal_elb,to_port,444,tcp
+/Users/brad.poulton/Documents/GitHub/xdr-terraform-modules/base/github/securitygroup-server.tf,ghe_server_inbound_mgmt_https_cidr,from_port,8443,tcp
+/Users/brad.poulton/Documents/GitHub/xdr-terraform-modules/base/github/securitygroup-server.tf,ghe_server_inbound_mgmt_https_cidr,to_port,8444,tcp
+/Users/brad.poulton/Documents/GitHub/xdr-terraform-modules/base/github/securitygroup-server.tf,ghe_server_inbound_mgmt_https_sgs,from_port,8443,tcp
+/Users/brad.poulton/Documents/GitHub/xdr-terraform-modules/base/github/securitygroup-server.tf,ghe_server_inbound_mgmt_https_sgs,to_port,8444,tcp
+/Users/brad.poulton/Documents/GitHub/xdr-terraform-modules/base/github/securitygroup-server.tf,ghe_server_inbound_mgmt_https_backup_sgs,from_port,8443,tcp
+/Users/brad.poulton/Documents/GitHub/xdr-terraform-modules/base/github/securitygroup-server.tf,ghe_server_inbound_mgmt_https_backup_sgs,to_port,8444,tcp
+/Users/brad.poulton/Documents/GitHub/xdr-terraform-modules/base/github/securitygroup-server.tf,ghe_server_inbound_https_internal_elb_8444,from_port,8443,tcp
+/Users/brad.poulton/Documents/GitHub/xdr-terraform-modules/base/github/securitygroup-server.tf,ghe_server_inbound_https_internal_elb_8444,to_port,8444,tcp
+/Users/brad.poulton/Documents/GitHub/xdr-terraform-modules/base/github/securitygroup-server.tf,ghe_server_inbound_https_external_elb_8444,from_port,8443,tcp
+/Users/brad.poulton/Documents/GitHub/xdr-terraform-modules/base/github/securitygroup-server.tf,ghe_server_inbound_https_external_elb_8444,to_port,8444,tcp
+/Users/brad.poulton/Documents/GitHub/xdr-terraform-modules/base/github/securitygroup-server.tf,ghe_server_outbound_http,from_port,80,tcp
+/Users/brad.poulton/Documents/GitHub/xdr-terraform-modules/base/github/securitygroup-server.tf,ghe_server_outbound_https,from_port,443,tcp
+/Users/brad.poulton/Documents/GitHub/xdr-terraform-modules/base/github/securitygroup-server.tf,ghe_server_outbound_syslog,from_port,1514,tcp
+/Users/brad.poulton/Documents/GitHub/xdr-terraform-modules/base/mailrelay/main.tf,smtp-in,from_port,25,tcp
+/Users/brad.poulton/Documents/GitHub/xdr-terraform-modules/base/mailrelay/main.tf,submission-out,from_port,587,tcp
+/Users/brad.poulton/Documents/GitHub/xdr-terraform-modules/base/teleport-single-instance/alb.tf,alb_to_server,from_port,3080,tcp
+/Users/brad.poulton/Documents/GitHub/xdr-terraform-modules/base/teleport-single-instance/alb-internal.tf,alb_internal-to_server,from_port,3080,tcp
+/Users/brad.poulton/Documents/GitHub/xdr-terraform-modules/base/proxy_server/main.tf,http-out,from_port,80,tcp
+/Users/brad.poulton/Documents/GitHub/xdr-terraform-modules/base/proxy_server/main.tf,https-out,from_port,443,tcp
+/Users/brad.poulton/Documents/GitHub/xdr-terraform-modules/base/nessus/instance_security_center/securitygroup-server.tf,security_center_inbound_443,from_port,443,tcp
+/Users/brad.poulton/Documents/GitHub/xdr-terraform-modules/base/nessus/instance_security_center/securitygroup-server.tf,security_center_inbound_443_from_alb,from_port,443,tcp
+/Users/brad.poulton/Documents/GitHub/xdr-terraform-modules/base/nessus/instance_security_center/securitygroup-server.tf,security_center_outbound_nessus,from_port,8834,tcp
+/Users/brad.poulton/Documents/GitHub/xdr-terraform-modules/base/nessus/instance_security_center/securitygroup-server.tf,security_center_outbound_nessus,to_port,8835,tcp
+/Users/brad.poulton/Documents/GitHub/xdr-terraform-modules/base/nessus/instance_nessus_scanner/securitygroup-server.tf,nessus_scanner_inbound_22,from_port,22,tcp
+/Users/brad.poulton/Documents/GitHub/xdr-terraform-modules/base/nessus/instance_nessus_scanner/securitygroup-server.tf,nessus_scanner_inbound_443,from_port,443,tcp
+/Users/brad.poulton/Documents/GitHub/xdr-terraform-modules/base/nessus/instance_nessus_scanner/securitygroup-server.tf,nessus_scanner_inbound_nessus,from_port,8834,tcp
+/Users/brad.poulton/Documents/GitHub/xdr-terraform-modules/base/nessus/instance_nessus_scanner/securitygroup-server.tf,nessus_scanner_inbound_nessus,to_port,8835,tcp
+/Users/brad.poulton/Documents/GitHub/xdr-terraform-modules/base/vmray_instances/security-groups.tf,vmray-ssh,from_port,22,tcp
+/Users/brad.poulton/Documents/GitHub/xdr-terraform-modules/base/vmray_instances/security-groups.tf,vmray-https,from_port,443,tcp
+/Users/brad.poulton/Documents/GitHub/xdr-terraform-modules/base/vmray_instances/security-groups.tf,vmray-egress,from_port,0,all
+/Users/brad.poulton/Documents/GitHub/xdr-terraform-modules/base/dns/resolver_instance/main.tf,dns-tcp,from_port,53,tcp
+/Users/brad.poulton/Documents/GitHub/xdr-terraform-modules/base/dns/resolver_instance/main.tf,dns-udp,from_port,53,udp
+/Users/brad.poulton/Documents/GitHub/xdr-terraform-modules/base/dns/resolver_instance/main.tf,dns_outbound_tcp,from_port,53,tcp
+/Users/brad.poulton/Documents/GitHub/xdr-terraform-modules/base/dns/resolver_instance/main.tf,dns_outbound_udp,from_port,53,udp
+/Users/brad.poulton/Documents/GitHub/xdr-terraform-modules/base/repo_server/main.tf,http-in,from_port,80,tcp
+/Users/brad.poulton/Documents/GitHub/xdr-terraform-modules/base/repo_server/main.tf,http-in-external-c2-users,from_port,80,tcp
+/Users/brad.poulton/Documents/GitHub/xdr-terraform-modules/base/repo_server/main.tf,https-in,from_port,443,tcp
+/Users/brad.poulton/Documents/GitHub/xdr-terraform-modules/base/repo_server/main.tf,https-in-external-c2-users,from_port,443,tcp
+/Users/brad.poulton/Documents/GitHub/xdr-terraform-modules/base/sensu/main.tf,sensu_ui,from_port,8000,tcp
+/Users/brad.poulton/Documents/GitHub/xdr-terraform-modules/base/sensu/main.tf,sensu_agent_internal,from_port,8081,tcp
+/Users/brad.poulton/Documents/GitHub/xdr-terraform-modules/base/sensu/main.tf,sensu_api,from_port,8080,tcp
+/Users/brad.poulton/Documents/GitHub/xdr-terraform-modules/base/sensu/main.tf,sensu_agent_external,from_port,8081,tcp
+/Users/brad.poulton/Documents/GitHub/xdr-terraform-modules/base/sensu/main.tf,sensu_api_external,from_port,8080,tcp
+/Users/brad.poulton/Documents/GitHub/xdr-terraform-modules/base/sensu/elb.tf,alb_to_sensu_server,from_port,8081,tcp
+/Users/brad.poulton/Documents/GitHub/xdr-terraform-modules/base/sensu/elb.tf,alb_to_sensu_health,from_port,8080,tcp
+/Users/brad.poulton/Documents/GitHub/xdr-terraform-modules/base/vault/main.tf,vault_server_from_alb,from_port,443,tcp
+/Users/brad.poulton/Documents/GitHub/xdr-terraform-modules/base/vault/main.tf,https-out,from_port,443,tcp
+/Users/brad.poulton/Documents/GitHub/xdr-terraform-modules/base/vault/main.tf,vault_server_to_alb,from_port,443,tcp
+/Users/brad.poulton/Documents/GitHub/xdr-terraform-modules/base/vault/main.tf,vault_server_egress_dynamodb,from_port,443,tcp
+/Users/brad.poulton/Documents/GitHub/xdr-terraform-modules/base/vault/elb.tf,vault_server_from_vpc,from_port,443,tcp
+/Users/brad.poulton/Documents/GitHub/xdr-terraform-modules/base/vault/elb.tf,alb_to_vault_server,from_port,443,tcp
+/Users/brad.poulton/Documents/GitHub/xdr-terraform-modules/base/phantom/securitygroup-server.tf,phantom_server_inbound_8888,from_port,8888,tcp
+/Users/brad.poulton/Documents/GitHub/xdr-terraform-modules/base/phantom/securitygroup-server.tf,phantom_server_inbound_alb_443,from_port,443,tcp
+/Users/brad.poulton/Documents/GitHub/xdr-terraform-modules/base/phantom/securitygroup-server.tf,phantom_server_inbound_alb_443_from_vpn,from_port,443,tcp
+/Users/brad.poulton/Documents/GitHub/xdr-terraform-modules/base/phantom/securitygroup-server.tf,phantom_server_outbound_postgres,from_port,8089,tcp
+/Users/brad.poulton/Documents/GitHub/xdr-terraform-modules/base/phantom/securitygroup-server.tf,phantom_server_outbound_udp_dns,from_port,53,tcp
+/Users/brad.poulton/Documents/GitHub/xdr-terraform-modules/base/phantom/securitygroup-server.tf,phantom_server_outbound_tcp_dns,from_port,53,udp
+/Users/brad.poulton/Documents/GitHub/xdr-terraform-modules/base/phantom/securitygroup-server.tf,phantom_server_outbound_http,from_port,80,tcp
+/Users/brad.poulton/Documents/GitHub/xdr-terraform-modules/base/phantom/securitygroup-server.tf,phantom_server_outbound_https,from_port,443,tcp
+/Users/brad.poulton/Documents/GitHub/xdr-terraform-modules/base/phantom/securitygroup-server.tf,phantom_server_outbound_ssh_to_legacy,from_port,22,tcp

+ 42 - 22
scripts/ports_scrapper.py

@@ -1,3 +1,10 @@
+#!/usr/local/bin/python3
+'''
+ports_scrapper.py
+The purpose of the script is to parse all .tf files in the xdr-terraform-modules git repo. 
+While parsing the .tf files, the open ports are gathered to help update the SSP. 
+'''
+
 import logging
 import sys
 import os
@@ -7,15 +14,10 @@ try:
 except ModuleNotFoundError:
     print("Module 'hcl2' is not installed. Try 'pip3 install python-hcl2'")
 
-'''
-#read in folders
-# loop through folders 
-#find .hcl files
-#pull out ports
-# display to user and write to csv file
-'''
+
 
 def get_current_dir():
+    '''Pulls current directory and returns it. Each user places the git repo at a diff location'''
     current_dir = os.path.dirname(os.path.realpath(__file__))
     relative_path = os.path.dirname(current_dir)
     correct_path = os.path.join(relative_path, "base/")
@@ -23,6 +25,7 @@ def get_current_dir():
     return correct_path
 
 def get_files(base_dir):
+    '''Gathers all the .tf files in a list and ignores some files '''
     print("Looking in " + base_dir +" for TF hcl files")
     r = []
     for root, dirs, files in os.walk(base_dir):
@@ -33,39 +36,56 @@ def get_files(base_dir):
     return r
 
 def parse_tf_files(files):
+    '''Parses the .tf files and returns the ports''' 
     my_resources = []
+    my_resources_dict = {}
     for file in files:
         #print(file)
         with open(file, 'r') as open_file:
             try:
                 dict = hcl2.load(open_file)
             except:
-                pass
+                print("WARNING: " + file + " This file did not load successfully and was skipped!")
 
         for resources in dict.get("resource", []):
             for resource in resources.keys():
                 for resource_name in resources[resource].keys():
                     for item_name in resources[resource][resource_name].keys():
-                        if item_name == "from_port" or item_name == "to_port":
-                            if resources[resource][resource_name]["from_port"] == resources[resource][resource_name]["to_port"]:
-                                my_resources.append(file + "," + resource_name + "," + item_name + "," + str(resources[resource][resource_name][item_name][0]) + "," + resources[resource][resource_name]["protocol"][0])
-                                break
-                            else:
-                                my_resources.append(file + "," + resource_name + "," + item_name + "," + str(resources[resource][resource_name][item_name][0]) + "," + resources[resource][resource_name]["protocol"][0])
+                        if item_name == "from_port" or item_name == "to_port": #only look at the ports
+                            if type(resources[resource][resource_name][item_name][0]) is int: #only grab it if is is a number
+                                if resources[resource][resource_name]["from_port"] == resources[resource][resource_name]["to_port"]:
+                                    #if the from_port and to_port are the same only grab it once 
+                                    my_resources.append(file + "," + resource_name + "," + item_name + "," + str(resources[resource][resource_name][item_name][0]) + "," + resources[resource][resource_name]["protocol"][0])
+                                    dict_input = file + "," + resource_name + "," + item_name + "," + str(resources[resource][resource_name][item_name][0]) + "," + resources[resource][resource_name]["protocol"][0]
+                                    my_resources_dict[resources[resource][resource_name][item_name][0]] = dict_input
+                                    break
+                                else:
+                                    #grab both from_port and to_port
+                                    my_resources.append(file + "," + resource_name + "," + item_name + "," + str(resources[resource][resource_name][item_name][0]) + "," + resources[resource][resource_name]["protocol"][0])
+                                    dict_input = file + "," + resource_name + "," + item_name + "," + str(resources[resource][resource_name][item_name][0]) + "," + resources[resource][resource_name]["protocol"][0]
+                                    my_resources_dict[resources[resource][resource_name][item_name][0]] = dict_input
     print("Found "+ str(len(my_resources)) + " port references.")
-    return my_resources
+    return my_resources, my_resources_dict
+
+def dedup_generate_csv(my_resources, my_resources_dict):
+    '''Cleans up the data and outputs it to a file.'''
 
-def dedup_generate_csv(my_resources):
-    #dedup_resources =
     with open('xdr_port_references.csv', mode='w') as xdr_port_references:
-        writer = csv.writer(xdr_port_references)
+        #writer = csv.writer(xdr_port_references)
+        for item in my_resources_dict:
+            xdr_port_references.write(my_resources_dict[item] + "\n")
+            #print(item)
+
+    with open('all_xdr_port_references.csv', mode='w') as xdr_port_references:
+        #writer = csv.writer(xdr_port_references)
         for item in my_resources:
-            writer.writerow(item)
-            print(item)
+            xdr_port_references.write(item+ "\n")
+            #print(item)
 
 if __name__ == "__main__":
     correct_path = get_current_dir()
     files = get_files(correct_path)
-    my_resources = parse_tf_files(files)
-    dedup_generate_csv(my_resources)
+    my_resources, my_resources_dict = parse_tf_files(files)
+    dedup_generate_csv(my_resources, my_resources_dict)
+    print("Completed Successfully. Please open csv files.") 
     sys.exit(0)

+ 38 - 0
scripts/xdr_port_references.csv

@@ -0,0 +1,38 @@
+/Users/brad.poulton/Documents/GitHub/xdr-terraform-modules/base/sensu/main.tf,sensu_ui,from_port,8000,tcp
+/Users/brad.poulton/Documents/GitHub/xdr-terraform-modules/base/splunk_servers/alsi/elb-hec.tf,alsi-alb-hec-8088-out,from_port,8088,tcp
+/Users/brad.poulton/Documents/GitHub/xdr-terraform-modules/base/phantom/securitygroup-server.tf,phantom_server_outbound_postgres,from_port,8089,tcp
+/Users/brad.poulton/Documents/GitHub/xdr-terraform-modules/base/splunk_servers/indexer_cluster/security-group-indexers.tf,splunk-idx-replication-out,from_port,9887,tcp
+/Users/brad.poulton/Documents/GitHub/xdr-terraform-modules/base/splunk_servers/alsi/master.tf,alsi-master-splunk-data,from_port,9997,tcp
+/Users/brad.poulton/Documents/GitHub/xdr-terraform-modules/base/splunk_servers/alsi/master.tf,alsi-master-splunk-data,to_port,9998,tcp
+/Users/brad.poulton/Documents/GitHub/xdr-terraform-modules/base/phantom/securitygroup-server.tf,phantom_server_outbound_https,from_port,443,tcp
+/Users/brad.poulton/Documents/GitHub/xdr-terraform-modules/base/phantom/securitygroup-server.tf,phantom_server_outbound_http,from_port,80,tcp
+/Users/brad.poulton/Documents/GitHub/xdr-terraform-modules/base/splunk_servers/alsi/workers.tf,alsi_worker_vpn_in2,from_port,9200,tcp
+/Users/brad.poulton/Documents/GitHub/xdr-terraform-modules/base/splunk_servers/alsi/master.tf,alsi-master-vpn-web-in,from_port,9000,tcp
+/Users/brad.poulton/Documents/GitHub/xdr-terraform-modules/base/splunk_servers/alsi/master.tf,alsi-master-interconnections,from_port,4200,tcp
+/Users/brad.poulton/Documents/GitHub/xdr-terraform-modules/base/phantom/securitygroup-server.tf,phantom_server_outbound_ssh_to_legacy,from_port,22,tcp
+/Users/brad.poulton/Documents/GitHub/xdr-terraform-modules/base/jira/rds_jira/security-groups.tf,jira_rds_in,from_port,5432,tcp
+/Users/brad.poulton/Documents/GitHub/xdr-terraform-modules/base/customer_portal/main.tf,customer_portal_smtps_outbound,from_port,465,tcp
+/Users/brad.poulton/Documents/GitHub/xdr-terraform-modules/base/interconnects/security-groups.tf,bgp_ingress,from_port,179,tcp
+/Users/brad.poulton/Documents/GitHub/xdr-terraform-modules/base/interconnects/security-groups.tf,ipsec_l2tp_ingress,from_port,1701,udp
+/Users/brad.poulton/Documents/GitHub/xdr-terraform-modules/base/interconnects/security-groups.tf,ipsec_ike_ingress,from_port,500,udp
+/Users/brad.poulton/Documents/GitHub/xdr-terraform-modules/base/interconnects/security-groups.tf,ipsec_ike_nat_t_ingress,from_port,4500,udp
+/Users/brad.poulton/Documents/GitHub/xdr-terraform-modules/base/vmray_instances/security-groups.tf,vmray-egress,from_port,0,all
+/Users/brad.poulton/Documents/GitHub/xdr-terraform-modules/base/openvpn/main.tf,openvpn-in,from_port,1194,udp
+/Users/brad.poulton/Documents/GitHub/xdr-terraform-modules/base/github/securitygroup-server.tf,ghe_server_inbound_https_external_elb_8444,from_port,8443,tcp
+/Users/brad.poulton/Documents/GitHub/xdr-terraform-modules/base/phantom/securitygroup-server.tf,phantom_server_inbound_8888,from_port,8888,tcp
+/Users/brad.poulton/Documents/GitHub/xdr-terraform-modules/base/github/securitygroup-server.tf,ghe_server_inbound_mgmt_ssh_backup_sgs,from_port,122,tcp
+/Users/brad.poulton/Documents/GitHub/xdr-terraform-modules/base/nessus/instance_nessus_scanner/securitygroup-server.tf,nessus_scanner_inbound_nessus,from_port,8834,tcp
+/Users/brad.poulton/Documents/GitHub/xdr-terraform-modules/base/nessus/instance_nessus_scanner/securitygroup-server.tf,nessus_scanner_inbound_nessus,to_port,8835,tcp
+/Users/brad.poulton/Documents/GitHub/xdr-terraform-modules/base/openvpn/main.tf,openvpn-ldap-out,from_port,636,tcp
+/Users/brad.poulton/Documents/GitHub/xdr-terraform-modules/base/sensu/elb.tf,alb_to_sensu_health,from_port,8080,tcp
+/Users/brad.poulton/Documents/GitHub/xdr-terraform-modules/base/jira/rds_jira/security-groups.tf,jira_security_in,to_port,65535,-1
+/Users/brad.poulton/Documents/GitHub/xdr-terraform-modules/base/github/securitygroup-backupserver.tf,ghe_backup_server_ingress_nfs,from_port,2049,tcp
+/Users/brad.poulton/Documents/GitHub/xdr-terraform-modules/base/github/securitygroup-server.tf,ghe_server_inbound_https_internal_elb,to_port,444,tcp
+/Users/brad.poulton/Documents/GitHub/xdr-terraform-modules/base/github/securitygroup-server.tf,ghe_server_inbound_https_internal_elb_23,from_port,23,tcp
+/Users/brad.poulton/Documents/GitHub/xdr-terraform-modules/base/github/securitygroup-server.tf,ghe_server_inbound_https_external_elb_8444,to_port,8444,tcp
+/Users/brad.poulton/Documents/GitHub/xdr-terraform-modules/base/github/securitygroup-server.tf,ghe_server_outbound_syslog,from_port,1514,tcp
+/Users/brad.poulton/Documents/GitHub/xdr-terraform-modules/base/mailrelay/main.tf,smtp-in,from_port,25,tcp
+/Users/brad.poulton/Documents/GitHub/xdr-terraform-modules/base/mailrelay/main.tf,submission-out,from_port,587,tcp
+/Users/brad.poulton/Documents/GitHub/xdr-terraform-modules/base/teleport-single-instance/alb-internal.tf,alb_internal-to_server,from_port,3080,tcp
+/Users/brad.poulton/Documents/GitHub/xdr-terraform-modules/base/phantom/securitygroup-server.tf,phantom_server_outbound_tcp_dns,from_port,53,udp
+/Users/brad.poulton/Documents/GitHub/xdr-terraform-modules/base/sensu/elb.tf,alb_to_sensu_server,from_port,8081,tcp