module.moose_cluster.module.indexer_cluster.module.indexer2.aws_launch_configuration.splunk_indexer


module.moose_cluster.module.indexer_cluster.module.indexer2.aws_autoscaling_group.splunk_indexer_asg



terraform destroy -target=module.moose_cluster.module.indexer_cluster.module.indexer1.aws_launch_configuration.splunk_indexer -target=module.moose_cluster.module.indexer_cluster.module.indexer1.aws_autoscaling_group.splunk_indexer_asg -target=module.moose_cluster.module.indexer_cluster.module.indexer2.aws_launch_configuration.splunk_indexer -target=module.moose_cluster.module.indexer_cluster.module.indexer2.aws_autoscaling_group.splunk_indexer_asg -target=module.moose_cluster.module.indexer_cluster.module.indexer0.aws_launch_configuration.splunk_indexer -target=module.moose_cluster.module.indexer_cluster.module.indexer0.aws_autoscaling_group.splunk_indexer_asg


terraform destroy -target=module.moose_cluster.module.indexer_cluster.module.indexer1.aws_launch_template.splunk_indexer -target=module.moose_cluster.module.indexer_cluster.module.indexer1.aws_autoscaling_group.splunk_indexer_asg -target=module.moose_cluster.module.indexer_cluster.module.indexer2.aws_launch_template.splunk_indexer -target=module.moose_cluster.module.indexer_cluster.module.indexer2.aws_autoscaling_group.splunk_indexer_asg -target=module.moose_cluster.module.indexer_cluster.module.indexer0.aws_launch_template.splunk_indexer -target=module.moose_cluster.module.indexer_cluster.module.indexer0.aws_autoscaling_group.splunk_indexer_asg


Current moose subnet: subnet-07312c554fb87e4b5  (main-infrastructure-public-us-east-1c)
ASG subnet: subnet-0b1e9d82bcd8c0a2c (main-infrastructure-public-us-east-1a)





moose-splunk-indexer-i-0a30e6cbd4d7461ba.msoc.defpoint.local:
    Data failed to compile:
----------
    The function "state.apply" is running as PID 11878 and was started at 2020, Apr 28 14:58:18.662679 with jid 20200428145818662679
moose-splunk-indexer-i-08151d5e4b73af430.msoc.defpoint.local:
    Data failed to compile:
----------
    The function "state.apply" is running as PID 11879 and was started at 2020, Apr 28 14:58:18.677356 with jid 20200428145818677356
moose-splunk-indexer-i-0e7519cfe60483af1.msoc.defpoint.local:
    Data failed to compile:
----------
    The function "state.apply" is running as PID 11817 and was started at 2020, Apr 28 14:58:19.465731 with jid 20200428145819465731







resource "aws_launch_configuration" "splunk_indexer" {
    name                        = "${var.launch_conf_name}"
    instance_type               = "${var.idx_instance_type}"
    image_id                    = "${var.ami}"
    user_data                   = "${var.user_data}"
    security_groups             = ["${var.indexer_security_group_ids}"]
    associate_public_ip_address = false
    key_name                    = "${var.key_name}"
    iam_instance_profile        = "${var.iam_instance_profile}"
    root_block_device           = "${var.root_block_device}"
    ebs_block_device            = "${local.ebs_block_device}"
    ebs_optimized               = true
    ephemeral_block_device = [
        {
        device_name  = "xvdaa"
        virtual_name = "ephemeral0"
        },
        {
        device_name = "xvdab"
        virtual_name = "ephemeral1"
        },
        {
        device_name = "xvdac"
        virtual_name = "ephemeral2"
        },
        {
        device_name = "xvdad"
        virtual_name = "ephemeral3"
        },
        {
        device_name = "xvdae"
        virtual_name = "ephemeral4"
        },
        {
        device_name = "xvdaf"
        virtual_name = "ephemeral5"
        },
        {
        device_name = "xvdag"
        virtual_name = "ephemeral6"
        },
        {
        device_name = "xvdah"
        virtual_name = "ephemeral7"
        },
        {
        device_name = "xvdai"
        virtual_name = "ephemeral8"
        },
        {
        device_name = "xvdaj"
        virtual_name = "ephemeral9"
        },
        {
        device_name = "xvdak"
        virtual_name = "ephemeral10"
        },
        {
        device_name = "xvdal"
        virtual_name = "ephemeral11"
        },
        {
        device_name = "xvdam"
        virtual_name = "ephemeral12"
        },
        {
        device_name = "xvdan"
        virtual_name = "ephemeral13"
        },
        {
        device_name = "xvdao"
        virtual_name = "ephemeral14"
        },
        {
        device_name = "xvdap"
        virtual_name = "ephemeral15"
        },
        {
        device_name = "xvdaq"
        virtual_name = "ephemeral16"
        },
        {
        device_name = "xvdar"
        virtual_name = "ephemeral17"
        },
        {
        device_name = "xvdas"
        virtual_name = "ephemeral18"
        },
        {
        device_name = "xvdat"
        virtual_name = "ephemeral19"
        },
        {
        device_name = "xvdau"
        virtual_name = "ephemeral20"
        },
        {
        device_name = "xvdav"
        virtual_name = "ephemeral21"
        },
        {
        device_name = "xvdaw"
        virtual_name = "ephemeral22"
        },
        {
        device_name = "xvdax"
        virtual_name = "ephemeral23"
        },
    ]
    lifecycle {
        create_before_destroy = true
    }
}

## ASG not launching
05/07/2020

ERROR:
* module.moose_cluster.module.indexer_cluster.module.indexer0.aws_autoscaling_group.splunk_indexer_asg: 1 error(s) occurred:

* aws_autoscaling_group.splunk_indexer_asg: "moose-splunk-asg-0": Waiting up to 10m0s: Need at least 1 healthy instances in ASG, have 0. Most recent activity: {
  ActivityId: "71d5c796-f6b8-7b06-600c-167c09da9b56",
  AutoScalingGroupName: "moose-splunk-asg-0",
  Cause: "At 2020-05-05T16:49:03Z an instance was started in response to a difference between desired and actual capacity, increasing the capacity from 0 to 1.",
  Description: "Launching a new EC2 instance.  Status Reason: The requested configuration is currently not supported. Please check the documentation for supported configurations. Launching EC2 instance failed.",
  Details: "{\"Subnet ID\":\"subnet-0b1e9d82bcd8c0a2c\",\"Availability Zone\":\"us-east-1a\"}",
  EndTime: 2020-05-05 16:49:05 +0000 UTC,
  Progress: 100,
  StartTime: 2020-05-05 16:49:05.566 +0000 UTC,
  StatusCode: "Failed",
  StatusMessage: "The requested configuration is currently not supported. Please check the documentation for supported configurations. Launching EC2 instance failed."
}

FIX: ebs optimize needs to be set to false for t2.small instance size. 

---
Salt issues
05/06/2020

RHEL subscription failing
salt-call state.sls os_modifications.rhel_registration

splunk install failing
salt-call state.sls splunk.new_install


salt moose-splunk-indexer-i* cmd.run 'systemctl restart splunkuf'

Run the salt high state twice

# load balancer
05/08/2020

The internal UFs are pointing to moose-splunk-indexer-1.msoc.defpoint.local:9998, moose-splunk-indexer-2.msoc.defpoint.local:9998, moose-splunk-indexer-3.msoc.defpoint.local:9998. This is not going to work for a ASG. Need to use the moose LB internally?