Adds GHE Key to Secrets Manager; Schedules App Builds; Adds GHE Secret to Codebuild Projects

Updates readme to change the way the GHE key is consumed. Removed the
manual step now that it's going to be needed in all accounts.

To be tagged v3.4.7

base/account_standards_c2/secrets.tf

@@ -1,7 +1,17 @@
 # Set up some basic secret configuration. We don't want the secrets themselves in here. They'll have to be hand-entered. But this will set up the scaffolding.
-#resource "aws_secretsmanager_secret" "ubuntu" {
-#  name = "Ubuntu"
-#  description = "Secrets required for provisioning Ubuntu systems."
-#  recovery_window_in_days = 30
-#  tags = merge(var.standard_tags, var.tags)
-#}
+output "secrets_manager_reminder" {
+  value = "REMINDER: If this is your first time, don't forget to update the secrets in secrets manager."
+}
+
+resource "aws_secretsmanager_secret" "codebuild_ghe_key" {
+  name = "GHE/mdr-aws-codebuild/key"
+  description = "GitHub Personal Access Key for the mdr-aws-codebuild account"
+  recovery_window_in_days = 30
+  tags = merge(var.standard_tags, var.tags)
+}
+
+# This just seeds an initial value. It will not be overwritten each update.
+resource "aws_secretsmanager_secret_version" "codebuild_ghe_secret_version" {
+  secret_id     = aws_secretsmanager_secret.codebuild_ghe_key.id
+  secret_string = "SETME"
+}

base/codebuild_artifact/README.md

@@ -10,9 +10,11 @@ This module should NOT create the github repo. That is a manual process. I am no
 
 ## Github Service Account ( mdr-aws-codebuild )
 
-AWS CodeBuild needs a Github Personal Access Token to pull code after the code in a repository has been updated. Terraform is capable of storing the Github Personal Access Token, but that is a bad idea. A better idea is a service account in Github that gives CodeBuild access to specific repositories. This user will need access to repositories in different organizations. The login credentials as well as the Personal Access Token for mdr-aws-codebuild are stored in Vault. 
+AWS CodeBuild needs a Github Personal Access Token to pull code after the code in a repository has been updated. This is the `mdr-aws-codebuild` account. The secret github token is stored in the C2 account and automatically added.
 
-The service account (mdr-aws-codebuild) needs to have a personal access token manually placed into the aws console. 
+Despite the Web Interface, there can be only one GHE token per account and region.
+
+The `mdr-aws-codebuild` user must have read access to the repositories you wish to build from.
 
 ## Creating a Personal Access Token
 

base/codebuild_ecr_customer_portal/README.md

@@ -12,6 +12,9 @@ This module should NOT create the github repo. That is a manual process. I am no
 
 ## Github Service Account ( mdr-aws-codebuild )
 
-AWS CodeBuild needs a Github Personal Access Token to pull code after the code in a repository has been updated. Terraform is capable of storing the Github Personal Access Token, but that is a bad idea. A better idea is a service account in Github that gives CodeBuild access to specific repositories. This user will need access to repositories in different organizations. The login credentials as well as the Personal Access Token for mdr-aws-codebuild are stored in Vault. 
+AWS CodeBuild needs a Github Personal Access Token to pull code after the code in a repository has been updated. This is the `mdr-aws-codebuild` account. The secret github token
+ is stored in the C2 account and automatically added.
 
-The service account (mdr-aws-codebuild) needs to have a personal access token manually placed into the aws console. 
+Despite the Web Interface, there can be only one GHE token per account and region.
+
+The `mdr-aws-codebuild` user must have read access to the repositories you wish to build from.

base/codebuild_ecr_customer_portal/ghe-key.tf

@@ -0,0 +1,27 @@
+data "aws_secretsmanager_secret" "ghe-key" {
+  name = "GHE/mdr-aws-codebuild/key"
+  provider = aws.c2
+}
+
+data "aws_secretsmanager_secret_version" "ghe-key" {
+  secret_id = data.aws_secretsmanager_secret.ghe-key.id
+  provider = aws.c2
+}
+
+#locals {
+#  If key was in json format, we would need to decode it.
+#  secret_ghe_key = jsondecode(data.aws_secretsmanager_secret_version.ghe-key.secret_string)
+#}
+
+
+# Note some AWS craziness here. The GitHub credential is not tied to a build, even though it _looks_
+# like it is in the Web UI. There can only be one GitHub credential per account+region::
+# https://docs.aws.amazon.com/cdk/api/v1/docs/@aws-cdk_aws-codebuild.GitHubSourceCredentials.html
+#
+# "Note: CodeBuild only allows a single credential for GitHub to be saved in a given AWS account 
+#        in a given region - any attempt to add more than one will result in an error."
+resource "aws_codebuild_source_credential" "github_token" {
+  auth_type   = "PERSONAL_ACCESS_TOKEN"
+  server_type = "GITHUB_ENTERPRISE"
+  token       = data.aws_secretsmanager_secret_version.ghe-key.secret_string
+}

base/codebuild_ecr_project/README.md

@@ -10,6 +10,8 @@ This module should NOT create the github repo. That is a manual process. I am no
 
 ## Github Service Account ( mdr-aws-codebuild )
 
-AWS CodeBuild needs a Github Personal Access Token to pull code after the code in a repository has been updated. Terraform is capable of storing the Github Personal Access Token, but that is a bad idea. A better idea is a service account in Github that gives CodeBuild access to specific repositories. This user will need access to repositories in different organizations. The login credentials as well as the Personal Access Token for mdr-aws-codebuild are stored in Vault. 
+AWS CodeBuild needs a Github Personal Access Token to pull code after the code in a repository has been updated. This is the `mdr-aws-codebuild` account. The secret github token is stored in the C2 account and automatically added.
 
-The service account (mdr-aws-codebuild) needs to have a personal access token manually placed into the aws console. 
+Despite the Web Interface, there can be only one GHE token per account and region.
+
+The `mdr-aws-codebuild` user must have read access to the repositories you wish to build from.

base/codebuild_ecr_project/ghe-key.tf

@@ -0,0 +1,27 @@
+data "aws_secretsmanager_secret" "ghe-key" {
+  name = "GHE/mdr-aws-codebuild/key"
+  provider = aws.c2
+}
+
+data "aws_secretsmanager_secret_version" "ghe-key" {
+  secret_id = data.aws_secretsmanager_secret.ghe-key.id
+  provider = aws.c2
+}
+
+#locals {
+#  If key was in json format, we would need to decode it.
+#  secret_ghe_key = jsondecode(data.aws_secretsmanager_secret_version.ghe-key.secret_string)
+#}
+
+
+# Note some AWS craziness here. The GitHub credential is not tied to a build, even though it _looks_
+# like it is in the Web UI. There can only be one GitHub credential per account+region::
+# https://docs.aws.amazon.com/cdk/api/v1/docs/@aws-cdk_aws-codebuild.GitHubSourceCredentials.html
+#
+# "Note: CodeBuild only allows a single credential for GitHub to be saved in a given AWS account 
+#        in a given region - any attempt to add more than one will result in an error."
+resource "aws_codebuild_source_credential" "github_token" {
+  auth_type   = "PERSONAL_ACCESS_TOKEN"
+  server_type = "GITHUB_ENTERPRISE"
+  token       = data.aws_secretsmanager_secret_version.ghe-key.secret_string
+}

base/codebuild_splunk_apps/README.md

@@ -16,9 +16,11 @@ This module should NOT create the github repo. That is a manual process. I am no
 
 ## Github Service Account ( mdr-aws-codebuild )
 
-AWS CodeBuild needs a Github Personal Access Token to pull code after the code in a repository has been updated. Terraform is capable of storing the Github Personal Access Token, but that is a bad idea. A better idea is a service account in Github that gives CodeBuild access to specific repositories. This user will need access to repositories in different organizations. The login credentials as well as the Personal Access Token for mdr-aws-codebuild are stored in Vault. 
+AWS CodeBuild needs a Github Personal Access Token to pull code after the code in a repository has been updated. This is the `mdr-aws-codebuild` account. The secret github token is stored in the C2 account and automatically added.
 
-The service account (mdr-aws-codebuild) needs to have a personal access token manually placed into the aws console. 
+Despite the Web Interface, there can be only one GHE token per account and region.
+
+The `mdr-aws-codebuild` user must have read access to the repositories you wish to build from.
 
 ## Creating a Personal Access Token
 

base/codebuild_splunk_apps/cloudwatch.tf

@@ -0,0 +1,98 @@
+# creates a role and schedules a build for each server type
+# 
+# Being polite aws users, we randomize the schedule to the beginning of the work day
+# (Between 9am and 1pm ET)
+resource "random_integer" "hour" {
+  min = 14 # 9 am ET
+  max = 17 # noon ET
+}
+
+resource "random_integer" "minute" {
+  min = 0
+  max = 59
+}
+
+resource "aws_cloudwatch_event_rule" "schedule_rule" {
+  for_each               = local.splunk_server_types
+
+  name = "scheduled_build_${each.value}"
+  schedule_expression = "cron(${random_integer.minute.result} ${random_integer.hour.result} * * ? *)"
+}
+
+resource "aws_iam_role" "codebuild_role" {
+  name = "splunk_apps_codebuild_role"
+  path     = "/aws_services/"
+
+  assume_role_policy = <<EOF
+{
+  "Version": "2012-10-17",
+  "Statement": [
+    {
+      "Effect": "Allow",
+      "Principal": {
+        "Service": [
+          "events.amazonaws.com",
+          "codebuild.amazonaws.com"
+        ]
+      },
+      "Action": "sts:AssumeRole"
+    }
+  ]
+}
+EOF
+}
+
+resource "aws_iam_policy" "codebuild_policy" {
+  name = "splunk_apps_policy"
+  path     = "/aws_services/"
+
+  policy = <<POLICY
+{
+    "Version": "2012-10-17",
+    "Statement": [
+        {
+            "Effect": "Allow",
+            "Resource": [
+                "arn:${var.aws_partition}:logs:${var.aws_region}:${var.aws_account_id}:log-group:/aws/codebuild/*"
+            ],
+            "Action": [
+                "logs:CreateLogGroup",
+                "logs:CreateLogStream",
+                "logs:PutLogEvents"
+            ]
+        },
+        {
+           "Action": [
+              "codebuild:StartBuild",
+              "codebuild:StopBuild",
+              "codebuild:BatchGet*",
+              "codebuild:Get*",
+              "codebuild:List*",
+              "codecommit:GetBranch",
+              "codecommit:GetCommit",
+              "codecommit:GetRepository",
+              "codecommit:ListBranches"
+            ],
+            "Effect": "Allow",
+            "Resource": "*"
+          }
+    ]
+}
+POLICY
+}
+
+resource "aws_iam_policy_attachment" "service_role_attachment" {
+  name = "splunk_apps_policy_attachment"
+  policy_arn = "${aws_iam_policy.codebuild_policy.arn}"
+  roles = ["${aws_iam_role.codebuild_role.id}"]
+}
+
+resource "aws_cloudwatch_event_target" "trigger_build" {
+  for_each = local.splunk_server_types
+
+  target_id = "trigger_build_${each.value}"
+  rule = "${aws_cloudwatch_event_rule.schedule_rule[each.value].name}"
+  arn = "${aws_codebuild_project.this[each.value].id}"
+
+  role_arn = "${aws_iam_role.codebuild_role.arn}"
+}

base/codebuild_splunk_apps/ghe-key.tf

@@ -0,0 +1,27 @@
+data "aws_secretsmanager_secret" "ghe-key" {
+  name = "GHE/mdr-aws-codebuild/key"
+  provider = aws.c2
+}
+
+data "aws_secretsmanager_secret_version" "ghe-key" {
+  secret_id = data.aws_secretsmanager_secret.ghe-key.id
+  provider = aws.c2
+}
+
+#locals {
+#  If key was in json format, we would need to decode it.
+#  secret_ghe_key = jsondecode(data.aws_secretsmanager_secret_version.ghe-key.secret_string)
+#}
+
+
+# Note some AWS craziness here. The GitHub credential is not tied to a build, even though it _looks_
+# like it is in the Web UI. There can only be one GitHub credential per account+region::
+# https://docs.aws.amazon.com/cdk/api/v1/docs/@aws-cdk_aws-codebuild.GitHubSourceCredentials.html
+#
+# "Note: CodeBuild only allows a single credential for GitHub to be saved in a given AWS account 
+#        in a given region - any attempt to add more than one will result in an error."
+resource "aws_codebuild_source_credential" "github_token" {
+  auth_type   = "PERSONAL_ACCESS_TOKEN"
+  server_type = "GITHUB_ENTERPRISE"
+  token       = data.aws_secretsmanager_secret_version.ghe-key.secret_string
+}

base/codebuild_splunk_apps/main.tf

@@ -2,15 +2,6 @@ data "github_repository" "this" {
     name    = "content_source"
 }
 
-#resource "aws_codebuild_source_credential" "github_token" {
-#  auth_type   = "PERSONAL_ACCESS_TOKEN"
-#  server_type = "GITHUB_ENTERPRISE"
-#  token       = "" # This could be used to make life easier, but it would be stored in the state in plaintext.
-#}
-output "Codebuild_AWS_Key_Reminder" {
-  value = "REMINDER: If this is a fresh deployment, you must manually enter the GITHUB token for 'mdr-aws-codebuild' (found in the vault) into one of the codebuild jobs."
-}
-
 resource "aws_codebuild_project" "this" {
   for_each               = local.splunk_server_types