infrastructure-notes/LCP in Azure.md

Notes on LCP in Azure

WALinuxAgent / Waluigi

Azure VMs "need" the WALinuxAgent. Home page is at Official MS Azure Repo but there's a standard RHEL shipped RPM for it too.

Name        : WALinuxAgent
Version     : 2.2.46
Release     : 2.el7_9
Architecture: noarch
Install Date: Thu Oct 15 14:28:33 2020
Group       : Development/Libraries
Size        : 1904125
License     : ASL 2.0
Signature   : RSA/SHA256, Thu Jun  4 05:33:33 2020, Key ID 199e2f91fd431d51
Source RPM  : WALinuxAgent-2.2.46-2.el7_9.src.rpm
Build Date  : Wed Jun  3 10:40:07 2020
Build Host  : x86-vm-26.build.eng.bos.redhat.com
Relocations : /usr
Packager    : Red Hat, Inc. <http://bugzilla.redhat.com/bugzilla>
Vendor      : Red Hat, Inc.
URL         : https://github.com/Azure/WALinuxAgent
Summary     : Microsoft Azure Linux Agent
Description :
The Azure Linux Agent supports the provisioning and running of Linux
VMs in the Azure cloud. This package should be installed on Linux disk
images that are built to run in the Azure environment.

Default configuration for Waluigi in /etc/waagent.conf includes this gem:

# Enable extension handling. Do not disable this unless you do not need password reset,
# backup, monitoring, or any extension handling whatsoever.
Extensions.Enabled=y

Extensions enable the Azure control plane to do .. pretty much whatever it wants. But, one of those things it can do is "enable backups" and clients like those so here we are.

OMS Agent

OMS Agent appears to be a log forwarder into Azure Log Analytics. Probably installed by an "extension". This appears to be the thing that is clobbering our syslog-ng configuration from time to time. It appears to be installed by an RPM:

Name        : omsagent
Version     : 1.13.35
Release     : 0
Architecture: x86_64
Install Date: Fri Mar  5 05:14:32 2021
Group       : Applications/System
Size        : 71658311
License     : none
Signature   : (none)
Source RPM  : omsagent-1.13.35-0.src.rpm
Build Date  : Wed Feb 17 04:23:20 2021
Build Host  : d808ccce382f
Relocations : (not relocatable)
Vendor      : http://www.microsoft.com
Summary     : Microsoft Operations Management Suite for UNIX/Linux agent
Description :
Provides agent for the Microsoft Operations Management Suite.

There's also a second-ish OMS Agent called "LAD" .. for "Azure Diagnostics Extension" see Azure Diagnostics Extension Overview and Use the Linux diagnostic extension 4.0 to monitor metrics and logs

I'm pretty sure the LAD version of the agent is the one wrecking everything...

Random

Somehow these systems have inspec on them too??

Name        : inspec
Version     : 2.2.61
Release     : 1.el7
Architecture: x86_64
Install Date: Tue Mar 10 21:05:53 2020
Group       : default
Size        : 175905773
License     : Apache-2.0
Signature   : DSA/SHA1, Thu Aug  9 14:19:53 2018, Key ID 2940aba983ef826a
Source RPM  : inspec-2.2.61-1.el7.src.rpm
Build Date  : Thu Aug  9 14:19:21 2018
Build Host  : inspec-rhel-7-builder-5d5788.cd.chef.co
Relocations : /
Packager    : Chef Software, Inc <maintainers@chef.io>
Vendor      : Omnibus <omnibus@getchef.com>
URL         : https://github.com/chef/inspec
Summary     : The full stack of inspec
Description :
The full stack of inspec

Cleaning up space

2021-06-03 FTD: afs-splunk-ds-4 was running out of space on /var, so I did the following:

• Moved some apparently redundant stuff from /var/lib/waagent to /opt/backups.from.var.lib.waagent.20210603/:

  drwxr-xr-x. 3 root root     65 Oct 21  2020 WALinuxAgent-2.2.52
  -rw-r--r--. 1 root root 553571 Oct 21  2020 WALinuxAgent-2.2.52.zip
  drwxr-xr-x. 3 root root     65 Feb 23 19:32 WALinuxAgent-2.2.53.1
  -rw-r--r--. 1 root root 598486 Feb 23 19:32 WALinuxAgent-2.2.53.1.zip
  drwxr-xr-x. 3 root root     65 Apr  2 00:27 WALinuxAgent-2.2.54.2
  -rw-r--r--. 1 root root 631616 Apr  2 00:27 WALinuxAgent-2.2.54.2.zip
  

• Deleted /var/opt/microsoft/omsagent/LAD/log/omsagent.log