Automate the Management of EC2 Inventory at Scale with Systems Manager

If you manage a lot of EC2 instances as a part of your day-to-day activities, you know the variety of tasks you need to take care of for all your instances, right from standardizing the creation of instances with golden images, to managing & maintaining them throughout their lifetime, and finally, cleaning up securely when they go out of use.

Each of these aspects includes a lot of detail.

Automating the creation of the golden images that your whole organization can use & keeping them always patched is in itself quite a sizeable piece of work, especially considering the sheer variety of operating systems & Linux flavors out there & the fact that different teams in your organization need different flavors.

Keeping track of all the instances you have, keeping them up-to-date, managing their maintenance & patching, and automating as much of this as possible, is where the bulk of your time will be spent.

People love to use & forget about their VMs, but you can’t! You’re the one looking out for idle instances, burning money for no reason, and cleaning them up as soon as possible.

As you see, there is enough work here to keep you busy forever, depending on the size of your enterprise.

Fortunately, there exist many tools to make our lives easier. AWS Systems Manager is one of them. But if you’re in a large enterprise, chances are, Systems Manager isn’t the only tool in your arsenal. Third-party tools come with some amazing features that AWS-native services simply don’t provide. CloudHealth by VMware is one such service.

Now that you have more systems involved in your cloud management journey, keeping them all in sync becomes yet another overhead. Here we explore the capabilities of Systems Manager that you can leverage to play well with other systems.


This article explores the inventory management capabilities of AWS Systems Manager by taking a very small & specific example: get the version of the OS running in all your EC2 instances.

EC2 Inventory

Even if all your instances are managed via Systems Manager, there is often a need to collect a detailed inventory of your instance fleet & either use it as a report for making decisions or import it into other tools & services for post-processing.

Since we’re talking about managing hundreds or thousands of instances here, automation is the only way to go.

As you would expect, AWS Systems Manager maintains a pretty detailed inventory & you can use the AWS CLI to automate the collection of any piece of information you need about your instances.

The describe-instance-information sub-command of the AWS CLI’s SSM command provides details about your instances. Here is an example:

> aws ssm describe-instance-information
{
    "InstanceInformationList": [
        {
            "InstanceId": "i-03197798fd71af36c",
            "PingStatus": "Online",
            "LastPingDateTime": "2021-10-15T17:29:41.444000+05:30",
            "AgentVersion": "3.1.338.0",
            "IsLatestVersion": false,
            "PlatformType": "Linux",
            "PlatformName": "CentOS Linux",
            "PlatformVersion": "7.9.2009",
            "ResourceType": "EC2Instance",
            "IPAddress": "10.69.11.143",
            "ComputerName": "ip-10-69-11-143.ec2.internal",
            "AssociationStatus": "Success",
            "LastAssociationExecutionDate": "2021-10-15T17:30:27.611000+05:30",
            "LastSuccessfulAssociationExecutionDate": "2021-10-15T17:30:27.611000+05:30",
            "AssociationOverview": {
                "DetailedStatus": "Success",
                "InstanceAssociationStatusAggregatedCount": {
                    "Success": 3
                }
            }
        }
    ]
}

That’s just the command’s default output, but you can change it a lot to meet your exact needs.

For example, the following Bash script retrieves only the OS name & version of the VMs whose IPs are passed to the script in a text file:

IP_FILE=$1
IPs=$(cat $IP_FILE)

for IP in $IPs; do
    aws ssm describe-instance-information \
    --query 'InstanceInformationList[?IPAddress==`'$IP'`].[PlatformName,PlatformVersion]' \
    --output text | xargs
done

It essentially just loops through the list of IPs in the input file & queries SSM for their details one by one.

The --query parameter of the SSM command is the star of the show here. It is filtering the output of all VMs to just the one we want with [?IPAddress=='$IP'] & then fetching the OS details with [PlatformName,PlatformVersion]. The xargs is just to remove the extra spaces in the output.

The --query parameter, in case you don’t know, can be used with any AWS CLI command. It takes a JMESPath expression as its value & filters the output of the command accordingly.

The above script takes an input file of IPs like this:

10.69.12.129
10.69.1.48
10.69.12.120
10.69.11.110
10.69.13.168
10.69.11.145
10.69.13.130
10.69.11.31

And returns their OS details like this:

CentOS Linux 7.9.2009
Amazon Linux 2
CentOS Linux 7.9.2009
CentOS Linux 7.9.2009
CentOS Linux 7.9.2009
CentOS Linux 7.9.2009
CentOS Linux 7.9.2009
Amazon Linux 2

Conclusion

By exploring a pretty niche example, we looked at the inventory management capabilities of the Systems Manager & how you can automate it at scale & customize it for your needs. I hope this article has added yet another skill to your skillset! 😊

About the Author ✍🏻

Harish KM is a Principal DevOps Engineer at QloudX & a top-ranked AWS Ambassador since 2020. 👨🏻‍💻

With over a decade of industry experience as everything from a full-stack engineer to a cloud architect, Harish has built many world-class solutions for clients around the world! 👷🏻‍♂️

With over 20 certifications in cloud (AWS, Azure, GCP), containers (Kubernetes, Docker) & DevOps (Terraform, Ansible, Jenkins), Harish is an expert in a multitude of technologies. 📚

These days, his focus is on the fascinating world of DevOps & how it can transform the way we do things! 🚀

Leave a Reply

Your email address will not be published. Required fields are marked *