This article describes an approach for correlating the business outcome of an AWS workload with the cost of its AWS resources.
The end goal of this activity is to arrive at a point where, given the input parameters of a workload (like data ingested), we should be able to predict the cost of AWS resources needed to run this workload.
We begin by running the workload several times, with varying input parameters. Each time, we measure both the input metrics of the workload, like amount of data ingested & the AWS bill it generates.
Eventually, we arrive at a correlation like this:
Input Metric #1
Input Metric #2
Input Metric #3
Total Cost of All AWS Resources Involved
Workload Run #1
Workload Run #2
Workload Run #3
Different AWS services are used to gather this data:
Standard CloudWatch service-level metrics or custom business-oriented metrics, like orders processed
AWS Cost & Usage Reports
Create AWS Cost & Usage Reports (CUR)
The AWS CUR contains the most comprehensive set of cost & usage data available. Reporting can be enabled on an account or organization-level. Reports can be configured to breakdown costs by the hour. Once enabled, the first report might take up to 24 hours to be delivered. Reports are delivered into an S3 bucket you choose. Reports are updated at least once a day & at most thrice a day.
Provide a report path prefix, select hourly granularity & enable Athena:
Complete the CUR creation wizard.
Activate Cost Allocation Tags
Since an AWS account could be running several workloads at any given time, one way to isolate a single instance of a workload is to tag all resources of that workload with a unique value. However, in order for any tags to appear in CUR, they must be activated as cost allocation tags. To do so:
Human-readable description of the charge. Examples: > $0.015 per provisioned shard-hour (Kinesis) > $0.005 per 1,000 PUT, COPY, POST, or LIST requests (S3) > AWS Lambda – Requests Free Tier – 1,000,000 Requests – US East (Northern Virginia)
The public price of the resource. You pay less based on various factors like credits, reservations & savings plans, agreements with AWS, support fees, etc
To create a correlation between the workload inputs & its cost, we can use a combination of CloudWatch metrics like PutRecordsBytes / IncomingBytes for a Kinesis stream & relevant fields from the CUR like line_item_usage_amount & line_item_blended_cost.
This article describes how you can use the AWS CUR to gain detailed insights into your AWS billing. Although it may not be practical to use the CUR directly for larger workloads, understanding it will give you an appreciation of how other tools & services in the market use the CUR as their data source & provide actionable insights.
Privacy & Cookies Policy
Necessary cookies are absolutely essential for the website to function properly. This category only includes cookies that ensures basic functionalities and security features of the website. These cookies do not store any personal information.
Any cookies that may not be particularly necessary for the website to function and is used specifically to collect user personal data via analytics, ads, other embedded contents are termed as non-necessary cookies. It is mandatory to procure user consent prior to running these cookies on your website.