Displaying AWS Spend In Turbonomic

Document created by dallas.demarr@vmturbo.com Expert on Mar 31, 2017Last modified by carlos.castellanos on Oct 22, 2019
Version 30Show Document
  • View in full screen mode

Turbonomic provides a complete picture of your environment. For those with AWS Targets, this includes spend amounts, which populate the Expenses -> Cost Breakdown views as seen in the "Cloud" tab. In order to display month-to-day spend, you must expose this information via placing the AWS Cost and Usage report in an S3 Bucket that Turbonomic will access.

 

To do this, you must do two things:

  • Create a Cost and Usage Report in AWS, saving it to an S3 bucket
  • Give Turbonomic access to the bucket in Target Configuration.

NOTE: This Guide assumes that the user performing these tasks has both the "AmazonS3FullAccess" and "Billing" permissions that will allow the user to create an S3 bucket to place a report into, and set report preferences for type of report (Daily Cost and Usage) plus location of the S3 bucket. For more details on Billing start here: Cost and Usage Report - AWS Billing and Cost Management 

DISCLAIMER: please keep in mind this document represents and should be read as "guidelines" on this procedure on the AWS side of things. We will do our best in keeping up to date steps and images that mimics to the best of our knowledge the most recent updates in AWS. Nevertheless, please refer to your provider latest official documentation for the most recent instructions or reach you Customer Success/Support representative for further assistance. 

Do you use Consolidated Billing (Consolidated Billing for Organizations - AWS Billing and Cost Management )?  Turbonomic will report the month-day spend summary by AWS Services, but in 5.9.0 cannot parse the different accounts and organizations.  You will need to add a AWS Target with an IAM User that is part of the Master Account Organization or outside Account which minimally has S3 Access to the bucket that is created in that organization. Also, we need to have EC2 instance ( VM either Powered OFF or ON ) on the Account that the IAM User is part of.

 

Create Cost and Usage Report

Creating reports for AWS budgeting

  1. Login
  2. Select a region (for instance US East (N. Virginia)), this is used in Turbonomic Operations Manager (TOM) later on
  3. Click on your account name in the upper left corner and select "My Billing Dashboard" from the drop down list 
  4. From the left menu select "Cost & Usage Reports"
  5. Click on "Create Report"
  6. Enter "Report name" and select "Include resource IDs" and click Next.
  7. You should be in the Delivery Options page. We need to configure an "S3 Bucket", the "Report path prefix", the "Time granularity". Other settings can be left at default value.
    • Let's first setup the easier settings already on screen:
      • Set the "Report path prefix" to "daily" 
      • Set the "Time granularity" to "Daily"
    • Now let's setup the required S3 Bucket. We will create a new bucket for these instructions (see section below to use an existing bucket)
      • Click on "Configure"
      • On Step 1, while you could select an existing bucket, we will use the "Create a bucket" section
        • Enter a unique "S3 bucket name". We will use "turbonomic.billing.reports"
        • Select a "Region". In this case "US East (N. Virginia)" should have been pre-selected (see #2 above)
          • Please take note of the "Region Name" value, you will need it later when adding the target to Turbonomic.
        • Click Next.
      • On Step 2 "Verify Policy", for now check "I have confirmed that  this policy is correct" and click Save.
    • Back in "Delivery Options" you should see the "S3 bucket" information now verified. Click Next.
  8. Please Review and if all is OK click on Review and Complete.
  9. The required Bucket and report are now present.

 

NOTE: AWS publishes the standard daily billing report twice a day, and may take up to 24 hours for the first report to be placed in the bucket.  The Cost Breakdown views in Turbonomic will start populating 1 - 2 hours after.

 

Specifying the Target

Navigate to the Target Configuration section (under Settings) in Turbonomic to add the target.

If you have already added the AWS target to Turbonomic, you may modify the existing target to add the bucket and report fields.  However you will need to re-enter the Secret Access Key.  If you target does not validate after modification, make sure you re-enter all credentials. For more information about adding an AWS Target, please see the latest Target Configuration Guide at https://greencircle.vmturbo.com/community/products/pages/documentation.

 

Note the IAM User (which belongs to the Access and Secret Access Key info) specified as the AWS Target needs to only have S3 access to use the report - Billing access is not required.

The three target fields needed for to access this report are: Cost and Usage Report Bucket Cost and Usage Report Path Cost and Usage Report Region.

  1. Cost and Usage Report Bucket: enter the full bucket name (for example, turbo.billing.reports)
  2. Cost and Usage Report Path: enter report path
    • You can copy the report path from the Billing Report details (see image below - enter path AS IS)
    • In this case it would be "daily/REPORT_AWS_daily"
  3. Cost and Usage Report Region: enter the Region of the S3 bucket (you should have taken note of this value on step #7 above)

 

EXAMPLE

AWS Target with AWS Billing Report named "turbonomic.billing.reports", created on an S3 Bucket named "Turbonomic_Cost_usage_report" created in the Canada Central Region (ca-cental-1):

 

After target discovery is complete, Turbonomic will display spend in the UI (please note that AWS reports make take up to 24hours to be created, only after they are created Turbonomic will start displaying detailed billing data)

 

 

[ALTERNATE USE CASE]

S3 Bucket (New) | Add S3 Bucket (Existing) to Report (New)

CREATING YOUR OWN S3 Bucket 

  • Use these steps If you need to create an S3 Bucket in which to store the report. 
  • Click on Services
    • Look S3 under the Storage section
    • or... simply type "S3" in the search box
  • Click on Create Bucket
  • Specify the required information (see image below)
    • "Bucket name" (e.g. "turbonomic.billing.reports.bucket")
    • "Region" (e.g. "US East (N. Virginia)") | Please take a note of the Region information.
    • Click Next.
  • Click Next on Configure Options (sample image below)
  • Click Next on "Set permissions" (sample image below) | We'll modify this later when defining a bucket for the report.
  • Click Create bucket in the Review section (disregard warnings regarding properties and permissions)

 

Adding an existing S3 Bucket to a new Report

  • Please start the process of creating a new Report (please refer to the procedure at the start of this document) 
  • On Step #7 (see above) when configuring the S3 Bucket for the report, instead of "Create a bucket" go to "Select existing bucket"
    • Select the S3 bucket you created from the drop-down list and click Next.
    • You can Verify Policy and click Save.
    • You may want to COPY this policy, but only for your own reference and to verify S3 bucket policy (see steps below)
  • You can continue with the rest of the report creation process as explain above (start of this article)
  • IMPORTANT NOTE: previous procedures may have required for you to COPY (from the report) and PASTE (to the S3 bucket) the default policy text. This is not needed anymore and the policy is injected automatically in the S3 bucket.
    • Let's verify this for our "turbonomic.billing.reports.bucket" previously created.
    • Go to Services > Storage > S3 and select to Edit the report.
    • Go to Permissions tab and click on Bucket Policy
    • Notice the policy from the Report has been copied here.

 

OTHER REFERENCES

Store and Retrieve a File (Create an S3 Bucket)

3 people found this helpful

Attachments

    Outcomes