Displaying AWS Spend In Turbonomic

Document created by dallas.demarr@vmturbo.com Expert on Mar 31, 2017Last modified by jacob.bendavid on Jun 6, 2018
Version 22Show Document
  • View in full screen mode

Turbonomic provides a complete picture of your environment. For those with AWS Targets, this includes spend amounts, which populate the Expenses -> Cost Breakdown views as seen in the "Cloud" tab. In order to display month-to-day spend, you must expose this information via placing the AWS Cost and Usage report in an S3 Bucket that Turbonomic will access.

 

To do this, you must do two things:

  • Create a Cost and Usage Report in AWS, saving it to an S3 bucket
  • Give Turbonomic access to the bucket in Target Configuration.

NOTE: This Guide assumes that the user performing these tasks has both the "AmazonS3FullAccess" and "Billing" permissions that will allow the user to create an S3 bucket to place a report into, and set report preferences for type of report (Daily Cost and Usage) plus location of the S3 bucket. For more details on Billing start here: Cost and Usage Report - AWS Billing and Cost Management 

Do you use Consolidated Billing (Consolidated Billing for Organizations - AWS Billing and Cost Management )?  Turbonomic will report the month-day spend summary by AWS Services, but in 5.9.0 cannot parse the different accounts and organizations.  You will need to add a AWS Target with an IAM User that is part of the Master Account Organization or outside Account which minimally has S3 Access to the bucket that is created in that organization. Also, we need to have EC2 instance ( VM either Powered OFF or ON ) on the Account that the IAM User is part of.

 

Create Cost and Usage Report

Creating reports for AWS budgeting

  1. Login
  2. Select a region (for instance eu-west-2), this is used in the Ops-Manager later on
  3. Click on your name in the upper left corner
  4. From the Drop down list select "My Billing Dashboard"
  5. From the left menu select "Reports"
  6. Click on "Create Report"
  7. Enter a Name for the report, select Daily
  8. Select Resource IDs
  9. Click Next
  10. Next, we need to create an S3 Bucket in which to store the report.  If you have an existing bucket you may skip this step
  11. Right click on "S3 Console" and open in a new Window
  12. Click on Create Bucket.  Make NOTE of the REGION this bucket is in.
  13. Enter a name for the bucket, as well as select a region.
  14. The name must be unique within all of S3 (must be universally unique)
  15. Click Next
  16. On "Set Properties", click Next
  17. On "Set Permissions", click Next. These will be modified in a future step
  18. On "Review", Click "Create Bucket"
  19. Back on the S3 page, your new Bucket will be created:
  20. Click on the bucket to open it
  21. Click on Permissions
  22. Click on "Bucket Policy"
  23. Now switch back on the Report Page
  24. Enter the s3 bucket name you created (or had pre-created) and click verify, you should see the error "X Invalid Bucket" - this is expected, this  will cause the sample policy to be adjusted to the correct bucket name)

  25. Next, click on the "Sample Policy" in the window
  26. Enter the Name of the bucket just created and then click on the link "sample policy"
  27. Copy the Contents of the Pop-up
  28. Note: if you are creating the bucket and do not have billing access to generate this json permissions policy, you can use Example 9 in this AWS document: Billing and Cost Management Permissions Reference - AWS Billing and Cost Management 
  29. Return to S3, and Paste the contents to the "Bucket Policy" editor
  30. Click Save
  31. Back in the report Window, click on Verify - now the bucket should be marked as Valid - Click next
  32. Under "Report Path prefix" type: daily - Click Next
  33. On the Review Page click "Review and Complete"
  34. The required Bucket and report are now present.

NOTE: AWS publishes the standard daily billing report twice a day, and may take up to 24 hours for the first report to be placed in the bucket.  The Cost Breakdown views in Turbonomic will start populating 1 - 2 hours after.

 

Specifying the Target

Navigate to the Target Configuration section (under Settings) in Turbonomic to add the target.

If you have already added the AWS target to Turbonomic, you may modify the existing target to add the bucket and report fields.  However you will need to re-enter the Secret Access Key.  If you target does not validate after modification, make sure you re-enter all credentials. For more information about adding an AWS Target, please see the latest Target Configuration Guide at https://greencircle.vmturbo.com/community/products/pages/documentation.

 

Note the IAM User (which belongs to the Access and Secret Access Key info) specified as the AWS Target needs to only have S3 access to use the report - Billing access is not required.

The three target fields needed for to access this report are:

  1. Cost and Usage Report Bucket: enter the full bucket name (for example, turbo.billing.report)
  2. Cost and Usage Report Path: enter report path (without a prefix or trailing /) - you can copy the report path from the BIlling Report details (see image below - note there are no / before or after)
  3. Cost and Usage Report Region: enter the region name of the S3 bucket (see this list for all AWS regions)

Example of AWS Target with AWS billing based on the above Billing Report, created on an S3 Bucket in Canada Central Region (ca-cental-1):

 

After target discovery is complete, Turbonomic will display spend in the UI (please note that AWS reports make take up to 24hours to be created, only after they are created Turbonomic will start displaying detailed billing data)

2 people found this helpful

Attachments

    Outcomes