lingenavd

Deployment recommendations by VMTurbo (Part 1)

Blog Post created by lingenavd on Feb 17, 2015

Introduction:

We have a VMware hypervisor environment and VMTurbo Operations Manager 5.1

We have many VMware clusters with separate data stores (LUNs) connected from SAN.

In our automated deployment process for virtual servers on VMware, all actions are orchestrated from the Microsoft SC Orchestrator product that fires a lot of Powershell towards Active Directory and VMware PowerCLI scripts towards VMware vCenter servers and some Web services (SOAP/REST) to service management products, etc.

 

After some initial discussions with the VMTurbo Customer Success team in the UK I was made aware of the integration possibilities with VMTurbo and how it might play a role in the automated deployment process.

 

The automated deployment process does not really differ much from a manual deployment.

As you may know in VMware the choice to deploy a VM from template will guide you through a wizard where you have to provide the VMware cluster (or specific host), the datastore (local / NAS or SAN storage), the vCenter folder etc.

In the automated server deployment process we need to know the same things, some provided by the requester and some provided by scripted intelligence.

 

I had built some intelligence into the PowerCLI scripts to get the best data store and a specific host in the cluster where the VM should be deployed. The VMTurbo Customer Success team told me we could use VMTurbo for that same recommendation. VMTurbo takes a lot of performance counters into consideration like IOPS, Latency, Storage Amount and Provisioned for data stores, but also CPU and Memory usage on VSphere ESX hosts to determine the optimal state for our infrastructure.

This will give better recommendations than any enthusiastic VMware system manager can give you at any time in my humble opinion, so I decided to give it a go.

 

In part 1 of this article I will describe the steps shared with me by the VMTurbo Customer Success team regarding the creation of the necessary groups, templates and deployment profiles from the GUI. As a result we will get some recommendations from the Deploy Tab in the GUI.

In part 2 of this article I will then describe my efforts to automate the recommendations from Powershell using the REST API services.

 

 

Background:

We use VMware SRM and therefore we made a naming convention for the data stores based on that functionality. For example: a server VM that needs to protected by VMware SRM is on data stores with the naming convention SR in the name. (Synchronous Replication)

Server VM’s that are not protected by SRM are on different data stores with “NR” in the name of the data store. (No Replication)

We have also standardized the server VM in 4 virtual hardware categories; Bronze, Silver, Gold and Platinum.

In the service portal where the IT personnel can order new VM’s a little questionnaire needs to be completed to gather some information for the administrative and technical process of delivering the VM to the requester.

 

Let’s get started:

To make it possible for VMTurbo to understand on what data stores SRM protected VM’s should reside we need to make some groups, templates and deployment profiles in VMTurbo, so let’s get started with the technical stuff.

 

Note!:

This is successfully implemented on VMTurbo Operations Manager 5.1.

It may work (or not) on prior versions.

 

Step 1: Create templates in VMTurbo, tab "Deploy" for the different
hardware profiles

 

Click the depicted icon below to list and add new templates and deployment profiles.

VMTurbo_config_button.png

 

VM Template Name

AutoDeploy-W2012R2-Bronze

Vendor:

 

Description:

Bronze - 1 CPU -1 GB - mem - 60 GB C: disk

VCPU

1

Virtual Memory [MB]

1024

Storage [GB]

60

Network Throughput used [MB /s]

Obviously a better recommendation if you know this from base lining

IOPS

Obviously a better recommendation if you know this from base lining

IO Throughput used [MB /s]

Obviously a better recommendation if you know this from base lining

 

 

Step 2: Create profiles in VMTurbo, tab "Deploy" to relate a deployment profile with a template

 

Deployment Profile Name

AutoDeploy-W2012R2-V98-ZWL

Checkbox “Select Cluster”

Checked

Dropdown box

Select the appropriate VMware cluster

Enter path of the ovf img / vhd file

I left this blank, because I don’t want to deploy fom VMTurbo

Related VM Templares:

Select on or multiple templates;

AutoDeploy-W2012R2-Bronze

 

 

Step 3: Create Dynamic Groups in VMTurbo in tab “Policy” - Group Management

In these steps we are using a regular expression against the Datastore name to distinguish between replicated (SR) and non-replicated (NR) storage

Create new group for NR Data Stores on cluster 1

 

Tab: Properties

Name

Autodeploy_NR_datastores_cluster1

Tab: Members

Checkbox: “Group entities by criteria”

Checked

Tab: Members

Dropdown box “Choose entity type…”

Storage

Tab: Members

Dropdown box “Choose search criteria…”

By name

Tab: Members

Dropdown box “equals”

Equals

Tab: Members

Regular expression

^NR\_GOLD.*cluster1

Tab: Members

Find matches

This should give you all NR data stores in the matched by the regular expression

Tab: Members

Create Group

Click on it to create the group

 

Create new group for SR Data Stores on cluster 1

 

Tab: Properties

Name

Autodeploy_SR_datastores_cluster1

Tab: Members

Checkbox: “Group entities by criteria”

Checked

Tab: Members

Dropdown box “Choose entity type…”

Storage

Tab: Members

Dropdown box “Choose search criteria…”

by name

Tab: Members

Dropdown box “equals”

Equals

Tab: Members

Regular expression

^SR\_GOLD.*cluster1

Tab: Members

Find matches

This should give you all SR data stores in the matched by the regular expression

Tab: Members

Create Group

Click on it to create the group

 

Create new reservation group for deployment recommendations on NR data stores in cluster 1

 

Tab: Properties

Name

Autodeploy_VM_on_NR_datastore_cluster1

Tab: Members

Checkbox: “Group entities by criteria”

Checked

Tab: Members

Dropdown box “Choose entity type…”

Virtual Machines

Tab: Members

Dropdown box “Choose search criteria…”

by name

Tab: Members

Dropdown box “equals”

Equals

Tab: Members

Regular expression

NR_Deploy.*

Tab: Members

Create Group

Click on it to create the group

 

Create new reservation group for deployment recommendations on SR data stores in cluster 1

 

Tab: Properties

Name

Autodeploy_VM_on_SR_datastore_cluster1

Tab: Members

Checkbox: “Group entities by criteria”

Checked

Tab: Members

Dropdown box “Choose entity type…”

Virtual Machines

Tab: Members

Dropdown box “Choose search criteria…”

by name

Tab: Members

Dropdown box “equals”

Equals

Tab: Members

Regular expression

SR_Deploy.*

Tab: Members

Create Group

Click on it to create the group

 

 

Step 4: Create Segments in VMTurbo in tab “Policy” - Workload placement

In these steps we are reflecting the constraint we have regarding the placement of VMs onto replicated or non-replicated storage through the creation of a workload placement policy in VMTurbo.

 

Create a segment to reflect the constraint that VMs in group “Autodeploy_VM_on_NR_datastore_cluster1” should only be placed on datastores in group “Autodeploy_NR_datastores_cluster1”

 

Name

Autodeploy_NR_Placement_cluster1

Checkbox “Place”

Checked

Dropdown box “Consumer Type”

Virtual machine

Consumer Group

Autodeploy_VM_on_NR_datastore_cluster1

Provider Group

Autodeploy_NR_datastores_cluster1

Checkbox “Limit workload entities to
  placement group”

Checked

Checkbox “Limit the number of workload
  entities per placement entity”

Not checked

Create

Click on it to create the segment

 

Create a segment to reflect the constraint that VMs in group “Autodeploy_VM_on_SR_datastore_cluster1” should only be placed on datastores in group “Autodeploy_SR_datastores_cluster1”

 

Name

Autodeploy_SR_Placement_cluster1

Checkbox “Place”

Checked

Dropdown box “Consumer Type”

Virtual machine

Consumer Group

Autodeploy_VM_on_SR_datastore_cluster1

Provider Group

Autodeploy_SR_datastores_cluster1

Checkbox “Limit workload entities to
  placement group”

Checked

Checkbox “Limit the number of workload
  entities per placement entity”

Not checked

Create

Click on it to create the segment

 

VMTurbo will now ensure that existing and new VMs will honour the constraints that have been reflected in the workload placement policies.

If you already have rules defined in vCenter then VMTurbo would automatically import those for you.

 

Step 5: Get a recommendation from VMTurbo in tab "Deploy"

 

In the left pane of the “Deploy tab” you have the reservation setup page:

Enter the following data into the fields to get a recommendation for one or multiple VM’s on NR data stores on cluster 1:

 

Reservation Name

NR_Deploy

VM Count

3

Reservation Date

Leave it on the current date

Deploy Date

Leave it on the current date

Dropdown box “Virtual machine Template”

AutoDeploy-W2012R2-Bronze

Dropdown box “Deployment Profile”

AutoDeploy-W2012R2-V98-ZWL

Request

Click on it to request a
  recommendation

 

 

The result:

As you can see different ESX hosts are recommended for each new VM. The NR data store is the same for all three new VM’s.

 

plaatje.png

 

Now the same for a SRM protected VM on SR data stores.

 

Reservation Name

SR_Deploy

VM Count

2

Reservation Date

Leave it on the current date

Deploy Date

Leave it on the current date

Dropdown box “Virtual machine
  Template”

AutoDeploy-W2012R2-Platinum

Dropdown box “Deployment Profile”

AutoDeploy-W2012R2-V98-ZWL

Request

Click on it to request a
  recommendation

 

 

The result:

As you can see different ESX hosts are recommended for each new VM, and now the data store is changed to SR data stores to accommodate SRM protected VM’s.

 

plaatje2.png

 

The placement recommendation made by VMTurbo is the result of the new VMs individually “asking the market” for the cheapest (and hence less utilized) place to live within the abstraction and economic scheduling engine of VMTurbo. As you can see the constraints I imposed on the market have been respected.

 

Watch for part 2 in the near future where I will describe my efforts to automate the recommendations from Powershell using the REST API services.

 

I hope this can help you to leave the smart thinking for deployment recommendations to VMTurbo.

 

With regards,

Andre van der Lingen

System Manager at SNS Bank

Outcomes