23 Aprile 2020 / Ultimo aggiornamento : 23 Aprile 2020 admin Cloud

Cloud-init with Terraform in vSphere environment… a step forward

I already write here about the ability to use Cloud-init in conjunction with Terraform to build an efficient and infrastructure-as-code system to cover infrastructure and configuration setup using the same deploy engine. Now. it’s time to take a step forward, showing how to use a for-each statement with Terraform to deploy different virtual machines using a common baseline, but specializing them according to their cloud-init specifications.

Terraform map and for-each usage

Coding the infrastructure assumes you write a “code” that describes your infrastructure introducing also some “tricky” element that increases the “agility” and like happens when you write code, you should ensure that all identical statements must be optimized using cycles, function calls, etc.

With Terraform it’s possible to put variables and declarations in two separate files: variables.tf and terraform.tfvars. The first is the “description” of the variables involved in the elements declared in the other files.

#cloud variables

variable "vsphere_env" {
    type = object({
        server = string
        user = string
        password = string
    })
    default = {
        server = "vcsa.local.lab"
        user = "administrator@vsphere.local"
        password = "SuperPassw0rd!"
    }
}

variable "k8s_master_env" {
    type = object({
        adminuser = string
        adminpwd = string
    })
    default = {
        adminuser = "admin"
        adminpwd = "admin"
    }
}

variable "vms" {
    type = map(object({
        vmname = string
        datastore = string
        network = string
        user = string
        password = string
        template = string
        cluster = string
        datacenter = string
        network = string
        hostname = string
    }))
}

#cloud variables

variable "vsphere_env" {

type = object({

server = string

user = string

password = string

})

default = {

server = "vcsa.local.lab"

user = "administrator@vsphere.local"

password = "SuperPassw0rd!"

}

variable "k8s_master_env" {

type = object({

adminuser = string

adminpwd = string

})

default = {

adminuser = "admin"

adminpwd = "admin"

}

variable "vms" {

type = map(object({

vmname = string

datastore = string

network = string

user = string

password = string

template = string

cluster = string

datacenter = string

network = string

hostname = string

}))

}

Let’s focus on “vms”: this is a map of objects that creates a dynamic structure that could be the source of an iteration. With the for_each statement is possible to iterate “vms” variable grabbing and using each key inside the resource brackets:

resource vsphere_virtual_machine "allvms" {
  for_each = var.vms

  resource_pool_id = data.vsphere_compute_cluster.this[each.key].resource_pool_id
  datastore_id     = data.vsphere_datastore.this[each.key].id

  ...
}

resource vsphere_virtual_machine "allvms" {

for_each = var.vms

resource_pool_id = data.vsphere_compute_cluster.this[each.key].resource_pool_id

datastore_id = data.vsphere_datastore.this[each.key].id

...

}

Variables declaration in terraform.tfvars should be:

vms = {
    rancher = {
        vmname = "rancher"
        datastore = "datastore1"
        datacenter = "HomeLabWorkload"
        network = "lablan"
        cluster = "workload"
        template = "ubuntu1804template"

        hostname = "rancher01"
        user = "local"
        password = "SuperPassword"
    }
    nfs = {
        vmname = "nfs"
        datastore = "datastore1"
        datacenter = "HomeLabWorkload"
        network = "lablan"
        cluster = "workload"
        template = "ubuntu1804template"

        hostname = "nfs01"
        user = "local"
        password = "SuperPassword"
    }
}

vms = {

rancher = {

vmname = "rancher"

datastore = "datastore1"

datacenter = "HomeLabWorkload"

network = "lablan"

cluster = "workload"

template = "ubuntu1804template"

hostname = "rancher01"

user = "local"

password = "SuperPassword"

}

nfs = {

vmname = "nfs"

datastore = "datastore1"

datacenter = "HomeLabWorkload"

network = "lablan"

cluster = "workload"

template = "ubuntu1804template"

hostname = "nfs01"

user = "local"

password = "SuperPassword"

}

You’ll find this example in my GitHub repo: https://github.com/linoproject/terraform/tree/master/rancher-lab

The configurations directories

Using the object-maps with the fore-each pattern is possible to dynamically pass variables but also pass entire configurations with cloud-init. We already saw in the previous post, how it’s easy to place a baseline configuration and a user-defined configuration which describes what specialization must be performed for every virtual machine.

In the using object element as a string pèattern for a multiple directory definition it’s possible to organize multiple cloud-init files that could be applied for every virtual machine:

resource vsphere_virtual_machine "allvms" {
 for_each = var.vms
 ...

 vapp {
   properties ={
     hostname = each.value.hostname
     user-data = base64encode(file("${path.module}/nodes/${each.value.vmname}/cloudinit/kickstart.yaml"))
   }
 }

 extra_config = {
   "guestinfo.metadata"          = base64encode(file("${path.module}/nodes/${each.value.vmname}/cloudinit/metadata.yaml"))
   "guestinfo.metadata.encoding" = "base64"
   "guestinfo.userdata"          = base64encode(file("${path.module}/nodes/${each.value.vmname}/cloudinit/userdata.yaml"))
   "guestinfo.userdata.encoding" = "base64"
 }
}

resource vsphere_virtual_machine "allvms" {

for_each = var.vms

...

vapp {

properties ={

hostname = each.value.hostname

user-data = base64encode(file("${path.module}/nodes/${each.value.vmname}/cloudinit/kickstart.yaml"))

}

extra_config = {

"guestinfo.metadata" = base64encode(file("${path.module}/nodes/${each.value.vmname}/cloudinit/metadata.yaml"))

"guestinfo.metadata.encoding" = "base64"

"guestinfo.userdata" = base64encode(file("${path.module}/nodes/${each.value.vmname}/cloudinit/userdata.yaml"))

"guestinfo.userdata.encoding" = "base64"

}

So user-data files should contain only what is “special” for this virtual machine like packages, IP hostname,…

Passing variables to cloud-init declarations

Let’s do a step more… In order to pass variables to the cloud-init file, it’s possible to use the template_file terraform data source. Then, it’s possible to implement the for-each statement to pass the specific values to the template, cycling to “vms” object list:

data template_file "metadataconfig" {
  for_each = var.vms

  # Main cloud-config configuration file.
  template = file("${path.module}/nodes/common/metadata.yaml")
  vars = {
    ip = "${each.value.ip}"
    hostname = "${each.value.hostname}"
    instance_id = "${each.value.vmname}"
    gw = "${var.vm_env.gw}"
    dns = "${var.vm_env.dns}"
  }
}

data template_file "metadataconfig" {

for_each = var.vms

# Main cloud-config configuration file.

template = file("${path.module}/nodes/common/metadata.yaml")

vars = {

ip = "${each.value.ip}"

hostname = "${each.value.hostname}"

instance_id = "${each.value.vmname}"

gw = "${var.vm_env.gw}"

dns = "${var.vm_env.dns}"

}

In this way, using this template mechanism opens the ability to reduce the number of cloud-init files separating the common element from the VM specific elements. In this example, kickstart.yaml and metadata.yaml could be transformed into two templates. Here the templates:

local-hostname: ${hostname}
instance-id: ${instance_id}
network:
  version: 2
  ethernets:
    ens192:
      dhcp4: false #true to use dhcp
      addresses:
        - ${ip}
      gateway4: ${gw} # Set gw here 
      nameservers:
        addresses:
          - ${dns} # Set DNS ip address here

local-hostname: ${hostname}

instance-id: ${instance_id}

network:

version: 2

ethernets:

ens192:

dhcp4: false #true to use dhcp

addresses:

- ${ip}

gateway4: ${gw} # Set gw here

nameservers:

addresses:

- ${dns} # Set DNS ip address here

Finally, in the main file the injection of the rendered template should be:

vapp {
    properties ={
      hostname = each.value.hostname
      user-data = base64encode(file("${path.module}/nodes/common/kickstart.yaml"))
    }
  }

  extra_config = {
    "guestinfo.metadata"          = base64encode(data.template_file.metadataconfig[each.key].rendered)
    "guestinfo.metadata.encoding" = "base64"
    "guestinfo.userdata"          = base64encode(file("${path.module}/nodes/${each.value.vmname}/cloudinit/userdata.yaml"))
    "guestinfo.userdata.encoding" = "base64"
  }