Cloud-init with Terraform in vSphere environment… a step forward

I already write here about the ability to use Cloud-init in conjunction with Terraform to build an efficient and infrastructure-as-code system to cover infrastructure and configuration setup using the same deploy engine. Now. it’s time to take a step forward, showing how to use a for-each statement with Terraform to deploy different virtual machines using a common baseline, but specializing them according to their cloud-init specifications.

Terraform map and for-each usage

Coding the infrastructure assumes you write a “code” that describes your infrastructure introducing also some “tricky” element that increases the “agility” and like happens when you write code, you should ensure that all identical statements must be optimized using cycles, function calls, etc.

With Terraform it’s possible to put variables and declarations in two separate files: variables.tf and terraform.tfvars. The first is the “description” of the variables involved in the elements declared in the other files.

#cloud variables

variable "vsphere_env" {
    type = object({
        server = string
        user = string
        password = string
    })
    default = {
        server = "vcsa.local.lab"
        user = "[email protected]"
        password = "SuperPassw0rd!"
    }
}

variable "k8s_master_env" {
    type = object({
        adminuser = string
        adminpwd = string
    })
    default = {
        adminuser = "admin"
        adminpwd = "admin"
    }
}

variable "vms" {
    type = map(object({
        vmname = string
        datastore = string
        network = string
        user = string
        password = string
        template = string
        cluster = string
        datacenter = string
        network = string
        hostname = string
    }))
}

Let’s focus on “vms”: this is a map of objects that creates a dynamic structure that could be the source of an iteration. With the for_each statement is possible to iterate “vms” variable grabbing and using each key inside the resource brackets:

resource vsphere_virtual_machine "allvms" {
  for_each = var.vms

  resource_pool_id = data.vsphere_compute_cluster.this[each.key].resource_pool_id
  datastore_id     = data.vsphere_datastore.this[each.key].id

  ...
}

Variables declaration in terraform.tfvars should be:

vms = {
    rancher = {
        vmname = "rancher"
        datastore = "datastore1"
        datacenter = "HomeLabWorkload"
        network = "lablan"
        cluster = "workload"
        template = "ubuntu1804template"

        hostname = "rancher01"
        user = "local"
        password = "SuperPassword"
    }
    nfs = {
        vmname = "nfs"
        datastore = "datastore1"
        datacenter = "HomeLabWorkload"
        network = "lablan"
        cluster = "workload"
        template = "ubuntu1804template"

        hostname = "nfs01"
        user = "local"
        password = "SuperPassword"
    }
}

You’ll find this example in my GitHub repo: https://github.com/linoproject/terraform/tree/master/rancher-lab

The configurations directories

Using the object-maps with the fore-each pattern is possible to dynamically pass variables but also pass entire configurations with cloud-init. We already saw in the previous post, how it’s easy to place a baseline configuration and a user-defined configuration which describes what specialization must be performed for every virtual machine.

In the using object element as a string pèattern for a multiple directory definition it’s possible to organize multiple cloud-init files that could be applied for every virtual machine:

resource vsphere_virtual_machine "allvms" {
 for_each = var.vms
 ...

 vapp {
   properties ={
     hostname = each.value.hostname
     user-data = base64encode(file("${path.module}/nodes/${each.value.vmname}/cloudinit/kickstart.yaml"))
   }
 }

 extra_config = {
   "guestinfo.metadata"          = base64encode(file("${path.module}/nodes/${each.value.vmname}/cloudinit/metadata.yaml"))
   "guestinfo.metadata.encoding" = "base64"
   "guestinfo.userdata"          = base64encode(file("${path.module}/nodes/${each.value.vmname}/cloudinit/userdata.yaml"))
   "guestinfo.userdata.encoding" = "base64"
 }
}

So user-data files should contain only what is “special” for this virtual machine like packages, IP hostname,…

Passing variables to cloud-init declarations

Let’s do a step more… In order to pass variables to the cloud-init file, it’s possible to use the template_file terraform data source. Then, it’s possible to implement the for-each statement to pass the specific values to the template, cycling to “vms” object list:

data template_file "metadataconfig" {
  for_each = var.vms

  # Main cloud-config configuration file.
  template = file("${path.module}/nodes/common/metadata.yaml")
  vars = {
    ip = "${each.value.ip}"
    hostname = "${each.value.hostname}"
    instance_id = "${each.value.vmname}"
    gw = "${var.vm_env.gw}"
    dns = "${var.vm_env.dns}"
  }
}

In this way, using this template mechanism opens the ability to reduce the number of cloud-init files separating the common element from the VM specific elements. In this example, kickstart.yaml and metadata.yaml could be transformed into two templates. Here the templates:

local-hostname: ${hostname}
instance-id: ${instance_id}
network:
  version: 2
  ethernets:
    ens192:
      dhcp4: false #true to use dhcp
      addresses:
        - ${ip}
      gateway4: ${gw} # Set gw here 
      nameservers:
        addresses:
          - ${dns} # Set DNS ip address here

Finally, in the main file the injection of the rendered template should be:

vapp {
    properties ={
      hostname = each.value.hostname
      user-data = base64encode(file("${path.module}/nodes/common/kickstart.yaml"))
    }
  }

  extra_config = {
    "guestinfo.metadata"          = base64encode(data.template_file.metadataconfig[each.key].rendered)
    "guestinfo.metadata.encoding" = "base64"
    "guestinfo.userdata"          = base64encode(file("${path.module}/nodes/${each.value.vmname}/cloudinit/userdata.yaml"))
    "guestinfo.userdata.encoding" = "base64"
  }

Then it’s possible to transform kickoff file in a template, in order to pass user/password credentials.

Time to show

You’ll find the fully functional example here: https://github.com/linoproject/terraform/tree/master/rancher-lab

Don’t forget to change/create terraform.tvars for your environment.