Sprinkle a little hiera data into a Puppet environment and you can easily maintain a single project targeting deployments across a range of VMs. If a Vagrant provisioned VM is one of your targets though, there will be a little extra work to do.
As something of a “devops” driven code monkey, keen to have my paws on everything from the code through to the live environments, it does seem to have been my “ops” half that’s had the most fun over the last couple of years as technologies like Docker and Puppet have come to greater prevalence and therefore, ultimately, to my attention.
My main project last year used a single Puppet 3 project to fully provision all of its target environments, from a standalone local VM for development right through to the nearly two-dozen VMs which comprised our production layout. Give me a choice between pages of installation instructions and random shell scripts or a single, integrated project with single-command, restartable, updatable provisioning and I’ll take the latter every time.
Kicking-off my current project in true Agile style, with a Sprint 1 target of a framework application and CI/CD chain to deliver it to our first-line test environment I was keen to start a Puppet project, version 4 this time, to install local development VMs (using Vagrant and VirtualBox) and also to provision the test environment.
This requires management of deployment specific settings and for that I’d use hiera – a YAML hierarchy to configure settings based on hosts, zones or any other type of local data you could want.
Which is where I hit a bit of a snag. Whilst fine in the test environment, just why wasn’t my Vagrant VM seeing my environment’s heira data?
Hiera data in a Puppet environment
My starter environment is as simple as possible so that I can establish that the hiera variables are picked up appropriately. I’ve got an environments folder with a single environment called dionysus in it (don’t ask!). Inside my environment I’ve got some starter hiera data files and a couple of manifests:-
My site.pp pulls in a single test manifest hello.pp, which just writes a message to the console during provisioning:-
class hello (
$message = 'default message'
) {
notify { $message :
}
}
I’m going to provision this using Vagrant and VirtualBox to a local development VM called vg-dionysus and therefore I’ve got a vg-dionysus YAML file in my nodes folder, from which I’m expecting a host specific override to $message to be picked up.
hello::message: Hello World!
And, for completeness, here’s my Vagrantfile to configure my development VM and run in the Puppet environment settings:-
# -*- mode: ruby -*- # vi: set ft=ruby : Vagrant.configure(2) do |config| config.vm.box = "puppetlabs/debian-8.2-64-puppet" config.vm.network "private_network", ip: "192.168.33.110" config.vm.network "public_network", bridge: "en0: Wi-Fi (AirPort)" config.vm.hostname = "vg-dionysus" config.vm.provider "virtualbox" do |vb| vb.memory = "2048" vb.name = "vg-dionysus" end config.vm.provision "puppet" do |puppet| puppet.environment = "dionysus" puppet.environment_path = "environments" puppet.working_directory = "/vagrant" end end
Expecting to see Hello World! when I ran vagrant up I was, to say the least, disappointed:-
==> default: Running provisioner: puppet... ==> default: Running Puppet with environment dionysus... ==> default: Notice: Compiled catalog for vg-dionysus in environment dionysus in 0.15 seconds ==> default: Notice: default message ==> default: Notice: /Stage[main]/Hello/Notify[default message]/message: defined 'message' as 'default message' ==> default: Notice: Applied catalog in 0.02 seconds
My hiera data was being totally ignored.
Seeing the Puppet strings
When Puppet provisioning isn’t behaving as expected, turning on verbose logging is really useful, and just requires an extra setting in the config.vm.provision “puppet” section of my Vagrantfile.
puppet.options = "--verbose --debug"
And in this instance the extra logging quickly points to the problem:-
==> default: Debug: hiera(): Looking up hello::message in YAML backend ==> default: Debug: hiera(): Looking for data source nodes/vg-dionysus ==> default: Debug: hiera(): Cannot find datafile /etc/puppetlabs/code/environments/dionysus/hieradata/nodes/vg-dionysus.yaml, skipping ==> default: Debug: hiera(): Looking for data source common ==> default: Debug: hiera(): Cannot find datafile /etc/puppetlabs/code/environments/dionysus/hieradata/common.yaml, skipping
The Puppet installation inside my VM is looking in /etc/puppetlabs/code/environments/ for YAML data. So, though I’ve pointed Puppet at my environment, since it’s not actually copied into the default location within the VM, the hiera processing isn’t seeing my YAML files.
Fixing the Puppet strings
Puppet’s default hiera rules are located in /etc/puppetlabs/code/hiera.yml. This file identifies the hierarchy which will be applied and allows the specification of a :datadir: setting to override where the hieradata will be located.
We ‘could’ perhaps change this file to point it at our Vagrant provided Puppet environment, but aside from feeling a bit brittle we’d need to automatically provision a change to the VM in order for our automatic provisioning to work.
Fortunately Vagrant’s Puppet integration includes the ability to configure a specific hiera.yml file to use instead of the default:-
puppet.hiera_config_path = "hiera.yml"
Setting this will cause Puppet to look for a hiera.yml in the same folder as our Vagrantfile to configure hiera processing. If we only want to change the location of our YAML files, then :datadir: is all we need to set, pointing it to the location of our Puppet environment within the VM:-
root@vg-dionysus: # df -k
Filesystem 1K-blocks Used Available Use% Mounted
/dev/dm-0 19382140 1260480 17130428 7% /
...
tmp_vagrant-puppet_environments 975929344 790454588 185474756 81%
/tmp/vagrant-puppet/environments
Here, our Puppet environment is mounted by Vagrant at /tmp/vagrant-puppet/environments, so our hiera.yml will look like this:-
:yaml:
:datadir: /tmp/vagrant-puppet/environments/%{environment}/hieradata
With this in place, a vagrant provision now gives me the node specific message I was looking for:-
==> default: Running provisioner: puppet... ==> default: Running Puppet with environment dionysus... ==> default: Notice: Compiled catalog for vg-dionysus in environment dionysus in 0.16 seconds ==> default: Notice: Hello World! ==> default: Notice: /Stage[main]/Hello/Notify[Hello World!]/message: defined 'message' as 'Hello World!' ==> default: Notice: Applied catalog in 0.01 seconds
