Managing your Puppetfile as Data

Written: 2017-08-04
Author: WhatsARanjit
Link: Gist

The problem

It’s time to learn and deploy Puppet. You’ve learned the Puppet DSL. Now you can write a class. You’ve learned what Facter is. Now you can put in conditional logic to make your classes dynamic. You’ve learned the params.pp pattern. Now you can write cleaner Puppet code. You’ve learned ERB. Now you can write templates. You’ve learned EPP. Now you write more templates. You’ve learned about how Hiera works. Now you can separate your data from your code. You’ve learned YAML. Now you can put your data in Hiera. You’ve learned rspec. Now you can write unit tests. You’ve learned ruby. Now you can write some custom facts and functions. All this and you haven’t even deployed anything yet.

You learn the roles & profiles pattern. Now you can organize your code in a mangeable fashion. You install a monolithic Puppet infrastructure. Now you can deploy your code to something. You teach yourself about r10k or Code Manager workflows. Now you have a process to update your code. Now it’s time to learn the format of a Puppetfile. It doesn’t seem like anything you’ve learned so far:

• Puppet DSL
• Facter
• Class inheritance
• ERB
• EPP
• Hiera
• YAML
• rspec-puppet
• ruby

Technically speaking, the Puppetfile is ruby. There are just a lot of methods pre-written to enable this file format that stems from librarian-puppet. For example:

librarian-puppet

mod 'node_manager',
  :git => 'https://github.com/WhatsARanjit/puppet-node_manager',
  :tag => '0.4.2'

ruby-ish

module_name = 'node_manager'
module_opts = {
  :git => 'https://github.com/WhatsARanjit/puppet-node_manager',
  :tag => '0.4.2'
}
mod(module_name, module_opts)

The latter example probably represents something closer to what you might see in ruby. It definitely looks more like code than it does a list of data. The thing is, the Puppetfile is just a list of data. We’ve learned to previously separate our data from our code. And we learned to put our data in YAML. Why not do the same thing here?

The fix

So our new idea is to represent a list of modules in YAML because all it is is data.

Original Puppetfile

mod 'puppetlabs/inifile'
mod 'puppetlabs/postgresql', '4.5.0'
mod 'puppetlabs/stdlib', '4.17.1'
mod 'node_manager',
    :git    => 'https://github.com/WhatsARanjit/puppet-node_manager',
    :branch => 'scripts'
mod 'share_data',
    :git => 'https://github.com/WhatsARanjit/puppet-share_data',
    :ref => '986a605e917515beaaa71846eba948d0b2a0e685'

Let’s list the modules in YAML instead.

puppetfile.yaml

---
- puppetlabs/inifile
- puppetlabs/postgresql: 4.5.0
- puppetlabs/stdlib:
    ref: 4.17.1
- node_manager:
    git: https://github.com/WhatsARanjit/puppet-node_manager
    branch: scripts
- share_data:
    git: https://github.com/vshn/puppet-gitlab
    ref: 986a605e917515beaaa71846eba948d0b2a0e685

Then because the Puppetfile is interpreted as ruby, we can use a script (that hopefully doesn’t change too much) that feeds your data list into the same mod() function.

Puppetfile

require 'yaml'
puppetfile = YAML.load_file('./puppetfile.yaml')

puppetfile.each do |m|
  if m.is_a?(String)
    mod(m)
  else m.is_a?(Hash)
    m_name = m.keys[0]
    if m_name =~ /[\/\-]/
      if m[m_name].is_a?(String)
        mod(m_name, m[m_name])
      else
        mod(m_name, m[m_name].values[0])
      end
    else
      msymbols = m[m_name].map { |k,v| [k.to_sym, v] }.to_h
      mod(m_name, msymbols)
    end
  end
end

I’ve made some additions here. In a traditional Puppetfile, you have to list :git and :ref as symbols, meaning they need a leading :. The above eliminates that need, because why do we need to know what ruby symbols are? The Puppetfile will break if you have a trailing comma, which you might be trained to add from learning Puppet DSL. YAML eliminates commas. This method elminates hash rockets, because if you’re neurotic like me, you always have to line them up. Also, it allows you to represent Forge modules like:

- puppetlabs/stdlib:
    ref: 4.17.1

…in case you want your Forge YAML to look similar to your Git YAML. I’m not sure of the delivery method for this. I’ve considered putting this code in a gem so that your Puppetfile says nothing but:

require 'puppetfileyaml'

…or something to that effect. But this means that you need to manually install the gem for you Puppetserver during your Puppet install process. That’s just another step. The simplest idea is just to have this Puppetfile be an example that you copy and paste it. Ideas are welcome.

P.S. The gist link contains code that secondarily looks for a puppetfile.json in case that’s your preference.