Sharing Data

Written: 2017-07-22
Author: WhatsARanjit
Link: WhatsARanjit/share_data

The problem

Many times we want to share something like a node’s IP address to be used in many places on another node. It could be a host entry, a white list, a firewall rule, a software configuration, etc. Every time we need this we write up a new exported resource from the node, it sends that resource to the mater, and then the target node will collect it.

The funny part about this is we have to export a resource for each thing we want to do, even in the end, we just want to share one value- the IP address. Also the model of exporting a resource requires you to know the destination immediately. You can’t share information that can be used any which way afterwards. Also, if you’d like to add an exported resource, you must change something on the source node to inflict a change on a target node. Intuitively, it would nice to edit the target node in order to change the target node.

The fix

So instead of exporting a resource, I wondered about exporting raw data. That data could be used in any way and not be locked in a single resource. We can do this using the WhatsARanjit/share_data module. First we start with a new defined type called share_data:

share_data { 'networking info':
  data  => {
    'ipaddress' => $ipaddress,
    'fqdn'      => $fqdn,
  },
  label => 'network',
}

You put this resource declaration once on the source node. The data you provide can be any format- String, Array, Hash, Boolean. You must provide a label which will help you to retrieve the data later. Then on the target node:

$data = share_data::retrieve('network')

That’s it. You retrieve the data by using your label. The $data variable now contains an Array of results by node. Each item in the array is your hash from that node. You can use and reuse it any way you need:

$data.each |$item| {
  host { $item['fqdn']:
    ensure => present,
    ip     => $item['ipaddress'],
  }
}
file { '/tmp/mywhitelist.txt':
  ensure  => file,
  content => inline_template('<% @data.each do |val| %><%= val["fqdn"] %>: <%= val["ipaddress"] %><% end %>'),
}

Now with the same $data variable, you have created host entries and a file with content like:

node1: 192.168.1.1
node2: 192.168.1.2
node3: 192.168.1.3

Furthermore, if you want to create more resources with these values on a target node, you only edit the target node. You do not have to make changes to code on a source node- not unless you want to export more data.

The lifecycle is the same. The source node has to export the data before it can be collected and used by the target node. In the backend of this thing, it’s using a single exported resource to ship data. So at least instead of exporting a number of resources from a node, you export one resource containing all your data. This presents some savings for PuppetDB and catalog sizes if you’re exporting a lot of things.