Dynamic hosts file using Chef

There are a number of ways to setup your infrastructure so that you can refer to machines by hostname. I currently prefer the “dynamically generated hosts file” approach because it’s simple to understand and setting up a DNS server is intimidating (as well as a single point of failure).

Shlomo Swidler has a great article comparing different DNS configurations as well as some sample code for dynamically updating hosts files. However, if you’re already using Chef you can achieve the same thing with a very simple cookbook.

First, create a new cookbook:

$ cd chef-repo/
$ rake new_cookbook COOKBOOK=hosts CB_PREFIX=site-

Place the following in site-cookbooks/hosts/recipes/default.rb:

# Find all nodes, sorting by Chef ID so their
# order doesn't change between runs.
hosts = search(:node, "*:*", "X_CHEF_id_CHEF_X asc")
 
template "/etc/hosts" do
  source "hosts.erb"
  owner "root"
  group "root"
  mode 0644
  variables(
    :hosts => hosts,
    :fqdn => node[:fqdn],
    :hostname => node[:hostname]
  )
end

Then create the template, site-cookbooks/hosts/templates/default/hosts.erb:

127.0.0.1   localhost
 
<% @hosts.keys.sort.each do |fqdn| %>
<%= @hosts[fqdn][:ipaddress] %> <%= fqdn %> <%= @hosts[fqdn][:hostname] %>
<% end %>
 
# The following lines are desirable for IPv6 capable hosts
::1 ip6-localhost ip6-loopback
fe00::0 ip6-localnet
ff00::0 ip6-mcastprefix
ff02::1 ip6-allnodes
ff02::2 ip6-allrouters
ff02::3 ip6-allhosts

Add the recipe to your node’s run_list and run chef-client and your /etc/hosts should contain all the Chef nodes on your network. Note that one downside to this approach is that updates will be slow (since chef-client only runs every 30 minutes by default).

Now what if you wanted to point at a specific service, like having chef.example.com point at your Chef master? Just search for it in your recipe (and add it to the variables list):

chef_server = search(:node, 'run_list:recipe\[chef\:\:server\]')

And add a line to the template:

# Chef master server
<%= @chef_server[:ipaddress] %> chef.example.com

Sure beats setting up a DNS server!

  • http://abulman.co.uk Alister

    just a point: but your hosts.erb file as show, doesn’t have a loop, or any dynamic output of the ‘hosts’ variable.

  • http://mocek.org/ Phil

    When I use this recipe, Chef fails with an error message indicating that we’ve tried to use the nonexistent “keys” method of an array, suggesting that search() is returning an array of hosts instead of a hash of them.

    Stack trace follows:

    Generated at Thu Jun 14 20:12:31 -0400 2012
    Chef::Mixin::Template::TemplateError:

    Chef::Mixin::Template::TemplateError (undefined method `keys’ for #) on line #3:

      1: 127.0.0.1  localhost.localdomain localhost
      2:
      3:
      4:
      5:

      (erubis):3:in `evaluate’
      /usr/lib64/ruby/gems/1.8/gems/erubis-2.7.0/lib/erubis/evaluator.rb:74:in `instance_eval’
      /usr/lib64/ruby/gems/1.8/gems/erubis-2.7.0/lib/erubis/evaluator.rb:74:in `evaluate’
      /usr/lib64/ruby/gems/1.8/gems/chef-0.10.10/bin/../lib/chef/mixin/template.rb:41:in `render_template’
      /usr/lib64/ruby/gems/1.8/gems/chef-0.10.10/bin/../lib/chef/provider/template.rb:99:in `render_with_context’
      /usr/lib64/ruby/gems/1.8/gems/chef-0.10.10/bin/../lib/chef/provider/template.rb:39:in `action_create’
      /usr/lib64/ruby/gems/1.8/gems/chef-0.10.10/bin/../lib/chef/resource.rb:454:in `send’
      /usr/lib64/ruby/gems/1.8/gems/chef-0.10.10/bin/../lib/chef/resource.rb:454:in `run_action’
      /usr/lib64/ruby/gems/1.8/gems/chef-0.10.10/bin/../lib/chef/runner.rb:49:in `run_action’
      /usr/lib64/ruby/gems/1.8/gems/chef-0.10.10/bin/../lib/chef/runner.rb:85:in `converge’
      /usr/lib64/ruby/gems/1.8/gems/chef-0.10.10/bin/../lib/chef/runner.rb:85:in `each’
      /usr/lib64/ruby/gems/1.8/gems/chef-0.10.10/bin/../lib/chef/runner.rb:85:in `converge’
      /usr/lib64/ruby/gems/1.8/gems/chef-0.10.10/bin/../lib/chef/resource_collection.rb:94:in `execute_each_resource’
      /usr/lib64/ruby/gems/1.8/gems/chef-0.10.10/bin/../lib/chef/resource_collection/stepable_iterator.rb:116:in `call’
      /usr/lib64/ruby/gems/1.8/gems/chef-0.10.10/bin/../lib/chef/resource_collection/stepable_iterator.rb:116:in `call_iterator_block’
      /usr/lib64/ruby/gems/1.8/gems/chef-0.10.10/bin/../lib/chef/resource_collection/stepable_iterator.rb:85:in `step’
      /usr/lib64/ruby/gems/1.8/gems/chef-0.10.10/bin/../lib/chef/resource_collection/stepable_iterator.rb:104:in `iterate’
      /usr/lib64/ruby/gems/1.8/gems/chef-0.10.10/bin/../lib/chef/resource_collection/stepable_iterator.rb:55:in `each_with_index’
      /usr/lib64/ruby/gems/1.8/gems/chef-0.10.10/bin/../lib/chef/resource_collection.rb:92:in `execute_each_resource’
      /usr/lib64/ruby/gems/1.8/gems/chef-0.10.10/bin/../lib/chef/runner.rb:80:in `converge’
      /usr/lib64/ruby/gems/1.8/gems/chef-0.10.10/bin/../lib/chef/client.rb:330:in `converge’
      /usr/lib64/ruby/gems/1.8/gems/chef-0.10.10/bin/../lib/chef/client.rb:163:in `run’
      /usr/lib64/ruby/gems/1.8/gems/chef-0.10.10/bin/../lib/chef/application/client.rb:254:in `run_application’
      /usr/lib64/ruby/gems/1.8/gems/chef-0.10.10/bin/../lib/chef/application/client.rb:241:in `loop’
      /usr/lib64/ruby/gems/1.8/gems/chef-0.10.10/bin/../lib/chef/application/client.rb:241:in `run_application’
      /usr/lib64/ruby/gems/1.8/gems/chef-0.10.10/bin/../lib/chef/application.rb:70:in `run’
      /usr/lib64/ruby/gems/1.8/gems/chef-0.10.10/bin/chef-client:26
      /usr/bin/chef-client:19:in `load’
      /usr/bin/chef-client:19

    /usr/lib64/ruby/gems/1.8/gems/chef-0.10.10/bin/../lib/chef/mixin/template.rb:43:in `render_template’
    /usr/lib64/ruby/gems/1.8/gems/chef-0.10.10/bin/../lib/chef/provider/template.rb:99:in `render_with_context’
    /usr/lib64/ruby/gems/1.8/gems/chef-0.10.10/bin/../lib/chef/provider/template.rb:39:in `action_create’
    /usr/lib64/ruby/gems/1.8/gems/chef-0.10.10/bin/../lib/chef/resource.rb:454:in `send’
    /usr/lib64/ruby/gems/1.8/gems/chef-0.10.10/bin/../lib/chef/resource.rb:454:in `run_action’
    /usr/lib64/ruby/gems/1.8/gems/chef-0.10.10/bin/../lib/chef/runner.rb:49:in `run_action’
    /usr/lib64/ruby/gems/1.8/gems/chef-0.10.10/bin/../lib/chef/runner.rb:85:in `converge’
    /usr/lib64/ruby/gems/1.8/gems/chef-0.10.10/bin/../lib/chef/runner.rb:85:in `each’
    /usr/lib64/ruby/gems/1.8/gems/chef-0.10.10/bin/../lib/chef/runner.rb:85:in `converge’
    /usr/lib64/ruby/gems/1.8/gems/chef-0.10.10/bin/../lib/chef/resource_collection.rb:94:in `execute_each_resource’
    /usr/lib64/ruby/gems/1.8/gems/chef-0.10.10/bin/../lib/chef/resource_collection/stepable_iterator.rb:116:in `call’
    /usr/lib64/ruby/gems/1.8/gems/chef-0.10.10/bin/../lib/chef/resource_collection/stepable_iterator.rb:116:in `call_iterator_block’
    /usr/lib64/ruby/gems/1.8/gems/chef-0.10.10/bin/../lib/chef/resource_collection/stepable_iterator.rb:85:in `step’
    /usr/lib64/ruby/gems/1.8/gems/chef-0.10.10/bin/../lib/chef/resource_collection/stepable_iterator.rb:104:in `iterate’
    /usr/lib64/ruby/gems/1.8/gems/chef-0.10.10/bin/../lib/chef/resource_collection/stepable_iterator.rb:55:in `each_with_index’
    /usr/lib64/ruby/gems/1.8/gems/chef-0.10.10/bin/../lib/chef/resource_collection.rb:92:in `execute_each_resource’
    /usr/lib64/ruby/gems/1.8/gems/chef-0.10.10/bin/../lib/chef/runner.rb:80:in `converge’
    /usr/lib64/ruby/gems/1.8/gems/chef-0.10.10/bin/../lib/chef/client.rb:330:in `converge’
    /usr/lib64/ruby/gems/1.8/gems/chef-0.10.10/bin/../lib/chef/client.rb:163:in `run’
    /usr/lib64/ruby/gems/1.8/gems/chef-0.10.10/bin/../lib/chef/application/client.rb:254:in `run_application’
    /usr/lib64/ruby/gems/1.8/gems/chef-0.10.10/bin/../lib/chef/application/client.rb:241:in `loop’
    /usr/lib64/ruby/gems/1.8/gems/chef-0.10.10/bin/../lib/chef/application/client.rb:241:in `run_application’
    /usr/lib64/ruby/gems/1.8/gems/chef-0.10.10/bin/../lib/chef/application.rb:70:in `run’
    /usr/lib64/ruby/gems/1.8/gems/chef-0.10.10/bin/chef-client:26
    /usr/bin/chef-client:19:in `load’
    /usr/bin/chef-client:19

  • http://anentropic.wordpress.com Anentropic

    you’re passing:

    :hosts => hosts,
    :fqdn => node[:fqdn],
    :hostname => node[:hostname]

    into the template but then using:

    did you mean to pass in :ipaddress ?

  • Peter Boers

    change the following :

    To this