Testing Applications

Last updated: 17 August 2021
Application management is only available on Puppet >= 4.3.0

Basic Test Structure

describe '<application name>' do
  let(:node) { '<host name>' }
  let(:title) { '<application instance title>' }
  let(:params) do
    {
      'nodes' => {
        ref('Node', node) => ref('<capitalised application name>', title),
      }
      # any additional app parameters
    }
  end

  # tests go here
end
  • The node definition is required to be set so that it can be later referenced in the nodes parameter.
  • Applications act like defined types and require a title to be defined.
  • The nodes parameter requires the use of node reference mappings to resource mappings. The ref method creates these references (a normal string will not suffice).
Cross-node support is not available at the moment and will return an error. Ensure that you model your tests to be single-node.

Configuring The Tests

Specifying parameters

If the object being tested takes parameters, these can be specified as a hash of values using let(:params).

let(:params) { {'ensure' => 'present', 'enable' => true} }

When passing undef as a parameter value, it should be passed as the symbol :undef.

let(:params) { {'user' => :undef} }

When passing a reference to a resource (e.g. Package['apache2']), it should be passed as a call to the ref helper (ref(<resource type>, <resource title>))

let(:params) { {'require' => ref('Package', 'apache2')} }

If have nested RSpec contexts to test the behaviour of different parameter values, you can partially override the parameters by merging the changed parameters into super() in your let(:params) block.

describe 'My::Class' do
  let(:params) do
    {
      'some_common_param' => 'value',
      'ensure'            => 'present',
    }
  end

  context 'with ensure => absent' do
    let(:params) do
      super().merge({ 'ensure' => 'absent' })
    end

    it { should compile }
  end
end

Specifying the FQDN of the test node

If the object being tested depends upon the node having a certain name, it can be specified using let(:node).

let(:node) { 'testhost.example.com' }

Specifying the environment name

If the manifest being tested expects to evaluate the environment name, it can be specified using let(:environment).

let(:environment) { 'production' }

Specifying node parameters

Node parameters (or top-scope variables) such as would be provided by an ENC can be specified as a hash of values using let(:node_params).

let(:node_params) { {'hostgroup' => 'web', 'rack' => 'KK04' } }

These node parameters will be merged into the default node parameters (if set), with these values taking precedence over the default node parameters in the event of a conflict.

If have nested RSpec contexts to test the behaviour of different node parameter values, you can partially override the node parameters by merging the changed parameters into super() in your let(:node_params) block.

describe 'My::Class' do
  let(:node_params) do
    {
      'some_common_param' => 'value',
      'role'              => 'default',
    }
  end

  context 'with role => web' do
    let(:node_params) do
      super().merge({ 'role' => 'web' })
    end

    it { should compile }
  end
end

Specifying code to include before

If the manifest being tested relies on some existing state (another class being included, variables to be set, etc), this can be specified using let(:pre_condition).

let(:pre_condition) { 'include some::other_class' }

The value may be a string or an array of strings that will be concatenated, and then be evaluated before the manifest being tested.

Specifying code to include after

If the manifest being tested depends on being evaluated before another manifest, this can be specified using let(:post_condition).

let(:post_condition) { 'include some::other_class::after' }

The value may be a string or an array of strings that will be concatenated, and then be evaluated after the manifest being tested.

Specifying facts

By default, the test environment contains only the hostname, domain, and fqdn facts (determined by the FQDN of the test node). Additional facts can be specified as a hash of values using let(:facts).

let(:facts) { {'operatingsystem' => 'Debian', 'ipaddress' => '192.168.0.1'} }

Facts may be expressed as a value (shown in the previous example) or a structure. Fact keys may be expressed as symbols or strings, and will be converted to a lower case string to align with the Facter standard.

let(:facts) do
  {
    'os' => {
      'family'  => 'RedHat',
      'release' => {
        'major' => '7',
        'minor' => '1',
        'full'  => '7.1.1503',
      }
    }
  }
end

These facts will be merged into the default facts (if set), with these values taking precedence over the default fact values in the event of a conflict.

If have nested RSpec contexts to test the behaviour of different fact values, you can partially override the parent facts by merging the changed facts into super() in your let(:facts) block.

describe 'My::Class' do
  let(:facts) do
    {
      'operatingsystem' => 'Debian',
      'role'            => 'default',
    }
  end

  context 'with role => web' do
    let(:facts) do
      super().merge({ 'role' => 'web' })
    end

    it { should compile }
  end
end
A common pattern is to use rspec-puppet-facts to automatically populate the facts for a specified operating system. Please read the rspec-puppet-facts documentation for more information.

Specifying trusted facts

When testing with Puppet >= 4.3, the trusted facts hash will have the standard trusted facts (certname, domain, and hostname) populated based on the node name. Those elements can only be set with the let(:node), not with this structure.

By default, the test environment contains no custom trusted facts (usually obtained from certificate extensions) and found in the extensions key. If the manifest being tested depends on the values from specific custom certificate extensions, they can be specified as a hash using let(:trusted_facts).

let(:trusted_facts) { {'pp_uuid' => '012345670-ABCD', 'some' => 'value'} }

These trusted facts will be merged into the default trusted facts (if set), with these values taking precedence over the default trusted facts in the event of a conflict.

If have nested RSpec contexts to test the behaviour of different trusted fact values, you can partially override the parent trusted facts by merging the changed facts into super() in your let(:trusted_facts) block.

describe 'My::Class' do
  let(:trusted_facts) do
    {
      'some_common_param' => 'value'
      'role'              => 'default',
    }
  end

  context 'with role => web' do
    let(:trusted_facts) do
      super().merge({ 'role' => 'web' })
    end

    it { should compile }
  end
end

Testing The Catalogue

Test that the catalogue compiles

This is the most basic test that can be done on a manifest. It will test that the manifest can be compiled into a catalogue, and that the catalogue has no dependency cycles between resources.

it { is_expected.to compile }

This matcher has an optional method that can be chained onto it in order to have rspec-puppet test that all relationships in the catalogue (as defined with require, notify, subscribe, before, or the chaining arrows) resolve to resources in the catalogue.

it { is_expected.to compile.with_all_deps }

Test for errors

When testing for an expected error (e.g. testing the behaviour of input validation), the and_raise_error method should be chained onto the compile matcher.

describe 'my::type' do
  context 'with ensure => present' do
    let(:params) { {'ensure' => 'present'} }

    it { is_expected.to compile }
  end

  context 'with ensure => whoopsiedoo' do
    let(:params) { {'ensure' => 'whoopsiedoo'} }

    it { is_expected.to compile.and_raise_error(/the expected error message/) }
  end
end

Test a resource

The presence of a resource in the catalogue can be tested using the generic contain_<resource type> matcher.

it { is_expected.to contain_service('apache2') }

If the <resource type> includes :: (e.g. the apache::vhost defined type), you must replace the :: with __ (two underscores) in the matcher name.

it { is_expected.to contain_apache__vhost('www.mysite.com') }

This can also be used to test if a class has been included in the catalogue.

it { is_expected.to contain_class('apache::vhosts') }
rspec-puppet does not do the class name parsing and lookup that the Puppet parser would do for you. The matcher only accepts fully qualified class names without any leading colons. This means that class foo::bar will only be matched by foo::bar, not by ::foo::bar or bar alone.

Test resource parameters

The values of a resource’s parameters can be tested by chaining with_<parameter name>(<value>) methods onto the contain_<resource type> matcher.

it { is_expected.to contain_apache__vhost('www.mysite.com').with_ensure('present') }

While you can chain multiple with_<parameter name> methods together, it may be cleaner for a large number of parameters to instead to chain the with method and pass a hash of expected parameters and values instead.

it { is_expected.to contain_service('apache').with('ensure' => 'present', 'enable' => true) }
# is equivalent to
it { is_expected.to contain_service('apache').with_ensure('present').with_enable(true) }

Testing parameters using with_<parameter name> or with will not take into account any other parameters that might be set on the resource. In order to test that only the specificied parameters have been set on a resource, the only_with_<parameter name> method can be chained onto the contain_<resource type> matcher.

# If any parameters have been set on Package[httpd] other than ensure, this test will fail.
it { is_expected.to contain_package('httpd').only_with_ensure('latest') }

Similarly to with_<parameter name>, there exists a way to specify multiple parameters at once, by chaining only_with onto the contain_<resource type> matcher and passing it a hash of expected parameters and values.

it { is_expected.to contain_service('apache').only_with('ensure' => 'running', 'enable' => true) }

Lastly, there are situations where it is necessary to test that certain parameters have not been set on a resource. This can be done by chaining without_<parameter name> methods onto the contain_<resource type> matcher.

it { is_expected.to contain_file('/tmp/testfile').without_mode }

As with the other parameter methods, there is a way to specify multiple undefined parameters at once by chaining the without method to the contain_<resource type> matcher and passing it an array of parameter names.

it { is_expected.to contain_service('apache').without(['restart', 'status']) }

Test resource parameter values for uniqueness

Use the have_unique_values_for_all matcher to test a specific resource parameter for uniqueness of values across the entire catalogue:

it { is_expected.to have_unique_values_for_all('user', 'uid')

Testing relationships between resources

The relationships between resources can be tested using the following methods, regardless of how the relationship has been defined. This mean that it doesn’t matter if it was defined using the relationship metaparameters (require, before, notify, subscribe) or the chaining arrows (->, <-, ~>, <~).

The values passed to these methods must be in the format used in the Puppet catalogue (which is slightly different to the way they're written in a Puppet manifest).
  • The resource titles must be unquoted (Package[apache] instead of Package['apache'])
  • One title per resource ([Package[apache], Package[htpasswd]] instead of Package[apache, htpasswd])
  • If referencing a class, it must be fully qualified and should not have a leading :: (Class[apache::service] instead of Class[::apache::service])
it { is_expected.to contain_file('a').that_requires('File[b]') }
it { is_expected.to contain_file('a').that_comes_before('File[b]') }
it { is_expected.to contain_file('a').that_notifies('File[b]') }
it { is_expected.to contain_file('a').that_subscribes_to('File[b]') }

An array can be passed if the resource has the same type of relationship to multiple resources.

it { is_expected.to contain_file('a').that_requires(['File[b]', 'File[c]']) }
it { is_expected.to contain_file('a').that_comes_before(['File[b]', 'File[c]']) }
it { is_expected.to contain_file('a').that_notifies(['File[b]', 'File[c]']) }
it { is_expected.to contain_file('a').that_subscribes_to(['File[b]', 'File[c]']) }

The relationships can be tested in either direction, so given the following manifest:

notify { 'a': }
notify { 'b':
  before => Notify['a'],
}

It can be tested that Notify[b] comes before Notify[a]

it { is_expected.to contain_notify('b').that_comes_before('Notify[a]') }

Or that Notify[a] requires Notify[b]

it { is_expected.to contain_notify('a').that_requires('Notify[b]') }

Testing the total number of resources

The total number of resources in the catalogue can be tested with the have_resource_count matcher.

it { is_expected.to have_resource_count(2) }

Testing the total number of classes

The total number of classes in the catalogue can be tested with the have_class_count matcher.

it { is_expected.to have_class_count(4) }

Testing the number of resources of a specific type

The number of resources of a specific type can be tested using the generic have_<resource type>_resource_count matcher.

it { is_expected.to have_exec_resource_count(1) }

As with the generic contain_<resource type> matcher, this matcher can also be used for defined types that contain :: in their name by replacing the :: with __ (two underscores).

it { is_expected.to have_apache__vhost_resource_count(3) }