Inspec: Retries/wait option (until service comes up)

Chef Infra (archive)
1

Hi all,

I want to verify that a service listens to a specific port. Unfortunately, startup of that service takes a couple of seconds, longer than kitchen verify takes between converge and verify.

Is there any way to let inspec retry a test or artificially pause it for a couple of seconds?

I was able to fix the issue by adding a line pause(10) to the test, but retry would be cleaner. Something similar to rspec-wait?

Thanks
Steffen

2

I am note 100% sure now, but if I remeber correctly Inspec is basically an extension of RSpec which means you can use a before block to check if the port is listening. Something like:

describe "App is running" do
  before do
    10.times do
      if PortIsListening
        return 0
      else
        sleep(1)
        next
      end
    end
    raise 'Port XY is not listening'
  end

  # inspec test here
end

All this is just ugly pseudo code, but I think you’ll get the idea. Docs here: https://www.relishapp.com/rspec/rspec-core/v/2-2/docs/hooks/before-and-after-hooks

3

Hi Joerg,

thanks a lot for your feedback. The before hook is indeed executed. However, the PortIsListening check is a bit challenging for me. I tried the following code:

describe port(8080) do
  before do
    30.times do
      if system("netstat -ntl | grep :8080 ")
        puts "listening"
        return 0
      else
        sleep(1)
        puts "not listening"
        next
      end
    end
  end
  it{ should be_listening }
end

Is the netstat in this case really executed inside the test-kitchen VM? The command itself works (also when I execute the system(..) within pry , but not when executed during kitchen verify.

Do you have any hint? It would be a lot cleaner, if I could replace PortIsListening with some call that triggers should_be listening. Any ideas?

Thanks
Steffen

4

Hi,

If your are really just looking in a retry for a specific test you could use rspec-retry [1] which would be a lot cleaner than the before block:

require 'rspec/retry'

describe port(8080) do
  it 'should succeed after a while', retry: 2, retry_wait: 7 do
    should be_listening
  end
end

The before block is just the generic approach which will work for “special” situations.

[1] https://github.com/NoRedInk/rspec-retry

5

Hi Jörg,
thanks for your help, I appreciate it.

Adding this doesn’t work unfortunately. After (retry x retry_wait) seconds pass by, the test fails. I verified that also with higher retry values. Adding a puts inside shows that the code is executed retry times, but it seems more like should be_listening is not re-evaluated. Is this possible?

Steffen

6

Hi,

You could test the code directly with inspec. What I did was simply putting the code block I posted to some file (/tmp/test_port_spec.rb) and running chef exec inspec exec /tmp/test_port_spec.rb.
Maybe even test the different scenarios with different ports to test if the code itself works:

  1. Run the test locally with against your local machine
  2. Run the test in the kitchen VM. You might need to install inspec into the chef installation (/opt/chef/embedded/bin/gem install inspec) and then you should find the inspec executable in /opt/chef/embedded/bin
  3. Run the test from your local machine in the VM with inspec exec test.rb -t ssh://user@hostname -i /path/to/key (key should be somewhere in the .kitchen directory if you are using Vagrant).
7

In case someone else is looking for this; and this was the only item that came up for me in a search; it is possible as follows:

  • gem install rspec-retry
  • then add a test as below (retrying every second for 60 seconds):
require 'rspec/retry'

describe 'Port 8080' do
  it 'should be listening', retry: 60, retry_wait: 1 do
    expect(port(8080).listening?).to eq true
  end
end
  • debug by adding the following to your test
RSpec.configure do |config|
  # show retry status in spec process
  config.verbose_retry = true
  # show exception that triggers a retry if verbose_retry is set to true
  config.display_try_failure_messages = true
end

h/t https://www.sidorenko.io/post/2016/11/writing-inspec-tests-in-the-rspec-style/