# Serverspec vs Mintest Feedback
**URL:** https://discourse.chef.io/t/serverspec-vs-mintest-feedback/6161
**Category:** Chef Infra (archive)
**Created:** [January 3, 2015, 8:00pm UTC](https://discourse.chef.io/t/serverspec-vs-mintest-feedback/6161 "2015-01-03T20:00:37Z")
**Posts on this page:** 10
**Page:** 1
### Author:  [@David\_Petzel](https://discourse.chef.io/u/David_Petzel)
#### Post date: [January 3, 2015, 8:00pm UTC](https://discourse.chef.io/t/serverspec-vs-mintest-feedback/6161/1 "2015-01-03T20:00:37Z")
TLDR - I’m interesting in knowing why folks are moving away from Minitest,
toward ServerSpec.
I’ve been noticing a lot of cookbooks recently have been migrating their
testings from minitest to ServerSpec. I’ve got nothing against ServerSpec,
but I’m curious why/where this trend is coming from.
One of the things I have always liked about minitest handler is that the
tests “follow” the cookbook. As an example we run the handler in our dev
and qa environments which we converge hourly, so as a by product we are
testing the integration of that cookbook in more places than just a
developers local workstation, and we are doing it on an hourly basis. As I
understand this is might be possible with ServerSpec, however its not as
simple, and comes with the lose of the node object data?
Maybe my question would be better asked in the context of an example…
Lets assume I am consuming a community cookbook which sets up some
application that has an HTTP port. There is an attribute that controls what
that HTTP port is.
In ServerSpec land, I think you need to hard code the HTTP port to test?
In Minitest, I have access to the node object, so my test is not hard coded
to a port, but instead uses the value of the node attribute.
For the sake of the discussion, lets say I run that application on port 78
in dev, 79 in qa, and 80 in production. While using minitests I’m able to
assert the application is working properly, because the minitest test would
be reading the node attribute. However from what I’ve seen with ServerSpec,
I don’t have that same ability to adapt the test to use the right port,
without manual intervention?
Have I totally missed the boat here on ServerSpec capabilities?
Thanks for any feedback/experience/war stories.
---
### Author:  [@jdunn](https://discourse.chef.io/u/jdunn)
#### Post date: [January 5, 2015, 8:31am UTC](https://discourse.chef.io/t/serverspec-vs-mintest-feedback/6161/2 "2015-01-05T08:31:02Z")
On Sat, Jan 3, 2015 at 12:00 PM, David Petzel [davidpetzel@gmail.com](mailto:davidpetzel@gmail.com) wrote:
> TLDR - I'm interesting in knowing why folks are moving away from Minitest,
> toward ServerSpec.
I think your statement:
" As I understand this is might be possible with ServerSpec, however
its not as simple, and comes with the lose of the node object data?"
is the crux of the reason why people are moving away from Minitest to
Serverspec -- because folks want the acceptance tests to not be
contaminated in any way by the mechanism by which it got there. They
see it as a /benefit/ that node state is not shared between the two.
- Julian
--
[Julian C. Dunn [jdunn@aquezada.com](mailto:jdunn@aquezada.com) \* Sorry, I'm ]
[WWW: [Julian Dunn's Blog - Commentary on media, technology, and everything in between.](http://www.aquezada.com/staff/julian) \* only Web 1.0 ]
[gopher://sdf.org/1/users/keymaker/ \* compliant!]
[PGP: 91B3 7A9D 683C 7C16 715F 442C 6065 D533 FDC2 05B9]
---
### Author:  [@Ranjib](https://discourse.chef.io/u/Ranjib)
#### Post date: [January 5, 2015, 8:38am UTC](https://discourse.chef.io/t/serverspec-vs-mintest-feedback/6161/3 "2015-01-05T08:38:11Z")
and you can always construct the node object as
Chef::Config.from\_file '/etc/chef/clinet.rb'
node = Chef::Node.load(Chef::Config[:node\_name])
..but as julian mentioned, chef is not a test harness system, but
rspec+serverspec is. Also, serverspec provides ssh and various other mode
of transport (specinfra subproject),
cheers
ranjib
On Mon, Jan 5, 2015 at 12:31 AM, Julian C. Dunn [jdunn@aquezada.com](mailto:jdunn@aquezada.com) wrote:
> On Sat, Jan 3, 2015 at 12:00 PM, David Petzel [davidpetzel@gmail.com](mailto:davidpetzel@gmail.com)
> wrote:
>
> > TLDR - I'm interesting in knowing why folks are moving away from
> > Minitest,
> > toward ServerSpec.
>
> I think your statement:
>
> " As I understand this is might be possible with ServerSpec, however
> its not as simple, and comes with the lose of the node object data?"
>
> is the crux of the reason why people are moving away from Minitest to
> Serverspec -- because folks want the acceptance tests to not be
> contaminated in any way by the mechanism by which it got there. They
> see it as a /benefit/ that node state is not shared between the two.
>
> - Julian
>
> --
> [Julian C. Dunn [jdunn@aquezada.com](mailto:jdunn@aquezada.com) \* Sorry, I'm ]
> [WWW: [Julian Dunn's Blog - Commentary on media, technology, and everything in between.](http://www.aquezada.com/staff/julian) \* only Web 1.0 ]
> [gopher://sdf.org/1/users/keymaker/ \* compliant!]
> [PGP: 91B3 7A9D 683C 7C16 715F 442C 6065 D533 FDC2 05B9]
---
### Author:  [@Taras\_Klym](https://discourse.chef.io/u/Taras_Klym)
#### Post date: [January 5, 2015, 12:55pm UTC](https://discourse.chef.io/t/serverspec-vs-mintest-feedback/6161/4 "2015-01-05T12:55:40Z")
Hi guys,
What I need is to change nodes run\_list basing on their tags and roles and names etc.
For example I have multiple nodes with different roles etc.
And I would like to assign some particular recipe to nodes which met some condition:
knife exec -E “nodes.find(:name =\> ‘_isec_’) { |node| node.run\_list \<\< “recipe[foo]”; n.save }”
It will work for nodes which have _isec_ in its names.
But what if I would like to use query something like:
knife exec -E “nodes.find(:name =\> ‘_isec_’ AND :tags =\> ‘tag\_name’ AND :role =\> ‘role\_name’) { |node| node.run\_list \<\< “recipe[foo]”; n.save }”
It fails. As it seems that “nodes.find” method only can take 1 statement. And right after first statement it is looking for ‘)’ instead of AND or other things… Or at least I have not found other usage.
Can you please help me with this issue?
Many thanks in advance.
Regards, Taras.
---
### Author:  [@jtimberman](https://discourse.chef.io/u/jtimberman)
#### Post date: [January 5, 2015, 3:48pm UTC](https://discourse.chef.io/t/serverspec-vs-mintest-feedback/6161/5 "2015-01-05T15:48:16Z")
Ohai!
On Sat, Jan 3, 2015 at 1:00 PM, David Petzel [davidpetzel@gmail.com](mailto:davidpetzel@gmail.com) wrote:
> TLDR - I'm interesting in knowing why folks are moving away from Minitest,
> toward ServerSpec.
As Julian said, there's an element of not wanting to "contaminate" the
tests. However, there's a couple other reasons.
> I've been noticing a lot of cookbooks recently have been migrating their
> testings from minitest to ServerSpec. I've got nothing against ServerSpec,
> but I'm curious why/where this trend is coming from.
Back in the before times, that is, before `test kitchen` and the notion of
`bussers`, the best (only?) way of reasonably testing nodes post
convergence was through the minitest chef handler. The early version of
test kitchen (pre 1.0) did run those, but the rewritten version, with the
bussers made it more flexible.
I think Serverspec is popular because Rspec is popular. It's not a value
judgment, just more widely used - Rspec is what CHEF uses for chef itself,
Ruby and Ruby on Rails programmers are familiar with Rspec. While minitest
is in the Ruby stdlib (and I like it more than Rspec for that reason), it's
simply not widespread. Also, Serverspec does an "outside in" kind of test
approach. I have more confidence that the tests are doing the right thing.
I can defer configurable options to attributes, and then set those
attributes on a per-suite basis.
Maybe my question would be better asked in the context of an example...
> Lets assume I am consuming a community cookbook which sets up some
> application that has an HTTP port. There is an attribute that controls what
> that HTTP port is.
> In ServerSpec land, I think you need to hard code the HTTP port to test?
> In Minitest, I have access to the node object, so my test is not hard
> coded to a port, but instead uses the value of the node attribute.
> For the sake of the discussion, lets say I run that application on port 78
> in dev, 79 in qa, and 80 in production. While using minitests I'm able to
> assert the application is working properly, because the minitest test would
> be reading the node attribute. However from what I've seen with ServerSpec,
> I don't have that same ability to adapt the test to use the right port,
> without manual intervention?
This is why I like, and use test kitchen. I can set up two suites.
suites:
- name: default
run\_list:
- recipe[webapp]
attributes:
- name: port78
run\_list:
- recipe[webapp]
attributes:
www:
port: 78
Then I can have tests for each suite, one that checks that my web app
responds on port 80 (presuming its the default), and responds on port 78. I
don't actually care about the different ports in different environments, I
want to test that the non-default port does the right thing. Adding more
suites that only change this attribute just tests that Chef works, which
we've already proved in the first case. In fact, if the port is an
attribute, I don't think the `default` suite is really _really_ necessary.
Cheers,
Joshua
---
### Author:  [@kallistec](https://discourse.chef.io/u/kallistec)
#### Post date: [January 5, 2015, 6:29pm UTC](https://discourse.chef.io/t/serverspec-vs-mintest-feedback/6161/6 "2015-01-05T18:29:26Z")
On Monday, January 5, 2015 at 4:55 AM, [klum\_tz@ukr.net](mailto:klum_tz@ukr.net) wrote:
> Hi guys,
>
> What I need is to change nodes run\_list basing on their tags and roles and names etc.
>
> For example I have multiple nodes with different roles etc.
>
> And I would like to assign some particular recipe to nodes which met some condition:
>
> knife exec -E "nodes.find(:name =\> '_isec_') { |node| node.run\_list \<\< "recipe[foo]"; n.save }"
>
> It will work for nodes which have _isec_ in its names.
>
> But what if I would like to use query something like:
>
> knife exec -E "nodes.find(:name =\> '_isec_' AND :tags =\> 'tag\_name' AND :role =\> 'role\_name') { |node| node.run\_list \<\< "recipe[foo]"; n.save }"
> You’re mixing ruby syntax and Solr query syntax here. If you want all the conditions to be AND’ed together, then you can pass a regular ruby Hash object, e.g., :name =\> “NAME\_QUERY”, :tags =\> “TAG\_QUERY”, …
> If you instead need to do something more complicated, you can pass a string to `nodes.find` using the same syntax as you would with `knife search`.
For reference, the code that generates the query from the `nodes.find` call is here: [https://github.com/opscode/chef/blob/005f75e158ce702bb28fd6199e9654b3d1115d3b/lib/chef/shell/model\_wrapper.rb#L86-L92](https://github.com/opscode/chef/blob/005f75e158ce702bb28fd6199e9654b3d1115d3b/lib/chef/shell/model_wrapper.rb#L86-L92)
HTH,
--
Daniel DeLeo
---
### Author:  [@tknerr](https://discourse.chef.io/u/tknerr)
#### Post date: [January 6, 2015, 12:17am UTC](https://discourse.chef.io/t/serverspec-vs-mintest-feedback/6161/7 "2015-01-06T00:17:42Z")
I always thought of minitest-handler and test-kitchen / serverspec as
being two orthogonal things:
- minitest-handler for "smoke tests" that run from the inside
everytime the system converges (e.g. check if processes are up and
running, etc.)
- kitchenci / serverspec for "integration tests" that still run from
the inside but not necessarily after each run (i.e. on demand via
`kitchen test`)
Actually, now that I'm writing it... the kitchenci tests actually run
inside the node, right?
What are the preferred frameworks for acceptance-level testing from the outside?
I had played with cucumber-nagios [0] once, but it doesn't seem
widespread in the chef community. Anyone having experience with
leibniz [1] btw?
Cheers,
Torben
[0] [cucumber-nagios](http://auxesis.github.io/cucumber-nagios/)
[1] [GitHub - Atalanta/leibniz: Leibniz is an integration testing framework for Chef](https://github.com/Atalanta/leibniz)
On Mon, Jan 5, 2015 at 4:48 PM, Joshua Timberman [joshua@chef.io](mailto:joshua@chef.io) wrote:
> Ohai!
>
> On Sat, Jan 3, 2015 at 1:00 PM, David Petzel [davidpetzel@gmail.com](mailto:davidpetzel@gmail.com) wrote:
>
> > TLDR - I'm interesting in knowing why folks are moving away from Minitest,
> > toward ServerSpec.
>
> As Julian said, there's an element of not wanting to "contaminate" the
> tests. However, there's a couple other reasons.
>
> > I've been noticing a lot of cookbooks recently have been migrating their
> > testings from minitest to ServerSpec. I've got nothing against ServerSpec,
> > but I'm curious why/where this trend is coming from.
>
> Back in the before times, that is, before `test kitchen` and the notion of
> `bussers`, the best (only?) way of reasonably testing nodes post convergence
> was through the minitest chef handler. The early version of test kitchen
> (pre 1.0) did run those, but the rewritten version, with the bussers made it
> more flexible.
>
> I think Serverspec is popular because Rspec is popular. It's not a value
> judgment, just more widely used - Rspec is what CHEF uses for chef itself,
> Ruby and Ruby on Rails programmers are familiar with Rspec. While minitest
> is in the Ruby stdlib (and I like it more than Rspec for that reason), it's
> simply not widespread. Also, Serverspec does an "outside in" kind of test
> approach. I have more confidence that the tests are doing the right thing. I
> can defer configurable options to attributes, and then set those attributes
> on a per-suite basis.
>
> > Maybe my question would be better asked in the context of an example...
> > Lets assume I am consuming a community cookbook which sets up some
> > application that has an HTTP port. There is an attribute that controls what
> > that HTTP port is.
> > In ServerSpec land, I think you need to hard code the HTTP port to test?
> > In Minitest, I have access to the node object, so my test is not hard
> > coded to a port, but instead uses the value of the node attribute.
> >
> > For the sake of the discussion, lets say I run that application on port 78
> > in dev, 79 in qa, and 80 in production. While using minitests I'm able to
> > assert the application is working properly, because the minitest test would
> > be reading the node attribute. However from what I've seen with ServerSpec,
> > I don't have that same ability to adapt the test to use the right port,
> > without manual intervention?
>
> This is why I like, and use test kitchen. I can set up two suites.
>
> suites:
>
> - name: default
> run\_list:
> - recipe[webapp]
> attributes:
>
> - name: port78
> run\_list:
> - recipe[webapp]
> attributes:
> www:
> port: 78
>
> Then I can have tests for each suite, one that checks that my web app
> responds on port 80 (presuming its the default), and responds on port 78. I
> don't actually care about the different ports in different environments, I
> want to test that the non-default port does the right thing. Adding more
> suites that only change this attribute just tests that Chef works, which
> we've already proved in the first case. In fact, if the port is an
> attribute, I don't think the `default` suite is really _really_ necessary.
>
> Cheers,
> Joshua
---
### Author:  [@coderanger](https://discourse.chef.io/u/coderanger)
#### Post date: [January 6, 2015, 12:24am UTC](https://discourse.chef.io/t/serverspec-vs-mintest-feedback/6161/8 "2015-01-06T00:24:43Z")
On Jan 5, 2015, at 4:17 PM, Torben Knerr [mail@tknerr.de](mailto:mail@tknerr.de) wrote:
> I always thought of minitest-handler and test-kitchen / serverspec as
> being two orthogonal things:
>
> - minitest-handler for "smoke tests" that run from the inside
> everytime the system converges (e.g. check if processes are up and
> running, etc.)
In it hasn't been mentioned that I've seen, check out [https://github.com/opscode/chef-rfc/blob/master/rfc035-audit-mode.md](https://github.com/opscode/chef-rfc/blob/master/rfc035-audit-mode.md) for a description of the next evolution of this idea.
--Noah
---
### Author:  [@tknerr](https://discourse.chef.io/u/tknerr)
#### Post date: [January 6, 2015, 12:31am UTC](https://discourse.chef.io/t/serverspec-vs-mintest-feedback/6161/9 "2015-01-06T00:31:25Z")
Thanks Noah!
audit-mode sounds super useful!
On Tue, Jan 6, 2015 at 1:24 AM, Noah Kantrowitz [noah@coderanger.net](mailto:noah@coderanger.net) wrote:
> On Jan 5, 2015, at 4:17 PM, Torben Knerr [mail@tknerr.de](mailto:mail@tknerr.de) wrote:
>
> > I always thought of minitest-handler and test-kitchen / serverspec as
> > being two orthogonal things:
> >
> > - minitest-handler for "smoke tests" that run from the inside
> > everytime the system converges (e.g. check if processes are up and
> > running, etc.)
>
> In it hasn't been mentioned that I've seen, check out [https://github.com/opscode/chef-rfc/blob/master/rfc035-audit-mode.md](https://github.com/opscode/chef-rfc/blob/master/rfc035-audit-mode.md) for a description of the next evolution of this idea.
>
> --Noah
---
### Author:  [@David\_Petzel](https://discourse.chef.io/u/David_Petzel)
#### Post date: [January 6, 2015, 1:36am UTC](https://discourse.chef.io/t/serverspec-vs-mintest-feedback/6161/10 "2015-01-06T01:36:54Z")
Thanks for all the feedback on this. I do have a better understanding now,
but it still feels like we “loosing something” in this transition. Maybe
I’m in the minority, but I like the fact that a cookbook author writes
minitests, and I can have those executed every single converge, not just
when someone decides to kick off a test-kitchen run.
On our team we are certainly making use of test-kitchen and bussers for
development and CI testing, but I can’t shake the feeling those are not
replacements for a validation test running on every converge.
In any case, thanks again for taking the time give me your perspectives.