How Can I Run a recipe twice on the same machine with different settings for each invocation?

Hi,

I’m wondering if it’s possible to apply a recipe twice on the same machine
with different settings.

The exact reason is that I have a build farm and each agent can only run a
single job at a time. I’d therefore like to install more than one agent on
each machine. Each agent needs slightly different settings so that they
don’t clash.

So I’d like to say, run the agent recipe passing it a parameter to say
"agentId=1" and run it again with “agentId=2” and so on.

Thanks in advance.
Tom


Tom Duckering

I asked this question to Opscode support earlier. They mentioned trying to
use LWRP or Definitions to apply a set of chef commands multiple times with
different settings. This of course did not help 100%, as I was hoping to
leverage already existing cookbooks and did not want to have to write my
cookbook to do something multiple times.

Another option is to setup multiple "nodes" on your system. You can do this
by running chef_client with the -N option for its name and -k for the client
key. Then you can adjust the node specific attributes use knife node edit
MY_NODE.

Bryan

On Tue, Oct 18, 2011 at 11:37 AM, Tom Duckering
tduckeri@thoughtworks.comwrote:

Hi,

I'm wondering if it's possible to apply a recipe twice on the same machine
with different settings.

The exact reason is that I have a build farm and each agent can only run a
single job at a time. I'd therefore like to install more than one agent on
each machine. Each agent needs slightly different settings so that they
don't clash.

So I'd like to say, run the agent recipe passing it a parameter to say
"agentId=1" and run it again with "agentId=2" and so on.

Thanks in advance.
Tom

--
Tom Duckering

Just starting to work on chef and not a ruby expert but I¹ve done something
similar like so:

attributes:
default[³agent_config²][³agent1²][³param²] = ³foo²
default[³agent_config²][³agent2²][³param²] = ³bar²

recipe:
node[³agent_config²].each do |agent_id|
node[³agent_config²][³#{agent_id}²].each do |agent_param|
execute ³#{agent_id}² do
command ³#{agent_id}.sh #{agent_param}²
action :run
end
end

On 10/18/11 11:51 AM, "Bryan Baugher" bjbq4d@gmail.com wrote:

I asked this question to Opscode support earlier. They mentioned trying to use
LWRP or Definitions to apply a set of chef commands multiple times with
different settings. This of course did not help 100%, as I was hoping to
leverage already existing cookbooks and did not want to have to write my
cookbook to do something multiple times.

Another option is to setup multiple "nodes" on your system. You can do this by
running chef_client with the -N option for its name and -k for the client key.
Then you can adjust the node specific attributes use knife node edit MY_NODE.

Bryan

On Tue, Oct 18, 2011 at 11:37 AM, Tom Duckering tduckeri@thoughtworks.com
wrote:

Hi,

I'm wondering if it's possible to apply a recipe twice on the same machine
with different settings.

The exact reason is that I have a build farm and each agent can only run a
single job at a time. I'd therefore like to install more than one agent on
each machine. Each agent needs slightly different settings so that they don't
clash.

So I'd like to say, run the agent recipe passing it a parameter to say
"agentId=1" and run it again with "agentId=2" and so on.

Thanks in advance.
Tom

A reasonable approach. The answer from support was the 'best' one, though, which is to take the recipe wrap it up in a definition or an LWRP.

If all you did was create a definition, paste the recipe in, and replace the 'agentid' with a parameter to the definition, you would wind up with:

agent "1"
agent "2"

In your recipe.

Best,
Adam

On Oct 18, 2011, at 10:33 AM, Joe Lind wrote:

Just starting to work on chef and not a ruby expert but I’ve done something similar like so:

attributes:
default[“agent_config”][“agent1”][“param”] = “foo”
default[“agent_config”][“agent2”][“param”] = “bar”

recipe:
node[“agent_config”].each do |agent_id|
node[“agent_config”][“#{agent_id}”].each do |agent_param|
execute “#{agent_id}” do
command “#{agent_id}.sh #{agent_param}”
action :run
end
end

On 10/18/11 11:51 AM, "Bryan Baugher" bjbq4d@gmail.com wrote:

I asked this question to Opscode support earlier. They mentioned trying to use LWRP or Definitions to apply a set of chef commands multiple times with different settings. This of course did not help 100%, as I was hoping to leverage already existing cookbooks and did not want to have to write my cookbook to do something multiple times.

Another option is to setup multiple "nodes" on your system. You can do this by running chef_client with the -N option for its name and -k for the client key. Then you can adjust the node specific attributes use knife node edit MY_NODE.

Bryan

On Tue, Oct 18, 2011 at 11:37 AM, Tom Duckering tduckeri@thoughtworks.com wrote:

Hi,

I'm wondering if it's possible to apply a recipe twice on the same machine with different settings.

The exact reason is that I have a build farm and each agent can only run a single job at a time. I'd therefore like to install more than one agent on each machine. Each agent needs slightly different settings so that they don't clash.

So I'd like to say, run the agent recipe passing it a parameter to say "agentId=1" and run it again with "agentId=2" and so on.

Thanks in advance.
Tom

Hey Tom,

You're explanation of the use case is a bit general that makes it hard to give specific advice. Do you have a specific community recipe your are interested in using in that manner, or your own?

During the same run, the same recipe will only be run once, but each run could have different inputs to tweak things as needed. You could use databags or node attributes.

I've got a 'similar' use case I've done for running multiple ruby active messaging pollers. I'm using the monit cookbook with the monitrc LWRP so in hopes that it provides inspiration I put some snippets below:

In my recipe:

monitrc "pollers", {:processor_groups => node[:processor_groups],
:app_dir => "#{node[:channels][(node[:channels][:app_name])][:current_path]}",
:rails_env => node[:channels][:environment],
:alert_email => node[:monit][:notify_email]}

the erb template used:

<% @processor_groups.keys.each do |group| %>
<% (0...@processor_groups[group][:num_pollers]).each do |i| %>
<% poller_group_name = group.sub('group', '') %>
CHECK PROCESS <%= poller_group_name %>poller<%= i %> WITH PIDFILE <%= File.join(@app_dir, "tmp", "pids", "#{poller_group_name}poller#{i}.pid") %> EVERY 5 CYCLES
START PROGRAM = "/bin/su - channels -c 'cd <%= @app_dir %> ; RAILS_ENV=<%= @rails_env %> bundle exec <%= @app_dir %>/script/poller <%=i%> start -- process-group=<%= group %>'" WITH TIMEOUT 300 seconds
STOP PROGRAM = "/bin/su - channels -c 'cd <%= @app_dir %> ; RAILS_ENV=<%= @rails_env %> bundle exec <%= @app_dir %>/script/poller <%=i%> stop -- process-group=<%= group %>'"
GROUP <%= group %>
GROUP messagebus
IF TOTALMEMORY > 250 MB FOR 10 CYCLES THEN restart
#IF 3 RESTARTS WITHIN 5 CYCLES THEN timeout
ALERT <%= @alert_email %> NOT {action, exec, nonexist, pid, ppid, resource}

<% end %>
<% end %>

example of the node attributes:

  :processor_groups => {
    :affiliate_agent_group => { :num_pollers => 1 },
    :introspect_video_group => { :num_pollers => 1 },
    :publisher_agent_group => { :num_pollers => 1 },
    :show_agent_group => { :num_pollers => 2 },
    :transcode_group => { :num_pollers => 1},
    :update_rss_feed_group  => { :num_pollers => 1 },
    :video_available_group => { :num_pollers => 1 }
  }

On Oct 18, 2011, at 9:37 AM, Tom Duckering wrote:

Hi,

I'm wondering if it's possible to apply a recipe twice on the same machine with different settings.

The exact reason is that I have a build farm and each agent can only run a single job at a time. I'd therefore like to install more than one agent on each machine. Each agent needs slightly different settings so that they don't clash.

So I'd like to say, run the agent recipe passing it a parameter to say "agentId=1" and run it again with "agentId=2" and so on.

Thanks in advance.
Tom

--
Tom Duckering

On Oct 18, 2011, at 9:37 AM, Tom Duckering wrote:

Hi,

I'm wondering if it's possible to apply a recipe twice on the same machine with different settings.

The exact reason is that I have a build farm and each agent can only run a single job at a time. I'd therefore like to install more than one agent on each machine. Each agent needs slightly different settings so that they don't clash.

So I'd like to say, run the agent recipe passing it a parameter to say "agentId=1" and run it again with "agentId=2" and so on.

No, recipes cannot be instantiated more than once. Fortunately LWRPs can and are very easy to write if you already have a working recipe :slight_smile:

--Noah