Getting chef to run actions in desired order, compile vs converge

Chef Infra (archive)
1

(was Subject: Re: [chef] NoMethodError for LWRP within a ruby_block?)

Thanks for the converge_by tip. Unfortunately I don't grok how to wield it for
my use case. Isn't converge_by only related to why-run mode? Which is a
"dry-run" mode? Is this use of converge_by "tricking" chef into running in the
correct order? I don't know enough to know if it's hacky or elegant or somewhere
in between.

I don't understand why it's so hard to get my actions running in the correct
order. Eventually I may get it and be able to create some elegant solution, but
right now it's taking me too long. I feel tempted to toss all of my logic and
order for this task into an external script and use chef's execute or script
resource to run it with an appropriate timeout. This makes me cringe in defeat.

But wait! A coworker and I experimented with run_context, and this worked:

::Chef::Recipe.send(:include, ::Cloner::Chef::DbSvc::Helper)
ruby_block "check mysql acl" do
block do
current_ip = %x( mysql --skip-column-names -hlocalhost -uroot -p#{cli_password} -e "SELECT Host FROM mysql.user WHERE User='cloner_admin'" )

if current_ip.empty?
  Chef::Log.debug("HIGGS-BOSON: current_ip is empty, applying grants.")
  ####run_context = Chef::RunContext.new(node, {})
  f = Chef::Resource::ClonerDbSvcMysqlacl.new("mysqlgrants", run_context)
  f.ipv4_address      quux_ip
  f.cli_password      cli_password
  f.cloner_admin_pass cloner_admin_pass
  f.run_action        :create_acl
end

end
end

Where "worked" means things ran in the right order: mysql start; wait for mysql
to accept conns; only then query mysql and take action on results.

/me falls over

kallen

On Thu, 05 Dec 2013, Lucas Hrabovsky wrote:

If your able to now get access to your LWRP and hitting the problem of things not running in
the converge, have a look at converge_by
http://docs.opscode.com/lwrp_custom_provider_ruby.html#converge-by

Here's an example from my app cookbook How to use converge_by in Chef LWRP's. This is from an app cookbook I forked from the main opscode application cookbook but stripped way down. ยท GitHub

On Thursday, December 5, 2013 at 2:17 PM, kallen@groknaut.net wrote:

Thanks for the response. I forgot chef DSL isn't available within a ruby_block.
Basically, I'm trying to get my recipe actions executing in the right order,
using the chef tools as I understand them so far. I'll probably try run_context
today.

Given my cookbook, the run order I desire is:

  1. Start mysql
  2. Wait for innodb recovery to complete (i.e. wait for mysql to respond to queries).
  3. Query a value from mysql
  4. Take action on that value using my LWRP

But the run order that actually happened before I tried putting 3 and 4 into
a ruby_block was:

  1. Query a value from mysql (got executed at compile phase)
  2. Take action on that value using my LWRP (got executed at compile phase)
  3. Start mysql (got executed at convergence phase)
  4. Wait for innodb recovery to complete (i.e. wait for mysql to respond to queries). (got executed at convergence phase)

Items executing at compile phase rather than convergence phase is working
against me, I feel. I'll keep trying.

thx,
kallen

On Wed, 04 Dec 2013, Brian Fletcher wrote:

Hi Callen,

When inside a ruby_block the chef DSL doesn't work. You need to use pure
ruby. Here is an example of how you can execute a resource from a
ruby_block. The cookbook is called 'test' and the lwrp is called 'mine'.

ruby_block 'dd' do
block do
mine = Chef::Resource::TestMine.new('bb', run_context)
mine.run_action(:create)
end
action :create
end

You will need something like the following. The lwrp name is turned into
camel case.

mysqlacl = Chef::Resource::ClonerDbSvcMysqlacl("mysqlgrants", run_context)
mysqlacl.ipv4_address(quux_ip)
?

mysqlacl.run_action(:create_acl)

Regarding whether its the right thing to do, I try to avoid them if
possible. This is because it is difficult to make assertions about the
actions executed within a ruby_block using chefspec (unit testing for
chef).

Thanks,

Brian

On 04/12/2013 19:06, "kallen@groknaut.net (mailto:kallen@groknaut.net)" <kallen@groknaut.net (mailto:kallen@groknaut.net)> wrote:

Hiya,

How do I make an LWRP that I created available in a recipe within a
ruby_block?
It's not working for me.

[2013-12-04T00:13:14+00:00] DEBUG: Re-raising exception: NoMethodError -
ruby_block[do stuff] (cloner::grants line 30) had an error:
NoMethodError: undefined method cloner_db_svc_mysqlacl' for Chef::Resource::RubyBlock /opt/chef/embedded/lib/ruby/gems/1.9.1/gems/chef-11.6.2/lib/chef/resource. rb:294:in method_missing'
/var/cache/chef/cookbooks/cloner/recipes/grants.rb:67:in `block (2
levels) in from_file'

/opt/chef/embedded/lib/ruby/gems/1.9.1/gems/chef-11.6.2/lib/chef/provider/
ruby_block.rb:33:in `call'

/opt/chef/embedded/lib/ruby/gems/1.9.1/gems/chef-11.6.2/lib/chef/provider/
ruby_block.rb:33:in `block in action_run'

/opt/chef/embedded/lib/ruby/gems/1.9.1/gems/chef-11.6.2/lib/chef/mixin/why
_run.rb:52:in `call'

/opt/chef/embedded/lib/ruby/gems/1.9.1/gems/chef-11.6.2/lib/chef/mixin/why
_run.rb:52:in `add_action'

/opt/chef/embedded/lib/ruby/gems/1.9.1/gems/chef-11.6.2/lib/chef/provider.
rb:149:in `converge_by'

/opt/chef/embedded/lib/ruby/gems/1.9.1/gems/chef-11.6.2/lib/chef/provider/
ruby_block.rb:32:in `action_run'

/opt/chef/embedded/lib/ruby/gems/1.9.1/gems/chef-11.6.2/lib/chef/provider.
rb:114:in `run_action'

/opt/chef/embedded/lib/ruby/gems/1.9.1/gems/chef-11.6.2/lib/chef/resource.
rb:625:in `run_action'
...

Afaict, the recipe code needs to execute during the convergence phase of
the
chef-client run. In my case, the code needs to (1) first start mysql, (2)
wait
for mysql to complete its startup (wait for innodb recovery to complete)
and (3)
then query a value from a table. (4) And then do another action depending
on the
value returned.

I tried doing the above outside a ruby_block my first time through, then
realized #3 was executing before #2 had completed. IOW #3 was executing
at the
compile phase. So now I'm trying to put all of this into a ruby_block,
but the
call to my LWRP is failing. (Is using a ruby_block for this the right
answer?)

The LWRP lives in 'cloner-db-svc' cookbook. The cookbook::recipe that
calls it
is 'cloner::grants'. The cloner cookbook depends on cloner-db-svc. When
the code
runs outside of a ruby_block, the recipe finds the LWRP method just fine,
but
the behavior I need doesn't occur.

::Chef::Recipe.send(:include, ::Cloner::Chef::DbSvc::Helper)

ruby_block "do stuff" do
block do
quux_ip = search(:node, 'roles:quux').map {|server|
server['ipaddress'] }.join(",")
cli_password = ::Cloner::Chef::DbSvc::Helper.fetchsecret( "secrets",
"#{node['cloner']['environment']}", "rootpass" )
current_ip = %x( mysql --skip-column-names -hlocalhost -uroot
-p#{cli_password} -e "SELECT Host FROM mysql.user WHERE
User='cloner_admin'" )

if current_ip.empty?
Chef::Log.debug("HIGGS-BOSON: current_ip is empty, applying
grants.")
cloner_admin_pass = ::Cloner::Chef::DbSvc::Helper.fetchsecret(
"secrets", "#{node['cloner']['environment']}", "cloner_admin_pass" )
cloner_proc_pass = ::Cloner::Chef::DbSvc::Helper.fetchsecret(
"secrets", "#{node['cloner']['environment']}", "cloner_proc_pass" )

cloner_db_svc_mysqlacl "mysqlgrants" do
action :create_acl
ipv4_address quux_ip
cli_password cli_password
cloner_admin_pass cloner_admin_pass
cloner_proc_pass cloner_proc_pass
end
current_ip = %x( mysql --skip-column-names -hlocalhost -uroot
-p#{cli_password} -e "SELECT Host FROM mysql.user WHERE
User='cloner_admin'" )
Chef::Log.debug("HIGGS-BOSON: After applying mysqlgrants, quux_ip
#{quux_ip}; current_ip #{current_ip}")

else # snipped for brevity
end
end
action :create
end

Thanks!
kallen