Why does my run fail wheh I use ignore_failure true?


#1

Why does my run fail when I use ignore_failure true?

I was under the impression that if I use ignore_failure and have a fail
will continue…

ec2-54-215-148-92.us-west-1.compute.amazonaws.com
ec2-54-215-148-92.us-west-1.compute.amazonaws.com
ec2-54-215-148-92.us-west-1.compute.amazonaws.com
ec2-54-215-148-92.us-west-1.compute.amazonaws.com
ec2-54-215-148-92.us-west-1.compute.amazonaws.com Relevant File Content:
ec2-54-215-148-92.us-west-1.compute.amazonaws.com
ec2-54-215-148-92.us-west-1.compute.amazonaws.com ----------------------
ec2-54-215-148-92.us-west-1.compute.amazonaws.com
ec2-54-215-148-92.us-west-1.compute.amazonaws.com/var/chef/cache/cookbooks/bootstrap/recipes/aws_monitor_tag.rb:
ec2-54-215-148-92.us-west-1.compute.amazonaws.com
ec2-54-215-148-92.us-west-1.compute.amazonaws.com 1: include_recipe "aws"
ec2-54-215-148-92.us-west-1.compute.amazonaws.com 2:
data_bag(“my_data_bag”)
ec2-54-215-148-92.us-west-1.compute.amazonaws.com 3: db =
data_bag_item(“my_data_bag”, “my”)
ec2-54-215-148-92.us-west-1.compute.amazonaws.com 4: aws = db[
node.chef_environment][‘aws’]
ec2-54-215-148-92.us-west-1.compute.amazonaws.com 5: AWS_ACCESS_KEY_ID =
aws[‘AWS_ACCESS_KEY_ID’]
ec2-54-215-148-92.us-west-1.compute.amazonaws.com 6:
AWS_SECRET_ACCESS_KEY = aws[‘AWS_SECRET_ACCESS_KEY’]
ec2-54-215-148-92.us-west-1.compute.amazonaws.com 7:
ec2-54-215-148-92.us-west-1.compute.amazonaws.com 8>> aws_resource_tag
node[‘ec2’][‘instance_id’] do
ec2-54-215-148-92.us-west-1.compute.amazonaws.com 9: aws_access_key
aws[‘AWS_ACCESS_KEY_ID’]
ec2-54-215-148-92.us-west-1.compute.amazonaws.com 10:
aws_secret_access_key aws[‘AWS_SECRET_ACCESS_KEY’]
ec2-54-215-148-92.us-west-1.compute.amazonaws.com 11: tags({“Name” =>
“Monitor”,
ec2-54-215-148-92.us-west-1.compute.amazonaws.com 12: “node_name”
=> “#{node.name}”,
ec2-54-215-148-92.us-west-1.compute.amazonaws.com 13:
“environment” => node.chef_environment})
ec2-54-215-148-92.us-west-1.compute.amazonaws.com 14: action :update
ec2-54-215-148-92.us-west-1.compute.amazonaws.com 15: ignore_failure
true
ec2-54-215-148-92.us-west-1.compute.amazonaws.com 16: end 17:
ec2-54-215-148-92.us-west-1.compute.amazonaws.com
ec2-54-215-148-92.us-west-1.compute.amazonaws.com
ec2-54-215-148-92.us-west-1.compute.amazonaws.com
ec2-54-215-148-92.us-west-1.compute.amazonaws.com
ec2-54-215-148-92.us-west-1.compute.amazonaws.com[2013-08-05T09:27:01+00:00]
ERROR: Running exception handlers
ec2-54-215-148-92.us-west-1.compute.amazonaws.com[2013-08-05T09:27:01+00:00]
ERROR: Exception handlers complete
ec2-54-215-148-92.us-west-1.compute.amazonaws.com[2013-08-05T09:27:01+00:00]
FATAL: Stacktrace dumped to
/var/chef/cache/chef-stacktrace.out
ec2-54-215-148-92.us-west-1.compute.amazonaws.com Chef Client failed. 0
resources updated
ec2-54-215-148-92.us-west-1.compute.amazonaws.com
ec2-54-215-148-92.us-west-1.compute.amazonaws.com[2013-08-05T09:27:01+00:00]
FATAL: Chef::Exceptions::ChildConvergeError:
Chef run process exited unsuccessfully (exit code 1)


#2

Can you paste a larger snippet of the failed run? It is possible it is
failing when it is parsing the code, not when it is running it, however it
is hard to tell from this message.

On Mon, Aug 5, 2013 at 5:31 AM, David Montgomery
davidmontgomery@gmail.comwrote:

Why does my run fail when I use ignore_failure true?

I was under the impression that if I use ignore_failure and have a fail
will continue…

ec2-54-215-148-92.us-west-1.compute.amazonaws.com
ec2-54-215-148-92.us-west-1.compute.amazonaws.com
ec2-54-215-148-92.us-west-1.compute.amazonaws.com
ec2-54-215-148-92.us-west-1.compute.amazonaws.com
ec2-54-215-148-92.us-west-1.compute.amazonaws.com Relevant File Content:
ec2-54-215-148-92.us-west-1.compute.amazonaws.com
ec2-54-215-148-92.us-west-1.compute.amazonaws.com ----------------------
ec2-54-215-148-92.us-west-1.compute.amazonaws.com
ec2-54-215-148-92.us-west-1.compute.amazonaws.com/var/chef/cache/cookbooks/bootstrap/recipes/aws_monitor_tag.rb:
ec2-54-215-148-92.us-west-1.compute.amazonaws.com
ec2-54-215-148-92.us-west-1.compute.amazonaws.com 1: include_recipe
"aws"
ec2-54-215-148-92.us-west-1.compute.amazonaws.com 2:
data_bag(“my_data_bag”)
ec2-54-215-148-92.us-west-1.compute.amazonaws.com 3: db =
data_bag_item(“my_data_bag”, “my”)
ec2-54-215-148-92.us-west-1.compute.amazonaws.com 4: aws = db[
node.chef_environment][‘aws’]
ec2-54-215-148-92.us-west-1.compute.amazonaws.com 5: AWS_ACCESS_KEY_ID
= aws[‘AWS_ACCESS_KEY_ID’]
ec2-54-215-148-92.us-west-1.compute.amazonaws.com 6:
AWS_SECRET_ACCESS_KEY = aws[‘AWS_SECRET_ACCESS_KEY’]
ec2-54-215-148-92.us-west-1.compute.amazonaws.com 7:
ec2-54-215-148-92.us-west-1.compute.amazonaws.com 8>> aws_resource_tag
node[‘ec2’][‘instance_id’] do
ec2-54-215-148-92.us-west-1.compute.amazonaws.com 9: aws_access_key
aws[‘AWS_ACCESS_KEY_ID’]
ec2-54-215-148-92.us-west-1.compute.amazonaws.com 10:
aws_secret_access_key aws[‘AWS_SECRET_ACCESS_KEY’]
ec2-54-215-148-92.us-west-1.compute.amazonaws.com 11: tags({“Name” =>
“Monitor”,
ec2-54-215-148-92.us-west-1.compute.amazonaws.com 12:
“node_name” => “#{node.name}”,
ec2-54-215-148-92.us-west-1.compute.amazonaws.com 13:
“environment” => node.chef_environment})
ec2-54-215-148-92.us-west-1.compute.amazonaws.com 14: action :update
ec2-54-215-148-92.us-west-1.compute.amazonaws.com 15: ignore_failure
true
ec2-54-215-148-92.us-west-1.compute.amazonaws.com 16: end 17:
ec2-54-215-148-92.us-west-1.compute.amazonaws.com
ec2-54-215-148-92.us-west-1.compute.amazonaws.com
ec2-54-215-148-92.us-west-1.compute.amazonaws.com
ec2-54-215-148-92.us-west-1.compute.amazonaws.com
ec2-54-215-148-92.us-west-1.compute.amazonaws.com[2013-08-05T09:27:01+00:00] ERROR: Running exception handlers
ec2-54-215-148-92.us-west-1.compute.amazonaws.com[2013-08-05T09:27:01+00:00] ERROR: Exception handlers complete
ec2-54-215-148-92.us-west-1.compute.amazonaws.com[2013-08-05T09:27:01+00:00] FATAL: Stacktrace dumped to
/var/chef/cache/chef-stacktrace.out
ec2-54-215-148-92.us-west-1.compute.amazonaws.com Chef Client failed. 0
resources updated
ec2-54-215-148-92.us-west-1.compute.amazonaws.com
ec2-54-215-148-92.us-west-1.compute.amazonaws.com[2013-08-05T09:27:01+00:00] FATAL: Chef::Exceptions::ChildConvergeError:
Chef run process exited unsuccessfully (exit code 1)


#3

On Monday, August 5, 2013 at 7:40 AM, Andrew Gross wrote:

Can you paste a larger snippet of the failed run? It is possible it is failing when it is parsing the code, not when it is running it, however it is hard to tell from this message.

This is definitely the case:
(snip…)

ec2-54-215-148-92.us-west-1.compute.amazonaws.com (http://ec2-54-215-148-92.us-west-1.compute.amazonaws.com) 8>> aws_resource_tag node[‘ec2’][‘instance_id’] do
Your error is in the compile phase right there. Probably you’re on VPC or a new EC2 account that is “VPC-ish”, so ohai can’t automatically detect EC2.

See: http://www.opscode.com/blog/2012/05/30/ohai-6-14-0-released/
And, when the ticket system comes back, see OHAI-310.

To create the ec2 “hint”, create /etc/chef/ohai/hints/ec.json (doesn’t need any content). The knife-ec2 plugin does this automatically, if you’re using a different provisioning workflow, you’ll probably want to add a step to create that file.


Daniel DeLeo


#4

To answer the larger question, “Why does my Chef run fail if I set
ignore_failure to true on a resource?”

Chef runs will fail if an exception is raised. Exceptions can be raised
within a resource provider, or they can be raised in the Ruby code you
write in between declaring resources.

Common culprits (in my experience):

  • Making assumptions about nested node data being present. (For example,
    ‘if node[:foo][:bar] == “baz”’ fails because node[:foo] isn’t defined)
  • Running external code (including Chef searches and data bag lookups!)
    without a rescue block.
  • Attempting to load a gem that isn’t installed on the system (yet, or
    ever).

Foodcritic rules for these situations would be awesome and I’d love to
write them, but looking into the inner workings of the Ruby AST parser code
gives me a serious case of brain-fry every time…

Hope this helps the OP understand the general situation (or, failing that,
helps someone Googling this question :smiley: ) …

On Mon, Aug 5, 2013 at 8:14 AM, Daniel DeLeo dan@kallistec.com wrote:

On Monday, August 5, 2013 at 7:40 AM, Andrew Gross wrote:

Can you paste a larger snippet of the failed run? It is possible it is
failing when it is parsing the code, not when it is running it, however it
is hard to tell from this message.

This is definitely the case:
(snip…)

ec2-54-215-148-92.us-west-1.compute.amazonaws.com 8>> aws_resource_tag
node[‘ec2’][‘instance_id’] do\

Your error is in the compile phase right there. Probably you’re on VPC or
a new EC2 account that is “VPC-ish”, so ohai can’t automatically detect
EC2.

See: http://www.opscode.com/blog/2012/05/30/ohai-6-14-0-released/
And, when the ticket system comes back, see OHAI-310.

To create the ec2 “hint”, create /etc/chef/ohai/hints/ec.json (doesn’t
need any content). The knife-ec2 plugin does this automatically, if you’re
using a different provisioning workflow, you’ll probably want to add a step
to create that file.


Daniel DeLeo


#5

Foodcritic rules for these situations would be awesome and I’d love to write
them, but looking into the inner workings of the Ruby AST parser code gives
me a serious case of brain-fry every time…
OT: This could be interesting, although I’m not certain how these
scenarios might be implemented as checks, since not every gem is named
the same as its packaging, et al.
Another thread on this topic of FC custom/extended rules might be useful.

-M

On Mon, Aug 5, 2013 at 1:19 PM, steve . leftathome@gmail.com wrote:

To answer the larger question, “Why does my Chef run fail if I set
ignore_failure to true on a resource?”

Chef runs will fail if an exception is raised. Exceptions can be raised
within a resource provider, or they can be raised in the Ruby code you write
in between declaring resources.

Common culprits (in my experience):

  • Making assumptions about nested node data being present. (For example,
    ‘if node[:foo][:bar] == “baz”’ fails because node[:foo] isn’t defined)
  • Running external code (including Chef searches and data bag lookups!)
    without a rescue block.
  • Attempting to load a gem that isn’t installed on the system (yet, or
    ever).

Foodcritic rules for these situations would be awesome and I’d love to write
them, but looking into the inner workings of the Ruby AST parser code gives
me a serious case of brain-fry every time…

Hope this helps the OP understand the general situation (or, failing that,
helps someone Googling this question :smiley: ) …

On Mon, Aug 5, 2013 at 8:14 AM, Daniel DeLeo dan@kallistec.com wrote:

On Monday, August 5, 2013 at 7:40 AM, Andrew Gross wrote:

Can you paste a larger snippet of the failed run? It is possible it is
failing when it is parsing the code, not when it is running it, however it
is hard to tell from this message.

This is definitely the case:
(snip…)

ec2-54-215-148-92.us-west-1.compute.amazonaws.com 8>> aws_resource_tag
node[‘ec2’][‘instance_id’] do\

Your error is in the compile phase right there. Probably you’re on VPC or
a new EC2 account that is “VPC-ish”, so ohai can’t automatically detect EC2.

See: http://www.opscode.com/blog/2012/05/30/ohai-6-14-0-released/
And, when the ticket system comes back, see OHAI-310.

To create the ec2 “hint”, create /etc/chef/ohai/hints/ec.json (doesn’t
need any content). The knife-ec2 plugin does this automatically, if you’re
using a different provisioning workflow, you’ll probably want to add a step
to create that file.


Daniel DeLeo


#6

Thats the the thing…I am using the /etc/chef/ohai/hints/
ec.json in a previous recipe before I try and add the tag. It did work on
an existing server.

On Mon, Aug 5, 2013 at 11:14 PM, Daniel DeLeo dan@kallistec.com wrote:

On Monday, August 5, 2013 at 7:40 AM, Andrew Gross wrote:

Can you paste a larger snippet of the failed run? It is possible it is
failing when it is parsing the code, not when it is running it, however it
is hard to tell from this message.

This is definitely the case:
(snip…)

ec2-54-215-148-92.us-west-1.compute.amazonaws.com 8>> aws_resource_tag
node[‘ec2’][‘instance_id’] do\

Your error is in the compile phase right there. Probably you’re on VPC or
a new EC2 account that is “VPC-ish”, so ohai can’t automatically detect
EC2.

See: http://www.opscode.com/blog/2012/05/30/ohai-6-14-0-released/
And, when the ticket system comes back, see OHAI-310.

To create the ec2 “hint”, create /etc/chef/ohai/hints/ec.json (doesn’t
need any content). The knife-ec2 plugin does this automatically, if you’re
using a different provisioning workflow, you’ll probably want to add a step
to create that file.


Daniel DeLeo


#7

You mean /etc/chef/ohai/hints/ec2.json right?

Thanks,
Matt Ray
Cloud Integrations Product Lead :: Opscodehttp://opscode.com
512.731.2218 :: matt@opscode.com
mattray :: GitHubhttp://github.com/mattray/ :: IRC :: Twitterhttp://twitter.com/mattray


From: David Montgomery davidmontgomery@gmail.com
Sent: Monday, August 05, 2013 1:15 PM
To: chef@lists.opscode.com
Subject: [chef] Re: Re: Re: Why does my run fail wheh I use ignore_failure true?

Thats the the thing…I am using the /etc/chef/ohai/hints/
ec.json in a previous recipe before I try and add the tag. It did work on an existing server.

On Mon, Aug 5, 2013 at 11:14 PM, Daniel DeLeo <dan@kallistec.commailto:dan@kallistec.com> wrote:
On Monday, August 5, 2013 at 7:40 AM, Andrew Gross wrote:
Can you paste a larger snippet of the failed run? It is possible it is failing when it is parsing the code, not when it is running it, however it is hard to tell from this message.
This is definitely the case:
(snip…)
ec2-54-215-148-92.us-west-1.compute.amazonaws.comhttp://ec2-54-215-148-92.us-west-1.compute.amazonaws.com 8>> aws_resource_tag node[‘ec2’][‘instance_id’] do
Your error is in the compile phase right there. Probably you’re on VPC or a new EC2 account that is “VPC-ish”, so ohai can’t automatically detect EC2.

See: http://www.opscode.com/blog/2012/05/30/ohai-6-14-0-released/
And, when the ticket system comes back, see OHAI-310.

To create the ec2 “hint”, create /etc/chef/ohai/hints/ec.json (doesn’t need any content). The knife-ec2 plugin does this automatically, if you’re using a different provisioning workflow, you’ll probably want to add a step to create that file.


Daniel DeLeo