RE: Re: override argument

Chef Infra (archive)
1

Did this behavior change in Chef Client 11?

We're still stuck on Chef 10 and the major gotcha we face with the use of -o is that all the attributes from other cookbooks disappear, disrupting node searches and such, just like you say. Essentially we've found that while -o might come in useful, it should generally be followed by a full run ASAP to prevent the corruption of the attributes and search-based relationships. As such we have structured the recipes which are generally -o material for us (recipes that do destructive upgrades, for example) such that they always include_recipe some top-level recipe at the end, forcing a more complete run to happen as part of the -o. More restrictive use of -o for "partial runs" has always caused more pain then it is worth for us.

In Chef 10 all the attributes get dropped at the beginning of the run, and then any new attributes from cookbooks that execute do get saved to the server AFAICT. When I eventually upgrade to Chef Client 11, can I expect all of my original attributes to remain unchanged after a -o run? That would be awesome except for the cases where we do a node.set in the recipe that is commonly used with -o....

-----Original Message-----
From: Noah Kantrowitz [mailto:noah@coderanger.net]
Sent: Tuesday, June 17, 2014 4:16 PM
To: chef@lists.opscode.com
Subject: [chef] Re: RE: Re: override argument

No, it is not amenable for general use. The biggest issue that override mode disables saving the node data back up to the server. This is done to avoid polluting the data on the server as not all recipes were executed, which might disrupt node searches and such. If you want to make conditional recipes, you can do that with something like:

return if node['skip_foo']

at the top of the recipe file.

--Noah

On Jun 17, 2014, at 9:47 AM, Florian Hehlen Florian.Hehlen@mri-group.com wrote:

HI,

Thank you Indra. I am aware of the feature. I am just wondering if it's safe to use -o or is it really only for testing.

Florian

From: Indra k [mailto:indra.k@cloudenablers.com]
Sent: 17 June 2014 13:10
To: chef@lists.opscode.com
Subject: [chef] Re: override argument

Hi Flo,

    Ya, it gets run.

chef-client -o "recipe[something]"

the above command runs a chef-client with given runlist .

You can also refer this by

"chef-client -h" ---which shows a what are all parameters we can run with chef-client.

Thanks,
Indra

From: Florian Hehlen Florian.Hehlen@mri-group.com
To: "chef@lists.opscode.com" chef@lists.opscode.com
Sent: Tuesday, 17 June 2014 3:42 PM
Subject: [chef] override argument

Hi,

What is the contract for the -o argument on chef-client? Can I expect that the only thing that changes between running 'chef-client' and 'chef-client -o "recipe[something]"', is the recipes that will get run?

Florian Hehlen

LEGAL DISCLAIMER
This communication and any attached documents are strictly confidential and/or legally privileged and they may not be used or disclosed by someone who is not a named recipient. If you have received this electronic communication in error please notify the sender by replying to this electronic communication inserting the word "misdirected" as the subject and delete this communication from your system.

LEGAL DISCLAIMER
This communication and any attached documents are strictly confidential and/or legally privileged and they may not be used or disclosed by someone who is not a named recipient. If you have received this electronic communication in error please notify the sender by replying to this electronic communication inserting the word "misdirected" as the subject and delete this communication from your system.

2

On Tuesday, June 17, 2014 at 2:26 PM, Chris Sibbitt wrote:

Did this behavior change in Chef Client 11?

We removed the node save in a recent 11.x release.

We're still stuck on Chef 10 and the major gotcha we face with the use of -o is that all the attributes from other cookbooks disappear, disrupting node searches and such, just like you say. Essentially we've found that while -o might come in useful, it should generally be followed by a full run ASAP to prevent the corruption of the attributes and search-based relationships. As such we have structured the recipes which are generally -o material for us (recipes that do destructive upgrades, for example) such that they always include_recipe some top-level recipe at the end, forcing a more complete run to happen as part of the -o. More restrictive use of -o for "partial runs" has always caused more pain then it is worth for us.

In Chef 10 all the attributes get dropped at the beginning of the run, and then any new attributes from cookbooks that execute do get saved to the server AFAICT. When I eventually upgrade to Chef Client 11, can I expect all of my original attributes to remain unchanged after a -o run? That would be awesome except for the cases where we do a node.set in the recipe that is commonly used with -o....

You can manually call node.save, but of course that puts you back where you started...

--
Daniel DeLeo

3

Hi.

Thanks for all the feedback. What is then the general practice that people follow with application life-cycle management?

Imagine you have the following recipes:

  • app::default (does maintenance things like rolling logs, restarting app if down)
  • app::install
  • app::upgrade
  • app::patch_db
  • app::backup
  • app::restore
  • app::clean

It makes no sense to run everything all of the time. Especially, if I am running chef-client every 15 minutes to monitor the node, which I don't do but I would like to. How do I run different combinations of these at different times? This is also really important in DEV or TEST environments where we constantly use the different combinations of recipes. At the moment I mostly rely on -o to do this but I have noticed that it does not always go as expected (which prompted this thread). So I don't use -o in prod but rather adjust the list of recipes on the node when I need to. This is clunky and dangerous I think a node's definition should be changed as rarely as possible.

Florian

-----Original Message-----
From: Daniel DeLeo [mailto:ddeleo@kallistec.com] On Behalf Of Daniel DeLeo
Sent: 18 June 2014 00:14
To: chef@lists.opscode.com
Subject: [chef] Re: RE: Re: override argument

On Tuesday, June 17, 2014 at 2:26 PM, Chris Sibbitt wrote:

Did this behavior change in Chef Client 11?

We removed the node save in a recent 11.x release.

We're still stuck on Chef 10 and the major gotcha we face with the use of -o is that all the attributes from other cookbooks disappear, disrupting node searches and such, just like you say. Essentially we've found that while -o might come in useful, it should generally be followed by a full run ASAP to prevent the corruption of the attributes and search-based relationships. As such we have structured the recipes which are generally -o material for us (recipes that do destructive upgrades, for example) such that they always include_recipe some top-level recipe at the end, forcing a more complete run to happen as part of the -o. More restrictive use of -o for "partial runs" has always caused more pain then it is worth for us.

In Chef 10 all the attributes get dropped at the beginning of the run, and then any new attributes from cookbooks that execute do get saved to the server AFAICT. When I eventually upgrade to Chef Client 11, can I expect all of my original attributes to remain unchanged after a -o run? That would be awesome except for the cases where we do a node.set in the recipe that is commonly used with -o....

You can manually call node.save, but of course that puts you back where you started...

--
Daniel DeLeo

LEGAL DISCLAIMER
This communication and any attached documents are strictly confidential and/or legally privileged and they may not be used or disclosed by someone who is not a named recipient. If you have received this electronic communication in error please notify the sender by replying to this electronic communication inserting the word "misdirected" as the subject and delete this communication from your system.

4

On a mobile keyboard so can't write as much as I would like, but this is fundamentally the wrong way to model app deployment with chef. Chef is a convergent system and so you need to thnk in terms of convergent units, not procedural steps like you have there. If you need additional procedural orchestration on top of the convergent bits, and you usually will, there are great tools for that like Fabric, Rundeck, or Opscode's own push jobs. These two pieces work together to manage deployments, chef converges state, fabric (et al) orchestrates procedural steps. --Noah

Hi.

Thanks for all the feedback. What is then the general practice that people follow with application life-cycle management?

Imagine you have the following recipes:

  • app::default (does maintenance things like rolling logs, restarting app if down)

  • app::install

  • app::upgrade

  • app::patch_db

  • app::backup

  • app::restore

  • app::clean

It makes no sense to run everything all of the time. Especially, if I am running chef-client every 15 minutes to monitor the node, which I don't do but I would like to. How do I run different combinations of these at different times? This is also really important in DEV or TEST environments where we constantly use the different combinations of recipes. At the moment I mostly rely on -o to do this but I have noticed that it does not always go as expected (which prompted this thread). So I don't use -o in prod but rather adjust the list of recipes on the node when I need to. This is clunky and dangerous I think a node's definition should be changed as rarely as possible.

Florian

-----Original Message-----

From: Daniel DeLeo [mailto:ddeleo@kallistec.com] On Behalf Of Daniel DeLeo

Sent: 18 June 2014 00:14

To: chef@lists.opscode.com

Subject: [chef] Re: RE: Re: override argument

On Tuesday, June 17, 2014 at 2:26 PM, Chris Sibbitt wrote:

Did this behavior change in Chef Client 11?

We removed the node save in a recent 11.x release.

We're still stuck on Chef 10 and the major gotcha we face with the use of -o is that all the attributes from other cookbooks disappear, disrupting node searches and such, just like you say. Essentially we've found that while -o might come in useful, it should generally be followed by a full run ASAP to prevent the corruption of the attributes and search-based relationships. As such we have structured the recipes which are generally -o material for us (recipes that do destructive upgrades, for example) such that they always include_recipe some top-level recipe at the end, forcing a more complete run to happen as part of the -o. More restrictive use of -o for "partial runs" has always caused more pain then it is worth for us.

In Chef 10 all the attributes get dropped at the beginning of the run, and then any new attributes from cookbooks that execute do get saved to the server AFAICT. When I eventually upgrade to Chef Client 11, can I expect all of my original attributes to remain unchanged after a -o run? That would be awesome except for the cases where we do a node.set in the recipe that is commonly used with -o....

You can manually call node.save, but of course that puts you back where you started...

--

Daniel DeLeo

LEGAL DISCLAIMER

This communication and any attached documents are strictly confidential and/or legally privileged and they may not be used or disclosed by someone who is not a named recipient. If you have received this electronic communication in error please notify the sender by replying to this electronic communication inserting the word "misdirected" as the subject and delete this communication from your system.

5

Noah,

I see your point. I am not sure I agree though. You seem to have a view that state convergence stops at a certain point. I think the only recipes below that aren’t about state convergence are ‘default’ and ‘backup’. So why should I use Chef to install/upgrade an OS level package, or a database, or a JVM but not to install/upgrade my application?

One way out of my problem is to have logical switches, activated through attributes, which will cause certain recipes to happen only during certain runs. That just seems a bit clunky and error-prone when some of the actions should only be done in very special cases.

I also find that Chef provides all the paradigms I need for managing an application’s life-cycle. Why add one more layer of indirection with Fabric, Rundeck, etal?

Florian

From: Noah Kantrowitz [mailto:noah@coderanger.net]
Sent: 18 June 2014 09:17
To: chef@lists.opscode.com
Subject: [chef] RE: Re: RE: Re: override argument

On a mobile keyboard so can't write as much as I would like, but this is fundamentally the wrong way to model app deployment with chef. Chef is a convergent system and so you need to thnk in terms of convergent units, not procedural steps like you have there. If you need additional procedural orchestration on top of the convergent bits, and you usually will, there are great tools for that like Fabric, Rundeck, or Opscode's own push jobs. These two pieces work together to manage deployments, chef converges state, fabric (et al) orchestrates procedural steps.

--Noah
On June 18, 2014 at 12:10:03 AM PDT, Florian Hehlen <Florian.Hehlen@mri-group.commailto:Florian.Hehlen@mri-group.com> wrote:

Hi.

Thanks for all the feedback. What is then the general practice that people follow with application life-cycle management?

Imagine you have the following recipes:

  • app::default (does maintenance things like rolling logs, restarting app if down)
  • app::install
  • app::upgrade
  • app::patch_db
  • app::backup
  • app::restore
  • app::clean

It makes no sense to run everything all of the time. Especially, if I am running chef-client every 15 minutes to monitor the node, which I don't do but I would like to. How do I run different combinations of these at different times? This is also really important in DEV or TEST environments where we constantly use the different combinations of recipes. At the moment I mostly rely on -o to do this but I have noticed that it does not always go as expected (which prompted this thread). So I don't use -o in prod but rather adjust the list of recipes on the node when I need to. This is clunky and dangerous I think a node's definition should be changed as rarely as possible.

Florian

-----Original Message-----
From: Daniel DeLeo [mailto:ddeleo@kallistec.com] On Behalf Of Daniel DeLeo
Sent: 18 June 2014 00:14
To: chef@lists.opscode.commailto:chef@lists.opscode.com
Subject: [chef] Re: RE: Re: override argument

On Tuesday, June 17, 2014 at 2:26 PM, Chris Sibbitt wrote:

Did this behavior change in Chef Client 11?

We removed the node save in a recent 11.x release.

We're still stuck on Chef 10 and the major gotcha we face with the use of -o is that all the attributes from other cookbooks disappear, disrupting node searches and such, just like you say. Essentially we've found that while -o might come in useful, it should generally be followed by a full run ASAP to prevent the corruption of the attributes and search-based relationships. As such we have structured the recipes which are generally -o material for us (recipes that do destructive upgrades, for example) such that they always include_recipe some top-level recipe at the end, forcing a more complete run to happen as part of the -o. More restrictive use of -o for "partial runs" has always caused more pain then it is worth for us.

In Chef 10 all the attributes get dropped at the beginning of the run, and then any new attributes from cookbooks that execute do get saved to the server AFAICT. When I eventually upgrade to Chef Client 11, can I expect all of my original attributes to remain unchanged after a -o run? That would be awesome except for the cases where we do a node.set in the recipe that is commonly used with -o....

You can manually call node.save, but of course that puts you back where you started...

--
Daniel DeLeo

LEGAL DISCLAIMER
This communication and any attached documents are strictly confidential and/or legally privileged and they may not be used or disclosed by someone who is not a named recipient. If you have received this electronic communication in error please notify the sender by replying to this electronic communication inserting the word "misdirected" as the subject and delete this communication from your system.

LEGAL DISCLAIMER
This communication and any attached documents are strictly confidential and/or legally privileged and they may not be used or disclosed by someone who is not a named recipient. If you have received this electronic communication in error please notify the sender by replying to this electronic communication inserting the word "misdirected" as the subject and delete this communication from your system.