Problem with chef-solo

Hi,

I’m currently only using chef-solo for deployment and found an issue:

It seems chef-solo doesn’t clear the cookbook directory before unpacking
the new cookbook retrieved via http.

This means if a file is removed from e.g. site-cookbooks in the remote
bundle, it will not be removed before chef-solo runs.

Should I file a ticket for this?

/Jeppe

Chef solo doesn't communicate with a remote server. Therefore any
changes on the server have no affect. Use chef client if you want
remote changes to be reflected on the node.

Sent from my phone

On Feb 27, 2010, at 6:58 AM, Jeppe Nejsum Madsen jeppe@ingolfs.dk
wrote:

Hi,

I'm currently only using chef-solo for deployment and found an issue:

It seems chef-solo doesn't clear the cookbook directory before
unpacking
the new cookbook retrieved via http.

This means if a file is removed from e.g. site-cookbooks in the remote
bundle, it will not be removed before chef-solo runs.

Should I file a ticket for this?

/Jeppe

Yep - if you are usig solo, it's up to you to clean up after yourself.

Sent from my iPhone

On Feb 27, 2010, at 11:08 AM, Alex Soto apsoto@gmail.com wrote:

Chef solo doesn't communicate with a remote server. Therefore any
changes on the server have no affect. Use chef client if you want
remote changes to be reflected on the node.

Sent from my phone

On Feb 27, 2010, at 6:58 AM, Jeppe Nejsum Madsen jeppe@ingolfs.dk
wrote:

Hi,

I'm currently only using chef-solo for deployment and found an issue:

It seems chef-solo doesn't clear the cookbook directory before
unpacking
the new cookbook retrieved via http.

This means if a file is removed from e.g. site-cookbooks in the
remote
bundle, it will not be removed before chef-solo runs.

Should I file a ticket for this?

/Jeppe

On Sat, Feb 27, 2010 at 10:59 PM, Adam Jacob adam@opscode.com wrote:

Yep - if you are usig solo, it's up to you to clean up after yourself.

Hmm it's not so much cleaning up after my self as making sure I get
what I ask for :slight_smile:

if I do a chef-solo -r http://somewhere

it downloads and extracts the cookbooks from a tgz file. But if files
exists in the cache dir that are not in the download cookbook (say,
from a previous run of chef-solo or because it's monday or....) those
older files will still be used in the new run.

Now that I know what the problem is, it's not a big deal to do an rm
-rf before running, but is there a reason why chef-solo can't just
wipe the file_cache_path before extracting the cookbooks?

/Jeppe

On Sat, Feb 27, 2010 at 2:13 PM, Jeppe Nejsum Madsen jeppe@ingolfs.dk wrote:

Now that I know what the problem is, it's not a big deal to do an rm
-rf before running, but is there a reason why chef-solo can't just
wipe the file_cache_path before extracting the cookbooks?

Hrm - I kind of look at solo as being pretty heavy on the 'roll your
own' end of things, in terms of exactly what you want the behavior to
be. It would be consistent with the way the chef-client works to go
ahead and wipe the cache first, but I can see a world where you want
to keep cookbooks around (for example, you want to actually pull from
multiple sources, and clean up after.)

I think we would be happy to accept a patch that lets this be a
configuration/command line option.

Adam

--
Opscode, Inc.
Adam Jacob, CTO
T: (206) 508-7449 E: adam@opscode.com

On Sun, Feb 28, 2010 at 12:41 AM, Adam Jacob adam@opscode.com wrote:

ahead and wipe the cache first, but I can see a world where you want
to keep cookbooks around (for example, you want to actually pull from
multiple sources, and clean up after.)

I think we would be happy to accept a patch that lets this be a
configuration/command line option.

Ah-ha!

Right. Well I also have a related feature to suggest. Its a case of
wanting to know whether you guys would accept a patch for this
particular feature. And if so, how the feature should be written
exactly.

Feature:
To make github tarballs unpacking supported with the chef-solo -r command.

Just look at this:

$ mkdir homebrew
$ curl -L http://github.com/mxcl/homebrew/tarball/master | tar xz
--strip 1 -C homebrew

...its easy!

So:
$ chef-solo -r "http://github.com/username/site-cookbooks/tarball/<branch_or_tag_name>"

Could fetch directly from github, and extract correctly as per usual
into "/tmp/cookbooks", or "/tmp/site-cookbooks".

Three reasons I believe this feature is worth having:

  • Nearly everybody already use github for their cookbooks.
  • It avoids any extra step realeasing your cookbooks into special
    tarballs onto amazon S3 or somewhere else.
  • This approach can also fully cater for your "multiple sources"
    scenario Holoway. Just pull in those sparse cookbooks together into a
    git branch (perhaps something we already do as part of our regular
    chef workflows?)

Needs clarification:
How should a user should be able to specify the downloading of both of
the cookbooks? That is to say, when we want "opscode/cookbooks" aswell
as "site-cookbooks" (because within site-cookbooks we may be just
overriding the odd file).

This i think its also useful for those people who want to gaurantee
having everything reproduced exactly the same as per their chef-server
configuration. Or just don't want the hassle of figuring out which
cookbooks to download. The way ive been doing it is to download them
beforehand with git, or sync them over ssh with unison. But it would
be just so much better to have it all done for me by the
chef-solo -r command. Like some kind of jedi-trick. Just downright
bling.

I don't believe this particular strategy has to necessarily mean there
are too many cookbooks being downloaded. For example all of
opscode/cookbooks is 2.4 megabytes, as a tarball. And its pretty
straightforward make up a seperate git for solo if you want to trim
things down to just whats necessary.

One more thing we might to check whether the commit for that url has
changed. To decide whether its necessary to download the tarball
again. But that should be pretty easy to do with github and standard
http requests.

Guess i should ask:
Had this feature been considered before? What was the conclusion?

dreamcat4
dreamcat4@gmail.com