# Thinking out loud

**URL:** <https://discourse.chef.io/t/thinking-out-loud/1239>\
**Category:** Chef Infra (archive)\
**Created:** [July 24, 2009, 7:58am UTC](https://discourse.chef.io/t/thinking-out-loud/1239 "2009-07-24T07:58:14Z")\
**Posts on this page:** 15\
**Page:** 1

<div class="post-metadata">

**Author:** ![Graeme\_Mathieson](https://sea2.discourse-cdn.com/flex016/user_avatar/discourse.chef.io/graeme_mathieson/32/914_2.png) [@Graeme\_Mathieson](https://discourse.chef.io/u/Graeme_Mathieson)\
**Post date:** [July 24, 2009, 7:58am UTC](https://discourse.chef.io/t/thinking-out-loud/1239/1 "2009-07-24T07:58:14Z")

</div>

Hi,

I’ve been messing with Chef for the past week and this morning I wrote  
out some notes on our private wiki about things I’d like to enhance/  
do. This is totally off my head and some of it is almost certainly as  
a result of missing out key bits of understanding. After writing it,  
I figured, why not share it, so here goes:

- Some sort of rspec/test-driven mechanism. At the very least, this  
should properly sanity check all the cookbooks, looking for typo’d  
attribute hashes, missing recipes and logic errors. I’m thinking that  
this could be built fairly simply just by stubbing parts of the  
framework and running each of the recipes. Second goal: get autospec  
working for it. Third goal: rspec matchers suitable for chef.

- Remove the needless verbosity of the attribute definitions, like  
creating new Mashes, and only defining attributes unless they already  
exist. It’s just messy.

- Remove the duplication in the metadata. While enhancing the  
attribute definitions, tweak them so that they include enough  
information to allow the metadata to be automatically generated.  
Dependencies can be calculated instead of specifying them manually.

- Allow cookbooks to be distributed as gems with gem dependencies on  
other cookbooks.

- Allow vendoring of these gems, because I don’t like relying on  
system gems, even in this sort of environment. Bootstrapping is key.

- Get rubigen to have generators for an empty repo, individual  
cookbooks, that kind of thing. In general, make the repo  
infrastructure feel more like a Rails app, just because that’s a good  
example of an app development environment. Individual recipes can be  
generated with: {{script/generate recipe ruby/gems}} which would  
automatically create the ‘ruby’ cookbook too, if it doesn’t already  
exist.

- A thin bootstrap tarball of chef solo that doesn’t rely on anything  
other than ruby being installed on the client system. No rubygems,  
nowt. That way, we can handle everything else through chef and have  
real bare-bones system images.

- Some sort of external, persistent information about nodes as well as  
roles in the repo. I kinda want the chef server to be easily  
replaceable, and everything about our infrastructure to be in version  
control. I can’t quite see how to do that with nodes yet, but perhaps  
I’m just missing something.

- Bootstrapping a node from a pristine image, pulling all its  
information from an external system. I’m (so far) not liking the idea  
of creating a node by passing in some JSON when it boots. It should  
boot then get that information from a persistent store so that, next  
time it’s imaged from afresh, it can start back where it left off  
without me having to separately document the JSON is was passed in on  
bootup. I think this is a repetition of the previous point. 🙂

- The equivalent of dpkg’s “Provides” for providing alternative  
implementations of packages. Say, for example, I have a  
ruby\_enterprise\_edition cookbook, which {{provides “ruby”}}. If  
another cookbook says {{include\_recipe “ruby”}} and we’ve already seen  
ruby\_enterprise\_edition, it will satisfy that dependency.

h3. Cookbooks I want

- Percona’s build of MySQL [[http://www.percona.com/mysql/5.0.82-b16/](http://www.percona.com/mysql/5.0.82-b16/)  
deb/]
- RabbitMQ
- ActiveMQ (until I make sure I can port our apps that use ActiveMQ  
across to a single message bus…)
- Confluence
- Bamboo
- Ruby Enterprise Edition with dpkg-diverts to neuter the existing  
ruby package, while leaving it around to satisfy dependencies…
- Variants of all the existing recipes that don’t use runit! (What can  
I say? I like consistency, and having a mixture of init.d and runit  
services isn’t consistent. And out of band monitoring with God is  
easier than shifting _everything_ across to runit…)
- Nanite infrastructure.

h3. Stuff I want

This is stuff that I want to implement, but I’m not yet sure whether  
it’s best done with chef, nanite or something else entirely.

- Rolling out software updates (in particular, package security  
updates) in a timely manner across our infrastructure, after having  
tested on our staging infrastructure.
- Upgrading an EC2 instance to a bigger instance type. Automagically.
- An overarching DSL for describing our infrastructure, from which all  
the node stuff is generated and deployed. What I’d _like_ to do is  
"draw" our infrastructure in Graphviz dot notation and for that to  
turn into physical, working, connected infrastructure. That’s my  
nearly-end-goal.

The end goal is to be able to sketch out these diagrams while relaxing  
on a beach in Malibu and hand them to a minion who transcribes them to  
dot notation and for that to turn into physical, working, connected  
infrastructure. 🙂

Cheers,

Graeme.

---

<div class="post-metadata">

**Author:** ![Adam\_Jacob](https://sea2.discourse-cdn.com/flex016/user_avatar/discourse.chef.io/adam_jacob/32/292_2.png) [@Adam\_Jacob](https://discourse.chef.io/u/Adam_Jacob)\
**Post date:** [July 25, 2009, 7:24pm UTC](https://discourse.chef.io/t/thinking-out-loud/1239/2 "2009-07-25T19:24:36Z")

</div>

On Fri, Jul 24, 2009 at 12:58 AM, Graeme Mathieson[mathie@woss.name](mailto:mathie@woss.name) wrote:

> I've been messing with Chef for the past week and this morning I wrote out  
> some notes on our private wiki about things I'd like to enhance/do. This is  
> totally off my head and some of it is almost certainly as a result of  
> missing out key bits of understanding. After writing it, I figured, why not  
> share it, so here goes:

Thank you so much for sharing this, especially with as much detail and  
as raw as it is. Awesome feedback.

> - Some sort of rspec/test-driven mechanism. At the very least, this should  
> properly sanity check all the cookbooks, looking for typo'd attribute  
> hashes, missing recipes and logic errors. I'm thinking that this could be  
> built fairly simply just by stubbing parts of the framework and running each  
> of the recipes. Second goal: get autospec working for it. Third goal: rspec  
> matchers suitable for chef.

We agree! The current plan here is to use Chef's own cucumber  
features, along with some nifty scaffolding, to perform integration  
tests for cookbooks. I'm on the fence about the utility of unit  
testing them - mocking out the responses from the Operating System is  
brittle... but I'm happy to be convinced. Some progress has been made  
towards this goal already, and we would all love the help.

> - Remove the needless verbosity of the attribute definitions, like creating  
> new Mashes, and only defining attributes unless they already exist. It's  
> just messy.

Agreed. Syntax suggestions?

> - Remove the duplication in the metadata. While enhancing the attribute  
> definitions, tweak them so that they include enough information to allow the  
> metadata to be automatically generated. Dependencies can be calculated  
> instead of specifying them manually.

Yeah - my only concern here is that there are going to be situations  
where the calculated dependencies can only be done at runtime (think  
having an array of recipes to include, for example)... so while we  
should put some more smarts in, we need to make sure you can always  
tweak the results by hand (and not have them blown away when the  
automated process swings by.)

> - Allow cookbooks to be distributed as gems with gem dependencies on other  
> cookbooks.

We're working on cookbook distribution very soon - it would be cool to  
have a mechanism like this.

> - Allow vendoring of these gems, because I don't like relying on system  
> gems, even in this sort of environment. Bootstrapping is key.

That is essentially what you have now with a normal chef repository -  
I imagine the vendoring process would be very simple.

> - Get rubigen to have generators for an empty repo, individual cookbooks,  
> that kind of thing. In general, make the repo infrastructure feel more like  
> a Rails app, just because that's a good example of an app development  
> environment. Individual recipes can be generated with: {{script/generate  
> recipe ruby/gems}} which would automatically create the 'ruby' cookbook too,  
> if it doesn't already exist.

Using rubigen to replace some of the custom Rakefile logic we use now  
is a good idea. File a ticket?

> - A thin bootstrap tarball of chef solo that doesn't rely on anything other  
> than ruby being installed on the client system. No rubygems, nowt. That way,  
> we can handle everything else through chef and have real bare-bones system  
> images.

This would really just be enough chef to get chef - which is pretty  
much the situation now.

> - Some sort of external, persistent information about nodes as well as roles  
> in the repo. I kinda want the chef server to be easily replaceable, and  
> everything about our infrastructure to be in version control. I can't quite  
> see how to do that with nodes yet, but perhaps I'm just missing something.

Yeah - we actually have the code to do this basically ready, we just  
need to rig it back up. I imagine this will re-appear in the near  
term future, and be very similar to the way we deal with Roles.

> - Bootstrapping a node from a pristine image, pulling all its information  
> from an external system. I'm (so far) not liking the idea of creating a node  
> by passing in some JSON when it boots. It should boot then get that  
> information from a persistent store so that, next time it's imaged from  
> afresh, it can start back where it left off without me having to separately  
> document the JSON is was passed in on bootup. I think this is a repetition  
> of the previous point. 🙂

You can put an URL in for the JSON and the Tarball, allowing you to  
build this system easily.

> - The equivalent of dpkg's "Provides" for providing alternative  
> implementations of packages. Say, for example, I have a  
> ruby\_enterprise\_edition cookbook, which {{provides "ruby"}}. If another  
> cookbook says {{include\_recipe "ruby"}} and we've already seen  
> ruby\_enterprise\_edition, it will satisfy that dependency.

Hrm - that's an interesting one. Right now you could have two  
cookbooks, which both provide a "ruby" recipe, one that install REE  
and one that uses the system ruby. The metadata would be what  
differentiates them, and things would work correctly. I hadn't  
thought about allowing cookbooks to essentially declare an alias...

> - Percona's build of MySQL [[http://www.percona.com/mysql/5.0.82-b16/deb/](http://www.percona.com/mysql/5.0.82-b16/deb/)]

Good idea! That should be an easy one.

> - RabbitMQ

We'll have RabbitMQ cookbooks soon, if you don't get to them first.

> - Variants of all the existing recipes that don't use runit! (What can I  
> say? I like consistency, and having a mixture of init.d and runit services  
> isn't consistent. And out of band monitoring with God is easier than  
> shifting _everything_ across to runit...)

Feel free to fork. 🙂

> - Nanite infrastructure.

Coming soon.

> - Rolling out software updates (in particular, package security updates) in  
> a timely manner across our infrastructure, after having tested on our  
> staging infrastructure.

You can do this now with Chef, and when a nanite agent is broadly  
available, trigger it into a more complex workflow.

> - Upgrading an EC2 instance to a bigger instance type. Automagically.

Should be easy, as long as all the persistent data is on an EBS store  
and all the ephemeral stuff comes from Chef.

> - An overarching DSL for describing our infrastructure, from which all the  
> node stuff is generated and deployed. What I'd _like_ to do is "draw" our  
> infrastructure in Graphviz dot notation and for that to turn into physical,  
> working, connected infrastructure. That's my nearly-end-goal.

We're working on this - not in Dot, but in general. Check out:

[http://wiki.opscode.com/display/chef/Roles+and+Infrastructures](http://wiki.opscode.com/display/chef/Roles+and+Infrastructures)

Thanks again for the great feedback, Graeme - glad to have you in the  
Chef community!

Regards,  
Adam

--  
Opscode, Inc.  
Adam Jacob, CTO  
T: (206) 508-7449 E: [adam@opscode.com](mailto:adam@opscode.com)

---

<div class="post-metadata">

**Author:** ![Miguel\_Cabeca](https://sea2.discourse-cdn.com/flex016/user_avatar/discourse.chef.io/miguel_cabeca/32/720_2.png) [@Miguel\_Cabeca](https://discourse.chef.io/u/Miguel_Cabeca)\
**Post date:** [July 28, 2009, 11:47am UTC](https://discourse.chef.io/t/thinking-out-loud/1239/3 "2009-07-28T11:47:33Z")

</div>

Hi,

> > - Remove the duplication in the metadata. While enhancing the  
> > attribute  
> > definitions, tweak them so that they include enough information to  
> > allow the  
> > metadata to be automatically generated. Dependencies can be  
> > calculated  
> > instead of specifying them manually.
> 
> Yeah - my only concern here is that there are going to be situations  
> where the calculated dependencies can only be done at runtime (think  
> having an array of recipes to include, for example)... so while we  
> should put some more smarts in, we need to make sure you can always  
> tweak the results by hand (and not have them blown away when the  
> automated process swings by.)

Right now the metadata (json) is always "compiled" from the  
metadata.rb file. How about a default generated metadata file from the  
cookbook itself with overrides in the metadata.rb file? That way if I  
don't like the generated dependencies I can always override them in  
the file.

> > - Allow cookbooks to be distributed as gems with gem dependencies  
> > on other  
> > cookbooks.
> 
> We're working on cookbook distribution very soon - it would be cool to  
> have a mechanism like this.

Oh my... Another distribution system? Is it realy necessary? I always  
download the external cookbooks and customize them somehow turning  
them into site-cookbooks. Does anyone use third-party cookbooks  
without any modification? Maybe I just like to make it hard on  
myself 😛

> > - Allow vendoring of these gems, because I don't like relying on  
> > system  
> > gems, even in this sort of environment. Bootstrapping is key.
> 
> That is essentially what you have now with a normal chef repository -  
> I imagine the vendoring process would be very simple.

I don't understand this part, mainly because i don't know the meaning  
of the 'vendoring' word. Will someone be kind enough to explain this  
to me?

> > - Some sort of external, persistent information about nodes as well  
> > as roles  
> > in the repo. I kinda want the chef server to be easily replaceable,  
> > and  
> > everything about our infrastructure to be in version control. I  
> > can't quite  
> > see how to do that with nodes yet, but perhaps I'm just missing  
> > something.
> 
> Yeah - we actually have the code to do this basically ready, we just  
> need to rig it back up. I imagine this will re-appear in the near  
> term future, and be very similar to the way we deal with Roles.

I like this idea. I too want to keep certain node information on VCS.

> > - The equivalent of dpkg's "Provides" for providing alternative  
> > implementations of packages. Say, for example, I have a  
> > ruby\_enterprise\_edition cookbook, which {{provides "ruby"}}. If  
> > another  
> > cookbook says {{include\_recipe "ruby"}} and we've already seen  
> > ruby\_enterprise\_edition, it will satisfy that dependency.
> 
> Hrm - that's an interesting one. Right now you could have two  
> cookbooks, which both provide a "ruby" recipe, one that install REE  
> and one that uses the system ruby. The metadata would be what  
> differentiates them, and things would work correctly. I hadn't  
> thought about allowing cookbooks to essentially declare an alias...

With a new distribution system comes a slew of features/problems that  
were adressed/solved over and over again by other distribution/package  
systems. Heck, why not just use dpkg for this if you're modeling  
after it? 😛

Best Regards

Miguel Cabeça

---

<div class="post-metadata">

**Author:** ![kallistec](https://sea2.discourse-cdn.com/flex016/user_avatar/discourse.chef.io/kallistec/32/23_2.png) [@kallistec](https://discourse.chef.io/u/kallistec)\
**Post date:** [July 28, 2009, 1:17pm UTC](https://discourse.chef.io/t/thinking-out-loud/1239/4 "2009-07-28T13:17:39Z")

</div>

On Tue, Jul 28, 2009 at 5:47 AM, Miguel Cabeça [cabeca@ist.utl.pt](mailto:cabeca@ist.utl.pt) wrote:

> - Allow vendoring of these gems, because I don't like relying on system
> 
> gems, even in this sort of environment. Bootstrapping is key.
> 
> That is essentially what you have now with a normal chef repository -  
> I imagine the vendoring process would be very simple.
> 
> I don't understand this part, mainly because i don't know the meaning of  
> the 'vendoring' word. Will someone be kind enough to explain this to me?

It comes from Rails, where a common and good practice is to get ruby gems  
the normal way, and then unpack a specific version of the rails (or other)  
gems to a my-rails-app/vendor/gems/ directory. Rails will prefer libraries  
in this directory over system gems. This locks the rails app to a specific  
version of the gems. You also get the benefit of being able to distribute  
the app and its dependencies in one big package that doesn't need anything  
other than ruby and the webserver to run.

> Best Regards
> 
> Miguel Cabeça

HTH,  
Dan DeLeo

---

<div class="post-metadata">

**Author:** ![kallistec](https://sea2.discourse-cdn.com/flex016/user_avatar/discourse.chef.io/kallistec/32/23_2.png) [@kallistec](https://discourse.chef.io/u/kallistec)\
**Post date:** [July 28, 2009, 1:40pm UTC](https://discourse.chef.io/t/thinking-out-loud/1239/5 "2009-07-28T13:40:34Z")

</div>

Also, as Adam said, if vendoring is the _only_ way to install cookbooks from  
gems, then the process is pretty easy. You'd just have a rake task like  
rake gem\_cookbook NAME="opscode-cookbooks"

or

rake gem\_cookbook:cherry\_pick NAME="opcode-cookbooks" BOOK="mysql"

Personally, I'd prefer thor for this because it handles arguments better,  
but I don't know if the additional dependency is worth it.

Dan DeLeo

On Tue, Jul 28, 2009 at 7:17 AM, Daniel DeLeo [dan@kallistec.com](mailto:dan@kallistec.com) wrote:

> On Tue, Jul 28, 2009 at 5:47 AM, Miguel Cabeça [cabeca@ist.utl.pt](mailto:cabeca@ist.utl.pt) wrote:
> 
> > - Allow vendoring of these gems, because I don't like relying on system
> > 
> > gems, even in this sort of environment. Bootstrapping is key.
> > 
> > That is essentially what you have now with a normal chef repository -  
> > I imagine the vendoring process would be very simple.
> > 
> > I don't understand this part, mainly because i don't know the meaning of  
> > the 'vendoring' word. Will someone be kind enough to explain this to me?
> 
> It comes from Rails, where a common and good practice is to get ruby gems  
> the normal way, and then unpack a specific version of the rails (or other)  
> gems to a my-rails-app/vendor/gems/ directory. Rails will prefer libraries  
> in this directory over system gems. This locks the rails app to a specific  
> version of the gems. You also get the benefit of being able to distribute  
> the app and its dependencies in one big package that doesn't need anything  
> other than ruby and the webserver to run.
> 
> > Best Regards
> > 
> > Miguel Cabeça
> 
> HTH,  
> Dan DeLeo

---

<div class="post-metadata">

**Author:** ![Graeme\_Mathieson](https://sea2.discourse-cdn.com/flex016/user_avatar/discourse.chef.io/graeme_mathieson/32/914_2.png) [@Graeme\_Mathieson](https://discourse.chef.io/u/Graeme_Mathieson)\
**Post date:** [July 29, 2009, 7:09am UTC](https://discourse.chef.io/t/thinking-out-loud/1239/6 "2009-07-29T07:09:23Z")

</div>

Hi,

Thanks for the detailed response, Adam, and apologies for the delay in  
following up. I've just got so many thoughts, ideas and directions  
that it's hard to know where to start! Well, that and I've got a  
production system to deliver ASAP, which is what I'm busting a gut on  
just now. 🙂

I'll tackle this one for now:

On 25 Jul 2009, at 20:24, Adam Jacob wrote:

> > - A thin bootstrap tarball of chef solo that doesn't rely on  
> > anything other  
> > than ruby being installed on the client system. No rubygems, nowt.  
> > That way,  
> > we can handle everything else through chef and have real bare-bones  
> > system  
> > images.
> 
> This would really just be enough chef to get chef - which is pretty  
> much the situation now.

But unfortunately, "just enough chef" is quite a lot. So I had a go  
at creating a "bootstrap" installation script for Chef. It works  
against a pristine Ubuntu 9.04 system and doesn't pollute the system  
itself with anything, leaving that to Chef itself. I've shifted the  
documentation (and the script itself) across to our public wiki:

```
 http://wiki.rubaidh.com/display/public/Bootstrapping+Chef

```

I'm sure it's utterly bug ridden and flawed, but it does appear to do  
the right thing. And, for bonus points, it seems to be somewhat  
idempotent in that it doesn't cause harm if you run it again. Not that  
it carries on where it left off, but at least everything happens in a  
temporary directory, or through Chef.

Thoughts?

## G

Graeme Mathieson  
Managing Director  
Rubaidh Ltd: Scottish for Ruby on Rails

Follow us on Twitter: [http://twitter.com/rubaidh](http://twitter.com/rubaidh) and \<[http://twitter.com/mathie](http://twitter.com/mathie)

> 

Web Site: [http://rubaidh.com/](http://rubaidh.com/)  
Blog: [http://woss.name/](http://woss.name/)  
Telephone: +44 (0)131 273 5271  
Mobile: +44 (0)7949 0777 44

Rubaidh Ltd is a limited company registered in Scotland with  
registration number SC297029 and VAT number GB 916 0341 53. The  
registered address is: Stuart House, Eskmills, Musselburgh, East  
Lothian, EH21 7PB, United Kingdom.

---

<div class="post-metadata">

**Author:** ![Joel\_Merrick](https://sea2.discourse-cdn.com/flex016/user_avatar/discourse.chef.io/joel_merrick/32/893_2.png) [@Joel\_Merrick](https://discourse.chef.io/u/Joel_Merrick)\
**Post date:** [July 29, 2009, 9:38am UTC](https://discourse.chef.io/t/thinking-out-loud/1239/7 "2009-07-29T09:38:15Z")

</div>

On Wed, Jul 29, 2009 at 8:09 AM, Graeme Mathieson[mathie@woss.name](mailto:mathie@woss.name) wrote:

> [http://wiki.rubaidh.com/display/public/Bootstrapping+Chef](http://wiki.rubaidh.com/display/public/Bootstrapping+Chef)

> Thoughts?

I'm yet to test is out, but this could be the 'missing link' between  
taking the FAI installation system I've been working on and giving the  
newly provisioned server a purpose (mail bastion, mysql server etc..)

Will let you know how it goes,

Cheers!  
Joel

--  
$ echo "kpfmAdpoofdufevq/dp/vl" | perl -pe 's/(.)/chr(ord($1)-1)/ge'

---

<div class="post-metadata">

**Author:** ![Adam\_Jacob](https://sea2.discourse-cdn.com/flex016/user_avatar/discourse.chef.io/adam_jacob/32/292_2.png) [@Adam\_Jacob](https://discourse.chef.io/u/Adam_Jacob)\
**Post date:** [July 29, 2009, 4:50pm UTC](https://discourse.chef.io/t/thinking-out-loud/1239/8 "2009-07-29T16:50:39Z")

</div>

On Wed, Jul 29, 2009 at 12:09 AM, Graeme Mathieson[mathie@woss.name](mailto:mathie@woss.name) wrote:

> Thanks for the detailed response, Adam, and apologies for the delay in  
> following up. I've just got so many thoughts, ideas and directions that it's  
> hard to know where to start! Well, that and I've got a production system to  
> deliver ASAP, which is what I'm busting a gut on just now. 🙂

Cool - take your time. We'll get Rome built, just not today.

> But unfortunately, "just enough chef" is quite a lot. So I had a go at  
> creating a "bootstrap" installation script for Chef. It works against a  
> pristine Ubuntu 9.04 system and doesn't pollute the system itself with  
> anything, leaving that to Chef itself. I've shifted the documentation (and  
> the script itself) across to our public wiki:
> 
> [http://wiki.rubaidh.com/display/public/Bootstrapping+Chef](http://wiki.rubaidh.com/display/public/Bootstrapping+Chef)
> 
> I'm sure it's utterly bug ridden and flawed, but it does appear to do the  
> right thing. And, for bonus points, it seems to be somewhat idempotent in  
> that it doesn't cause harm if you run it again. Not that it carries on where  
> it left off, but at least everything happens in a temporary directory, or  
> through Chef.
> 
> Thoughts?

I dig it! It's already pretty generic, but seeing something like this  
for all the various platforms we support is a great idea. Reminds me  
of the old Gnome installer from back in the day. Another variant on  
this would be using Capistrano to do it remotely..

Adam

--  
Opscode, Inc.  
Adam Jacob, CTO  
T: (206) 508-7449 E: [adam@opscode.com](mailto:adam@opscode.com)

---

<div class="post-metadata">

**Author:** ![Ian\_Ragsdale](https://sea2.discourse-cdn.com/flex016/user_avatar/discourse.chef.io/ian_ragsdale/32/915_2.png) [@Ian\_Ragsdale](https://discourse.chef.io/u/Ian_Ragsdale)\
**Post date:** [July 29, 2009, 4:58pm UTC](https://discourse.chef.io/t/thinking-out-loud/1239/9 "2009-07-29T16:58:10Z")

</div>

I actually did something similar myself, but totally in ruby. It  
creates a Net::SSH connection over which it can run commands, runs the  
commands remotely, and evaluates the results locally.

It figures out if apt-get or yum are installed, installs the necessary  
packages to run chef-solo, then copies over the chef-solo config (the  
really cool thing is that Net::SCP can happen over the same SSH  
connection) and runs the specified recipe.

- Ian

On Jul 29, 2009, at 11:50 AM, Adam Jacob wrote:

> I dig it! It's already pretty generic, but seeing something like this  
> for all the various platforms we support is a great idea. Reminds me  
> of the old Gnome installer from back in the day. Another variant on  
> this would be using Capistrano to do it remotely..
> 
> Adam
> 
> --  
> Opscode, Inc.  
> Adam Jacob, CTO  
> T: (206) 508-7449 E: [adam@opscode.com](mailto:adam@opscode.com)

---

<div class="post-metadata">

**Author:** ![Sander\_van\_Zoest](https://sea2.discourse-cdn.com/flex016/user_avatar/discourse.chef.io/sander_van_zoest/32/48_2.png) [@Sander\_van\_Zoest](https://discourse.chef.io/u/Sander_van_Zoest)\
**Post date:** [July 29, 2009, 6:04pm UTC](https://discourse.chef.io/t/thinking-out-loud/1239/10 "2009-07-29T18:04:53Z")

</div>

On Wed, Jul 29, 2009 at 9:50 AM, Adam Jacob [adam@opscode.com](mailto:adam@opscode.com) wrote:

> \> http://wiki.rubaidh.com/display/public/Bootstrapping+Chef \> \> I'm sure it's utterly bug ridden and flawed, but it does appear to do the \> right thing. And, for bonus points, it seems to be somewhat idempotent in \> that it doesn't cause harm if you run it again. Not that it carries on where \> it left off, but at least everything happens in a temporary directory, or \> through Chef. \> \> Thoughts?
> 
> I dig it! It's already pretty generic, but seeing something like this  
> for all the various platforms we support is a great idea. Reminds me  
> of the old Gnome installer from back in the day. Another variant on  
> this would be using Capistrano to do it remotely..

I am using something similar, but a bit more basic as a rake task in the  
chef-repo.

[http://gist.github.com/158298](http://gist.github.com/158298)

--  
Sander van Zoest  
[sander@vanzoest.com](mailto:sander@vanzoest.com)  
San Diego, CA, USA  
[http://Sander.vanZoest.com/](http://Sander.vanZoest.com/)

---

<div class="post-metadata">

**Author:** ![kallistec](https://sea2.discourse-cdn.com/flex016/user_avatar/discourse.chef.io/kallistec/32/23_2.png) [@kallistec](https://discourse.chef.io/u/kallistec)\
**Post date:** [July 29, 2009, 6:48pm UTC](https://discourse.chef.io/t/thinking-out-loud/1239/11 "2009-07-29T18:48:57Z")

</div>

I think this SSH integration is super awesome. At the Chef preso at Denver's  
derailed, everyone I talked to seemed pretty hot on the idea of making the  
client install as close to 100% automated as possible.  
Graeme et. al.,  
Your script is pretty impressive, but I think there might be a simpler way.  
I'm currently looking at making the chef client distributable as a tarball  
using Wycats' new bundler project.

> **[GitHub - wycats/bundler](https://github.com/wycats/bundler/tree/master)**
>
> master

My first crack at this (embarrassingly simple) is here:  
[Gemfile · GitHub](http://gist.github.com/158274) If you put that somewhere and name it  
``Gemfile'' and install bundler (github clone only for now), then you can  
run gem\_bundler and it will get all the gems, and vendor them for you,  
resulting in something you could tar up and download. I'm pretty excited  
about this, as it would allow for some magical possibilities like having  
chef itself install rubygems.

The caveats right now are:

- The resulting vendored gems are architecture-dependent. This is because  
the JSON gem has C extensions. I'll be exploring workarounds for this.
- The executables expect themselves to be located in a particular directory.  
For example, the chef-solo command generated by bundler looks like this:

#!/usr/bin/env ruby1.8  
require "/home/ddeleo/ruby/travelling-chef/vendor/gems/environments/default"  
load  
"/home/ddeleo/ruby/travelling-chef/vendor/gems/gems/chef-0.7.4/bin/chef-solo"

I got in touch with wycats about this, but no resolution yet. The best way  
to deal with this for now is to do everything in /tmp.

Cheers,  
Dan DeLeo

On Wed, Jul 29, 2009 at 10:58 AM, Ian Ragsdale [ian.ragsdale@gmail.com](mailto:ian.ragsdale@gmail.com)wrote:

> I actually did something similar myself, but totally in ruby. It creates a  
> Net::SSH connection over which it can run commands, runs the commands  
> remotely, and evaluates the results locally.
> 
> It figures out if apt-get or yum are installed, installs the necessary  
> packages to run chef-solo, then copies over the chef-solo config (the really  
> cool thing is that Net::SCP can happen over the same SSH connection) and  
> runs the specified recipe.
> 
> - Ian
> 
> On Jul 29, 2009, at 11:50 AM, Adam Jacob wrote:
> 
> > I dig it! It's already pretty generic, but seeing something like this  
> > for all the various platforms we support is a great idea. Reminds me  
> > of the old Gnome installer from back in the day. Another variant on  
> > this would be using Capistrano to do it remotely..
> > 
> > Adam
> > 
> > --  
> > Opscode, Inc.  
> > Adam Jacob, CTO  
> > T: (206) 508-7449 E: [adam@opscode.com](mailto:adam@opscode.com)

---

<div class="post-metadata">

**Author:** ![Graeme\_Mathieson](https://sea2.discourse-cdn.com/flex016/user_avatar/discourse.chef.io/graeme_mathieson/32/914_2.png) [@Graeme\_Mathieson](https://discourse.chef.io/u/Graeme_Mathieson)\
**Post date:** [August 9, 2009, 7:08am UTC](https://discourse.chef.io/t/thinking-out-loud/1239/12 "2009-08-09T07:08:04Z")

</div>

Hi,

Just cherry picking again. 🙂

On 25 Jul 2009, at 21:24, Adam Jacob wrote:

> > - Get rubigen to have generators for an empty repo, individual  
> > cookbooks,  
> > that kind of thing.
> 
> Using rubigen to replace some of the custom Rakefile logic we use now  
> is a good idea. File a ticket?

First attempt of this is at:

```
http://github.com/rubaidh/chef/tree/master

```

So far, it only generates the bare repository, but I do have a mostly-  
finished cookbook generator that I'll merge in later on today, too.

G

---

<div class="post-metadata">

**Author:** ![Adam\_Jacob](https://sea2.discourse-cdn.com/flex016/user_avatar/discourse.chef.io/adam_jacob/32/292_2.png) [@Adam\_Jacob](https://discourse.chef.io/u/Adam_Jacob)\
**Post date:** [July 29, 2009, 7:28pm UTC](https://discourse.chef.io/t/thinking-out-loud/1239/13 "2009-07-29T19:28:46Z")

</div>

We're also working on native packages for more platforms - CentOS has  
great native packages, Debian and Ubuntu are coming soon.

Adam

On Wed, Jul 29, 2009 at 11:04 AM, Sander van Zoest[sander@vanzoest.com](mailto:sander@vanzoest.com) wrote:

> On Wed, Jul 29, 2009 at 9:50 AM, Adam Jacob [adam@opscode.com](mailto:adam@opscode.com) wrote:
> 
> > \> http://wiki.rubaidh.com/display/public/Bootstrapping+Chef \> \> I'm sure it's utterly bug ridden and flawed, but it does appear to do \> the \> right thing. And, for bonus points, it seems to be somewhat idempotent \> in \> that it doesn't cause harm if you run it again. Not that it carries on \> where \> it left off, but at least everything happens in a temporary directory, \> or \> through Chef. \> \> Thoughts?
> > 
> > I dig it! It's already pretty generic, but seeing something like this  
> > for all the various platforms we support is a great idea. Reminds me  
> > of the old Gnome installer from back in the day. Another variant on  
> > this would be using Capistrano to do it remotely..
> 
> I am using something similar, but a bit more basic as a rake task in the  
> chef-repo.
> 
> [http://gist.github.com/158298](http://gist.github.com/158298)
> 
> --  
> Sander van Zoest  
> [sander@vanzoest.com](mailto:sander@vanzoest.com)  
> San Diego, CA, USA  
> [http://Sander.vanZoest.com/](http://Sander.vanZoest.com/)

--  
Opscode, Inc.  
Adam Jacob, CTO  
T: (206) 508-7449 E: [adam@opscode.com](mailto:adam@opscode.com)

---

<div class="post-metadata">

**Author:** ![Graeme\_Mathieson](https://sea2.discourse-cdn.com/flex016/user_avatar/discourse.chef.io/graeme_mathieson/32/914_2.png) [@Graeme\_Mathieson](https://discourse.chef.io/u/Graeme_Mathieson)\
**Post date:** [July 30, 2009, 7:14am UTC](https://discourse.chef.io/t/thinking-out-loud/1239/14 "2009-07-30T07:14:58Z")

</div>

Hi,

On 29 Jul 2009, at 19:48, Daniel DeLeo wrote:

> Your script is pretty impressive, but I think there might be a  
> simpler way. I'm currently looking at making the chef client  
> distributable as a tarball using Wycats' new bundler project.

Oh, my. Yep, that's a lot simpler. 🙂 No more fiddling with  
bootstrapping for me today, but I'm definitely going to check out  
rewriting the script in terms of wycat's bundler over the weekend.

G

---

<div class="post-metadata">

**Author:** ![snacktime](https://avatars.discourse-cdn.com/v4/letter/s/7feea3/32.png) [@snacktime](https://discourse.chef.io/u/snacktime)\
**Post date:** [July 30, 2009, 7:29am UTC](https://discourse.chef.io/t/thinking-out-loud/1239/15 "2009-07-30T07:29:36Z")

</div>

Makeself has been around for quite a while and is great for stuff like this.  
It creates a self extracting archive that will extract itself and then run  
a shell script. I use it to install chef solo and our own client on ec2  
instances.  
Chris

On Thu, Jul 30, 2009 at 12:14 AM, Graeme Mathieson [mathie@woss.name](mailto:mathie@woss.name) wrote:

> Hi,  
> On 29 Jul 2009, at 19:48, Daniel DeLeo wrote:
> 
> Your script is pretty impressive, but I think there might be a simpler way.  
> I'm currently looking at making the chef client distributable as a tarball  
> using Wycats' new bundler project.
> 
> Oh, my. Yep, that's a lot simpler. 🙂 No more fiddling with bootstrapping  
> for me today, but I'm definitely going to check out rewriting the script in  
> terms of wycat's bundler over the weekend.
> 
> G