Workflows, librarian, cookbooks as individual git repositories, and other headaches

[Bryan_Berry]

I have pretty much settled on using librarian to manage to my cookbooks/
directory, holding all of my ‘vendor’ cookbooks, i.e. cookbooks I don’t
actively modify and putting those cookbooks that I do actively edit into
site-cookbooks/

Each cookbook in site-cookbooks/ has its own git repository

The pain arises in the following situations

1. When I want to sync my setup from one workstation to another, i.e. b/w
   my workstation in the office and my laptop at home
2. End of the day I want to make that all my commits across various
   cookbooks in site-cookbooks are pushed to origin
3. I want my colleague to quickly be working on the same set of cookbooks
   that I am

This doesn’t seem like super hard stuff, but like so many problems in the
Chef world, someone has probably already invented a cleverer solution than
I would invent on my own.

I have in mind a Rake task that searches site-cookbooks and pushs the
latest commits to master

But how would I make sure that my colleague and I have the same set of
site-cookbooks/ ? Perhaps something like Cheffile or an addition to
Cheffile that pulls specified cookbooks into site-cookbooks but doesn’t
take over the entire directory like librarian does with cookbooks/.

[realityforge]

Hi,

On Fri, Apr 13, 2012 at 4:20 PM, Bryan Berry bryan.berry@gmail.com wrote:

I have pretty much settled on using librarian to manage to my cookbooks/
directory, holding all of my 'vendor' cookbooks, i.e. cookbooks I don't
actively modify and putting those cookbooks that I do actively edit into
site-cookbooks/

Each cookbook in site-cookbooks/ has its own git repository

The pain arises in the following situations

1. When I want to sync my setup from one workstation to another, i.e. b/w my
   workstation in the office and my laptop at home
2. End of the day I want to make that all my commits across various
   cookbooks in site-cookbooks are pushed to origin
3. I want my colleague to quickly be working on the same set of cookbooks
   that I am

This doesn't seem like super hard stuff, but like so many problems in the
Chef world, someone has probably already invented a cleverer solution than I
would invent on my own.

I have in mind a Rake task that searches site-cookbooks and pushs the latest
commits to master

But how would I make sure that my colleague and I have the same set of
site-cookbooks/ ? Perhaps something like Cheffile or an addition to Cheffile
that pulls specified cookbooks into site-cookbooks but doesn't take over the
entire directory like librarian does with cookbooks/.

If you are working in a git repository I would highly recommend the
use of braid. With it you can essentially merge a remote repository
into a local repository (I believe under the covers it is just a git
subtree merge). Thus your local repository always has the complete
state of your system. If you want to push changes back to their origin
then that is possible to just push. These changes would appear in both
the local repository and the origin repository. Braid may not be chef
specific (and may not understand dependencies and versions) but it is
designed for this workflow and I think it is a far easier to use for
that sort of thing rather than a collection of more complex tools.

For example...

import original data

braid add git@github.com:realityforge-cookbooks/glassfish.git
cookbooks/glassfish
...

update from original repository as it has changed

braid update cookbooks/glassfish
...

make some changes in local repository

vi cookbooks/glassfish/...
git commit -a
...

examine changes from origin repository

braid diff cookbooks/glassfish
...

optionally push changes back to origin repository

braid push cookbooks/glassfish

I tend to not directly push back from the repository but may generate
diffs and manually apply the changes to the upstream cookbook. This
allows me to clean uyp patches when necessary and remove some cruft.

--
Cheers,

Peter Donald

[Bryan_Berry]

tks Peter, braid looks like it could be an interesting wrapper around git.
I probably could build a rake task around it that meets my needs, will have
to think about it some more

On Fri, Apr 13, 2012 at 8:47 AM, Peter Donald peter@realityforge.orgwrote:

Hi,

On Fri, Apr 13, 2012 at 4:20 PM, Bryan Berry bryan.berry@gmail.com
wrote:

I have pretty much settled on using librarian to manage to my cookbooks/
directory, holding all of my 'vendor' cookbooks, i.e. cookbooks I don't
actively modify and putting those cookbooks that I do actively edit into
site-cookbooks/

Each cookbook in site-cookbooks/ has its own git repository

The pain arises in the following situations

1. When I want to sync my setup from one workstation to another, i.e.
   b/w my
   workstation in the office and my laptop at home
2. End of the day I want to make that all my commits across various
   cookbooks in site-cookbooks are pushed to origin
3. I want my colleague to quickly be working on the same set of cookbooks
   that I am

This doesn't seem like super hard stuff, but like so many problems in the
Chef world, someone has probably already invented a cleverer solution
than I
would invent on my own.

I have in mind a Rake task that searches site-cookbooks and pushs the
latest
commits to master

But how would I make sure that my colleague and I have the same set of
site-cookbooks/ ? Perhaps something like Cheffile or an addition to
Cheffile
that pulls specified cookbooks into site-cookbooks but doesn't take over
the
entire directory like librarian does with cookbooks/.

If you are working in a git repository I would highly recommend the
use of braid. With it you can essentially merge a remote repository
into a local repository (I believe under the covers it is just a git
subtree merge). Thus your local repository always has the complete
state of your system. If you want to push changes back to their origin
then that is possible to just push. These changes would appear in both
the local repository and the origin repository. Braid may not be chef
specific (and may not understand dependencies and versions) but it is
designed for this workflow and I think it is a far easier to use for
that sort of thing rather than a collection of more complex tools.

For example...

import original data

braid add git@github.com:realityforge-cookbooks/glassfish.git
cookbooks/glassfish
...

update from original repository as it has changed

braid update cookbooks/glassfish
...

make some changes in local repository

vi cookbooks/glassfish/...
git commit -a
...

examine changes from origin repository

braid diff cookbooks/glassfish
...

optionally push changes back to origin repository

braid push cookbooks/glassfish

I tend to not directly push back from the repository but may generate
diffs and manually apply the changes to the upstream cookbook. This
allows me to clean uyp patches when necessary and remove some cruft.

--
Cheers,

Peter Donald

[Craig_Cook]

I am new to chef…

Some questions/feedback for this page

http://wiki.opscode.com/display/chef/Installing+Chef+Server+Manually#InstallingChefServerManually-ConfiguretheCommandLineClient

Configure the Command Line Client

Should this line
"First, create the ~/.chef directory and copy the required certificates created by the server."

Be changed to something like this:

In this example we have logged in as the user “bob”. Where you see “bob” in the following examples replace it with your username.

First, create the ~/.chef directory and copy the required certificates created by the server.

sudo chown -R bob ~/.chef

further down to this section
"Create a Knife Client for Your Laptop/Desktop"

Make sure you’ve configured knife on your chef server as described above before proceeding with this step.

Create a private certificate that you will transfer to your Laptop/Desktop (Should the private cert be deleted after you have copied if to your laptop?)

chef-server> knife client create bob -n -a -f /tmp/bob.pem
INFO: Created (or updated) client[bob]

i.e. change all references for “my-username” to “bob”

Or, am I not understanding what is going on here…

This section is not clear

your-laptop > knife client list
chef-webui
my-username
bob
chef-validator

Obviously by this stage your local workstation is communicating to the chef-server.

It shows a client called bob and also my-username, if you worked through everything on this page shouldn’t you just have “bob” as the client? Is “bob” meant to be “bobs_workstation”? That would make “my-username” the local username I logged into my local workstation with…

Craig

[Jeremiah_Snapp]

I don't have a rake task to handle making sure all my cookbooks have been
pushed.

But I do have a way to manage my cookbooks that could solve the problem of
making sure a colleague knows what's in the site-cookbooks directory.
I really like Librarian for managing cookbooks but I use it a little
different than it's documentation suggests.

Among other things I don't include site-cookbooks in my knife
cookbook_path. Instead I put git hosted cookbooks outside my chef-repo and
source them using :path in Cheffile while testing them. I also keep a
commented :git source to the cookbooks in Cheffile so once I push changes
to github I comment the :path entries and uncomment the :git entries.

I describe this more completely here.

jeremiahsnapp/chef-repo

I hope that helps.

-Jeremiah

[Jason_Boxman]

On Fri, Apr 20, 2012 at 3:52 PM, Jeremiah Snapp
jeremiah.snapp@gmail.com wrote:

I don't have a rake task to handle making sure all my cookbooks have been
pushed.

But I do have a way to manage my cookbooks that could solve the problem of
making sure a colleague knows what's in the site-cookbooks directory.
I really like Librarian for managing cookbooks but I use it a little
different than it's documentation suggests.

Among other things I don't include site-cookbooks in my knife cookbook_path.
Instead I put git hosted cookbooks outside my chef-repo and source them
using :path in Cheffile while testing them. I also keep a commented :git
source to the cookbooks in Cheffile so once I push changes to github I
comment the :path entries and uncomment the :git entries.

I describe this more completely here.

Chef Cookbook Management · jeremiahsnapp/chef-repo Wiki · GitHub

That's a great workflow and I do something very similar myself. Being
lazy, I abuse rb-inotify under Linux to monitor changes to my external
cookbooks directory so any file adds or deletions by me are hardlinked
inside my site-cookbooks directory, rather than forcing librarian to
update every time I update a file. I then share out my chef repo via
VBox for testing.

--
Thursday next week is indeed time for a thorough insight
into the swirling red debian vortex. — leeta