About Opscode's cookbook repo changes


#1

Hi all,

last month there was a post regarding changes in the opscode cookbooks repo:

While I can understand the reasons that made Opscode remove the
metadata.json from all cookbooks, it does pose a big problem for
non-ruby tools like LittleChef, which depended on JSON to
inter-operate.

Is there a way to generate metadata.json for all cookbooks that
doesn’t need a Chef Server?
“knife cookbook metadata” doesn’t classify there, unfortunately.

Cheers,
Miquel


#2

On Friday, June 17, 2011 at 2:10 AM, Miquel Torres wrote:

Hi all,

last month there was a post regarding changes in the opscode cookbooks repo:
http://www.opscode.com/blog/2011/05/24/update-on-the-future-of-opscodes-cookbooks/

While I can understand the reasons that made Opscode remove the
metadata.json from all cookbooks, it does pose a big problem for
non-ruby tools like LittleChef, which depended on JSON to
inter-operate.

Is there a way to generate metadata.json for all cookbooks that
doesn’t need a Chef Server?
“knife cookbook metadata” doesn’t classify there, unfortunately.

Cheers,
Miquel
knife cookbook metadata doesn’t need to talk to the server to work, but the config checker will error out if you don’t have a key. So the best solution is to patch knife so that the config checker only cares about your private key when running a command that requires it.


Dan DeLeo


#3

That would be a possible solution. It is not a good one, because you
need to install the whole chef package plus dependencies to use knife,
and you even get a chef-client running, which you definitely don’t
want in your desktop.

Isn’t it possible to for example use a rake task, and only need to
install a chef “light” package or something?

2011/6/17 Daniel DeLeo dan@kallistec.com:

On Friday, June 17, 2011 at 2:10 AM, Miquel Torres wrote:

Hi all,

last month there was a post regarding changes in the opscode cookbooks repo:
http://www.opscode.com/blog/2011/05/24/update-on-the-future-of-opscodes-cookbooks/

While I can understand the reasons that made Opscode remove the
metadata.json from all cookbooks, it does pose a big problem for
non-ruby tools like LittleChef, which depended on JSON to
inter-operate.

Is there a way to generate metadata.json for all cookbooks that
doesn’t need a Chef Server?
“knife cookbook metadata” doesn’t classify there, unfortunately.

Cheers,
Miquel
knife cookbook metadata doesn’t need to talk to the server to work, but the config checker will error out if you don’t have a key. So the best solution is to patch knife so that the config checker only cares about your private key when running a command that requires it.


Dan DeLeo


#4

On Friday, June 17, 2011 at 9:34 AM, Miquel Torres wrote:

That would be a possible solution. It is not a good one, because you
need to install the whole chef package plus dependencies to use knife,
and you even get a chef-client running, which you definitely don’t
want in your desktop.

Isn’t it possible to for example use a rake task, and only need to
install a chef “light” package or something?

Mostly not. Whether or not a given packaging system starts a chef-client daemon is a packaging issue. I’ll defer to the apt experts for instructions on how to do this on Ubuntu, but the rubygems packages, for example, do not start chef-client.

Could you explain more about what issues are caused by installing the “whole chef package plus dependencies”? Are there conflicts with other packages?


Dan DeLeo

2011/6/17 Daniel DeLeo <dan@kallistec.com (mailto:dan@kallistec.com)>:

On Friday, June 17, 2011 at 2:10 AM, Miquel Torres wrote:

Hi all,

last month there was a post regarding changes in the opscode cookbooks repo:
http://www.opscode.com/blog/2011/05/24/update-on-the-future-of-opscodes-cookbooks/

While I can understand the reasons that made Opscode remove the
metadata.json from all cookbooks, it does pose a big problem for
non-ruby tools like LittleChef, which depended on JSON to
inter-operate.

Is there a way to generate metadata.json for all cookbooks that
doesn’t need a Chef Server?
“knife cookbook metadata” doesn’t classify there, unfortunately.

Cheers,
Miquel
knife cookbook metadata doesn’t need to talk to the server to work, but the config checker will error out if you don’t have a key. So the best solution is to patch knife so that the config checker only cares about your private key when running a command that requires it.


Dan DeLeo


#5

there are no conflicts, and certainly can be done. It just has many
dependencies for what in LittleChef’s case would be a small use case
(translating ruby to JSON, basically).

Anyway, like you said the solution would be to patch knife so that it
doesn’t require a server for tasks like “cookbook metadata”. I hope
someone wants to implement that! (hint, hint :slight_smile:

Independently of the need for that knife change, another solution
would be to reintroduce the metadata.json files in the repo, or to
create a “json” branch, which is always in sync with master but
includes the metadata.json files. That can even be automatically done
with some git commit hooks.

Thanks for the answer,
Miquel

2011/6/17 Daniel DeLeo dan@kallistec.com:

On Friday, June 17, 2011 at 9:34 AM, Miquel Torres wrote:

That would be a possible solution. It is not a good one, because you
need to install the whole chef package plus dependencies to use knife,
and you even get a chef-client running, which you definitely don’t
want in your desktop.

Isn’t it possible to for example use a rake task, and only need to
install a chef “light” package or something?

Mostly not. Whether or not a given packaging system starts a chef-client daemon is a packaging issue. I’ll defer to the apt experts for instructions on how to do this on Ubuntu, but the rubygems packages, for example, do not start chef-client.

Could you explain more about what issues are caused by installing the “whole chef package plus dependencies”? Are there conflicts with other packages?


Dan DeLeo

2011/6/17 Daniel DeLeo <dan@kallistec.com (mailto:dan@kallistec.com)>:

On Friday, June 17, 2011 at 2:10 AM, Miquel Torres wrote:

Hi all,

last month there was a post regarding changes in the opscode cookbooks repo:
http://www.opscode.com/blog/2011/05/24/update-on-the-future-of-opscodes-cookbooks/

While I can understand the reasons that made Opscode remove the
metadata.json from all cookbooks, it does pose a big problem for
non-ruby tools like LittleChef, which depended on JSON to
inter-operate.

Is there a way to generate metadata.json for all cookbooks that
doesn’t need a Chef Server?
“knife cookbook metadata” doesn’t classify there, unfortunately.

Cheers,
Miquel
knife cookbook metadata doesn’t need to talk to the server to work, but the config checker will error out if you don’t have a key. So the best solution is to patch knife so that the config checker only cares about your private key when running a command that requires it.


Dan DeLeo


#6

Miquel,

On Fri, Jun 17, 2011 at 10:54 AM, Miquel Torres tobami@googlemail.com wrote:

there are no conflicts, and certainly can be done. It just has many
dependencies for what in LittleChef’s case would be a small use case
(translating ruby to JSON, basically).

This comes down to what the repositories are actually for. We’re
hosting the code in development - those repositories make no promises
about functionality, release quality, or anything else. The packages
you can retrieve from community.opscode.com, on the other hand, do
have compiled metadata.json files - because they are released
artifacts that are meant to be consumed.

Mandating changes to developer workflows is, in my opinion, the wrong
way to go about it - I think we should be standardizing on a mechanism
for retrieval of stable artifacts. We do need to make it easier to
integrate those with the developer workflow, but the difference is
important - and this case is why. As the maintainers, we made an easy
choice - putting those generated artifacts in the source repository
meant a needless amount of churn, and we always rely on the version
generated. The metadata is all there in the released artifacts - it’s
the marriage of dev workflow with the tooling that is the problem
here.

Adam


Opscode, Inc.
Adam Jacob, Chief Product Officer
T: (206) 619-7151 E: adam@opscode.com


#7

On Sat, Jun 18, 2011 at 3:54 AM, Miquel Torres tobami@googlemail.com wrote:

there are no conflicts, and certainly can be done. It just has many
dependencies for what in LittleChef’s case would be a small use case
(translating ruby to JSON, basically).

Anyway, like you said the solution would be to patch knife so that it
doesn’t require a server for tasks like “cookbook metadata”. I hope
someone wants to implement that! (hint, hint :slight_smile:

Independently of the need for that knife change, another solution
would be to reintroduce the metadata.json files in the repo, or to
create a “json” branch, which is always in sync with master but
includes the metadata.json files. That can even be automatically done
with some git commit hooks.

The github.com/cookbooks account tracks the opscode repo and will
likely start tracking other cookbooks as soon a jnewland id happy with
some changes I made to the cookbooks-bot.
When we update the cookbook we do generate the json file - using full
chef. This works for us at the moment.

jnewland was setting up a server to make the updates run daily - I
haven’t tracked that over the last week or so, but he is on irc if you
want to enquire about when the update is scheduled to run.

HTH?

Thanks for the answer,
Miquel

2011/6/17 Daniel DeLeo dan@kallistec.com:

On Friday, June 17, 2011 at 9:34 AM, Miquel Torres wrote:

That would be a possible solution. It is not a good one, because you
need to install the whole chef package plus dependencies to use knife,
and you even get a chef-client running, which you definitely don’t
want in your desktop.

Isn’t it possible to for example use a rake task, and only need to
install a chef “light” package or something?

Mostly not. Whether or not a given packaging system starts a chef-client daemon is a packaging issue. I’ll defer to the apt experts for instructions on how to do this on Ubuntu, but the rubygems packages, for example, do not start chef-client.

Could you explain more about what issues are caused by installing the “whole chef package plus dependencies”? Are there conflicts with other packages?


Dan DeLeo

2011/6/17 Daniel DeLeo <dan@kallistec.com (mailto:dan@kallistec.com)>:

On Friday, June 17, 2011 at 2:10 AM, Miquel Torres wrote:

Hi all,

last month there was a post regarding changes in the opscode cookbooks repo:
http://www.opscode.com/blog/2011/05/24/update-on-the-future-of-opscodes-cookbooks/

While I can understand the reasons that made Opscode remove the
metadata.json from all cookbooks, it does pose a big problem for
non-ruby tools like LittleChef, which depended on JSON to
inter-operate.

Is there a way to generate metadata.json for all cookbooks that
doesn’t need a Chef Server?
“knife cookbook metadata” doesn’t classify there, unfortunately.

Cheers,
Miquel
knife cookbook metadata doesn’t need to talk to the server to work, but the config checker will error out if you don’t have a key. So the best solution is to patch knife so that the config checker only cares about your private key when running a command that requires it.


Dan DeLeo


πόλλ’ οἶδ ἀλώπηξ, ἀλλ’ ἐχῖνος ἓν μέγα
[The fox knows many things, but the hedgehog knows one big thing.]
Archilochus, Greek poet (c. 680 BC – c. 645 BC)
http://hedgehogshiatus.com


#8

Hi Adam,

thanks for the detailed answer. It seems then that the way to go is to
use http://community.opscode.com/cookbooks/
either manually downloads or through a client that interfaces with the REST API.

I guess we are just used to cloning a Github repo, because it is so
convenient. But that will do.

That does not change the fact that patching knife so that it doesn’t
require a Server for “local” operations is still a good idea. Cookbook
developers that don’t use a Chef Server may still want to generate the
metadata.json for their own cookbooks (and .json for roles).

Miquel

2011/6/18 Hedge Hog hedgehogshiatus@gmail.com:

On Sat, Jun 18, 2011 at 3:54 AM, Miquel Torres tobami@googlemail.com wrote:

there are no conflicts, and certainly can be done. It just has many
dependencies for what in LittleChef’s case would be a small use case
(translating ruby to JSON, basically).

Anyway, like you said the solution would be to patch knife so that it
doesn’t require a server for tasks like “cookbook metadata”. I hope
someone wants to implement that! (hint, hint :slight_smile:

Independently of the need for that knife change, another solution
would be to reintroduce the metadata.json files in the repo, or to
create a “json” branch, which is always in sync with master but
includes the metadata.json files. That can even be automatically done
with some git commit hooks.

The github.com/cookbooks account tracks the opscode repo and will
likely start tracking other cookbooks as soon a jnewland id happy with
some changes I made to the cookbooks-bot.
When we update the cookbook we do generate the json file - using full
chef. This works for us at the moment.

jnewland was setting up a server to make the updates run daily - I
haven’t tracked that over the last week or so, but he is on irc if you
want to enquire about when the update is scheduled to run.

HTH?

Thanks for the answer,
Miquel

2011/6/17 Daniel DeLeo dan@kallistec.com:

On Friday, June 17, 2011 at 9:34 AM, Miquel Torres wrote:

That would be a possible solution. It is not a good one, because you
need to install the whole chef package plus dependencies to use knife,
and you even get a chef-client running, which you definitely don’t
want in your desktop.

Isn’t it possible to for example use a rake task, and only need to
install a chef “light” package or something?

Mostly not. Whether or not a given packaging system starts a chef-client daemon is a packaging issue. I’ll defer to the apt experts for instructions on how to do this on Ubuntu, but the rubygems packages, for example, do not start chef-client.

Could you explain more about what issues are caused by installing the “whole chef package plus dependencies”? Are there conflicts with other packages?


Dan DeLeo

2011/6/17 Daniel DeLeo <dan@kallistec.com (mailto:dan@kallistec.com)>:

On Friday, June 17, 2011 at 2:10 AM, Miquel Torres wrote:

Hi all,

last month there was a post regarding changes in the opscode cookbooks repo:
http://www.opscode.com/blog/2011/05/24/update-on-the-future-of-opscodes-cookbooks/

While I can understand the reasons that made Opscode remove the
metadata.json from all cookbooks, it does pose a big problem for
non-ruby tools like LittleChef, which depended on JSON to
inter-operate.

Is there a way to generate metadata.json for all cookbooks that
doesn’t need a Chef Server?
“knife cookbook metadata” doesn’t classify there, unfortunately.

Cheers,
Miquel
knife cookbook metadata doesn’t need to talk to the server to work, but the config checker will error out if you don’t have a key. So the best solution is to patch knife so that the config checker only cares about your private key when running a command that requires it.


Dan DeLeo


πόλλ’ οἶδ ἀλώπηξ, ἀλλ’ ἐχῖνος ἓν μέγα
[The fox knows many things, but the hedgehog knows one big thing.]
Archilochus, Greek poet (c. 680 BC – c. 645 BC)
http://hedgehogshiatus.com


#9

When we update the cookbook we do generate the json file - using full
chef. This works for us at the moment.
I had a look and recently updated cookbooks do not have a
metadata.json (coming when jnewland finishes?)

So yes, github.com/cookbooks looks like another solution, though
cloning dozens of repos is not nearly as convenient.
Thanks!

2011/6/18 Hedge Hog hedgehogshiatus@gmail.com:

On Sat, Jun 18, 2011 at 3:54 AM, Miquel Torres tobami@googlemail.com wrote:

there are no conflicts, and certainly can be done. It just has many
dependencies for what in LittleChef’s case would be a small use case
(translating ruby to JSON, basically).

Anyway, like you said the solution would be to patch knife so that it
doesn’t require a server for tasks like “cookbook metadata”. I hope
someone wants to implement that! (hint, hint :slight_smile:

Independently of the need for that knife change, another solution
would be to reintroduce the metadata.json files in the repo, or to
create a “json” branch, which is always in sync with master but
includes the metadata.json files. That can even be automatically done
with some git commit hooks.

The github.com/cookbooks account tracks the opscode repo and will
likely start tracking other cookbooks as soon a jnewland id happy with
some changes I made to the cookbooks-bot.
When we update the cookbook we do generate the json file - using full
chef. This works for us at the moment.

jnewland was setting up a server to make the updates run daily - I
haven’t tracked that over the last week or so, but he is on irc if you
want to enquire about when the update is scheduled to run.

HTH?

Thanks for the answer,
Miquel

2011/6/17 Daniel DeLeo dan@kallistec.com:

On Friday, June 17, 2011 at 9:34 AM, Miquel Torres wrote:

That would be a possible solution. It is not a good one, because you
need to install the whole chef package plus dependencies to use knife,
and you even get a chef-client running, which you definitely don’t
want in your desktop.

Isn’t it possible to for example use a rake task, and only need to
install a chef “light” package or something?

Mostly not. Whether or not a given packaging system starts a chef-client daemon is a packaging issue. I’ll defer to the apt experts for instructions on how to do this on Ubuntu, but the rubygems packages, for example, do not start chef-client.

Could you explain more about what issues are caused by installing the “whole chef package plus dependencies”? Are there conflicts with other packages?


Dan DeLeo

2011/6/17 Daniel DeLeo <dan@kallistec.com (mailto:dan@kallistec.com)>:

On Friday, June 17, 2011 at 2:10 AM, Miquel Torres wrote:

Hi all,

last month there was a post regarding changes in the opscode cookbooks repo:
http://www.opscode.com/blog/2011/05/24/update-on-the-future-of-opscodes-cookbooks/

While I can understand the reasons that made Opscode remove the
metadata.json from all cookbooks, it does pose a big problem for
non-ruby tools like LittleChef, which depended on JSON to
inter-operate.

Is there a way to generate metadata.json for all cookbooks that
doesn’t need a Chef Server?
“knife cookbook metadata” doesn’t classify there, unfortunately.

Cheers,
Miquel
knife cookbook metadata doesn’t need to talk to the server to work, but the config checker will error out if you don’t have a key. So the best solution is to patch knife so that the config checker only cares about your private key when running a command that requires it.


Dan DeLeo


πόλλ’ οἶδ ἀλώπηξ, ἀλλ’ ἐχῖνος ἓν μέγα
[The fox knows many things, but the hedgehog knows one big thing.]
Archilochus, Greek poet (c. 680 BC – c. 645 BC)
http://hedgehogshiatus.com


#10

What is this LittleChef?

On Jun 17, 2011, at 4:10 AM, Miquel Torres wrote:

Hi all,

last month there was a post regarding changes in the opscode cookbooks repo:
http://www.opscode.com/blog/2011/05/24/update-on-the-future-of-opscodes-cookbooks/

While I can understand the reasons that made Opscode remove the
metadata.json from all cookbooks, it does pose a big problem for
non-ruby tools like LittleChef, which depended on JSON to
inter-operate.

Is there a way to generate metadata.json for all cookbooks that
doesn’t need a Chef Server?
“knife cookbook metadata” doesn’t classify there, unfortunately.

Cheers,
Miquel


#11

It’s a Python-based package that allows you to build nodes
withchef-solo by pushing cookbooks to the nodes via Fabric.


On Tue, Jun 21, 2011 at 9:33 AM, Jeffrey E. Sussna jes@ingineering.it wrote:

What is this LittleChef?

On Jun 17, 2011, at 4:10 AM, Miquel Torres wrote:

Hi all,

last month there was a post regarding changes in the opscode cookbooks repo:
http://www.opscode.com/blog/2011/05/24/update-on-the-future-of-opscodes-cookbooks/

While I can understand the reasons that made Opscode remove the
metadata.json from all cookbooks, it does pose a big problem for
non-ruby tools like LittleChef, which depended on JSON to
inter-operate.

Is there a way to generate metadata.json for all cookbooks that
doesn’t need a Chef Server?
“knife cookbook metadata” doesn’t classify there, unfortunately.

Cheers,
Miquel


#12

I’d rather say, “via ssh” :wink:

2011/6/21 Grig Gheorghiu grig.gheorghiu@gmail.com:

It’s a Python-based package that allows you to build nodes
withchef-solo by pushing cookbooks to the nodes via Fabric.

https://github.com/tobami/littlechef
http://sysadvent.blogspot.com/2010/12/day-9-automated-deployments-with.html

On Tue, Jun 21, 2011 at 9:33 AM, Jeffrey E. Sussna jes@ingineering.it wrote:

What is this LittleChef?

On Jun 17, 2011, at 4:10 AM, Miquel Torres wrote:

Hi all,

last month there was a post regarding changes in the opscode cookbooks repo:
http://www.opscode.com/blog/2011/05/24/update-on-the-future-of-opscodes-cookbooks/

While I can understand the reasons that made Opscode remove the
metadata.json from all cookbooks, it does pose a big problem for
non-ruby tools like LittleChef, which depended on JSON to
inter-operate.

Is there a way to generate metadata.json for all cookbooks that
doesn’t need a Chef Server?
“knife cookbook metadata” doesn’t classify there, unfortunately.

Cheers,
Miquel


#13

@Daniel, @all: a user pointed out that you can in fact use knife cookbook metadata without a Server nor knife.rb configuration.

I investigated a bit and Chef 0.8 didn’t allow the command
(server-less), but Chef 0.10 does. It prints out “WARNING: No knife
configuration file found”, but the metadata.json are created out of
the metadata.rb.

That definitely settles the matter :wink:

Thanks a lot!
Miquel

2011/6/21 Miquel Torres tobami@googlemail.com:

I’d rather say, “via ssh” :wink:

2011/6/21 Grig Gheorghiu grig.gheorghiu@gmail.com:

It’s a Python-based package that allows you to build nodes
withchef-solo by pushing cookbooks to the nodes via Fabric.

https://github.com/tobami/littlechef
http://sysadvent.blogspot.com/2010/12/day-9-automated-deployments-with.html

On Tue, Jun 21, 2011 at 9:33 AM, Jeffrey E. Sussna jes@ingineering.it wrote:

What is this LittleChef?

On Jun 17, 2011, at 4:10 AM, Miquel Torres wrote:

Hi all,

last month there was a post regarding changes in the opscode cookbooks repo:
http://www.opscode.com/blog/2011/05/24/update-on-the-future-of-opscodes-cookbooks/

While I can understand the reasons that made Opscode remove the
metadata.json from all cookbooks, it does pose a big problem for
non-ruby tools like LittleChef, which depended on JSON to
inter-operate.

Is there a way to generate metadata.json for all cookbooks that
doesn’t need a Chef Server?
“knife cookbook metadata” doesn’t classify there, unfortunately.

Cheers,
Miquel