Versioning forked cookbooks


#1

Ohai Chefs,

I have a question about how to version forked community cookbooks. In our environment, we require a patch to the cron community cookbook, since it doesn’t exactly meet our needs. Since a patch has also been submitted to upstream, eventually the community version will increase its version, possibly conflicting with ours. As well, at some point in the future, we would like to switch back to the community version once the patch is accepted.

Given this background, how would you suggest to version the cron cookbook, as well as cookbooks that include it as a dependency?

Thanks!
Andrew


#2

generally, we use librarian (or berks) to manage the community cookbooks,
and point it to our forked repo till the patch is being merged and revert
back to the upstream version when its available. This same workflow as any
other rubygem + bundler
regards
ranjib

On Fri, Aug 8, 2014 at 12:18 PM, Andrew Brown anbrown@blackberry.com
wrote:

Ohai Chefs,

I have a question about how to version forked community cookbooks. In
our environment, we require a patch to the cron community cookbook, since
it doesn’t exactly meet our needs. Since a patch has also been submitted
to upstream, eventually the community version will increase its version,
possibly conflicting with ours. As well, at some point in the future, we
would like to switch back to the community version once the patch is
accepted.

Given this background, how would you suggest to version the cron
cookbook, as well as cookbooks that include it as a dependency?

Thanks!
Andrew


#3

We’re using berks as well, which is why the problem arrives. :slight_smile:
For example, berks uploads my forked cron 1.4.1 to my chef server, and then freezes it.
Community also releases cron 1.4.1, and then I revert my Berksfile change to use metadata again.
The next time Berkshelf runs, the community upload of 1.4.1 fails, as it is already frozen in my Chef server.

Cheers,
Andrew

On 2014-08-08, 3:34 PM, “Ranjib Dey” <dey.ranjib@gmail.commailto:dey.ranjib@gmail.com> wrote:

generally, we use librarian (or berks) to manage the community cookbooks, and point it to our forked repo till the patch is being merged and revert back to the upstream version when its available. This same workflow as any other rubygem + bundler
regards
ranjib

On Fri, Aug 8, 2014 at 12:18 PM, Andrew Brown <anbrown@blackberry.commailto:anbrown@blackberry.com> wrote:
Ohai Chefs,

I have a question about how to version forked community cookbooks. In our environment, we require a patch to the cron community cookbook, since it doesn’t exactly meet our needs. Since a patch has also been submitted to upstream, eventually the community version will increase its version, possibly conflicting with ours. As well, at some point in the future, we would like to switch back to the community version once the patch is accepted.

Given this background, how would you suggest to version the cron cookbook, as well as cookbooks that include it as a dependency?

Thanks!
Andrew


#4

This is currently an unsolved problem.
There’s a lot of contention around versioning and Chef, and there’s no
clear workflow for “what works best”.

The flow Ranjib mentioned is one we do as well, and we don’t freeze
cookbooks on upload for this reason.

Is it perfect? Nope. But it works for now, at least until we can introduce
better version identifiers in to Chef, chef-client, knife, and any plugins
that we leverage for that to speak the same versioning language.

-M

On Fri, Aug 8, 2014 at 3:39 PM, Andrew Brown anbrown@blackberry.com wrote:

We’re using berks as well, which is why the problem arrives. :slight_smile:
For example, berks uploads my forked cron 1.4.1 to my chef server, and
then freezes it.
Community also releases cron 1.4.1, and then I revert my Berksfile change
to use metadata again.
The next time Berkshelf runs, the community upload of 1.4.1 fails, as it
is already frozen in my Chef server.

Cheers,
Andrew

On 2014-08-08, 3:34 PM, “Ranjib Dey” dey.ranjib@gmail.com wrote:

generally, we use librarian (or berks) to manage the community
cookbooks, and point it to our forked repo till the patch is being merged
and revert back to the upstream version when its available. This same
workflow as any other rubygem + bundler
regards
ranjib

On Fri, Aug 8, 2014 at 12:18 PM, Andrew Brown anbrown@blackberry.com
wrote:

Ohai Chefs,

I have a question about how to version forked community cookbooks. In
our environment, we require a patch to the cron community cookbook, since
it doesn’t exactly meet our needs. Since a patch has also been submitted
to upstream, eventually the community version will increase its version,
possibly conflicting with ours. As well, at some point in the future, we
would like to switch back to the community version once the patch is
accepted.

Given this background, how would you suggest to version the cron
cookbook, as well as cookbooks that include it as a dependency?

Thanks!
Andrew


#5

On Friday, August 8, 2014 at 12:42 PM, Mike wrote:

This is currently an unsolved problem.
There’s a lot of contention around versioning and Chef, and there’s no clear workflow for “what works best”.

The flow Ranjib mentioned is one we do as well, and we don’t freeze cookbooks on upload for this reason.

Is it perfect? Nope. But it works for now, at least until we can introduce better version identifiers in to Chef, chef-client, knife, and any plugins that we leverage for that to speak the same versioning language.

-M
Not an immediate solution, but we’re working on it. See https://github.com/opscode/chef-dk/blob/master/POLICYFILE_README.md and https://github.com/opscode/chef-rfc/pull/20


Daniel DeLeo


#6

oh, we dont upload multiple versions of same cookbooks (neither do we lock
any cookbook version, since theres no 2 copies). I guess, you can either
tell berks not to freeze it, or bump up version (1.4.2) in your fork. I
like the second approach, but this will require you to explicitly delete
the cookbook when you change you berkfile to point to upstream again

On Fri, Aug 8, 2014 at 12:39 PM, Andrew Brown anbrown@blackberry.com
wrote:

We’re using berks as well, which is why the problem arrives. :slight_smile:
For example, berks uploads my forked cron 1.4.1 to my chef server, and
then freezes it.
Community also releases cron 1.4.1, and then I revert my Berksfile change
to use metadata again.
The next time Berkshelf runs, the community upload of 1.4.1 fails, as it
is already frozen in my Chef server.

Cheers,
Andrew

On 2014-08-08, 3:34 PM, “Ranjib Dey” dey.ranjib@gmail.com wrote:

generally, we use librarian (or berks) to manage the community
cookbooks, and point it to our forked repo till the patch is being merged
and revert back to the upstream version when its available. This same
workflow as any other rubygem + bundler
regards
ranjib

On Fri, Aug 8, 2014 at 12:18 PM, Andrew Brown anbrown@blackberry.com
wrote:

Ohai Chefs,

I have a question about how to version forked community cookbooks. In
our environment, we require a patch to the cron community cookbook, since
it doesn’t exactly meet our needs. Since a patch has also been submitted
to upstream, eventually the community version will increase its version,
possibly conflicting with ours. As well, at some point in the future, we
would like to switch back to the community version once the patch is
accepted.

Given this background, how would you suggest to version the cron
cookbook, as well as cookbooks that include it as a dependency?

Thanks!
Andrew


#7

Thank you!

On 2014-08-08, 3:44 PM, “Daniel DeLeo” dan@kallistec.com wrote:

On Friday, August 8, 2014 at 12:42 PM, Mike wrote:

This is currently an unsolved problem.
There’s a lot of contention around versioning and Chef, and there’s no
clear workflow for “what works best”.

The flow Ranjib mentioned is one we do as well, and we don’t freeze
cookbooks on upload for this reason.

Is it perfect? Nope. But it works for now, at least until we can
introduce better version identifiers in to Chef, chef-client, knife, and
any plugins that we leverage for that to speak the same versioning
language.

-M
Not an immediate solution, but we¹re working on it. See
https://github.com/opscode/chef-dk/blob/master/POLICYFILE_README.md and
https://github.com/opscode/chef-rfc/pull/20


Daniel DeLeo


#8

There was once https://tickets.opscode.com/browse/CHEF-4027 to support
SemVer with prerelease version identifiers, that would help in this
situation.

Not sure if it will ever come, though…

On Fri, Aug 8, 2014 at 10:38 PM, Andrew Brown anbrown@blackberry.com wrote:

Thank you!

On 2014-08-08, 3:44 PM, “Daniel DeLeo” dan@kallistec.com wrote:

On Friday, August 8, 2014 at 12:42 PM, Mike wrote:

This is currently an unsolved problem.
There’s a lot of contention around versioning and Chef, and there’s no
clear workflow for “what works best”.

The flow Ranjib mentioned is one we do as well, and we don’t freeze
cookbooks on upload for this reason.

Is it perfect? Nope. But it works for now, at least until we can
introduce better version identifiers in to Chef, chef-client, knife, and
any plugins that we leverage for that to speak the same versioning
language.

-M
Not an immediate solution, but we¹re working on it. See
https://github.com/opscode/chef-dk/blob/master/POLICYFILE_README.md and
https://github.com/opscode/chef-rfc/pull/20


Daniel DeLeo


#9

For forked cookbooks I use submodules on chef-repo instead of specifying on
the Berksfile.

Cheers,
Braulio
Em 08/08/2014 16:18, “Andrew Brown” anbrown@blackberry.com escreveu:

Ohai Chefs,

I have a question about how to version forked community cookbooks. In
our environment, we require a patch to the cron community cookbook, since
it doesn’t exactly meet our needs. Since a patch has also been submitted
to upstream, eventually the community version will increase its version,
possibly conflicting with ours. As well, at some point in the future, we
would like to switch back to the community version once the patch is
accepted.

Given this background, how would you suggest to version the cron
cookbook, as well as cookbooks that include it as a dependency?

Thanks!
Andrew


#10

I’m afraid that there are too many options and too many practices. Since there are only 3 fields for release numbers, there is no option to have a “1.0.0.1” for a locally modified version of the upstream 1.0.0. And when you modify version “1.0.0” for a local patch, and call it “1.0.1” to get it to auto-update, you now have a conflict.

Personally, I like being a dirty rotten scoundrel and back numbering my in-house version of a cookbook, to say “0.9.0”. Leave space for a few revisions, and lock the version of the cookbook in the environments. Then upstream updates can be published and upated as desired without modifying the in-house verson. And when upstream gets the patches, just release the lock.

If you have to keep both cookbooks alongside each other, neither librarian-chef nor Berkshelf support deploying multiple versions of the same cookbook from the same ‘Berksfile.lock’ or ‘Cheffile.lock’. You need a different directory with a different Berksfile or a different Cheffile with the old version locked in, deploying only the relevant older cookbook and older dependencies. But this is true for any chef environment that serves multiple cookbook versions for multiple QA or Production environments from the same server.

You cannot rely on “Oh, I installed that before with Berkshelf or Libratian-chef so it will always be there.”, not unless you have a very thorough backup and lockdown system. It’s too easy to have a cookbook that conflicts in number with a new, desired upstream if you just increment numbers, and then you’re stick trying to support two cookbooks with the same name and the same number!


Nico Kadel-Garcia
Senior Systems Consultant
Email: nkadelgarcia-consultant@scholastic.com
Cell Phone: +1.339.368.2428


From: Ranjib Dey dey.ranjib@gmail.com
Sent: Friday, August 08, 2014 3:45 PM
To: chef@lists.opscode.com
Subject: [chef] Re: Re: Re: Versioning forked cookbooks

oh, we dont upload multiple versions of same cookbooks (neither do we lock any cookbook version, since theres no 2 copies). I guess, you can either tell berks not to freeze it, or bump up version (1.4.2) in your fork. I like the second approach, but this will require you to explicitly delete the cookbook when you change you berkfile to point to upstream again

On Fri, Aug 8, 2014 at 12:39 PM, Andrew Brown <anbrown@blackberry.commailto:anbrown@blackberry.com> wrote:
We’re using berks as well, which is why the problem arrives. :slight_smile:
For example, berks uploads my forked cron 1.4.1 to my chef server, and then freezes it.
Community also releases cron 1.4.1, and then I revert my Berksfile change to use metadata again.
The next time Berkshelf runs, the community upload of 1.4.1 fails, as it is already frozen in my Chef server.

Cheers,
Andrew

On 2014-08-08, 3:34 PM, “Ranjib Dey” <dey.ranjib@gmail.commailto:dey.ranjib@gmail.com> wrote:

generally, we use librarian (or berks) to manage the community cookbooks, and point it to our forked repo till the patch is being merged and revert back to the upstream version when its available. This same workflow as any other rubygem + bundler
regards
ranjib

On Fri, Aug 8, 2014 at 12:18 PM, Andrew Brown <anbrown@blackberry.commailto:anbrown@blackberry.com> wrote:
Ohai Chefs,

I have a question about how to version forked community cookbooks. In our environment, we require a patch to the cron community cookbook, since it doesn’t exactly meet our needs. Since a patch has also been submitted to upstream, eventually the community version will increase its version, possibly conflicting with ours. As well, at some point in the future, we would like to switch back to the community version once the patch is accepted.

Given this background, how would you suggest to version the cron cookbook, as well as cookbooks that include it as a dependency?

Thanks!
Andrew


#11

When any knife to a previously
known good chef server
fails, yet the node is up and
I can ssh into the chef server…

In /var/log/chef/client.log on
the server I see ECONN refused
errors connecting to local host
port 4000 with stack trace dumps.

I see no error in server.log.

Thoughts on how to proceed?

On Aug 10, 2014, at 12:30 PM, Torben Knerr mail@tknerr.de wrote:

There was once https://tickets.opscode.com/browse/CHEF-4027 to support
SemVer with prerelease version identifiers, that would help in this
situation.

Not sure if it will ever come, though…

On Fri, Aug 8, 2014 at 10:38 PM, Andrew Brown anbrown@blackberry.com wrote:
Thank you!

On 2014-08-08, 3:44 PM, “Daniel DeLeo” dan@kallistec.com wrote:

On Friday, August 8, 2014 at 12:42 PM, Mike wrote:

This is currently an unsolved problem.
There’s a lot of contention around versioning and Chef, and there’s no
clear workflow for “what works best”.

The flow Ranjib mentioned is one we do as well, and we don’t freeze
cookbooks on upload for this reason.

Is it perfect? Nope. But it works for now, at least until we can
introduce better version identifiers in to Chef, chef-client, knife, and
any plugins that we leverage for that to speak the same versioning
language.

-M
Not an immediate solution, but we¹re working on it. See
https://github.com/opscode/chef-dk/blob/master/POLICYFILE_README.md and
https://github.com/opscode/chef-rfc/pull/20


Daniel DeLeo


#12

On Monday, August 11, 2014 at 9:26 AM, Stuart Cracraft wrote:

When any knife to a previously
known good chef server
fails, yet the node is up and
I can ssh into the chef server…

In /var/log/chef/client.log on
the server I see ECONN refused
errors connecting to local host
port 4000 with stack trace dumps.

I see no error in server.log.

Thoughts on how to proceed?
Most likely your config is bad, http://localhost:4000 is/was a default setting for chef_server_url.


Daniel DeLeo


#13

Hi Stuart,

When you run chef-server-ctl status from the Chef Server, what does it show?

-Stephen

On Mon, Aug 11, 2014 at 9:26 AM, Stuart Cracraft smcracraft@me.com wrote:

When any knife to a previously
known good chef server
fails, yet the node is up and
I can ssh into the chef server…

In /var/log/chef/client.log on
the server I see ECONN refused
errors connecting to local host
port 4000 with stack trace dumps.

I see no error in server.log.

Thoughts on how to proceed?

On Aug 10, 2014, at 12:30 PM, Torben Knerr mail@tknerr.de wrote:

There was once https://tickets.opscode.com/browse/CHEF-4027 to support
SemVer with prerelease version identifiers, that would help in this
situation.

Not sure if it will ever come, though…

On Fri, Aug 8, 2014 at 10:38 PM, Andrew Brown anbrown@blackberry.com
wrote:

Thank you!

On 2014-08-08, 3:44 PM, “Daniel DeLeo” dan@kallistec.com wrote:

On Friday, August 8, 2014 at 12:42 PM, Mike wrote:

This is currently an unsolved problem.
There’s a lot of contention around versioning and Chef, and there’s no
clear workflow for “what works best”.

The flow Ranjib mentioned is one we do as well, and we don’t freeze
cookbooks on upload for this reason.

Is it perfect? Nope. But it works for now, at least until we can
introduce better version identifiers in to Chef, chef-client, knife,
and

any plugins that we leverage for that to speak the same versioning
language.

-M
Not an immediate solution, but we¹re working on it. See
https://github.com/opscode/chef-dk/blob/master/POLICYFILE_README.md
and

https://github.com/opscode/chef-rfc/pull/20


Daniel DeLeo


Stephen Delano
Software Development Engineer
Opscode, Inc.
1008 Western Avenue
Suite 601
Seattle, WA 98104