How to push to upstream cookbook-vendor branch?

Bryan_Berry · October 15, 2011, 6:42pm

I am trying to figure out how to “vendor branch” technique to push my
changes upstream cookbook.

Here is the workflow that I have in mind
knife cookbook site install <cookbook> vim cookbooks/ #make changes
git checkout <cookbook>-vendor git checkout master path/to/modified-cookbook
$ git commit -am ‘merging select files into vendor-branch’

However, when I do $ git checkout chef--vendor && git ls-files
git seems to be tracking everything in this branch, not just the files for
the cookbook.

I have read through
http://wiki.opscode.com/display/chef/Working+with+Git+and+Cookbooks, but it
still isn’t clear to me how to push my changes to the upstream cookbook.
Perhaps I am missing something completely obvious. Can someone enlighten me?

The only other alternative I see is to track down the source of cookbook,
fork it, create a patch from my code, apply it to my fork, send a pull
request.

coderanger · October 15, 2011, 7:19pm

On Oct 15, 2011, at 11:42 AM, Bryan Berry wrote:

I am trying to figure out how to "vendor branch" technique to push my changes upstream cookbook.

Here is the workflow that I have in mind
$ knife cookbook site install
$ vim cookbooks/ #make changes
$ git checkout -vendor
$ git checkout master path/to/modified-cookbook
$ git commit -am 'merging select files into vendor-branch'

However, when I do $ git checkout chef--vendor && git ls-files
git seems to be tracking everything in this branch, not just the files for the cookbook.

I have read through http://wiki.opscode.com/display/chef/Working+with+Git+and+Cookbooks, but it still isn't clear to me how to push my changes to the upstream cookbook. Perhaps I am missing something completely obvious. Can someone enlighten me?

The only other alternative I see is to track down the source of cookbook, fork it, create a patch from my code, apply it to my fork, send a pull request.

As I explained on IRC, that is indeed the way to do it. The management in the cookbook repo is purely as an installed artifact right now.

--Noah

Bryan_Berry · October 15, 2011, 7:21pm

damn, i was hoping that 1) i misunderstood u on irc and 2) there was magic
that I was missing

On Sat, Oct 15, 2011 at 9:19 PM, Noah Kantrowitz noah@coderanger.netwrote:

On Oct 15, 2011, at 11:42 AM, Bryan Berry wrote:

I am trying to figure out how to "vendor branch" technique to push my
changes upstream cookbook.

Here is the workflow that I have in mind
$ knife cookbook site install
$ vim cookbooks/ #make changes
$ git checkout -vendor
$ git checkout master path/to/modified-cookbook
$ git commit -am 'merging select files into vendor-branch'

However, when I do $ git checkout chef--vendor && git
ls-files
git seems to be tracking everything in this branch, not just the files
for the cookbook.

I have read through
http://wiki.opscode.com/display/chef/Working+with+Git+and+Cookbooks, but
it still isn't clear to me how to push my changes to the upstream cookbook.
Perhaps I am missing something completely obvious. Can someone enlighten me?

The only other alternative I see is to track down the source of cookbook,
fork it, create a patch from my code, apply it to my fork, send a pull
request.

As I explained on IRC, that is indeed the way to do it. The management in
the cookbook repo is purely as an installed artifact right now.

--Noah
-----BEGIN PGP SIGNATURE-----

iEYEARECAAYFAk6Z3LIACgkQA9fEp2eRsU/9uQCg71c3GrXSTO3X0dgzUUvrD9qF
If4AoJdpH43XifeRWWcLn0F+x4nlmpBe
=22IR
-----END PGP SIGNATURE-----

Eric_G_Wolfe · October 16, 2011, 7:49pm

You don't have to fork a whole copy of each user's fork of cookbooks to
collaborate on community cookbooks, however. An easy way to do this, is
set up a remote for the user you want to fetch from, add a tracking
branch, push your changes back to your own origin, and finally do a pull
request. It may be a little confusing but its not as hard as it sounds.

First go find the user's cookbook repo you want to fork, there you can
find the read-only git link for that user's fork. Since you brought up
my yumrepo branch earlier on the mailing list, I'll use that as an
example here. Assuming you already have a local clone of your own
opscode/cookbooks fork, then you can go into your local repository and
manipulate a branch to point

Add a remote named "apenguin"

git remote add apenguin git://github.com/atomic-penguin/cookbooks.git

Fetch all the tags and branches from the user's upstream cookbook repo.

git fetch apenguin

Add a local tracking branch for apenguin/yumrepo branch

git branch --track yumrepo apenguin/yumrepo

Switch to your newly tracked local branch. Do a pull to update your

local tracking branch.
4) git checkout yumrepo; git pull yumrepo

Make change, stage change, commit change

git add .; git commit -m "Making changes"

Push change to a new branch on your own fork of opscode/cookbooks.

Assuming your own remote is "origin".
6) git push origin yumrepo:yumrepo

At which point you can proceed to do a pull request through the github
Web UI.

Eric G. Wolfe
Senior Linux Administrator,
IT Infrastructure Systems

Marshall University Computing Services
Drinko Library 428-K
One John Marshall Dr.
Huntington, WV 25755
Phone: 304.696.3428
Email: eric.wolfe@marshall.edu

Civilization is the limitless multiplication of unnecessary necessities.
-- Mark Twain

On 10/15/2011 03:21 PM, Bryan Berry wrote:

damn, i was hoping that 1) i misunderstood u on irc and 2) there was
magic that I was missing

On Sat, Oct 15, 2011 at 9:19 PM, Noah Kantrowitz <noah@coderanger.net
mailto:noah@coderanger.net> wrote:

On Oct 15, 2011, at 11:42 AM, Bryan Berry wrote:

> I am trying to figure out how to "vendor branch" technique to
push my changes upstream cookbook.
>
> Here is the workflow that I have in mind
> $ knife cookbook site install <cookbook>
> $ vim cookbooks/<cookbook>  #make changes
> $ git checkout <cookbook>-vendor
> $ git checkout master path/to/modified-cookbook
> $ git commit -am 'merging select files into vendor-branch'
>
> However, when I  do  $ git checkout chef-<cookbook>-vendor &&
git ls-files
> git seems to be tracking everything in this branch, not just the
files for the cookbook.
>
> I have read through
http://wiki.opscode.com/display/chef/Working+with+Git+and+Cookbooks,
but it still isn't clear to me how to push my changes to the
upstream cookbook. Perhaps I am missing something completely
obvious. Can someone enlighten me?
>
> The only other alternative I see is to track down the source of
cookbook, fork it, create a patch from my code, apply it to my
fork, send a pull request.

As I explained on IRC, that is indeed the way to do it. The
management in the cookbook repo is purely as an installed artifact
right now.

--Noah
-----BEGIN PGP SIGNATURE-----

iEYEARECAAYFAk6Z3LIACgkQA9fEp2eRsU/9uQCg71c3GrXSTO3X0dgzUUvrD9qF
If4AoJdpH43XifeRWWcLn0F+x4nlmpBe
=22IR
-----END PGP SIGNATURE-----

Adam_Jacob · October 17, 2011, 4:19am

I honestly think it's important to call out that there are two, sometimes very, different workflows here. One is the consumption of a released cookbook - as an end user. The community site is geared for that workflow, and not really for the second workflow: collaboration and contribution. In particular, we don't want to be forcing peoples workflows for the software they choose to open source - we do want to provide a place for people who need them to get copies. Community is closer to CPAN, Rubygems, or EPEL than it is to GitHub, and that's by design.

We are adding source code browsing, though, and better integration for people with common workflows (like the Github workflow) for collaboration to make it easier to collaborate.

Best,
Adam

On Oct 15, 2011, at 12:21 PM, Bryan Berry wrote:

damn, i was hoping that 1) i misunderstood u on irc and 2) there was magic that I was missing

On Sat, Oct 15, 2011 at 9:19 PM, Noah Kantrowitz noah@coderanger.net wrote:

On Oct 15, 2011, at 11:42 AM, Bryan Berry wrote:

I am trying to figure out how to "vendor branch" technique to push my changes upstream cookbook.

Here is the workflow that I have in mind
$ knife cookbook site install
$ vim cookbooks/ #make changes
$ git checkout -vendor
$ git checkout master path/to/modified-cookbook
$ git commit -am 'merging select files into vendor-branch'

However, when I do $ git checkout chef--vendor && git ls-files
git seems to be tracking everything in this branch, not just the files for the cookbook.

I have read through http://wiki.opscode.com/display/chef/Working+with+Git+and+Cookbooks, but it still isn't clear to me how to push my changes to the upstream cookbook. Perhaps I am missing something completely obvious. Can someone enlighten me?

The only other alternative I see is to track down the source of cookbook, fork it, create a patch from my code, apply it to my fork, send a pull request.

As I explained on IRC, that is indeed the way to do it. The management in the cookbook repo is purely as an installed artifact right now.

--Noah
-----BEGIN PGP SIGNATURE-----

iEYEARECAAYFAk6Z3LIACgkQA9fEp2eRsU/9uQCg71c3GrXSTO3X0dgzUUvrD9qF
If4AoJdpH43XifeRWWcLn0F+x4nlmpBe
=22IR
-----END PGP SIGNATURE-----

Bryan_Berry · October 17, 2011, 5:17am

thanks Eric, I will try this out.

Do you think that it would be best to make yumrepo a separate github repo as
Adam and Noah seem to suggest?

Back to my original mail. Do you think that my corp-epel would be useful to
other sysadmins? And do you think a data bag "urls" is the best way to
access that repo?

On Sun, Oct 16, 2011 at 9:49 PM, Eric G. Wolfe eric.wolfe@marshall.eduwrote:

**
You don't have to fork a whole copy of each user's fork of cookbooks to
collaborate on community cookbooks, however. An easy way to do this, is set
up a remote for the user you want to fetch from, add a tracking branch, push
your changes back to your own origin, and finally do a pull request. It may
be a little confusing but its not as hard as it sounds.

First go find the user's cookbook repo you want to fork, there you can find
the read-only git link for that user's fork. Since you brought up my
yumrepo branch earlier on the mailing list, I'll use that as an example
here. Assuming you already have a local clone of your own opscode/cookbooks
fork, then you can go into your local repository and manipulate a branch to
point

Add a remote named "apenguin"

git remote add apenguin git://github.com/atomic-penguin/cookbooks.git

Fetch all the tags and branches from the user's upstream cookbook repo.

git fetch apenguin

Add a local tracking branch for apenguin/yumrepo branch

git branch --track yumrepo apenguin/yumrepo

Switch to your newly tracked local branch. Do a pull to update your

local tracking branch.
4) git checkout yumrepo; git pull yumrepo

Make change, stage change, commit change

git add .; git commit -m "Making changes"

Push change to a new branch on your own fork of opscode/cookbooks.

Assuming your own remote is "origin".
6) git push origin yumrepo:yumrepo

At which point you can proceed to do a pull request through the github Web
UI.

Eric G. Wolfe
Senior Linux Administrator,
IT Infrastructure Systems

Marshall University Computing Services
Drinko Library 428-K
One John Marshall Dr.
Huntington, WV 25755
Phone: 304.696.3428
Email: eric.wolfe@marshall.edu

Civilization is the limitless multiplication of unnecessary necessities.
-- Mark Twain

On 10/15/2011 03:21 PM, Bryan Berry wrote:

damn, i was hoping that 1) i misunderstood u on irc and 2) there was magic
that I was missing

On Sat, Oct 15, 2011 at 9:19 PM, Noah Kantrowitz noah@coderanger.netwrote:

On Oct 15, 2011, at 11:42 AM, Bryan Berry wrote:

I am trying to figure out how to "vendor branch" technique to push my
changes upstream cookbook.

Here is the workflow that I have in mind
$ knife cookbook site install
$ vim cookbooks/ #make changes
$ git checkout -vendor
$ git checkout master path/to/modified-cookbook
$ git commit -am 'merging select files into vendor-branch'

However, when I do $ git checkout chef--vendor && git
ls-files
git seems to be tracking everything in this branch, not just the files
for the cookbook.

I have read through
http://wiki.opscode.com/display/chef/Working+with+Git+and+Cookbooks, but
it still isn't clear to me how to push my changes to the upstream cookbook.
Perhaps I am missing something completely obvious. Can someone enlighten me?

The only other alternative I see is to track down the source of cookbook,
fork it, create a patch from my code, apply it to my fork, send a pull
request.

As I explained on IRC, that is indeed the way to do it. The management in
the cookbook repo is purely as an installed artifact right now.

--Noah
-----BEGIN PGP SIGNATURE-----

iEYEARECAAYFAk6Z3LIACgkQA9fEp2eRsU/9uQCg71c3GrXSTO3X0dgzUUvrD9qF
If4AoJdpH43XifeRWWcLn0F+x4nlmpBe
=22IR
-----END PGP SIGNATURE-----

zts · October 17, 2011, 11:11am

Hi Adam,

On Mon, Oct 17, 2011 at 5:19 AM, Adam Jacob adam@opscode.com wrote:

I honestly think it's important to call out that there are two, sometimes
very, different workflows here. One is the consumption of a released
cookbook - as an end user. The community site is geared for that workflow,
and not really for the second workflow: collaboration and contribution. In
particular, we don't want to be forcing peoples workflows for the software
they choose to open source - we do want to provide a place for people who
need them to get copies. Community is closer to CPAN, Rubygems, or EPEL than
it is to GitHub, and that's by design.

Thanks for calling out those two workflows - though I'm now curious as
to whether my experience using community cookbooks is atypical. While
a good handful have been suitable "off the shelf", the larger portion
have required work to be usable.

Do users on Ubuntu (or other non-Centos platforms) tend to find that
community cookbooks need no changes beyond their own site-specific
customisations?

The two main reasons we've had to make changes have been:

patchy support for Centos, and,
unparameterised configuration which wasn't suitable for our environment.

It's really easy to fix these things in situ (ie, in the cookbook
directory that's just been installed), but feels much more complicated
to take those changes and submit them upstream. This is frustrating -
while it doesn't prevent me from achieving my goals, I feel like I
should be contributing my own learning back to the repository to make
the next person's job that bit easier.

Does this suggest a third workflow? A low-friction process for
submitting fixes benefits the quality of cookbooks on offer to the
community, and may appeal to a different population of users than
those interested in releasing cookbooks they've created from scratch.

We are adding source code browsing, though, and better integration for
people with common workflows (like the Github workflow) for collaboration to
make it easier to collaborate.

This is great to hear, and it's nice to see CPAN mentioned in the list
of analogous communities. The Perl community have got an amazing
number of things right there, and the details are often overlooked.

Thankyou,

Zac

Eric_G_Wolfe · October 17, 2011, 2:00pm

Responses inline.

On 10/17/2011 01:17 AM, Bryan Berry wrote:

thanks Eric, I will try this out.

Do you think that it would be best to make yumrepo a separate github
repo as Adam and Noah seem to suggest?
Do whatever comes easiest for you. I myself have rsync'd files from one
git work tree to another, rather than using some git workflow to do a
real merge.

Back to my original mail. Do you think that my corp-epel would be
useful to other sysadmins? And do you think a data bag "urls" is the
best way to access that repo?
I don't know. If all you're trying to do is manage a repo file to point
servers at an internal yum repo, then that should be an easy addition.
I do think there are probably a few people that may need something like
that. While there are probably others that just shove the one-off rpms
into cookbooks and deploy them with cookbook_file and rpm package
resources. You could do a recipe for an internal repo, which shouldn't
be incredibly difficult using the yum_repository LWRP. I might call it
something other than CORP-EPEL to avoid confusion with the Fedora EPEL.

If you're trying to manage the repo itself, thats different. I
personally don't have a lot of experience with that. That may be
something which is complex enough to warrant its own separate cookbook.

On Sun, Oct 16, 2011 at 9:49 PM, Eric G. Wolfe
<eric.wolfe@marshall.edu mailto:eric.wolfe@marshall.edu> wrote:

You don't have to fork a whole copy of each user's fork of
cookbooks to collaborate on community cookbooks, however.  An easy
way to do this, is set up a remote for the user you want to fetch
from, add a tracking branch, push your changes back to your own
origin, and finally do a pull request.  It may be a little
confusing but its not as hard as it sounds.

First go find the user's cookbook repo you want to fork, there you
can find the read-only git link for that user's fork.  Since you
brought up my yumrepo branch earlier on the mailing list, I'll use
that as an example here.  Assuming you already have a local clone
of your own opscode/cookbooks fork, then you can go into your
local repository and manipulate a branch to point

# Add a remote named "apenguin"
1) git remote add apenguin
git://github.com/atomic-penguin/cookbooks.git
<http://github.com/atomic-penguin/cookbooks.git>

# Fetch all the tags and branches from the user's upstream
cookbook repo.
2) git fetch apenguin

# Add a local tracking branch for apenguin/yumrepo branch
3) git branch --track yumrepo apenguin/yumrepo

# Switch to your newly tracked local branch.  Do a pull to update
your local tracking branch.
4) git checkout yumrepo; git pull yumrepo

# Make change, stage change, commit change
5) git add .; git commit -m "Making changes"

# Push change to a new branch on your own fork of
opscode/cookbooks.  Assuming your own remote is "origin".
6) git push origin yumrepo:yumrepo

At which point you can proceed to do a pull request through the
github Web UI.

Eric G. Wolfe
Senior Linux Administrator,
IT Infrastructure Systems
--------------------------------------
Marshall University Computing Services
Drinko Library 428-K
One John Marshall Dr.
Huntington, WV 25755
Phone:304.696.3428  <tel:304.696.3428>
Email:eric.wolfe@marshall.edu  <mailto:eric.wolfe@marshall.edu>

Civilization is the limitless multiplication of unnecessary necessities.
		-- Mark Twain


On 10/15/2011 03:21 PM, Bryan Berry wrote:

damn, i was hoping that 1) i misunderstood u on irc and 2) there
was magic that I was missing :(

On Sat, Oct 15, 2011 at 9:19 PM, Noah Kantrowitz
<noah@coderanger.net <mailto:noah@coderanger.net>> wrote:


    On Oct 15, 2011, at 11:42 AM, Bryan Berry wrote:

    > I am trying to figure out how to "vendor branch" technique
    to push my changes upstream cookbook.
    >
    > Here is the workflow that I have in mind
    > $ knife cookbook site install <cookbook>
    > $ vim cookbooks/<cookbook>  #make changes
    > $ git checkout <cookbook>-vendor
    > $ git checkout master path/to/modified-cookbook
    > $ git commit -am 'merging select files into vendor-branch'
    >
    > However, when I  do  $ git checkout chef-<cookbook>-vendor
    && git ls-files
    > git seems to be tracking everything in this branch, not
    just the files for the cookbook.
    >
    > I have read through
    http://wiki.opscode.com/display/chef/Working+with+Git+and+Cookbooks,
    but it still isn't clear to me how to push my changes to the
    upstream cookbook. Perhaps I am missing something completely
    obvious. Can someone enlighten me?
    >
    > The only other alternative I see is to track down the
    source of cookbook, fork it, create a patch from my code,
    apply it to my fork, send a pull request.

    As I explained on IRC, that is indeed the way to do it. The
    management in the cookbook repo is purely as an installed
    artifact right now.

    --Noah
    -----BEGIN PGP SIGNATURE-----

    iEYEARECAAYFAk6Z3LIACgkQA9fEp2eRsU/9uQCg71c3GrXSTO3X0dgzUUvrD9qF
    If4AoJdpH43XifeRWWcLn0F+x4nlmpBe
    =22IR
    -----END PGP SIGNATURE-----

Joshua_Timberman · October 17, 2011, 4:53pm

Hi Zac!

On Mon, Oct 17, 2011 at 11:11 AM, Zac Stevens zts@cryptocracy.com wrote:

Thanks for calling out those two workflows - though I'm now curious as
to whether my experience using community cookbooks is atypical. While
a good handful have been suitable "off the shelf", the larger portion
have required work to be usable.

Do users on Ubuntu (or other non-Centos platforms) tend to find that
community cookbooks need no changes beyond their own site-specific
customisations?

The two main reasons we've had to make changes have been:

patchy support for Centos, and,

unparameterised configuration which wasn't suitable for our environment.

When it comes to the Opscode cookbooks that are shared on the
community site[0], we're working on what it means to support various
platforms, including a list of which platforms we do support. This
will come in the form of documentation and an announcement.

Out of this, we will also gain:

Improvements to the cookbooks we share for the platforms we
officially support.
More "parameterization" of configuration through attributes and
data bags (depending on context).

Does this suggest a third workflow? A low-friction process for
submitting fixes benefits the quality of cookbooks on offer to the
community, and may appeal to a different population of users than
those interested in releasing cookbooks they've created from scratch.

The workflow for contributing fixes to the Opscode cookbooks follows
the similar workflow to the Chef project itself[1], and the wiki
documents the general process.

Workflow for contributing fixes to other cookbooks on the site depends
on what the authors of those cookbooks wants.

[0]: Only 116 out of 322, or 36% of the cookbooks on the community
site are published by Opscode.
[1]: http://wiki.opscode.com/display/chef/Working+with+Git

Topic		Replies	Views
Tips for injecting cookbooks from chef-repo to git repo Chef Infra (archive)	1	512	May 20, 2016
Working with vendor branches Chef Infra (archive)	2	294	March 15, 2011
Chef code version control Chef Infra (archive)	3	396	January 22, 2021
Customizing cookbooks Chef Infra (archive)	2	229	October 23, 2012
Better workflow for actively-developed cookbooks Chef Infra (archive)	19	397	June 25, 2012

How to push to upstream cookbook-vendor branch?

Add a remote named "apenguin"

Fetch all the tags and branches from the user's upstream cookbook repo.

Add a local tracking branch for apenguin/yumrepo branch

Switch to your newly tracked local branch. Do a pull to update your

Make change, stage change, commit change

Push change to a new branch on your own fork of opscode/cookbooks.

Eric G. Wolfe Senior Linux Administrator, IT Infrastructure Systems

Add a remote named "apenguin"

Fetch all the tags and branches from the user's upstream cookbook repo.

Add a local tracking branch for apenguin/yumrepo branch

Switch to your newly tracked local branch. Do a pull to update your

Make change, stage change, commit change

Push change to a new branch on your own fork of opscode/cookbooks.

Eric G. Wolfe Senior Linux Administrator, IT Infrastructure Systems

Related Topics

Eric G. Wolfe
Senior Linux Administrator,
IT Infrastructure Systems

Eric G. Wolfe
Senior Linux Administrator,
IT Infrastructure Systems