Source repositories as resources?

#1

Hello all. One of the things that I’d like chef to be able to do is check out
a certain version of our source to be installed. This makes more sense to me
than using something like Capistrano to maintain the correct version of the
source.

I’m sure that I could use a script to do this, but it seems like it would make
more sense to have Chef treat a source repository as a resource, which could
have providers for git, subversion, etc. Then you could easily set a
repository URL, authentication info, and revision numbers or branches, etc.

Is anybody else using chef to manage a source repository? How are other people
handling this?

Thanks,
Ian

#2

On 1/07/2009, at 10:18 AM, ian.ragsdale@gmail.com wrote:

Hello all. One of the things that I’d like chef to be able to do is
check out
a certain version of our source to be installed. This makes more
sense to me
than using something like Capistrano to maintain the correct version
of the
source.

I’m sure that I could use a script to do this, but it seems like it
would make
more sense to have Chef treat a source repository as a resource,
which could
have providers for git, subversion, etc. Then you could easily set a
repository URL, authentication info, and revision numbers or
branches, etc.

Is anybody else using chef to manage a source repository? How are
other people
handling this?

Ezra from EngineYard did great work with Chef-Deploy, a Capistrano
replacement driven by Chef. It is used throughout EY’s Solo and Flex
platforms, supports awesome push-based-hooking and other magic.

Hope this helps! :slight_smile:

–
AJ Christensen, Software Engineer
Opscode, Inc.
E: aj@opscode.com

#3

Ian,

Is anybody else using chef to manage a source repository? How are
other people
handling this?

In my subversion cookbook I’ve created a subversion_export definition
that I can use on any recipe like this:

subversion_export “webmail” do
url "https://xpto.com/svn/webmail/trunk/web"
path "/var/www/mail.ist.utl.pt"
revision 33
end

The definition is backed by a custom script uploaded to the host by
the subversion recipe.

It’s been working form me, but a repo resource would be a very nice
addition indeed.

Best Regards

Miguel Cabeça

#4

Ok, it sounds like this would be useful to at least a few people,
perhaps even to the chef-deploy project, as it might allow them to
simplify their code. Since I probably need it myself, I’m probably
gonna end up writing it soon. Before I do, I have two questions for
the opscode guys.

  1. Would you consider this for inclusion in Chef? I’d be happy to
    donate it and would love to not have to maintain and install it
    separately.
  2. If the answer to #1 is yes, would you prefer this be written using
    the grit gem, or would you prefer to have it call git on the command
    line? I can see good arguments for both, so if you guys would want to
    include it then I’d go with whatever you prefer.
  • Ian

On Jul 1, 2009, at 5:47 PM, Miguel Cabeça wrote:

Ian,

Is anybody else using chef to manage a source repository? How are
other people
handling this?

In my subversion cookbook I’ve created a subversion_export
definition that I can use on any recipe like this:

subversion_export “webmail” do
url "https://xpto.com/svn/webmail/trunk/web"
path "/var/www/mail.ist.utl.pt"
revision 33
end

The definition is backed by a custom script uploaded to the host by
the subversion recipe.

It’s been working form me, but a repo resource would be a very nice
addition indeed.

Best Regards

Miguel Cabeça

#5

On Wed, Jul 1, 2009 at 4:23 PM, Ian Ragsdaleian.ragsdale@gmail.com wrote:

Ok, it sounds like this would be useful to at least a few people, perhaps
even to the chef-deploy project, as it might allow them to simplify their
code. Since I probably need it myself, I’m probably gonna end up writing it
soon. Before I do, I have two questions for the opscode guys.

  1. Would you consider this for inclusion in Chef? I’d be happy to donate it
    and would love to not have to maintain and install it separately.

Absolutely. File a ticket for the new feature, and then publish it up
to Github.

  1. If the answer to #1 is yes, would you prefer this be written using the
    grit gem, or would you prefer to have it call git on the command line? I
    can see good arguments for both, so if you guys would want to include it
    then I’d go with whatever you prefer.

I think it’s fine to use the Grit gem - you can probably also include
some logic in the Git provider to auto-install the gem if you don’t
have it already on first use of the provider.

You’ll wind up building:

  1. The repo resource, which all the other resources will inherit from
  2. Helper resources for the various providers, a-la gem_package, etc.
  3. Providers for each VCS you want to support.

We’ll be happy to help with any details of how to implement this.

Best,
Adam

–
Opscode, Inc.
Adam Jacob, CTO
T: (206) 508-4759 E: adam@opscode.com

#6

On Jul 1, 2009, at 6:28 PM, Adam Jacob wrote:

On Wed, Jul 1, 2009 at 4:23 PM, Ian Ragsdaleian.ragsdale@gmail.com
wrote:

Ok, it sounds like this would be useful to at least a few people,
perhaps
even to the chef-deploy project, as it might allow them to simplify
their
code. Since I probably need it myself, I’m probably gonna end up
writing it
soon. Before I do, I have two questions for the opscode guys.

  1. Would you consider this for inclusion in Chef? I’d be happy to
    donate it
    and would love to not have to maintain and install it separately.

Absolutely. File a ticket for the new feature, and then publish it up
to Github.

Excellent. I’ll probably begin work on this tomorrow.

  1. If the answer to #1 is yes, would you prefer this be written
    using the
    grit gem, or would you prefer to have it call git on the command
    line? I
    can see good arguments for both, so if you guys would want to
    include it
    then I’d go with whatever you prefer.

I think it’s fine to use the Grit gem - you can probably also include
some logic in the Git provider to auto-install the gem if you don’t
have it already on first use of the provider.

Great - that probably makes for much cleaner code. It looks like Grit
also shells out to the command line for some actions as well - should
I do the same for the git package? Are there there other places in
the code where this is done?

You’ll wind up building:

  1. The repo resource, which all the other resources will inherit from
  2. Helper resources for the various providers, a-la gem_package, etc.
  3. Providers for each VCS you want to support.

We’ll be happy to help with any details of how to implement this.

That all makes plenty of sense - should I just mail this list if I
have questions?

Thanks,
Ian

#7

On 1/07/2009, at 4:28 PM, Adam Jacob wrote:

On Wed, Jul 1, 2009 at 4:23 PM, Ian Ragsdaleian.ragsdale@gmail.com
wrote:

Ok, it sounds like this would be useful to at least a few people,
perhaps
even to the chef-deploy project, as it might allow them to simplify
their
code. Since I probably need it myself, I’m probably gonna end up
writing it
soon. Before I do, I have two questions for the opscode guys.

  1. Would you consider this for inclusion in Chef? I’d be happy to
    donate it
    and would love to not have to maintain and install it separately.

Absolutely. File a ticket for the new feature, and then publish it up
to Github.

Someone filed one when this thread started up:

http://tickets.opscode.com/browse/CHEF-419

“Create SCM resource and providers for git & svn”

  1. If the answer to #1 is yes, would you prefer this be written
    using the
    grit gem, or would you prefer to have it call git on the command
    line? I
    can see good arguments for both, so if you guys would want to
    include it
    then I’d go with whatever you prefer.

I think it’s fine to use the Grit gem - you can probably also include
some logic in the Git provider to auto-install the gem if you don’t
have it already on first use of the provider.

Maybe a generic Git provider that abstracts both Grit and the Git
binaries away, then you can ‘provider Chef::Provider::Git::Grit’ in
your resources.

The magic sauce definitely would be using gem_package internally to
get Grit if you don’t have it :slight_smile:

You’ll wind up building:

  1. The repo resource, which all the other resources will inherit from
  2. Helper resources for the various providers, a-la gem_package, etc.
  3. Providers for each VCS you want to support.

We’ll be happy to help with any details of how to implement this.

Best,
Adam

–
Opscode, Inc.
Adam Jacob, CTO
T: (206) 508-4759 E: adam@opscode.com

–
AJ Christensen, Software Engineer
Opscode, Inc.
E: aj@opscode.com

#8

On 1/07/2009, at 4:35 PM, Ian Ragsdale wrote:

On Jul 1, 2009, at 6:28 PM, Adam Jacob wrote:

On Wed, Jul 1, 2009 at 4:23 PM, Ian
Ragsdaleian.ragsdale@gmail.com wrote:

Ok, it sounds like this would be useful to at least a few people,
perhaps
even to the chef-deploy project, as it might allow them to
simplify their
code. Since I probably need it myself, I’m probably gonna end up
writing it
soon. Before I do, I have two questions for the opscode guys.

  1. Would you consider this for inclusion in Chef? I’d be happy to
    donate it
    and would love to not have to maintain and install it separately.

Absolutely. File a ticket for the new feature, and then publish it
up
to Github.

Excellent. I’ll probably begin work on this tomorrow.

Before starting anything, I notice we haven’t got an individual CLA
from you (and not sure if you are covered under any of our approved
CCLA’s).

Have a read over http://wiki.opscode.com/display/opscode/Contributing :slight_smile:

  1. If the answer to #1 is yes, would you prefer this be written
    using the
    grit gem, or would you prefer to have it call git on the command
    line? I
    can see good arguments for both, so if you guys would want to
    include it
    then I’d go with whatever you prefer.

I think it’s fine to use the Grit gem - you can probably also include
some logic in the Git provider to auto-install the gem if you don’t
have it already on first use of the provider.

Great - that probably makes for much cleaner code. It looks like
Grit also shells out to the command line for some actions as well -
should I do the same for the git package? Are there there other
places in the code where this is done?

The package provider supports preseeding by making use of a
remote_file resource to fetch the appropriate preseed file from the
Chef Server, which is similar to what you’ll want to be achieving here.

You’ll wind up building:

  1. The repo resource, which all the other resources will inherit from
  2. Helper resources for the various providers, a-la gem_package, etc.
  3. Providers for each VCS you want to support.

We’ll be happy to help with any details of how to implement this.

That all makes plenty of sense - should I just mail this list if I
have questions?

Absolutely, we’re active on IRC, too: irc://irc.freenode.net/chef :slight_smile:

–
AJ Christensen, Software Engineer
Opscode, Inc.
E: aj@opscode.com