Re: Single chef server vs. multiple chef servers - pros and cons?

Chef Infra (archive)
1

I explored Berkshelf but it does NOT work with SVN though, which is what we have. So I can’t do this in my Berksfile

cookbook ‘my_company_cookbook’, svn: 'https://my_svn_repo.com/sompath/my_company_cookbook’

Chris

From: Yauhen Artsiukhou [mailto:jsirex@gmail.com]
Sent: Friday, October 24, 2014 6:01 AM
To: chef@lists.opscode.com
Subject: [chef] Re: RE: Re: RE: Re: RE: Re: Single chef server vs. multiple chef servers - pros and cons?

After first read, policy file looks very complicated. Especially if I have many cookbooks with its own policyfiles. How it would be merged?

To solve environment-cookbook pattern with berkshelf I've decided to remove cookbook itself and leave only Berksfile with list of top-level cookbooks.
This Berkfile has two sources:

  1. Berkshelf API (proxy to my Chef Server, hope berkshelf will work with chef server api directly in next releases)
  2. Community Site

Now you can do something like 'berks install', 'berks upload' for updating cookbook information.
Then, stage by stage you can do:

  1. berks apply dev
  2. knife ssh "chef_environment:dev" "sudo chef-client # force global reconfiguration
  4. berks apply qa
  5. knife ssh "chef_environment:qa" "sudo chef-client # force global reconfiguration
  6. Test all
  7. berks apply production

2014-10-23 21:13 GMT+03:00 Fouts, Chris <Chris.Fouts@sensus.commailto:Chris.Fouts@sensus.com>:
Yes, I'll look into the PolicyFile feature. Thanks!

Chris

-----Original Message-----
From: Daniel DeLeo [mailto:ddeleo@kallistec.commailto:ddeleo@kallistec.com] On Behalf Of Daniel DeLeo
Sent: Thursday, October 23, 2014 1:47 PM
To: chef@lists.opscode.commailto:chef@lists.opscode.com
Subject: [chef] Re: RE: Re: RE: Re: Single chef server vs. multiple chef servers - pros and cons?

On Thursday, October 23, 2014 at 10:38 AM, Fouts, Chris wrote:

Thanks Phil, I’m doing this NOW, and yes, I’m disciplined enough to update my metadata.rb files, and to pick up the correct version in my environments. However, one flaw of Chef (IMO) is that environments and roles are not versioned, so someone can upload an environment with a bad cookbook version to the Chef server. IOW, there is no built-in Chef mechanism to catch this “easy to screw up” process.

Chris

This isn’t viable in the short term, but we’re working on some features to address these problems (the “Policyfile” feature mentioned earlier in this thread). There’s a lot of information in the release announcement for ChefDK 0.3.0: ChefDK 0.3.0 Released! Introducing Policyfiles - Chef Blog | Chef

I can’t give you a timeline for when we expect this to be production ready, but it’s something you might wish to consider if you’re considering making drastic changes to your workflow.

--
Daniel DeLeo

2

Chris, I'm not using svn/git or other such sources in environment
Berksfile. Depends on your workflow Berksfile has only source to Chef
Server or to Chef Server and community Chef Supermarket. Thats all.

First case workflow:

  1. You develop/test and upload to chef server each cookbook separately with
    its dependencies.
  2. When you're ready you just say with Berkshelf to your chef server: "Hey,
    here is list of cookbooks I need with its version. Chef Server, look at
    your uploaded cookbooks, solve dependencies and apply it."

Here is Sample of Pseudo Environment Berksfile:
source "http://my-chef-server-via-berkshelf-api:4000"

cookbook "my-cookbook1", "= 1.2.3"
cookbook "my-cookbook2", "= 2.3.4"
cookbook "my-base", "~> 1.0" # also allowed, but I prefer to use explicit
version
cookbook "apache2", "~>2.1"

If some of required cookbook or its dependencies cannot be solved you'll
have to manually fix it (upload required cookbooks)

For me this is pretty new workflow and it still contains some minor bugs,
but it already works.
Hope it helps somebody.

2014-10-24 16:39 GMT+03:00 Fouts, Chris Chris.Fouts@sensus.com:

I explored Berkshelf but it does NOT work with SVN though, which is what
we have. So I can’t do this in my Berksfile

cookbook ‘my_company_cookbook’, svn: 'https://my_svn_repo.com/sompath/my_company_cookbook’

Chris

From: Yauhen Artsiukhou [mailto:jsirex@gmail.com]
Sent: Friday, October 24, 2014 6:01 AM
To: chef@lists.opscode.com
Subject: [chef] Re: RE: Re: RE: Re: RE: Re: Single chef server vs.
multiple chef servers - pros and cons?

After first read, policy file looks very complicated. Especially if I have
many cookbooks with its own policyfiles. How it would be merged?

To solve environment-cookbook pattern with berkshelf I've decided to
remove cookbook itself and leave only Berksfile with list of top-level
cookbooks.

This Berkfile has two sources:

  1. Berkshelf API (proxy to my Chef Server, hope berkshelf will work with
    chef server api directly in next releases)

  2. Community Site

Now you can do something like 'berks install', 'berks upload' for updating
cookbook information.

Then, stage by stage you can do:

  1. berks apply dev

  2. knife ssh "chef_environment:dev" "sudo chef-client # force global
    reconfiguration

  4. berks apply qa

  5. knife ssh "chef_environment:qa" "sudo chef-client # force global
    reconfiguration

  6. Test all

  7. berks apply production

2014-10-23 21:13 GMT+03:00 Fouts, Chris Chris.Fouts@sensus.com:

Yes, I'll look into the PolicyFile feature. Thanks!

Chris

-----Original Message-----
From: Daniel DeLeo [mailto:ddeleo@kallistec.com] On Behalf Of Daniel DeLeo
Sent: Thursday, October 23, 2014 1:47 PM
To: chef@lists.opscode.com
Subject: [chef] Re: RE: Re: RE: Re: Single chef server vs. multiple chef
servers - pros and cons?

On Thursday, October 23, 2014 at 10:38 AM, Fouts, Chris wrote:

Thanks Phil, I’m doing this NOW, and yes, I’m disciplined enough to
update my metadata.rb files, and to pick up the correct version in my
environments. However, one flaw of Chef (IMO) is that environments and
roles are not versioned, so someone can upload an environment with a bad
cookbook version to the Chef server. IOW, there is no built-in Chef
mechanism to catch this “easy to screw up” process.

Chris

This isn’t viable in the short term, but we’re working on some features to
address these problems (the “Policyfile” feature mentioned earlier in this
thread). There’s a lot of information in the release announcement for
ChefDK 0.3.0:
ChefDK 0.3.0 Released! Introducing Policyfiles - Chef Blog | Chef

I can’t give you a timeline for when we expect this to be production
ready, but it’s something you might wish to consider if you’re considering
making drastic changes to your workflow.

--
Daniel DeLeo

3

Thank you!

In your example, if “my_cookbook1” (or the others) are NOT in my local workstation but are in some file repository (SVN in my case), how, where, and when will Berkshelf fetched (cleckout) from my repository to my workstation, and then eventually uploaded to the chef server. I’m missing this workflow using Berkshelf

File repo <-----> Chef workstation <----> Chef server

Chris

From: Yauhen Artsiukhou [mailto:jsirex@gmail.com]
Sent: Saturday, October 25, 2014 8:51 AM
To: chef@lists.opscode.com
Subject: [chef] Re: Re: Single chef server vs. multiple chef servers - pros and cons?

Chris, I'm not using svn/git or other such sources in environment Berksfile. Depends on your workflow Berksfile has only source to Chef Server or to Chef Server and community Chef Supermarket. Thats all.

First case workflow:

  1. You develop/test and upload to chef server each cookbook separately with its dependencies.
  2. When you're ready you just say with Berkshelf to your chef server: "Hey, here is list of cookbooks I need with its version. Chef Server, look at your uploaded cookbooks, solve dependencies and apply it."

Here is Sample of Pseudo Environment Berksfile:
source "http://my-chef-server-via-berkshelf-api:4000"

cookbook "my-cookbook1", "= 1.2.3"
cookbook "my-cookbook2", "= 2.3.4"
cookbook "my-base", "~> 1.0" # also allowed, but I prefer to use explicit version
cookbook "apache2", "~>2.1"

If some of required cookbook or its dependencies cannot be solved you'll have to manually fix it (upload required cookbooks)

For me this is pretty new workflow and it still contains some minor bugs, but it already works.
Hope it helps somebody.

2014-10-24 16:39 GMT+03:00 Fouts, Chris <Chris.Fouts@sensus.commailto:Chris.Fouts@sensus.com>:
I explored Berkshelf but it does NOT work with SVN though, which is what we have. So I can’t do this in my Berksfile

cookbook ‘my_company_cookbook’, svn: 'https://my_svn_repo.com/sompath/my_company_cookbook’

Chris

From: Yauhen Artsiukhou [mailto:jsirex@gmail.commailto:jsirex@gmail.com]
Sent: Friday, October 24, 2014 6:01 AM
To: chef@lists.opscode.commailto:chef@lists.opscode.com
Subject: [chef] Re: RE: Re: RE: Re: RE: Re: Single chef server vs. multiple chef servers - pros and cons?

After first read, policy file looks very complicated. Especially if I have many cookbooks with its own policyfiles. How it would be merged?

To solve environment-cookbook pattern with berkshelf I've decided to remove cookbook itself and leave only Berksfile with list of top-level cookbooks.
This Berkfile has two sources:

  1. Berkshelf API (proxy to my Chef Server, hope berkshelf will work with chef server api directly in next releases)
  2. Community Site

Now you can do something like 'berks install', 'berks upload' for updating cookbook information.
Then, stage by stage you can do:

  1. berks apply dev
  2. knife ssh "chef_environment:dev" "sudo chef-client # force global reconfiguration
  4. berks apply qa
  5. knife ssh "chef_environment:qa" "sudo chef-client # force global reconfiguration
  6. Test all
  7. berks apply production

2014-10-23 21:13 GMT+03:00 Fouts, Chris <Chris.Fouts@sensus.commailto:Chris.Fouts@sensus.com>:
Yes, I'll look into the PolicyFile feature. Thanks!

Chris

-----Original Message-----
From: Daniel DeLeo [mailto:ddeleo@kallistec.commailto:ddeleo@kallistec.com] On Behalf Of Daniel DeLeo
Sent: Thursday, October 23, 2014 1:47 PM
To: chef@lists.opscode.commailto:chef@lists.opscode.com
Subject: [chef] Re: RE: Re: RE: Re: Single chef server vs. multiple chef servers - pros and cons?

On Thursday, October 23, 2014 at 10:38 AM, Fouts, Chris wrote:

Thanks Phil, I’m doing this NOW, and yes, I’m disciplined enough to update my metadata.rb files, and to pick up the correct version in my environments. However, one flaw of Chef (IMO) is that environments and roles are not versioned, so someone can upload an environment with a bad cookbook version to the Chef server. IOW, there is no built-in Chef mechanism to catch this “easy to screw up” process.

Chris

This isn’t viable in the short term, but we’re working on some features to address these problems (the “Policyfile” feature mentioned earlier in this thread). There’s a lot of information in the release announcement for ChefDK 0.3.0: ChefDK 0.3.0 Released! Introducing Policyfiles - Chef Blog | Chef

I can’t give you a timeline for when we expect this to be production ready, but it’s something you might wish to consider if you’re considering making drastic changes to your workflow.

--
Daniel DeLeo

4

I was enlightened on this during the Chef Intermediate class I just took. If I want to use Berkshelf, I would have to do in two steps since I have SVN, on what would normally be done with one step with a git repo. I’ll have to

  •    svn checkout my cookbooks to a local directory structure

  •    Use my Berksfile to refer to the the paths in above directory structure

Chris

From: Fouts, Chris [mailto:Chris.Fouts@Sensus.com]
Sent: Thursday, October 30, 2014 8:45 AM
To: chef@lists.opscode.com
Subject: [chef] RE: Re: Re: Single chef server vs. multiple chef servers - pros and cons?

Thank you!

In your example, if “my_cookbook1” (or the others) are NOT in my local workstation but are in some file repository (SVN in my case), how, where, and when will Berkshelf fetched (cleckout) from my repository to my workstation, and then eventually uploaded to the chef server. I’m missing this workflow using Berkshelf

File repo <-----> Chef workstation <----> Chef server

Chris

From: Yauhen Artsiukhou [mailto:jsirex@gmail.com]
Sent: Saturday, October 25, 2014 8:51 AM
To: chef@lists.opscode.commailto:chef@lists.opscode.com
Subject: [chef] Re: Re: Single chef server vs. multiple chef servers - pros and cons?

Chris, I'm not using svn/git or other such sources in environment Berksfile. Depends on your workflow Berksfile has only source to Chef Server or to Chef Server and community Chef Supermarket. Thats all.

First case workflow:

  1. You develop/test and upload to chef server each cookbook separately with its dependencies.
  2. When you're ready you just say with Berkshelf to your chef server: "Hey, here is list of cookbooks I need with its version. Chef Server, look at your uploaded cookbooks, solve dependencies and apply it."

Here is Sample of Pseudo Environment Berksfile:
source "http://my-chef-server-via-berkshelf-api:4000"

cookbook "my-cookbook1", "= 1.2.3"
cookbook "my-cookbook2", "= 2.3.4"
cookbook "my-base", "~> 1.0" # also allowed, but I prefer to use explicit version
cookbook "apache2", "~>2.1"

If some of required cookbook or its dependencies cannot be solved you'll have to manually fix it (upload required cookbooks)

For me this is pretty new workflow and it still contains some minor bugs, but it already works.
Hope it helps somebody.

2014-10-24 16:39 GMT+03:00 Fouts, Chris <Chris.Fouts@sensus.commailto:Chris.Fouts@sensus.com>:
I explored Berkshelf but it does NOT work with SVN though, which is what we have. So I can’t do this in my Berksfile

cookbook ‘my_company_cookbook’, svn: 'https://my_svn_repo.com/sompath/my_company_cookbook’

Chris

From: Yauhen Artsiukhou [mailto:jsirex@gmail.commailto:jsirex@gmail.com]
Sent: Friday, October 24, 2014 6:01 AM
To: chef@lists.opscode.commailto:chef@lists.opscode.com
Subject: [chef] Re: RE: Re: RE: Re: RE: Re: Single chef server vs. multiple chef servers - pros and cons?

After first read, policy file looks very complicated. Especially if I have many cookbooks with its own policyfiles. How it would be merged?

To solve environment-cookbook pattern with berkshelf I've decided to remove cookbook itself and leave only Berksfile with list of top-level cookbooks.
This Berkfile has two sources:

  1. Berkshelf API (proxy to my Chef Server, hope berkshelf will work with chef server api directly in next releases)
  2. Community Site

Now you can do something like 'berks install', 'berks upload' for updating cookbook information.
Then, stage by stage you can do:

  1. berks apply dev
  2. knife ssh "chef_environment:dev" "sudo chef-client # force global reconfiguration
  4. berks apply qa
  5. knife ssh "chef_environment:qa" "sudo chef-client # force global reconfiguration
  6. Test all
  7. berks apply production

2014-10-23 21:13 GMT+03:00 Fouts, Chris <Chris.Fouts@sensus.commailto:Chris.Fouts@sensus.com>:
Yes, I'll look into the PolicyFile feature. Thanks!

Chris

-----Original Message-----
From: Daniel DeLeo [mailto:ddeleo@kallistec.commailto:ddeleo@kallistec.com] On Behalf Of Daniel DeLeo
Sent: Thursday, October 23, 2014 1:47 PM
To: chef@lists.opscode.commailto:chef@lists.opscode.com
Subject: [chef] Re: RE: Re: RE: Re: Single chef server vs. multiple chef servers - pros and cons?

On Thursday, October 23, 2014 at 10:38 AM, Fouts, Chris wrote:

Thanks Phil, I’m doing this NOW, and yes, I’m disciplined enough to update my metadata.rb files, and to pick up the correct version in my environments. However, one flaw of Chef (IMO) is that environments and roles are not versioned, so someone can upload an environment with a bad cookbook version to the Chef server. IOW, there is no built-in Chef mechanism to catch this “easy to screw up” process.

Chris

This isn’t viable in the short term, but we’re working on some features to address these problems (the “Policyfile” feature mentioned earlier in this thread). There’s a lot of information in the release announcement for ChefDK 0.3.0: ChefDK 0.3.0 Released! Introducing Policyfiles - Chef Blog | Chef

I can’t give you a timeline for when we expect this to be production ready, but it’s something you might wish to consider if you’re considering making drastic changes to your workflow.

--
Daniel DeLeo