What techniques do you use for controlling the deployment of recipe
changes via Chef server? We’re looking for a way to automate canary
deployments, rolling changes to first one, then a few, and then the
rest of the affected nodes.
New templates etc. can take advantage of file specificity (though it’d
be nice to have other options there for giving an arbitrary set of
nodes the same non-default file), but it’s not much help if you have a
new version of the actual recipe code or a whole cookbook or a role.
One option is to give the new version a different name and shuffle run
lists around, then move it into place and shuffle back, but that can
be awkward to automate, especially if you’re modifying something that
gets pulled in by other roles that depend on it.
You can copy cookbooks out and try them with chef-solo, but I don’t
trust that to get the same results as a full chef-client run against
the server; it feels like more of a unit test than a deployment step.
We’ve thought about migrating nodes between two different Chef servers
(or two different Opscode Platform accounts), but that seems like an
awfully heavyweight solution.
Other ideas?
–
Mark J. Reed markjreed@gmail.com
Mark,
I think the next major version of Chef will include the ability to have
environments. This will probably help you roll things out like you're
describing. You can track some of this work here:
http://github.com/sdelano/chef/tree/environments
On Tue, Oct 19, 2010 at 12:30 PM, Mark J. Reed markjreed@gmail.com wrote:
What techniques do you use for controlling the deployment of recipe
changes via Chef server? We're looking for a way to automate canary
deployments, rolling changes to first one, then a few, and then the
rest of the affected nodes.
New templates etc. can take advantage of file specificity (though it'd
be nice to have other options there for giving an arbitrary set of
nodes the same non-default file), but it's not much help if you have a
new version of the actual recipe code or a whole cookbook or a role.
One option is to give the new version a different name and shuffle run
lists around, then move it into place and shuffle back, but that can
be awkward to automate, especially if you're modifying something that
gets pulled in by other roles that depend on it.
You can copy cookbooks out and try them with chef-solo, but I don't
trust that to get the same results as a full chef-client run against
the server; it feels like more of a unit test than a deployment step.
We've thought about migrating nodes between two different Chef servers
(or two different Opscode Platform accounts), but that seems like an
awfully heavyweight solution.
Other ideas?
--
Mark J. Reed markjreed@gmail.com
--
Charles Sullivan
charlie.sullivan@gmail.com
Hey Mark,
In the mean time, I use multiple opscode platforms which is equivalent to multiple chef-servers for each 'environment'. I use a single cookbook git repo with qa/staging/production branches and promote cookbooks through the branches.
I also have my knife config determine what branch I'm on and configure the knife client to communicate with the corresponding server, setup correct keys, etc.
Alex
On Oct 19, 2010, at 12:11 PM, Charles Sullivan wrote:
Mark,
I think the next major version of Chef will include the ability to have environments. This will probably help you roll things out like you're describing. You can track some of this work here: http://github.com/sdelano/chef/tree/environments
On Tue, Oct 19, 2010 at 12:30 PM, Mark J. Reed markjreed@gmail.com wrote:
What techniques do you use for controlling the deployment of recipe
changes via Chef server? We're looking for a way to automate canary
deployments, rolling changes to first one, then a few, and then the
rest of the affected nodes.
New templates etc. can take advantage of file specificity (though it'd
be nice to have other options there for giving an arbitrary set of
nodes the same non-default file), but it's not much help if you have a
new version of the actual recipe code or a whole cookbook or a role.
One option is to give the new version a different name and shuffle run
lists around, then move it into place and shuffle back, but that can
be awkward to automate, especially if you're modifying something that
gets pulled in by other roles that depend on it.
You can copy cookbooks out and try them with chef-solo, but I don't
trust that to get the same results as a full chef-client run against
the server; it feels like more of a unit test than a deployment step.
We've thought about migrating nodes between two different Chef servers
(or two different Opscode Platform accounts), but that seems like an
awfully heavyweight solution.
Other ideas?
--
Mark J. Reed markjreed@gmail.com
--
Charles Sullivan
charlie.sullivan@gmail.com
On Tue, Oct 19, 2010 at 3:20 PM, Alex Soto apsoto@gmail.com wrote:
In the mean time, I use multiple opscode platforms which is equivalent to
multiple chef-servers for each 'environment'. I use a single cookbook git
repo with qa/staging/production branches and promote cookbooks through the
branches.
Do you swing nodes between platforms?
On 20/10/2010, at 06:20 , Alex Soto wrote:
...
I also have my knife config determine what branch I'm on and configure the knife client to communicate with the corresponding server, setup correct keys, etc.
That sounds interesting. How do you do that? Have you got any code you can share?
--
cheers,
Mike Williams
On Tue, Oct 19, 2010 at 3:34 PM, Mike Williams
mike@cogentconsulting.com.au wrote:
On 20/10/2010, at 06:20 , Alex Soto wrote:
I also have my knife config determine what branch I'm on and configure the knife client to communicate with the corresponding server, setup correct keys, etc.
That sounds interesting. How do you do that? Have you got any code you can share?
Here's what some of us do at Opscode using a directory structure:
mkdir ~/platform # create a base directory
cd ~/platform
git clone git://github.com/opscode/chef-repo.git ORGANIZATION
mkdir ORGANIZATION/.chef
cp /path/to/YOUR_KEY.pem ORGANIZATION/.chef/$USER.pem
cp /path/to/ORGANIZATION-validator.pem
ORGANIZATION/.chef/ORGANIZATION-validator.pem
cat > ORGANIZATION/.chef/knife.rb << EOF
config_dir = File.dirname(FILE)
cookbook_dir = File.join(config_dir, ".." ,"cookbooks")
organization = File.basename(File.expand_path(File.join(File.dirname(FILE),
"..")))
log_level :info
log_location STDOUT
node_name ENV['USER']
client_key "#{config_dir}/#{ENV['USER']}.pem"
chef_server_url "Sign In - Chef Manage"
cache_type 'BasicFile'
cache_options( :path => "#{config_dir}/checksums" )
cookbook_path [ cookbook_dir ]
validation_client_name "#{organization}-validator"
validation_key "#{config_dir}/#{organization}-validator.pem"
EOF
repeat this process for each organization, which allows you to switch
organizations by changing between the directories in a tree like:
platform/
platform/sprocket-prod/
platform/sprocket-preprod/
platform/personal-barnacles/
platform/sandbox-of-dewm/
Of course this is setup for using the platform using the directory
structure as a variable, but the knife configuration file is ruby so
you can choose different chef_server_urls than the platform based on
other factors.
Bryan
No I don't. Each environment has it's own nodes.
A side effect I kind of like is the node list is less cluttered with nodes that don't gave anything to do with each others environments.
Sent from my phone
On Oct 19, 2010, at 2:24 PM, Brian Akins brian@akins.org wrote:
On Tue, Oct 19, 2010 at 3:20 PM, Alex Soto apsoto@gmail.com wrote:
In the mean time, I use multiple opscode platforms which is equivalent to multiple chef-servers for each 'environment'. I use a single cookbook git repo with qa/staging/production branches and promote cookbooks through the branches.
Do you swing nodes between platforms?
-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1
Hello!
On Oct 19, 2010, at 4:34 PM, Mike Williams wrote:
On 20/10/2010, at 06:20 , Alex Soto wrote:
I also have my knife config determine what branch I'm on and configure the knife client to communicate with the corresponding server, setup correct keys, etc.
That sounds interesting. How do you do that? Have you got any code you can share?
I do it with this knife.rb for my local environment:
Opscode, Inc
Joshua Timberman, Technical Evangelist
IRC, Skype, Twitter, Github: jtimberman
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.10 (Darwin)
iEYEARECAAYFAky+LPQACgkQO97WSdVpzT0FKgCgmmwttxNKGi+92ZRf6NB/gqVg
Q/0AnR5fVH1ZV34atPFCWNxG2XZGI9ot
=ZzE4
-----END PGP SIGNATURE-----
Sure, the idea is a knife.rb that evaluates a corresponding knife-config.BRANCHNAME.rb to override the config based on the git branch you are on. Great example of the design win of having ruby as the config language. You could DRY this up a bit, but this is version 1. Looking forward to any 'patches'.
Alex
.chef/knife.rb
Default to 'QA' environment
validation_client_name 'channels-qa-validator'
validation_key './.chef/channels-qa-validator.pem'
chef_server_url 'Sign In - Chef Manage'
cache_options( :path => './.chef/channels-qa-checksums' )
log_level :debug
log_location STDOUT
node_name 'apsoto'
client_key '/Users/asoto/.chef/apsoto.pem'
cache_type 'BasicFile'
cookbook_path './cookbooks','./site-cookbooks'
Read in any environment specific config
branch_cmd = %q{git branch --no-color 2> /dev/null | sed -e '//d' -e 's/* (.*)/\1/'}
branch = #{branch_cmd}.strip
environment_config = File.join(File.dirname(FILE), "knife-config.#{branch}.rb")
if File.exists?(environment_config)
eval(IO.read(environment_config))
Chef::Log.info("Loaded environment specific configuration from #{environment_config}")
else
Chef::Log.info("No environment specific config file found for branch #{branch}")
end
.chef/knife-config.master.rb
log_level :info
log_location STDOUT
validation_client_name 'channels-qa-validator'
validation_key './.chef/channels-qa-validator.pem'
chef_server_url 'Sign In - Chef Manage'
cache_options( :path => './.chef/channels-qa-checksums' )
.chef/knife-config.staging.rb
log_level :debug
log_location STDOUT
validation_client_name 'channels-staging-validator'
validation_key './.chef/channels-staging-validator.pem'
chef_server_url 'Sign In - Chef Manage'
cache_options( :path => './.chef/channels-staging-checksums' )
On Oct 19, 2010, at 3:34 PM, Mike Williams wrote:
On 20/10/2010, at 06:20 , Alex Soto wrote:
...
I also have my knife config determine what branch I'm on and configure the knife client to communicate with the corresponding server, setup correct keys, etc.
That sounds interesting. How do you do that? Have you got any code you can share?
--
cheers,
Mike Williams