Chef-provisioning best practices for supporting multiple configurations

StFS · April 9, 2015, 6:42pm

Hello.

We’re starting to look into using chef-provisioning and things are looking promising except maybe for lack of documentation on best practices. The documentation at https://docs.chef.io/provisioning.html is good but it doesn’t really explain the best ways to structure your chef-provisioning code.

We have a system that can be set up on a cluster but it can also be set up on a single machine. We would like to use chef-provisioning to be able to set up multiple types of setups (single node, multiple nodes, using vagrant, using AWS, using our own EC2 compatable private cloud etc.). We are going to use these scripts to perform automatic tests, so when a new release of our software is issued, we want an automatic process to start and set everything up for us.

What I currently have are basically just some ruby files and an environment directory with json “chef environment” files.

The ruby files I have are:

aws-minimal.rb
vagrant-minimal.rb
myapp-1node.rb

The myapp-single-node.rb is very simple, it sets up a single “machine” and applies the necessary recipes to it. The idea later on is to have myapp-4node.rb etc to set our system up on more nodes as needed.

The aws-minimal.rb file looks like this:

require ‘chef/provisioning/fog_driver/driver’

with_machine_options({
:bootstrap_options => {
:flavor_id => ‘t2.medium’,
},
:ssh_username => ‘centos’,
:image_id => ‘ami-96a818fe’
})

Basically this sets everything up for AWS as you can see.

The way I run this is:

chef-client -z provision/aws-minimal.rb provision/myapp-1node.rb

I also have a vagrant-minimal.rb file that looks like this:

require ‘chef/provisioning/vagrant_driver’

vagrant_box ‘centos7’ do
url 'https://cdn.nextcode.com/contrib/nextcode_centos-7-x86_64_virtualbox.box’
end

with_machine_options :vagrant_options => {
‘vm.box’ => ‘centos7’
}

And if I want to run my myapp-1node.rb and use vagrant instead of AWS I run this:
?chef-client -z provision/vagrant-minimal.rb provision/myapp-1node.rb

So as you can see, I’m keeping the driver configuration separate from my provisioning scripts, hoping that they can stay agnostic to what kind of machine is being set up and in what environment (vagrant or AWS).

So far so good, everything above is working for me.

But here’s my question (finally!):

If I want to make a multiple node setup using chef-provisioning and some nodes need to be more powerful than others, how do I do that properly?

I assume I would be able to run something like the following:

??chef-client -z aws-minimal.rb myapp-4node-db_web.rb aws-medium.rb myapp-4node-backend_worker.rb

The idea being that aws-minimal.rb sets up the machine type just as before, then the first two of the 4 machines would be set up (db and web) which only need a “minimal type” of server, then the AWS configuration is changed to use a beefier type of AWS server with aws-medium.rb and then finally the other two machines (backend and worker) would be set up using myapp-4node-backend_worker.rb which would then be set up on the more powerful AWS servers.

This seems a bit convoluted to me and I’m not really all that happy with it (and actually I haven’t tested if this in fact works). For example, I am splitting my machine definitions into files depending on which machines should have the same specs and assuming that something came in between and set these specs correctly.

Does anybody have a better way to do this?

-Kind regards,

Stefan Freyr.?

Christine_Draper · April 10, 2015, 2:39am

That's an interesting question. I've also been struggling with that
separation.

Perhaps if the provisioning recipes were in a cookbook, then the
machine_options to use for each node might be defined using attributes,
and looked up and set on each machine. But I haven't tried it (so far only
running with recipe files rather than a run list).

Regards,
Christine

On Thu, Apr 9, 2015 at 1:42 PM, Stefán Freyr Stefánsson <
stefan@wuxinextcode.com> wrote:

Hello.

We're starting to look into using chef-provisioning and things are
looking promising except maybe for lack of documentation on best practices.
The documentation at https://docs.chef.io/provisioning.html is good but
it doesn't really explain the best ways to structure your chef-provisioning
code.

We have a system that can be set up on a cluster but it can also be set
up on a single machine. We would like to use chef-provisioning to be able
to set up multiple types of setups (single node, multiple nodes, using
vagrant, using AWS, using our own EC2 compatable private cloud etc.). We
are going to use these scripts to perform automatic tests, so when a new
release of our software is issued, we want an automatic process to start
and set everything up for us.

What I currently have are basically just some ruby files and an
environment directory with json "chef environment" files.

The ruby files I have are:

aws-minimal.rb

vagrant-minimal.rb

myapp-1node.rb

The myapp-single-node.rb is very simple, it sets up a single "machine"
and applies the necessary recipes to it. The idea later on is to have
myapp-4node.rb etc to set our system up on more nodes as needed.

The aws-minimal.rb file looks like this:

require 'chef/provisioning/fog_driver/driver'

with_machine_options({
:bootstrap_options => {
:flavor_id => 't2.medium',
},
:ssh_username => 'centos',
:image_id => 'ami-96a818fe'
})

Basically this sets everything up for AWS as you can see.

The way I run this is:

chef-client -z provision/aws-minimal.rb provision/myapp-1node.rb

I also have a vagrant-minimal.rb file that looks like this:

require 'chef/provisioning/vagrant_driver'

vagrant_box 'centos7' do
url '
https://cdn.nextcode.com/contrib/nextcode_centos-7-x86_64_virtualbox.box'
end

with_machine_options :vagrant_options => {
'vm.box' => 'centos7'
}

And if I want to run my myapp-1node.rb and use vagrant instead of AWS I
run this:

chef-client -z provision/vagrant-minimal.rb provision/myapp-1node.rb

So as you can see, I'm keeping the driver configuration separate from my
provisioning scripts, hoping that they can stay agnostic to what kind of
machine is being set up and in what environment (vagrant or AWS).

So far so good, everything above is working for me.

But here's my question (finally!):

If I want to make a multiple node setup using chef-provisioning and some
nodes need to be more powerful than others, how do I do that properly?

I assume I would be able to run something like the following:

chef-client -z aws-minimal.rb myapp-4node-db_web.rb aws-medium.rb myapp
-4node-backend_worker.rb

The idea being that aws-minimal.rb sets up the machine type just as
before, then the first two of the 4 machines would be set up (db and web)
which only need a "minimal type" of server, then the AWS configuration is
changed to use a beefier type of AWS server with aws-medium.rb and then
finally the other two machines (backend and worker) would be set up using
myapp-4node-backend_worker.rb which would then be set up on the more
powerful AWS servers.

This seems a bit convoluted to me and I'm not really all that happy with
it (and actually I haven't tested if this in fact works). For example, I am
splitting my machine definitions into files depending on which machines
should have the same specs and assuming that something came in between and
set these specs correctly.

Does anybody have a better way to do this?

-Kind regards,

Stefan Freyr.

Gabriel_Rosendorf · April 10, 2015, 1:47pm

I'm also starting to look into chef-provisioning, and specifically
chef-provisioning-aws. One thing that I've sort of struggled with is what
that workflow is going to look like. The direction that I'm heading is this:

Each deployment (collection of vpc, subnets, instances, etc) will
reside in it's own chef repo, separate from where we store our normal
cookbooks, environments, etc.
Each region that we'll deploy to is represented as an environment,
e.g. us-east-1.json.
When we converge, we point chef-client to a cookbook and an
environment, e.g. chef-client -z -o provisioning-cookbook -E us-west-2

This allows us to control attributes per region, subnet CIDRs for example.
Also, since the outputs are stored as data bags, this method of using
separate chef repos per deployment type keeps those separated in source
control.

This is my first crack at chef-provisioning, so this may be a horrible idea
of reasons that I haven't encountered yet, but for now it seems to make
sense. I'd love to hear how other folks are doing it!

Best,
Gabriel

On Thu, Apr 9, 2015 at 10:39 PM Christine Draper <
christine_draper@thirdwaveinsights.com> wrote:

That's an interesting question. I've also been struggling with that
separation.

Perhaps if the provisioning recipes were in a cookbook, then the
machine_options to use for each node might be defined using attributes,
and looked up and set on each machine. But I haven't tried it (so far only
running with recipe files rather than a run list).

Regards,
Christine

On Thu, Apr 9, 2015 at 1:42 PM, Stefán Freyr Stefánsson <
stefan@wuxinextcode.com> wrote:

Hello.

We're starting to look into using chef-provisioning and things are
looking promising except maybe for lack of documentation on best practices.
The documentation at https://docs.chef.io/provisioning.html is good but
it doesn't really explain the best ways to structure your chef-provisioning
code.

We have a system that can be set up on a cluster but it can also be set
up on a single machine. We would like to use chef-provisioning to be able
to set up multiple types of setups (single node, multiple nodes, using
vagrant, using AWS, using our own EC2 compatable private cloud etc.). We
are going to use these scripts to perform automatic tests, so when a new
release of our software is issued, we want an automatic process to start
and set everything up for us.

What I currently have are basically just some ruby files and an
environment directory with json "chef environment" files.

The ruby files I have are:

aws-minimal.rb

vagrant-minimal.rb

myapp-1node.rb

The myapp-single-node.rb is very simple, it sets up a single "machine"
and applies the necessary recipes to it. The idea later on is to have
myapp-4node.rb etc to set our system up on more nodes as needed.

The aws-minimal.rb file looks like this:

require 'chef/provisioning/fog_driver/driver'

with_machine_options({
:bootstrap_options => {
:flavor_id => 't2.medium',
},
:ssh_username => 'centos',
:image_id => 'ami-96a818fe'
})

Basically this sets everything up for AWS as you can see.

The way I run this is:

chef-client -z provision/aws-minimal.rb provision/myapp-1node.rb

I also have a vagrant-minimal.rb file that looks like this:

require 'chef/provisioning/vagrant_driver'

vagrant_box 'centos7' do
url '
https://cdn.nextcode.com/contrib/nextcode_centos-7-x86_64_virtualbox.box'
end

with_machine_options :vagrant_options => {
'vm.box' => 'centos7'
}

And if I want to run my myapp-1node.rb and use vagrant instead of AWS I
run this:

chef-client -z provision/vagrant-minimal.rb provision/myapp-1node.rb

So as you can see, I'm keeping the driver configuration separate from
my provisioning scripts, hoping that they can stay agnostic to what kind of
machine is being set up and in what environment (vagrant or AWS).

So far so good, everything above is working for me.

But here's my question (finally!):

If I want to make a multiple node setup using chef-provisioning and
some nodes need to be more powerful than others, how do I do that properly?

I assume I would be able to run something like the following:

chef-client -z aws-minimal.rb myapp-4node-db_web.rb aws-medium.rb myapp
-4node-backend_worker.rb

The idea being that aws-minimal.rb sets up the machine type just as
before, then the first two of the 4 machines would be set up (db and web)
which only need a "minimal type" of server, then the AWS configuration is
changed to use a beefier type of AWS server with aws-medium.rb and then
finally the other two machines (backend and worker) would be set up using
myapp-4node-backend_worker.rb which would then be set up on the more
powerful AWS servers.

This seems a bit convoluted to me and I'm not really all that happy
with it (and actually I haven't tested if this in fact works). For example,
I am splitting my machine definitions into files depending on which
machines should have the same specs and assuming that something came in
between and set these specs correctly.

Does anybody have a better way to do this?

-Kind regards,

Stefan Freyr.

Tyler_Ball · April 13, 2015, 4:25pm

For creating different types of machines, you can directly set machine_options as an attribute on machine:

require 'chef/provisioning/aws_driver'

with_driver 'aws::eu-west-1'

machine_batch ‘my cluster’ do
machine ‘small’ do
machine_options :bootstrap_options {
instance_type: ‘m3.medium’
}
end
machine ‘large’ do
machine_options :bootstrap_options {
instance_type ‘c4.large’
}
end
end

If you are provisioning in AWS, you should also be using chef-provisioning-aws instead of chef-provisioning-fog.

The hash you set on :bootstrap_options can come from an attribute, set at a role or environment level.

Your provisioning run_list looks fine to me - what about running multiple recipes (a base recipe and different recipes for different machine sizes) makes you uncomfortable?

-T

On Apr 10, 2015, at 6:47 AM, Gabriel Rosendorf grosendorf@gmail.com wrote:

I'm also starting to look into chef-provisioning, and specifically chef-provisioning-aws. One thing that I've sort of struggled with is what that workflow is going to look like. The direction that I'm heading is this:

Each deployment (collection of vpc, subnets, instances, etc) will reside in it's own chef repo, separate from where we store our normal cookbooks, environments, etc.
Each region that we'll deploy to is represented as an environment, e.g. us-east-1.json.
When we converge, we point chef-client to a cookbook and an environment, e.g. chef-client -z -o provisioning-cookbook -E us-west-2
This allows us to control attributes per region, subnet CIDRs for example. Also, since the outputs are stored as data bags, this method of using separate chef repos per deployment type keeps those separated in source control.

This is my first crack at chef-provisioning, so this may be a horrible idea of reasons that I haven't encountered yet, but for now it seems to make sense. I'd love to hear how other folks are doing it!

Best,
Gabriel

On Thu, Apr 9, 2015 at 10:39 PM Christine Draper <christine_draper@thirdwaveinsights.com mailto:christine_draper@thirdwaveinsights.com> wrote:
That's an interesting question. I've also been struggling with that separation.

Perhaps if the provisioning recipes were in a cookbook, then the machine_options to use for each node might be defined using attributes, and looked up and set on each machine. But I haven't tried it (so far only running with recipe files rather than a run list).

Regards,
Christine

On Thu, Apr 9, 2015 at 1:42 PM, Stefán Freyr Stefánsson <stefan@wuxinextcode.com mailto:stefan@wuxinextcode.com> wrote:
Hello.

We're starting to look into using chef-provisioning and things are looking promising except maybe for lack of documentation on best practices. The documentation at https://docs.chef.io/provisioning.html https://docs.chef.io/provisioning.html is good but it doesn't really explain the best ways to structure your chef-provisioning code.

We have a system that can be set up on a cluster but it can also be set up on a single machine. We would like to use chef-provisioning to be able to set up multiple types of setups (single node, multiple nodes, using vagrant, using AWS, using our own EC2 compatable private cloud etc.). We are going to use these scripts to perform automatic tests, so when a new release of our software is issued, we want an automatic process to start and set everything up for us.

What I currently have are basically just some ruby files and an environment directory with json "chef environment" files.

The ruby files I have are:
aws-minimal.rb
vagrant-minimal.rb
myapp-1node.rb

The myapp-single-node.rb is very simple, it sets up a single "machine" and applies the necessary recipes to it. The idea later on is to have myapp-4node.rb etc to set our system up on more nodes as needed.

The aws-minimal.rb file looks like this:
require 'chef/provisioning/fog_driver/driver'

with_machine_options({
:bootstrap_options => {
:flavor_id => 't2.medium',
},
:ssh_username => 'centos',
:image_id => 'ami-96a818fe'
})

Basically this sets everything up for AWS as you can see.

The way I run this is:

chef-client -z provision/aws-minimal.rb provision/myapp-1node.rb

I also have a vagrant-minimal.rb file that looks like this:

require 'chef/provisioning/vagrant_driver'

vagrant_box 'centos7' do
url 'https://cdn.nextcode.com/contrib/nextcode_centos-7-x86_64_virtualbox.box https://cdn.nextcode.com/contrib/nextcode_centos-7-x86_64_virtualbox.box'
end

with_machine_options :vagrant_options => {
'vm.box' => 'centos7'
}

And if I want to run my myapp-1node.rb and use vagrant instead of AWS I run this:
chef-client -z provision/vagrant-minimal.rb provision/myapp-1node.rb

So as you can see, I'm keeping the driver configuration separate from my provisioning scripts, hoping that they can stay agnostic to what kind of machine is being set up and in what environment (vagrant or AWS).

So far so good, everything above is working for me.

But here's my question (finally!):

If I want to make a multiple node setup using chef-provisioning and some nodes need to be more powerful than others, how do I do that properly?

I assume I would be able to run something like the following:

chef-client -z aws-minimal.rb myapp-4node-db_web.rb aws-medium.rb myapp-4node-backend_worker.rb

The idea being that aws-minimal.rb sets up the machine type just as before, then the first two of the 4 machines would be set up (db and web) which only need a "minimal type" of server, then the AWS configuration is changed to use a beefier type of AWS server with aws-medium.rb and then finally the other two machines (backend and worker) would be set up using myapp-4node-backend_worker.rb which would then be set up on the more powerful AWS servers.

This seems a bit convoluted to me and I'm not really all that happy with it (and actually I haven't tested if this in fact works). For example, I am splitting my machine definitions into files depending on which machines should have the same specs and assuming that something came in between and set these specs correctly.

Does anybody have a better way to do this?

-Kind regards,

Stefan Freyr.

StFS · April 14, 2015, 5:51pm

Hi.

Thanks for the answer but there seems to be a little misunderstanding.

What I was mostly complaining about was that I want to separate the AWS specific things from the definition of what machines I want to build.

In other words, I want to be able to build a 7 machine cluster and with relative ease, I want to be able to do that on more than one environment (say AWS, our private EC2 compliant cloud, Rackspace, etc).

In your example, you’ve fixed the driver to aws inside the “machine definition ruby script”.

I’m going to play around with what you said about passing attributes into these scripts, If I do that I can probably have a set of attributes for each driver type. But I’m not quite sure how to proceed since my machine building ruby script isn’t actually a recipe, it’s just a ruby file that’s not a part of a cookbook. Maybe that’s part of my problem, should this be an actual recipe?

At any rate, I think the time has come for some documentation about what the proper ways to do chef-provisioning are. At least I haven’t been able to find it.

But I’m curious, why should I use the aws_driver instead of fog? Is fog being deprecated? I haven’t seen anything about that?

-Stefan Freyr.

From: Tyler Ball tball@chef.io
Sent: Monday, April 13, 2015 4:25 PM
To: chef@lists.opscode.com
Subject: [chef] Re: chef-provisioning best practices for supporting multiple configurations

For creating different types of machines, you can directly set machine_options as an attribute on machine:

require ‘chef/provisioning/aws_driver’

with_driver ‘aws::eu-west-1’

machine_batch ‘my cluster’ do
machine ‘small’ do
machine_options :bootstrap_options {
instance_type: ‘m3.medium’
}
end
machine ‘large’ do
machine_options :bootstrap_options {
instance_type ‘c4.large’
}
end
end

If you are provisioning in AWS, you should also be using chef-provisioning-aws instead of chef-provisioning-fog.

The hash you set on :bootstrap_options can come from an attribute, set at a role or environment level.

Your provisioning run_list looks fine to me - what about running multiple recipes (a base recipe and different recipes for different machine sizes) makes you uncomfortable?

-T

On Apr 10, 2015, at 6:47 AM, Gabriel Rosendorf <grosendorf@gmail.com mailto:grosendorf@gmail.com> wrote:

I’m also starting to look into chef-provisioning, and specifically chef-provisioning-aws. One thing that I’ve sort of struggled with is what that workflow is going to look like. The direction that I’m heading is this:

Each deployment (collection of vpc, subnets, instances, etc) will reside in it’s own chef repo, separate from where we store our normal cookbooks, environments, etc.
Each region that we’ll deploy to is represented as an environment, e.g. us-east-1.json.
When we converge, we point chef-client to a cookbook and an environment, e.g. chef-client -z -o provisioning-cookbook -E us-west-2

This allows us to control attributes per region, subnet CIDRs for example. Also, since the outputs are stored as data bags, this method of using separate chef repos per deployment type keeps those separated in source control.

This is my first crack at chef-provisioning, so this may be a horrible idea of reasons that I haven’t encountered yet, but for now it seems to make sense. I’d love to hear how other folks are doing it!

Best,
Gabriel

On Thu, Apr 9, 2015 at 10:39 PM Christine Draper <christine_draper@thirdwaveinsights.com mailto:christine_draper@thirdwaveinsights.com> wrote:
That’s an interesting question. I’ve also been struggling with that separation.

Perhaps if the provisioning recipes were in a cookbook, then the machine_options to use for each node might be defined using attributes, and looked up and set on each machine. But I haven’t tried it (so far only running with recipe files rather than a run list).

Regards,
Christine

On Thu, Apr 9, 2015 at 1:42 PM, Stefán Freyr Stefánsson <stefan@wuxinextcode.com mailto:stefan@wuxinextcode.com> wrote:

Hello.

We’re starting to look into using chef-provisioning and things are looking promising except maybe for lack of documentation on best practices. The documentation at https://docs.chef.io/provisioning.html is good but it doesn’t really explain the best ways to structure your chef-provisioning code.

We have a system that can be set up on a cluster but it can also be set up on a single machine. We would like to use chef-provisioning to be able to set up multiple types of setups (single node, multiple nodes, using vagrant, using AWS, using our own EC2 compatable private cloud etc.). We are going to use these scripts to perform automatic tests, so when a new release of our software is issued, we want an automatic process to start and set everything up for us.

What I currently have are basically just some ruby files and an environment directory with json “chef environment” files.

The ruby files I have are:

aws-minimal.rb
vagrant-minimal.rb
myapp-1node.rb

The myapp-single-node.rb is very simple, it sets up a single “machine” and applies the necessary recipes to it. The idea later on is to have myapp-4node.rb etc to set our system up on more nodes as needed.

The aws-minimal.rb file looks like this:

require ‘chef/provisioning/fog_driver/driver’

with_machine_options({
:bootstrap_options => {
:flavor_id => ‘t2.medium’,
},
:ssh_username => ‘centos’,
:image_id => ‘ami-96a818fe’
})

Basically this sets everything up for AWS as you can see.

The way I run this is:

chef-client -z provision/aws-minimal.rb provision/myapp-1node.rb

I also have a vagrant-minimal.rb file that looks like this:

require ‘chef/provisioning/vagrant_driver’

vagrant_box ‘centos7’ do
url 'https://cdn.nextcode.com/contrib/nextcode_centos-7-x86_64_virtualbox.box’
end

with_machine_options :vagrant_options => {
‘vm.box’ => ‘centos7’
}

And if I want to run my myapp-1node.rb and use vagrant instead of AWS I run this:
?chef-client -z provision/vagrant-minimal.rb provision/myapp-1node.rb

So as you can see, I’m keeping the driver configuration separate from my provisioning scripts, hoping that they can stay agnostic to what kind of machine is being set up and in what environment (vagrant or AWS).

So far so good, everything above is working for me.

But here’s my question (finally!):

If I want to make a multiple node setup using chef-provisioning and some nodes need to be more powerful than others, how do I do that properly?

I assume I would be able to run something like the following:

??chef-client -z aws-minimal.rb myapp-4node-db_web.rb aws-medium.rb myapp-4node-backend_worker.rb

The idea being that aws-minimal.rb sets up the machine type just as before, then the first two of the 4 machines would be set up (db and web) which only need a “minimal type” of server, then the AWS configuration is changed to use a beefier type of AWS server with aws-medium.rb and then finally the other two machines (backend and worker) would be set up using myapp-4node-backend_worker.rb which would then be set up on the more powerful AWS servers.

This seems a bit convoluted to me and I’m not really all that happy with it (and actually I haven’t tested if this in fact works). For example, I am splitting my machine definitions into files depending on which machines should have the same specs and assuming that something came in between and set these specs correctly.

Does anybody have a better way to do this?

-Kind regards,

Stefan Freyr.?

Tyler_Ball · April 14, 2015, 6:51pm

Hey Stefan,

Check out the chef-server-cluster https://github.com/opscode-cookbooks/chef-server-cluster cookbook. In particular, look at the cluster-provision https://github.com/opscode-cookbooks/chef-server-cluster/blob/master/recipes/cluster-provision.rb recipe then the setup-provisioner https://github.com/opscode-cookbooks/chef-server-cluster/blob/master/recipes/setup-provisioner.rb recipe.

The setup recipe sets with_driver and with_machine_options from attributes. So using that single cluster-provision recipe you could provision machines in AWS or machines in Azure using only different attributes (which you could set in an environment, or policy file, etc.).

I see something similar fulfilling your needs (as I understand them - I could still be misunderstanding). By machine building ruby script do you mean the script which contacts AWS and provisions the machine? Or do you mean a script that installs Chef once the machine is provisioned?

In the first case, I think you should convert that script to a recipe which leverages chef-provisioning to provision machines. In the second case, I would need to understand the use case better.

You are right - we need some minimum viable product cookbooks and documentation which show the right way to use chef-provisioning.

Finally, yes, AWS via chef-provisioning-fog is going to be deprecated. The reason is that Fog doesn’t provide as many AWS resources as direct interaction with the AWS SDK does.

Unfortunately the best way to find information about chef-provisioning is to ask questions in the gitter https://gitter.im/chef/chef-provisioning or here on the mailing list. We’re moving so fast on this stuff that it is hard to keep any documentation current. The examples https://github.com/chef/chef-provisioning-aws/tree/master/docs/examples folder in the cookbook contains examples that we run manually, so they should be the most-up-to-date examples.

-T

On Apr 14, 2015, at 10:51 AM, Stefán Freyr Stefánsson stefan@wuxinextcode.com wrote:

Hi.

Thanks for the answer but there seems to be a little misunderstanding.

What I was mostly complaining about was that I want to separate the AWS specific things from the definition of what machines I want to build.

In other words, I want to be able to build a 7 machine cluster and with relative ease, I want to be able to do that on more than one environment (say AWS, our private EC2 compliant cloud, Rackspace, etc).

In your example, you've fixed the driver to aws inside the "machine definition ruby script".

I'm going to play around with what you said about passing attributes into these scripts, If I do that I can probably have a set of attributes for each driver type. But I'm not quite sure how to proceed since my machine building ruby script isn't actually a recipe, it's just a ruby file that's not a part of a cookbook. Maybe that's part of my problem, should this be an actual recipe?

At any rate, I think the time has come for some documentation about what the proper ways to do chef-provisioning are. At least I haven't been able to find it.

But I'm curious, why should I use the aws_driver instead of fog? Is fog being deprecated? I haven't seen anything about that?

-Stefan Freyr.

From: Tyler Ball tball@chef.io
Sent: Monday, April 13, 2015 4:25 PM
To: chef@lists.opscode.com
Subject: [chef] Re: chef-provisioning best practices for supporting multiple configurations

For creating different types of machines, you can directly set machine_options as an attribute on machine:

require 'chef/provisioning/aws_driver'

with_driver 'aws::eu-west-1'

machine_batch ‘my cluster’ do
machine ‘small’ do
machine_options :bootstrap_options {
instance_type: ‘m3.medium’
}
end
machine ‘large’ do
machine_options :bootstrap_options {
instance_type ‘c4.large’
}
end
end

If you are provisioning in AWS, you should also be using chef-provisioning-aws instead of chef-provisioning-fog.

The hash you set on :bootstrap_options can come from an attribute, set at a role or environment level.

Your provisioning run_list looks fine to me - what about running multiple recipes (a base recipe and different recipes for different machine sizes) makes you uncomfortable?

-T

On Apr 10, 2015, at 6:47 AM, Gabriel Rosendorf <grosendorf@gmail.com mailto:grosendorf@gmail.com> wrote:

I'm also starting to look into chef-provisioning, and specifically chef-provisioning-aws. One thing that I've sort of struggled with is what that workflow is going to look like. The direction that I'm heading is this:

Each deployment (collection of vpc, subnets, instances, etc) will reside in it's own chef repo, separate from where we store our normal cookbooks, environments, etc.
Each region that we'll deploy to is represented as an environment, e.g. us-east-1.json.
When we converge, we point chef-client to a cookbook and an environment, e.g. chef-client -z -o provisioning-cookbook -E us-west-2
This allows us to control attributes per region, subnet CIDRs for example. Also, since the outputs are stored as data bags, this method of using separate chef repos per deployment type keeps those separated in source control.

This is my first crack at chef-provisioning, so this may be a horrible idea of reasons that I haven't encountered yet, but for now it seems to make sense. I'd love to hear how other folks are doing it!

Best,
Gabriel

On Thu, Apr 9, 2015 at 10:39 PM Christine Draper <christine_draper@thirdwaveinsights.com mailto:christine_draper@thirdwaveinsights.com> wrote:
That's an interesting question. I've also been struggling with that separation.

Perhaps if the provisioning recipes were in a cookbook, then the machine_options to use for each node might be defined using attributes, and looked up and set on each machine. But I haven't tried it (so far only running with recipe files rather than a run list).

Regards,
Christine

On Thu, Apr 9, 2015 at 1:42 PM, Stefán Freyr Stefánsson <stefan@wuxinextcode.com mailto:stefan@wuxinextcode.com> wrote:
Hello.

We're starting to look into using chef-provisioning and things are looking promising except maybe for lack of documentation on best practices. The documentation at https://docs.chef.io/provisioning.html https://docs.chef.io/provisioning.html is good but it doesn't really explain the best ways to structure your chef-provisioning code.

We have a system that can be set up on a cluster but it can also be set up on a single machine. We would like to use chef-provisioning to be able to set up multiple types of setups (single node, multiple nodes, using vagrant, using AWS, using our own EC2 compatable private cloud etc.). We are going to use these scripts to perform automatic tests, so when a new release of our software is issued, we want an automatic process to start and set everything up for us.

What I currently have are basically just some ruby files and an environment directory with json "chef environment" files.

The ruby files I have are:
aws-minimal.rb
vagrant-minimal.rb
myapp-1node.rb

The myapp-single-node.rb is very simple, it sets up a single "machine" and applies the necessary recipes to it. The idea later on is to have myapp-4node.rb etc to set our system up on more nodes as needed.

The aws-minimal.rb file looks like this:
require 'chef/provisioning/fog_driver/driver'

with_machine_options({
:bootstrap_options => {
:flavor_id => 't2.medium',
},
:ssh_username => 'centos',
:image_id => 'ami-96a818fe'
})

Basically this sets everything up for AWS as you can see.

The way I run this is:
chef-client -z provision/aws-minimal.rb provision/myapp-1node.rb

I also have a vagrant-minimal.rb file that looks like this:
require 'chef/provisioning/vagrant_driver'

vagrant_box 'centos7' do
url 'https://cdn.nextcode.com/contrib/nextcode_centos-7-x86_64_virtualbox.box https://cdn.nextcode.com/contrib/nextcode_centos-7-x86_64_virtualbox.box'
end

with_machine_options :vagrant_options => {
'vm.box' => 'centos7'
}

And if I want to run my myapp-1node.rb and use vagrant instead of AWS I run this:
chef-client -z provision/vagrant-minimal.rb provision/myapp-1node.rb

So as you can see, I'm keeping the driver configuration separate from my provisioning scripts, hoping that they can stay agnostic to what kind of machine is being set up and in what environment (vagrant or AWS).

So far so good, everything above is working for me.

But here's my question (finally!):

If I want to make a multiple node setup using chef-provisioning and some nodes need to be more powerful than others, how do I do that properly?

I assume I would be able to run something like the following:
chef-client -z aws-minimal.rb myapp-4node-db_web.rb aws-medium.rb myapp-4node-backend_worker.rb

The idea being that aws-minimal.rb sets up the machine type just as before, then the first two of the 4 machines would be set up (db and web) which only need a "minimal type" of server, then the AWS configuration is changed to use a beefier type of AWS server with aws-medium.rb and then finally the other two machines (backend and worker) would be set up using myapp-4node-backend_worker.rb which would then be set up on the more powerful AWS servers.

This seems a bit convoluted to me and I'm not really all that happy with it (and actually I haven't tested if this in fact works). For example, I am splitting my machine definitions into files depending on which machines should have the same specs and assuming that something came in between and set these specs correctly.

Does anybody have a better way to do this?

-Kind regards,
Stefan Freyr.

StFS · April 16, 2015, 5:17pm

Thanks a lot for your answer Tyler.

I’m going to take a look at the cookbook you pointed me to.

Regarding the deprecation of fog-aws, that makes me quite sad

Basically I’d like to see fog-ec2 kept alive in order to be able to use it against providers that have an EC2 compatable API like QStack.

chef-provisioning-aws doesn’t seem to support changing the ec2 endpoint for example, so that will be unusable for us unfortunately.

Just my 2 cents… I hope you take this into some consideration regarding deprecating fog-aws (which as I mentioned might be better named fog-ec2).

Kind regards,

Stefan Freyr.

From: Tyler Ball tball@chef.io
Sent: Tuesday, April 14, 2015 6:51 PM
To: Stefán Freyr Stefánsson
Cc: chef@lists.opscode.com
Subject: Re: [chef] chef-provisioning best practices for supporting multiple configurations

Hey Stefan,

Check out the chef-server-clusterhttps://github.com/opscode-cookbooks/chef-server-cluster cookbook. In particular, look at the cluster-provisionhttps://github.com/opscode-cookbooks/chef-server-cluster/blob/master/recipes/cluster-provision.rb recipe then the setup-provisionerhttps://github.com/opscode-cookbooks/chef-server-cluster/blob/master/recipes/setup-provisioner.rb recipe.

The setup recipe sets with_driver and with_machine_options from attributes. So using that single cluster-provision recipe you could provision machines in AWS or machines in Azure using only different attributes (which you could set in an environment, or policy file, etc.).

I see something similar fulfilling your needs (as I understand them - I could still be misunderstanding). By machine building ruby script do you mean the script which contacts AWS and provisions the machine? Or do you mean a script that installs Chef once the machine is provisioned?

In the first case, I think you should convert that script to a recipe which leverages chef-provisioning to provision machines. In the second case, I would need to understand the use case better.

You are right - we need some minimum viable product cookbooks and documentation which show the right way to use chef-provisioning.

Finally, yes, AWS via chef-provisioning-fog is going to be deprecated. The reason is that Fog doesn’t provide as many AWS resources as direct interaction with the AWS SDK does.

Unfortunately the best way to find information about chef-provisioning is to ask questions in the gitterhttps://gitter.im/chef/chef-provisioning or here on the mailing list. We’re moving so fast on this stuff that it is hard to keep any documentation current. The exampleshttps://github.com/chef/chef-provisioning-aws/tree/master/docs/examples folder in the cookbook contains examples that we run manually, so they should be the most-up-to-date examples.

-T

On Apr 14, 2015, at 10:51 AM, Stefán Freyr Stefánsson <stefan@wuxinextcode.com mailto:stefan@wuxinextcode.com> wrote:

Hi.

Thanks for the answer but there seems to be a little misunderstanding.

What I was mostly complaining about was that I want to separate the AWS specific things from the definition of what machines I want to build.

In other words, I want to be able to build a 7 machine cluster and with relative ease, I want to be able to do that on more than one environment (say AWS, our private EC2 compliant cloud, Rackspace, etc).

In your example, you’ve fixed the driver to aws inside the “machine definition ruby script”.

I’m going to play around with what you said about passing attributes into these scripts, If I do that I can probably have a set of attributes for each driver type. But I’m not quite sure how to proceed since my machine building ruby script isn’t actually a recipe, it’s just a ruby file that’s not a part of a cookbook. Maybe that’s part of my problem, should this be an actual recipe?

At any rate, I think the time has come for some documentation about what the proper ways to do chef-provisioning are. At least I haven’t been able to find it.

But I’m curious, why should I use the aws_driver instead of fog? Is fog being deprecated? I haven’t seen anything about that?

-Stefan Freyr.

From: Tyler Ball <tball@chef.io mailto:tball@chef.io>
Sent: Monday, April 13, 2015 4:25 PM
To: chef@lists.opscode.com mailto:chef@lists.opscode.com
Subject: [chef] Re: chef-provisioning best practices for supporting multiple configurations

For creating different types of machines, you can directly set machine_options as an attribute on machine:

require ‘chef/provisioning/aws_driver’

with_driver ‘aws::eu-west-1’

machine_batch ‘my cluster’ do
machine ‘small’ do
machine_options :bootstrap_options {
instance_type: ‘m3.medium’
}
end
machine ‘large’ do
machine_options :bootstrap_options {
instance_type ‘c4.large’
}
end
end

If you are provisioning in AWS, you should also be using chef-provisioning-aws instead of chef-provisioning-fog.

The hash you set on :bootstrap_options can come from an attribute, set at a role or environment level.

Your provisioning run_list looks fine to me - what about running multiple recipes (a base recipe and different recipes for different machine sizes) makes you uncomfortable?

-T

On Apr 10, 2015, at 6:47 AM, Gabriel Rosendorf <grosendorf@gmail.com mailto:grosendorf@gmail.com> wrote:

I’m also starting to look into chef-provisioning, and specifically chef-provisioning-aws. One thing that I’ve sort of struggled with is what that workflow is going to look like. The direction that I’m heading is this:

Each deployment (collection of vpc, subnets, instances, etc) will reside in it’s own chef repo, separate from where we store our normal cookbooks, environments, etc.
Each region that we’ll deploy to is represented as an environment, e.g. us-east-1.json.
When we converge, we point chef-client to a cookbook and an environment, e.g. chef-client -z -o provisioning-cookbook -E us-west-2

This allows us to control attributes per region, subnet CIDRs for example. Also, since the outputs are stored as data bags, this method of using separate chef repos per deployment type keeps those separated in source control.

This is my first crack at chef-provisioning, so this may be a horrible idea of reasons that I haven’t encountered yet, but for now it seems to make sense. I’d love to hear how other folks are doing it!

Best,
Gabriel

On Thu, Apr 9, 2015 at 10:39 PM Christine Draper <christine_draper@thirdwaveinsights.com mailto:christine_draper@thirdwaveinsights.com> wrote:
That’s an interesting question. I’ve also been struggling with that separation.

Perhaps if the provisioning recipes were in a cookbook, then the machine_options to use for each node might be defined using attributes, and looked up and set on each machine. But I haven’t tried it (so far only running with recipe files rather than a run list).

Regards,
Christine

On Thu, Apr 9, 2015 at 1:42 PM, Stefán Freyr Stefánsson <stefan@wuxinextcode.com mailto:stefan@wuxinextcode.com> wrote:
Hello.

We’re starting to look into using chef-provisioning and things are looking promising except maybe for lack of documentation on best practices. The documentation at https://docs.chef.io/provisioning.html is good but it doesn’t really explain the best ways to structure your chef-provisioning code.

We have a system that can be set up on a cluster but it can also be set up on a single machine. We would like to use chef-provisioning to be able to set up multiple types of setups (single node, multiple nodes, using vagrant, using AWS, using our own EC2 compatable private cloud etc.). We are going to use these scripts to perform automatic tests, so when a new release of our software is issued, we want an automatic process to start and set everything up for us.

What I currently have are basically just some ruby files and an environment directory with json “chef environment” files.

The ruby files I have are:

aws-minimal.rb
vagrant-minimal.rb
myapp-1node.rb

The myapp-single-node.rb is very simple, it sets up a single “machine” and applies the necessary recipes to it. The idea later on is to have myapp-4node.rb etc to set our system up on more nodes as needed.

The aws-minimal.rb file looks like this:
require ‘chef/provisioning/fog_driver/driver’

with_machine_options({
:bootstrap_options => {
:flavor_id => ‘t2.medium’,
},
:ssh_username => ‘centos’,
:image_id => ‘ami-96a818fe’
})

Basically this sets everything up for AWS as you can see.

The way I run this is:
chef-client -z provision/aws-minimal.rb provision/myapp-1node.rb

I also have a vagrant-minimal.rb file that looks like this:
require ‘chef/provisioning/vagrant_driver’

vagrant_box ‘centos7’ do
url 'https://cdn.nextcode.com/contrib/nextcode_centos-7-x86_64_virtualbox.box’
end

with_machine_options :vagrant_options => {
‘vm.box’ => ‘centos7’
}

And if I want to run my myapp-1node.rb and use vagrant instead of AWS I run this:
?chef-client -z provision/vagrant-minimal.rb provision/myapp-1node.rb

So as you can see, I’m keeping the driver configuration separate from my provisioning scripts, hoping that they can stay agnostic to what kind of machine is being set up and in what environment (vagrant or AWS).

So far so good, everything above is working for me.

But here’s my question (finally!):

If I want to make a multiple node setup using chef-provisioning and some nodes need to be more powerful than others, how do I do that properly?

I assume I would be able to run something like the following:
??chef-client -z aws-minimal.rb myapp-4node-db_web.rb aws-medium.rb myapp-4node-backend_worker.rb

The idea being that aws-minimal.rb sets up the machine type just as before, then the first two of the 4 machines would be set up (db and web) which only need a “minimal type” of server, then the AWS configuration is changed to use a beefier type of AWS server with aws-medium.rb and then finally the other two machines (backend and worker) would be set up using myapp-4node-backend_worker.rb which would then be set up on the more powerful AWS servers.

This seems a bit convoluted to me and I’m not really all that happy with it (and actually I haven’t tested if this in fact works). For example, I am splitting my machine definitions into files depending on which machines should have the same specs and assuming that something came in between and set these specs correctly.

Does anybody have a better way to do this?

-Kind regards,
Stefan Freyr.?

Topic		Replies	Views
Configure/deploy multiples apps per cookbook, how? Chef Infra (archive)	10	413	September 10, 2012
Chef-provisioning and test kitchen Chef Infra (archive)	3	600	January 30, 2018
Chef-Provisioning Future Plans Chef Infra (archive)	4	691	June 13, 2018
How can I supply different bootstrap_options to a chef-provisioning machine? Chef Infra (archive)	0	577	March 16, 2016
On Chef patterns - ways to organize a Chef kitchen Chef Infra (archive)	3	321	February 3, 2014

Chef-provisioning best practices for supporting multiple configurations

Related Topics