Chef Metal vs. Vagrant


#1

Ohai Chefs!

with the recent blog post about “Chef Provisioning” I believe this
means that chef-metal went live:

Something I was always wondering about: there seems to be a huge
overlap between Chef Metal and Vagrant – is this by intent? Which
gaps does chef-metal try to fill and what can I do with chef-metal
that I can’t do well with vagrant?

Cheers,
Torben


#2

On Mon, Nov 17, 2014 at 12:21 PM, Torben Knerr mail@tknerr.de wrote:

with the recent blog post about “Chef Provisioning” I believe this
means that chef-metal went live:
https://www.getchef.com/blog/2014/11/12/chef-provisioning-infrastructure-as-code

Yes, just to be clear, “Chef Provisioning” is what was previously
known as “Chef Metal”. We renamed it because there was confusion in
that it isn’t just to automate bare metal.

Something I was always wondering about: there seems to be a huge
overlap between Chef Metal and Vagrant – is this by intent? Which
gaps does chef-metal try to fill and what can I do with chef-metal
that I can’t do well with vagrant?

Chef Provisioning is intended to be a platform that’ll allow you to
automate all aspects of your stack. For example, if you’re in AWS,
various AWS objects like SNS topics, SQS queues, IAM users, ELBs, etc.
can be managed. Vagrant doesn’t have those capabilities; it deals only
with machines.

There are also other features like parallelization (via the
machine_batch resource) that Vagrant won’t have.

Those are some of the things I can think of, off the top of my head.

  • Julian


[ Julian C. Dunn jdunn@aquezada.com * Sorry, I’m ]
[ WWW: http://www.aquezada.com/staff/julian * only Web 1.0 ]
[ gopher://sdf.org/1/users/keymaker/ * compliant! ]
[ PGP: 91B3 7A9D 683C 7C16 715F 442C 6065 D533 FDC2 05B9 ]


#3

What’s the ETA on other things like, security groups and so on?

Doug.

On Mon, Nov 17, 2014 at 1:33 PM, Julian C. Dunn jdunn@aquezada.com wrote:

On Mon, Nov 17, 2014 at 12:21 PM, Torben Knerr mail@tknerr.de wrote:

with the recent blog post about “Chef Provisioning” I believe this
means that chef-metal went live:

https://www.getchef.com/blog/2014/11/12/chef-provisioning-infrastructure-as-code

Yes, just to be clear, “Chef Provisioning” is what was previously
known as “Chef Metal”. We renamed it because there was confusion in
that it isn’t just to automate bare metal.

Something I was always wondering about: there seems to be a huge
overlap between Chef Metal and Vagrant – is this by intent? Which
gaps does chef-metal try to fill and what can I do with chef-metal
that I can’t do well with vagrant?

Chef Provisioning is intended to be a platform that’ll allow you to
automate all aspects of your stack. For example, if you’re in AWS,
various AWS objects like SNS topics, SQS queues, IAM users, ELBs, etc.
can be managed. Vagrant doesn’t have those capabilities; it deals only
with machines.

There are also other features like parallelization (via the
machine_batch resource) that Vagrant won’t have.

Those are some of the things I can think of, off the top of my head.

  • Julian


[ Julian C. Dunn jdunn@aquezada.com * Sorry, I’m ]
[ WWW: http://www.aquezada.com/staff/julian * only Web 1.0 ]
[ gopher://sdf.org/1/users/keymaker/ * compliant! ]
[ PGP: 91B3 7A9D 683C 7C16 715F 442C 6065 D533 FDC2 05B9 ]


Regards,

Douglas Garstang
http://www.linkedin.com/in/garstang
Email: doug.garstang@gmail.com
Cell: +1-805-340-5627


#4

Hi Julian,

thanks, indeed Vagrant is focused on virtual machines only, good to
know where Chef Provisioning is heading to.

While at first I couldn’t find the additional resources in
opscode/chef-provisioning itself, I found them in
opscode/chef-provisioning-aws [1].

Is the intention to provide some common abstractions (e.g for cloud
queues, object storage, load balancers, etc) like Vagrant does for
virtual machines too? Or is the aim to simply provide a higher level
configuration DSL in favor of the specific commandline tools?

Cheers,
Torben

[1] https://github.com/opscode/chef-provisioning-aws/tree/master/lib/chef/resource

On Mon, Nov 17, 2014 at 10:33 PM, Julian C. Dunn jdunn@aquezada.com wrote:

On Mon, Nov 17, 2014 at 12:21 PM, Torben Knerr mail@tknerr.de wrote:

with the recent blog post about “Chef Provisioning” I believe this
means that chef-metal went live:
https://www.getchef.com/blog/2014/11/12/chef-provisioning-infrastructure-as-code

Yes, just to be clear, “Chef Provisioning” is what was previously
known as “Chef Metal”. We renamed it because there was confusion in
that it isn’t just to automate bare metal.

Something I was always wondering about: there seems to be a huge
overlap between Chef Metal and Vagrant – is this by intent? Which
gaps does chef-metal try to fill and what can I do with chef-metal
that I can’t do well with vagrant?

Chef Provisioning is intended to be a platform that’ll allow you to
automate all aspects of your stack. For example, if you’re in AWS,
various AWS objects like SNS topics, SQS queues, IAM users, ELBs, etc.
can be managed. Vagrant doesn’t have those capabilities; it deals only
with machines.

There are also other features like parallelization (via the
machine_batch resource) that Vagrant won’t have.

Those are some of the things I can think of, off the top of my head.

  • Julian


[ Julian C. Dunn jdunn@aquezada.com * Sorry, I’m ]
[ WWW: http://www.aquezada.com/staff/julian * only Web 1.0 ]
[ gopher://sdf.org/1/users/keymaker/ * compliant! ]
[ PGP: 91B3 7A9D 683C 7C16 715F 442C 6065 D533 FDC2 05B9 ]


#5

On Mon, Nov 17, 2014 at 6:12 PM, Torben Knerr mail@tknerr.de wrote:

Is the intention to provide some common abstractions (e.g for cloud
queues, object storage, load balancers, etc) like Vagrant does for
virtual machines too? Or is the aim to simply provide a higher level
configuration DSL in favor of the specific commandline tools?

I’m not sure. John Keiser would be the best person to answer.
However… I have talked to some folks about this and they’ve warned
us off the idea, because there is actually less commonality than you
might think amongst the cloud providers.

Since I don’t have broad operational experience with anything beyond
AWS I thought I’d merely relate that. If anyone can weigh in, we’d
certainly welcome the feedback.

  • Julian


[ Julian C. Dunn jdunn@aquezada.com * Sorry, I’m ]
[ WWW: http://www.aquezada.com/staff/julian * only Web 1.0 ]
[ gopher://sdf.org/1/users/keymaker/ * compliant! ]
[ PGP: 91B3 7A9D 683C 7C16 715F 442C 6065 D533 FDC2 05B9 ]


#6

Yes, would be interesting to hear what John Keiser thinks on this.

There is also the Heat DSL (driven by the OpenStack community) which aims
exactly at an abstract definition of cloud infrastructures (think of a more
generic AWS cloudformation).

Cheers, Torben
Am 18.11.2014 03:56 schrieb “Julian C. Dunn” jdunn@aquezada.com:

On Mon, Nov 17, 2014 at 6:12 PM, Torben Knerr mail@tknerr.de wrote:

Is the intention to provide some common abstractions (e.g for cloud
queues, object storage, load balancers, etc) like Vagrant does for
virtual machines too? Or is the aim to simply provide a higher level
configuration DSL in favor of the specific commandline tools?

I’m not sure. John Keiser would be the best person to answer.
However… I have talked to some folks about this and they’ve warned
us off the idea, because there is actually less commonality than you
might think amongst the cloud providers.

Since I don’t have broad operational experience with anything beyond
AWS I thought I’d merely relate that. If anyone can weigh in, we’d
certainly welcome the feedback.

  • Julian


[ Julian C. Dunn jdunn@aquezada.com * Sorry, I’m ]
[ WWW: http://www.aquezada.com/staff/julian * only Web 1.0 ]
[ gopher://sdf.org/1/users/keymaker/ * compliant! ]
[ PGP: 91B3 7A9D 683C 7C16 715F 442C 6065 D533 FDC2 05B9 ]


#7

Hi,

furthermore there is Terraform (http://terraform.io) by Hashicorp that
comes a little closer to Chef Provisioning / Heat / CloudFormation than
Vagrant. It has multiple providers (AWS, Digital Ocean) and can run
provisioning on each of the nodes (just like Vagrant), including Chef as an
option. Still under heavy development but I am currently using it in a
project and am quite happy.

Sören

2014-11-18 7:10 GMT+01:00 Torben Knerr mail@tknerr.de:

Yes, would be interesting to hear what John Keiser thinks on this.

There is also the Heat DSL (driven by the OpenStack community) which aims
exactly at an abstract definition of cloud infrastructures (think of a more
generic AWS cloudformation).

Cheers, Torben
Am 18.11.2014 03:56 schrieb “Julian C. Dunn” jdunn@aquezada.com:

On Mon, Nov 17, 2014 at 6:12 PM, Torben Knerr mail@tknerr.de wrote:

Is the intention to provide some common abstractions (e.g for cloud
queues, object storage, load balancers, etc) like Vagrant does for
virtual machines too? Or is the aim to simply provide a higher level
configuration DSL in favor of the specific commandline tools?

I’m not sure. John Keiser would be the best person to answer.
However… I have talked to some folks about this and they’ve warned
us off the idea, because there is actually less commonality than you
might think amongst the cloud providers.

Since I don’t have broad operational experience with anything beyond
AWS I thought I’d merely relate that. If anyone can weigh in, we’d
certainly welcome the feedback.

  • Julian


[ Julian C. Dunn jdunn@aquezada.com * Sorry, I’m ]
[ WWW: http://www.aquezada.com/staff/julian * only Web 1.0 ]
[ gopher://sdf.org/1/users/keymaker/ * compliant! ]
[ PGP: 91B3 7A9D 683C 7C16 715F 442C 6065 D533 FDC2 05B9 ]


#8

Y, I mentioned these other Hashicorp projects while at Chef Community
Summit, didn’t really come away with an answer of what I should consider
the differentiator of where Chef Inc and Hashicorp are headed.

Time will tell!

Best,
Martin

On Tue, Nov 18, 2014 at 2:13 AM, Sören Blom soeren.blom@gmail.com wrote:

Hi,

furthermore there is Terraform (http://terraform.io) by Hashicorp that
comes a little closer to Chef Provisioning / Heat / CloudFormation than
Vagrant. It has multiple providers (AWS, Digital Ocean) and can run
provisioning on each of the nodes (just like Vagrant), including Chef as an
option. Still under heavy development but I am currently using it in a
project and am quite happy.

Sören

2014-11-18 7:10 GMT+01:00 Torben Knerr mail@tknerr.de:

Yes, would be interesting to hear what John Keiser thinks on this.

There is also the Heat DSL (driven by the OpenStack community) which aims
exactly at an abstract definition of cloud infrastructures (think of a more
generic AWS cloudformation).

Cheers, Torben
Am 18.11.2014 03:56 schrieb “Julian C. Dunn” jdunn@aquezada.com:

On Mon, Nov 17, 2014 at 6:12 PM, Torben Knerr mail@tknerr.de wrote:

Is the intention to provide some common abstractions (e.g for cloud
queues, object storage, load balancers, etc) like Vagrant does for
virtual machines too? Or is the aim to simply provide a higher level
configuration DSL in favor of the specific commandline tools?

I’m not sure. John Keiser would be the best person to answer.
However… I have talked to some folks about this and they’ve warned
us off the idea, because there is actually less commonality than you
might think amongst the cloud providers.

Since I don’t have broad operational experience with anything beyond
AWS I thought I’d merely relate that. If anyone can weigh in, we’d
certainly welcome the feedback.

  • Julian


[ Julian C. Dunn jdunn@aquezada.com * Sorry, I’m ]
[ WWW: http://www.aquezada.com/staff/julian * only Web 1.0 ]
[ gopher://sdf.org/1/users/keymaker/ * compliant! ]
[ PGP: 91B3 7A9D 683C 7C16 715F 442C 6065 D533 FDC2 05B9 ]


#9

I’ll explain the thinking and then the plan :slight_smile: Two of Chef’s big goals are
to let you:

  1. Describe your whole infrastructure in recipes.
  2. Reuse the same recipes across the dev/test/deploy workflow wherever
    possible.

This means that we want to have resources for everything; but that they
don’t all have to be abstractions.

Chef Provisioning implements the secondary goal with abstractions like
machine; but as Julian says, not everything can (or should) be abstracted.
The plan is to provide abstractions where they are meaningful, and to
provide cloud-, container- or vm-specific resources to manage things that
don’t fit that model.

On Mon, Nov 17, 2014 at 6:55 PM, Julian C. Dunn jdunn@aquezada.com wrote:

On Mon, Nov 17, 2014 at 6:12 PM, Torben Knerr mail@tknerr.de wrote:

Is the intention to provide some common abstractions (e.g for cloud
queues, object storage, load balancers, etc) like Vagrant does for
virtual machines too? Or is the aim to simply provide a higher level
configuration DSL in favor of the specific commandline tools?

I’m not sure. John Keiser would be the best person to answer.
However… I have talked to some folks about this and they’ve warned
us off the idea, because there is actually less commonality than you
might think amongst the cloud providers.

Since I don’t have broad operational experience with anything beyond
AWS I thought I’d merely relate that. If anyone can weigh in, we’d
certainly welcome the feedback.

  • Julian


[ Julian C. Dunn jdunn@aquezada.com * Sorry, I’m ]
[ WWW: http://www.aquezada.com/staff/julian * only Web 1.0 ]
[ gopher://sdf.org/1/users/keymaker/ * compliant! ]
[ PGP: 91B3 7A9D 683C 7C16 715F 442C 6065 D533 FDC2 05B9 ]


#10

Thanks everybody for the responses!

@Sören, Martin - thanks for pointing me to terraform again. I looked
at it once in the early stages but then basically forgot about it…

@JulianDunn, JohnKeiser - makes sense, much clearer now. +1 for
providing abstractions only where meaningful rather than being leaky
:wink:

On Tue, Nov 18, 2014 at 7:20 PM, John Keiser jkeiser@getchef.com wrote:

I’ll explain the thinking and then the plan :slight_smile: Two of Chef’s big goals are
to let you:

  1. Describe your whole infrastructure in recipes.
  2. Reuse the same recipes across the dev/test/deploy workflow wherever
    possible.

This means that we want to have resources for everything; but that they
don’t all have to be abstractions.

Chef Provisioning implements the secondary goal with abstractions like
machine; but as Julian says, not everything can (or should) be abstracted.
The plan is to provide abstractions where they are meaningful, and to
provide cloud-, container- or vm-specific resources to manage things that
don’t fit that model.

On Mon, Nov 17, 2014 at 6:55 PM, Julian C. Dunn jdunn@aquezada.com wrote:

On Mon, Nov 17, 2014 at 6:12 PM, Torben Knerr mail@tknerr.de wrote:

Is the intention to provide some common abstractions (e.g for cloud
queues, object storage, load balancers, etc) like Vagrant does for
virtual machines too? Or is the aim to simply provide a higher level
configuration DSL in favor of the specific commandline tools?

I’m not sure. John Keiser would be the best person to answer.
However… I have talked to some folks about this and they’ve warned
us off the idea, because there is actually less commonality than you
might think amongst the cloud providers.

Since I don’t have broad operational experience with anything beyond
AWS I thought I’d merely relate that. If anyone can weigh in, we’d
certainly welcome the feedback.

  • Julian


[ Julian C. Dunn jdunn@aquezada.com * Sorry, I’m ]
[ WWW: http://www.aquezada.com/staff/julian * only Web 1.0 ]
[ gopher://sdf.org/1/users/keymaker/ * compliant! ]
[ PGP: 91B3 7A9D 683C 7C16 715F 442C 6065 D533 FDC2 05B9 ]


#11

There still seems be a huge gap in the bare metal area. Something like what puppets razor module has done . Some of us still have data centers and have to deploy physical hosts :slight_smile: . Using chef end to end to deploy an ESXi cluster and configure it would be amazing .
I basically have a small puppet vm to run razor because it works so well with our current Cisco UCS deployment . Would love to get rid of it

Sent from my iPhone

On Nov 18, 2014, at 11:20 AM, John Keiser jkeiser@getchef.com wrote:

I’ll explain the thinking and then the plan :slight_smile: Two of Chef’s big goals are to let you:

  1. Describe your whole infrastructure in recipes.
  2. Reuse the same recipes across the dev/test/deploy workflow wherever possible.

This means that we want to have resources for everything; but that they don’t all have to be abstractions.

Chef Provisioning implements the secondary goal with abstractions like machine; but as Julian says, not everything can (or should) be abstracted. The plan is to provide abstractions where they are meaningful, and to provide cloud-, container- or vm-specific resources to manage things that don’t fit that model.

On Mon, Nov 17, 2014 at 6:55 PM, Julian C. Dunn jdunn@aquezada.com wrote:
On Mon, Nov 17, 2014 at 6:12 PM, Torben Knerr mail@tknerr.de wrote:

Is the intention to provide some common abstractions (e.g for cloud
queues, object storage, load balancers, etc) like Vagrant does for
virtual machines too? Or is the aim to simply provide a higher level
configuration DSL in favor of the specific commandline tools?

I’m not sure. John Keiser would be the best person to answer.
However… I have talked to some folks about this and they’ve warned
us off the idea, because there is actually less commonality than you
might think amongst the cloud providers.

Since I don’t have broad operational experience with anything beyond
AWS I thought I’d merely relate that. If anyone can weigh in, we’d
certainly welcome the feedback.

  • Julian


[ Julian C. Dunn jdunn@aquezada.com * Sorry, I’m ]
[ WWW: http://www.aquezada.com/staff/julian * only Web 1.0 ]
[ gopher://sdf.org/1/users/keymaker/ * compliant! ]
[ PGP: 91B3 7A9D 683C 7C16 715F 442C 6065 D533 FDC2 05B9 ]


#12

There are several answers with metal, including openstack and vsphere, but
the closest to what you have is Hanlon, with the chef-provisioning-hanlon
driver. It is more beta than some of the rest of provisioning, but
improving rapidly.
On Nov 18, 2014 9:49 PM, “Ryan Kelley” rykelley@gmail.com wrote:

There still seems be a huge gap in the bare metal area. Something like
what puppets razor module has done . Some of us still have data centers and
have to deploy physical hosts :slight_smile: . Using chef end to end to deploy an ESXi
cluster and configure it would be amazing .
I basically have a small puppet vm to run razor because it works so well
with our current Cisco UCS deployment . Would love to get rid of it

Sent from my iPhone

On Nov 18, 2014, at 11:20 AM, John Keiser jkeiser@getchef.com wrote:

I’ll explain the thinking and then the plan :slight_smile: Two of Chef’s big goals
are to let you:

  1. Describe your whole infrastructure in recipes.
  2. Reuse the same recipes across the dev/test/deploy workflow wherever
    possible.

This means that we want to have resources for everything; but that they
don’t all have to be abstractions.

Chef Provisioning implements the secondary goal with abstractions like
machine; but as Julian says, not everything can (or should) be abstracted.
The plan is to provide abstractions where they are meaningful, and to
provide cloud-, container- or vm-specific resources to manage things that
don’t fit that model.

On Mon, Nov 17, 2014 at 6:55 PM, Julian C. Dunn jdunn@aquezada.com
wrote:

On Mon, Nov 17, 2014 at 6:12 PM, Torben Knerr mail@tknerr.de wrote:

Is the intention to provide some common abstractions (e.g for cloud
queues, object storage, load balancers, etc) like Vagrant does for
virtual machines too? Or is the aim to simply provide a higher level
configuration DSL in favor of the specific commandline tools?

I’m not sure. John Keiser would be the best person to answer.
However… I have talked to some folks about this and they’ve warned
us off the idea, because there is actually less commonality than you
might think amongst the cloud providers.

Since I don’t have broad operational experience with anything beyond
AWS I thought I’d merely relate that. If anyone can weigh in, we’d
certainly welcome the feedback.

  • Julian


[ Julian C. Dunn jdunn@aquezada.com * Sorry, I’m ]
[ WWW: http://www.aquezada.com/staff/julian * only Web 1.0 ]
[ gopher://sdf.org/1/users/keymaker/ * compliant! ]
[ PGP: 91B3 7A9D 683C 7C16 715F 442C 6065 D533 FDC2 05B9 ]