Using chef to provision/deprovision chef server users

Chef Infra (archive)
1

Ohai Chefs –

Im curious what others have done to tackle the provision/deprovision of
users within a given Chef Server.

My use case is having to own multiple company product’s disparate
open-source chef servers. Im envisioning a workflow where a top-level
chef-server manages the foo product’s chef server(s), satisfying dev+ops+qe
for the foo product are provisioned/deprovisioned. The same goes for the
bar product’s chef server(s) + team, the baz product’s chef server(s) +
team, etc. This would allow for onboarding already spun-up chef servers
and continuing to provide airspace segregation between products.

  • Is anyone doing anything like this currently?
  • How is it working out? Wonderfully? Terribly?
  • Can alternatives like Private Chef solve this function?

Curious what the community is doing. Thanks.

-nick

2

On Oct 15, 2013, at 2:06 PM, Nick Silkey nick@silkey.org wrote:

Im curious what others have done to tackle the provision/deprovision of users within a given Chef Server.

My use case is having to own multiple company product's disparate open-source chef servers. Im envisioning a workflow where a top-level chef-server manages the foo product's chef server(s), satisfying dev+ops+qe for the foo product are provisioned/deprovisioned. The same goes for the bar product's chef server(s) + team, the baz product's chef server(s) + team, etc. This would allow for onboarding already spun-up chef servers and continuing to provide airspace segregation between products.

Just checking -- this is a large-scale multi-tenant environment, right? Not just people from different teams in the same company, but actual different paying customers, some of whom might be competitors for some of your other customers. And we're potentially talking about many thousands of such customers, right?

Maybe not Facebook or Netflix or Amazon scale, but still what would be considered "large scale".

  • Is anyone doing anything like this currently?
  • How is it working out? Wonderfully? Terribly?
  • Can alternatives like Private Chef solve this function?

What I have heard is that Private Chef 11.x was developed in part based on Hosted Chef and the sorts of things that Opscode had to do to handle their own multi-tenant environment. My understanding is that Hosted Chef is now running on pretty much the same codebase as everyone else who is using Enterprise Chef 11.x, and is the largest known installation of EC. Well, the largest known installation that anyone can talk about, so far as we know.

So, if using the same chef server with RBAC that Opscode is using internally for Hosted Chef would be a suitable solution for you, I think that's a pretty easy answer. Of course, actually administering a large scale multi-tenant Hosted Chef type of environment is not exactly a walk in the park, but I suspect that you'll find some people on this list who can give you some advice towards that end.

Now, if a Hosted Chef type of solution is not adequate for you, and you would actually need separate chef servers for each environment in order to provide that "air gap" level of security, then I'm not sure how you would scale that -- we know what Opscode did (mostly), but I haven't heard of any other solutions in that space on that scale.

--
Brad Knowles brad@shub-internet.org
LinkedIn Profile: http://tinyurl.com/y8kpxu

3

We do small scale (5 chef servers) multi-tenancy with a master/central chef
server managing others. We didn't go the private chef way because it just
worked easily that way.

The reason we do it is to have separation of teams, workflow, git, releases
of code... Etc. Couldn't really do it otherwise with open source chef
although that'd be great! Maybe in the future we can try implementing this.
It would help for aggregation/reporting for example.
On Oct 15, 2013 9:21 PM, "Brad Knowles" brad@shub-internet.org wrote:

On Oct 15, 2013, at 2:06 PM, Nick Silkey nick@silkey.org wrote:

Im curious what others have done to tackle the provision/deprovision of
users within a given Chef Server.

My use case is having to own multiple company product's disparate
open-source chef servers. Im envisioning a workflow where a top-level
chef-server manages the foo product's chef server(s), satisfying dev+ops+qe
for the foo product are provisioned/deprovisioned. The same goes for the
bar product's chef server(s) + team, the baz product's chef server(s) +
team, etc. This would allow for onboarding already spun-up chef servers
and continuing to provide airspace segregation between products.

Just checking -- this is a large-scale multi-tenant environment, right?
Not just people from different teams in the same company, but actual
different paying customers, some of whom might be competitors for some of
your other customers. And we're potentially talking about many thousands
of such customers, right?

Maybe not Facebook or Netflix or Amazon scale, but still what would be
considered "large scale".

  • Is anyone doing anything like this currently?
  • How is it working out? Wonderfully? Terribly?
  • Can alternatives like Private Chef solve this function?

What I have heard is that Private Chef 11.x was developed in part based on
Hosted Chef and the sorts of things that Opscode had to do to handle their
own multi-tenant environment. My understanding is that Hosted Chef is now
running on pretty much the same codebase as everyone else who is using
Enterprise Chef 11.x, and is the largest known installation of EC. Well,
the largest known installation that anyone can talk about, so far as we
know.

So, if using the same chef server with RBAC that Opscode is using
internally for Hosted Chef would be a suitable solution for you, I think
that's a pretty easy answer. Of course, actually administering a large
scale multi-tenant Hosted Chef type of environment is not exactly a walk in
the park, but I suspect that you'll find some people on this list who can
give you some advice towards that end.

Now, if a Hosted Chef type of solution is not adequate for you, and you
would actually need separate chef servers for each environment in order to
provide that "air gap" level of security, then I'm not sure how you would
scale that -- we know what Opscode did (mostly), but I haven't heard of any
other solutions in that space on that scale.

--
Brad Knowles brad@shub-internet.org
LinkedIn Profile: http://tinyurl.com/y8kpxu

4

We paid for Enterprise Chef. At the time, spinning up a Chef server wasn't
fun, so it seemed like a good idea.

I wrote some Chef code that drives the new org provisioning process through
mechanize since using knife-opc wasn't an option (apparently it requires
the big nasty superuser do-everything key?) ...

On Tue, Oct 15, 2013 at 2:17 PM, Maxime Brugidou
maxime.brugidou@gmail.comwrote:

We do small scale (5 chef servers) multi-tenancy with a master/central
chef server managing others. We didn't go the private chef way because it
just worked easily that way.

The reason we do it is to have separation of teams, workflow, git,
releases of code... Etc. Couldn't really do it otherwise with open source
chef although that'd be great! Maybe in the future we can try implementing
this. It would help for aggregation/reporting for example.
On Oct 15, 2013 9:21 PM, "Brad Knowles" brad@shub-internet.org wrote:

On Oct 15, 2013, at 2:06 PM, Nick Silkey nick@silkey.org wrote:

Im curious what others have done to tackle the provision/deprovision of
users within a given Chef Server.

My use case is having to own multiple company product's disparate
open-source chef servers. Im envisioning a workflow where a top-level
chef-server manages the foo product's chef server(s), satisfying dev+ops+qe
for the foo product are provisioned/deprovisioned. The same goes for the
bar product's chef server(s) + team, the baz product's chef server(s) +
team, etc. This would allow for onboarding already spun-up chef servers
and continuing to provide airspace segregation between products.

Just checking -- this is a large-scale multi-tenant environment, right?
Not just people from different teams in the same company, but actual
different paying customers, some of whom might be competitors for some of
your other customers. And we're potentially talking about many thousands
of such customers, right?

Maybe not Facebook or Netflix or Amazon scale, but still what would be
considered "large scale".

  • Is anyone doing anything like this currently?
  • How is it working out? Wonderfully? Terribly?
  • Can alternatives like Private Chef solve this function?

What I have heard is that Private Chef 11.x was developed in part based
on Hosted Chef and the sorts of things that Opscode had to do to handle
their own multi-tenant environment. My understanding is that Hosted Chef
is now running on pretty much the same codebase as everyone else who is
using Enterprise Chef 11.x, and is the largest known installation of EC.
Well, the largest known installation that anyone can talk about, so far as
we know.

So, if using the same chef server with RBAC that Opscode is using
internally for Hosted Chef would be a suitable solution for you, I think
that's a pretty easy answer. Of course, actually administering a large
scale multi-tenant Hosted Chef type of environment is not exactly a walk in
the park, but I suspect that you'll find some people on this list who can
give you some advice towards that end.

Now, if a Hosted Chef type of solution is not adequate for you, and you
would actually need separate chef servers for each environment in order to
provide that "air gap" level of security, then I'm not sure how you would
scale that -- we know what Opscode did (mostly), but I haven't heard of any
other solutions in that space on that scale.

--
Brad Knowles brad@shub-internet.org
LinkedIn Profile: http://tinyurl.com/y8kpxu

5

we use three tools

  1. knife-server , to spawn new chef servers
  2. a bash script, called restore_chef.sh, which deletes all cookbooks,
    databags, roles, environments and upload them from local git repo. (you can
    pass -c knife.rb to this script)
  3. A custom knife plugin to backup/restore (only nodes, clients, databags)
    chef artifact, gzip it, and store it in s3.

we use a combination of these three tools to

  1. spawn fresh chef server, and sync them with existing ones.
  2. to keep two distant chef servers in sync
  3. to run our integratin tests ( we spawn chef-zero locally, use the
    restore_chef.sh and knife chefserver restore to populate data ), and then
    converge lxc against it. this allows us testing chef cookbooks/knife
    plugins locally, yest inside the production environment (logically, as all
    search calls return same values as they would do in production environment)

we have three chef servers in total (Production, Staging, Ops), and all of
them are controlled by themselves, except Staging invokes the 2 & 3rd tool
against Ops (for continuous deployment).

my advice would put more effort in testing the mechanism (on daily basis,
as part of your CI/test harness), and let that requirement drive the actual
choice of tools. There are ample options in the opensource world, also
writing new ones, or customizing existing ones is not a lot of effort.

On Tue, Oct 15, 2013 at 12:06 PM, Nick Silkey nick@silkey.org wrote:

Ohai Chefs --

Im curious what others have done to tackle the provision/deprovision of
users within a given Chef Server.

My use case is having to own multiple company product's disparate
open-source chef servers. Im envisioning a workflow where a top-level
chef-server manages the foo product's chef server(s), satisfying dev+ops+qe
for the foo product are provisioned/deprovisioned. The same goes for the
bar product's chef server(s) + team, the baz product's chef server(s) +
team, etc. This would allow for onboarding already spun-up chef servers
and continuing to provide airspace segregation between products.

  • Is anyone doing anything like this currently?
  • How is it working out? Wonderfully? Terribly?
  • Can alternatives like Private Chef solve this function?

Curious what the community is doing. Thanks.

-nick