RE: Re: RE: Correct use of Chef


#1

A node is simply a server. It sounds like in your situation, you may have multiple servers for each customer. So the statement about “map a node to a given customer and stage and keep the config values there” doesn’t seem to make too much sense to me.

I actually use data bags for that purpose - one data bag per customer, in your situation. Then write a “dispatcher” cookbook that reads the information from the data bag, and uses it to determine how to configure this particular node.

Editing files is something chef is extremely poor at. Doesn’t matter if it’s prop, xml, … It’s just not in chef’s philosophy (although there are some ugly ways to do it if you really, really, need to).

Chef is organized around the concept of components/packages/… Basically, you create a cookbook (or download an existing one from supermarket.getchef.com) to completely manage a particular component of your system from start to finish. For instance, postfix. Generally, the cookbook will throw away any existing configuration files, and generate a new one from scratch.

Kevin Keane

The NetTech

760-721-8339

http://www.4nettech.com

Our values: Privacy, Liberty, Justice

See https://www.4nettech.com/corp/the-nettech-values.html

-----Original message-----
From: Vikas Roonwal pvikasroonwal@gmail.com
Sent: Tuesday 11th November 2014 3:12
To: chef@lists.opscode.com
Subject: [chef] Re: RE: Correct use of Chef

Hi Kevin,
Thanks for reinforcing my belief in Chef.
The deployments and nodes are managed by a single admin team as the customizations are hosted and managed by us.
My worries were mainly about managing the databags/environment specific values - but a bit more of googling led to me believe that this is one use of environment variables/databags.
Until we move to a situation where complete files can be managed and pushed out for each stage and client I think it is OK to map a node to a given customer and stage and keep the config values there.
Only hurdle to cross is finding a cookbook/recipe which would handle prop and xml file edits-clumsy but something we have to do.
I was afraid our usage was more of a wrapper on ANT like approach - something that chef enthusiasts might question.

~Vikas

On Nov 11, 2014 4:09 PM, “Kevin Keane Subscription” <subscription@kkeane.com mailto:subscription@kkeane.com > wrote:
It sounds like chef is indeed a good choice for you.

The only question is how isolated the environments need to be. Generally speaking, you would probably want to set up ONE chef server for all your clients. That means that each client can see the configuration information for all the other clients.

You could also set up a separate chef server for each client. It can be difficult to keep all these servers consistent, though.

Or you can use a single Chef server with multi-tenant support to ensure isolation. The hosted chef server supports it, and I hear that the new version chef server 12 also has that feature.

Kevin Keane

The NetTech

http://www.4nettech.com http://www.4nettech.com

Our values: Privacy, Liberty, Justice

See https://www.4nettech.com/corp/the-nettech-values.html

-----Original message-----
From: Vikas Roonwal <pvikasroonwal@gmail.com mailto:pvikasroonwal@gmail.com >
Sent: Monday 10th November 2014 22:43
To: chef@lists.opscode.com mailto:chef@lists.opscode.com
Subject: [chef] Correct use of Chef

Hi,
This is more of a newbie’s confusion than a dev’s request for help. Just trying to ascertain if Chef is a good choice in the given scenario.
I am part of a services team which customizes a product for client usage.
There are multiple clients and each has their own environment for QA, Staging, Production and Dev.
Since the customization is done on a similar product the build and deployment options are same across.
We have been using ANT and Shell Scripts till now - with a lot of manual intervention and have looked at chef.
Are the following assumptions and usage pattern correct -

  1. Create 1 environment per customer + environment [QA, Staging,etc.]
  2. Tie the servers/machines or nodes to these environments
  3. Use Environment specific databags or properties to pass config data to nodes
  4. Enable different teams like DB, Networking to populate the relevant config values
  5. Get each node to install required apps like JBOSS, MySql, etc. and update the config files based on entries in databags [ basically edit xml or prop files]

Any insight or pointers to user experience would be of great help.

Thanks,
Vikas


#2

Yes - we do have multiple machines for a customer - say 1 each for QA,
Staging, etc.
The machines are mapped i.e. dedicated for that purpose alone.
We can either have a databag like Customer 1{prod…, qa…} or create
environments like customer1_prod with its own props.
The pain point as you mentioned is file editing, we are trying to move away
from it - but might take some time getting there.

~Vikas
On Nov 11, 2014 5:29 PM, “Kevin Keane Subscription” subscription@kkeane.com
wrote:

A node is simply a server. It sounds like in your situation, you may
have multiple servers for each customer. So the statement about “map a node
to a given customer and stage and keep the config values there” doesn’t
seem to make too much sense to me.

I actually use data bags for that purpose - one data bag per customer, in
your situation. Then write a “dispatcher” cookbook that reads the
information from the data bag, and uses it to determine how to configure
this particular node.

Editing files is something chef is extremely poor at. Doesn’t matter if
it’s prop, xml, … It’s just not in chef’s philosophy (although there are
some ugly ways to do it if you really, really, need to).

Chef is organized around the concept of components/packages/… Basically,
you create a cookbook (or download an existing one from
supermarket.getchef.com) to completely manage a particular component of
your system from start to finish. For instance, postfix. Generally, the
cookbook will throw away any existing configuration files, and generate a
new one from scratch.

Kevin Keane

The NetTech

760-721-8339

http://www.4nettech.com

Our values: Privacy, Liberty, Justice

See https://www.4nettech.com/corp/the-nettech-values.html

-----Original message-----
From: Vikas Roonwal pvikasroonwal@gmail.com
Sent: Tuesday 11th November 2014 3:12
To: chef@lists.opscode.com
Subject: [chef] Re: RE: Correct use of Chef

Hi Kevin,
Thanks for reinforcing my belief in Chef.
The deployments and nodes are managed by a single admin team as the
customizations are hosted and managed by us.
My worries were mainly about managing the databags/environment specific
values - but a bit more of googling led to me believe that this is one use
of environment variables/databags.
Until we move to a situation where complete files can be managed and
pushed out for each stage and client I think it is OK to map a node to a
given customer and stage and keep the config values there.
Only hurdle to cross is finding a cookbook/recipe which would handle prop
and xml file edits-clumsy but something we have to do.
I was afraid our usage was more of a wrapper on ANT like approach -
something that chef enthusiasts might question.

~Vikas
On Nov 11, 2014 4:09 PM, “Kevin Keane Subscription” <
subscription@kkeane.com> wrote:

It sounds like chef is indeed a good choice for you.

The only question is how isolated the environments need to be. Generally
speaking, you would probably want to set up ONE chef server for all your
clients. That means that each client can see the configuration information
for all the other clients.

You could also set up a separate chef server for each client. It can be
difficult to keep all these servers consistent, though.

Or you can use a single Chef server with multi-tenant support to ensure
isolation. The hosted chef server supports it, and I hear that the new
version chef server 12 also has that feature.

Kevin Keane

The NetTech

http://www.4nettech.com

Our values: Privacy, Liberty, Justice

See https://www.4nettech.com/corp/the-nettech-values.html

-----Original message-----
From: Vikas Roonwal pvikasroonwal@gmail.com
Sent: Monday 10th November 2014 22:43
To: chef@lists.opscode.com
Subject: [chef] Correct use of Chef

Hi,
This is more of a newbie’s confusion than a dev’s request for help. Just
trying to ascertain if Chef is a good choice in the given scenario.
I am part of a services team which customizes a product for client usage.
There are multiple clients and each has their own environment for QA,
Staging, Production and Dev.
Since the customization is done on a similar product the build and
deployment options are same across.
We have been using ANT and Shell Scripts till now - with a lot of manual
intervention and have looked at chef.
Are the following assumptions and usage pattern correct -

  1. Create 1 environment per customer + environment [QA, Staging,etc.]
  2. Tie the servers/machines or nodes to these environments
  3. Use Environment specific databags or properties to pass config data to
    nodes
  4. Enable different teams like DB, Networking to populate the relevant
    config values
  5. Get each node to install required apps like JBOSS, MySql, etc. and
    update the config files based on entries in databags [ basically edit xml
    or prop files]

Any insight or pointers to user experience would be of great help.

Thanks,
Vikas


#3

I’d recommend you look at doing one organization per tenant, not one
environment per tenant. Data bags are organization-wide and you’ll see
easier code re-use if you keep the namespace to one customer.

On Tue, Nov 11, 2014 at 5:24 AM, Vikas Roonwal pvikasroonwal@gmail.com
wrote:

Yes - we do have multiple machines for a customer - say 1 each for QA,
Staging, etc.
The machines are mapped i.e. dedicated for that purpose alone.
We can either have a databag like Customer 1{prod…, qa…} or create
environments like customer1_prod with its own props.
The pain point as you mentioned is file editing, we are trying to move
away from it - but might take some time getting there.

~Vikas
On Nov 11, 2014 5:29 PM, “Kevin Keane Subscription” <
subscription@kkeane.com> wrote:

A node is simply a server. It sounds like in your situation, you may
have multiple servers for each customer. So the statement about “map a node
to a given customer and stage and keep the config values there” doesn’t
seem to make too much sense to me.

I actually use data bags for that purpose - one data bag per customer, in
your situation. Then write a “dispatcher” cookbook that reads the
information from the data bag, and uses it to determine how to configure
this particular node.

Editing files is something chef is extremely poor at. Doesn’t matter if
it’s prop, xml, … It’s just not in chef’s philosophy (although there are
some ugly ways to do it if you really, really, need to).

Chef is organized around the concept of components/packages/…
Basically, you create a cookbook (or download an existing one from
supermarket.getchef.com) to completely manage a particular component of
your system from start to finish. For instance, postfix. Generally, the
cookbook will throw away any existing configuration files, and generate a
new one from scratch.

Kevin Keane

The NetTech

760-721-8339

http://www.4nettech.com

Our values: Privacy, Liberty, Justice

See https://www.4nettech.com/corp/the-nettech-values.html

-----Original message-----
From: Vikas Roonwal pvikasroonwal@gmail.com
Sent: Tuesday 11th November 2014 3:12
To: chef@lists.opscode.com
Subject: [chef] Re: RE: Correct use of Chef

Hi Kevin,
Thanks for reinforcing my belief in Chef.
The deployments and nodes are managed by a single admin team as the
customizations are hosted and managed by us.
My worries were mainly about managing the databags/environment specific
values - but a bit more of googling led to me believe that this is one use
of environment variables/databags.
Until we move to a situation where complete files can be managed and
pushed out for each stage and client I think it is OK to map a node to a
given customer and stage and keep the config values there.
Only hurdle to cross is finding a cookbook/recipe which would handle prop
and xml file edits-clumsy but something we have to do.
I was afraid our usage was more of a wrapper on ANT like approach -
something that chef enthusiasts might question.

~Vikas
On Nov 11, 2014 4:09 PM, “Kevin Keane Subscription” <
subscription@kkeane.com> wrote:

It sounds like chef is indeed a good choice for you.

The only question is how isolated the environments need to be. Generally
speaking, you would probably want to set up ONE chef server for all your
clients. That means that each client can see the configuration information
for all the other clients.

You could also set up a separate chef server for each client. It can be
difficult to keep all these servers consistent, though.

Or you can use a single Chef server with multi-tenant support to ensure
isolation. The hosted chef server supports it, and I hear that the new
version chef server 12 also has that feature.

Kevin Keane

The NetTech

http://www.4nettech.com

Our values: Privacy, Liberty, Justice

See https://www.4nettech.com/corp/the-nettech-values.html

-----Original message-----
From: Vikas Roonwal pvikasroonwal@gmail.com
Sent: Monday 10th November 2014 22:43
To: chef@lists.opscode.com
Subject: [chef] Correct use of Chef

Hi,
This is more of a newbie’s confusion than a dev’s request for help. Just
trying to ascertain if Chef is a good choice in the given scenario.
I am part of a services team which customizes a product for client usage.
There are multiple clients and each has their own environment for QA,
Staging, Production and Dev.
Since the customization is done on a similar product the build and
deployment options are same across.
We have been using ANT and Shell Scripts till now - with a lot of manual
intervention and have looked at chef.
Are the following assumptions and usage pattern correct -

  1. Create 1 environment per customer + environment [QA, Staging,etc.]
  2. Tie the servers/machines or nodes to these environments
  3. Use Environment specific databags or properties to pass config data
    to nodes
  4. Enable different teams like DB, Networking to populate the relevant
    config values
  5. Get each node to install required apps like JBOSS, MySql, etc. and
    update the config files based on entries in databags [ basically edit xml
    or prop files]

Any insight or pointers to user experience would be of great help.

Thanks,
Vikas


Justin Dossey
Practice Owner
New Context Services, Inc


#4

There are not many good Chef tools for editing small parts of an existing file, but Chef is very good at managing the entire contents of a file. See the template resource https://docs.getchef.com/resource_template.html for information. It is not always possible to manage the entire file, but if you can it makes it a lot easier.

-T

On Nov 11, 2014, at 4:04 AM, Kevin Keane Subscription subscription@kkeane.com wrote:

A node is simply a server. It sounds like in your situation, you may have multiple servers for each customer. So the statement about “map a node to a given customer and stage and keep the config values there” doesn’t seem to make too much sense to me.

I actually use data bags for that purpose - one data bag per customer, in your situation. Then write a “dispatcher” cookbook that reads the information from the data bag, and uses it to determine how to configure this particular node.

Editing files is something chef is extremely poor at. Doesn’t matter if it’s prop, xml, … It’s just not in chef’s philosophy (although there are some ugly ways to do it if you really, really, need to).

Chef is organized around the concept of components/packages/… Basically, you create a cookbook (or download an existing one from supermarket.getchef.com) to completely manage a particular component of your system from start to finish. For instance, postfix. Generally, the cookbook will throw away any existing configuration files, and generate a new one from scratch.

Kevin Keane

The NetTech

760-721-8339

http://www.4nettech.com

Our values: Privacy, Liberty, Justice

See https://www.4nettech.com/corp/the-nettech-values.html

-----Original message-----
From: Vikas Roonwal pvikasroonwal@gmail.com
Sent: Tuesday 11th November 2014 3:12
To: chef@lists.opscode.com
Subject: [chef] Re: RE: Correct use of Chef

Hi Kevin,
Thanks for reinforcing my belief in Chef.
The deployments and nodes are managed by a single admin team as the customizations are hosted and managed by us.
My worries were mainly about managing the databags/environment specific values - but a bit more of googling led to me believe that this is one use of environment variables/databags.
Until we move to a situation where complete files can be managed and pushed out for each stage and client I think it is OK to map a node to a given customer and stage and keep the config values there.
Only hurdle to cross is finding a cookbook/recipe which would handle prop and xml file edits-clumsy but something we have to do.
I was afraid our usage was more of a wrapper on ANT like approach - something that chef enthusiasts might question.

~Vikas

On Nov 11, 2014 4:09 PM, “Kevin Keane Subscription” <subscription@kkeane.com mailto:subscription@kkeane.com> wrote:
It sounds like chef is indeed a good choice for you.

The only question is how isolated the environments need to be. Generally speaking, you would probably want to set up ONE chef server for all your clients. That means that each client can see the configuration information for all the other clients.

You could also set up a separate chef server for each client. It can be difficult to keep all these servers consistent, though.

Or you can use a single Chef server with multi-tenant support to ensure isolation. The hosted chef server supports it, and I hear that the new version chef server 12 also has that feature.

Kevin Keane

The NetTech

http://www.4nettech.com http://www.4nettech.com/
Our values: Privacy, Liberty, Justice

See https://www.4nettech.com/corp/the-nettech-values.html https://www.4nettech.com/corp/the-nettech-values.html

-----Original message-----
From: Vikas Roonwal <pvikasroonwal@gmail.com mailto:pvikasroonwal@gmail.com>
Sent: Monday 10th November 2014 22:43
To: chef@lists.opscode.com mailto:chef@lists.opscode.com
Subject: [chef] Correct use of Chef

Hi,
This is more of a newbie’s confusion than a dev’s request for help. Just trying to ascertain if Chef is a good choice in the given scenario.
I am part of a services team which customizes a product for client usage.
There are multiple clients and each has their own environment for QA, Staging, Production and Dev.
Since the customization is done on a similar product the build and deployment options are same across.
We have been using ANT and Shell Scripts till now - with a lot of manual intervention and have looked at chef.
Are the following assumptions and usage pattern correct -

  1. Create 1 environment per customer + environment [QA, Staging,etc.]
  2. Tie the servers/machines or nodes to these environments
  3. Use Environment specific databags or properties to pass config data to nodes
  4. Enable different teams like DB, Networking to populate the relevant config values
  5. Get each node to install required apps like JBOSS, MySql, etc. and update the config files based on entries in databags [ basically edit xml or prop files]

Any insight or pointers to user experience would be of great help.

Thanks,
Vikas


#5

The pain point as you mentioned is file editing, we are trying to move
away from it - but might take some time getting there.

https://supermarket.getchef.com/cookbooks/line/


Jeff Blaine
kickflop.net
PGP/GnuPG Key ID: 0x0C8EDD02