Hola!
I’m evaluating the introduction of Chef in an environment with about 40
different types of server, delivering about 70 different applications
which are running partly different configurations. In other words … a
big mess … or should I say, a big challenge? 
Any way, we’re trying to figure out how to define roles for our
colourful environment. We came up with this:
roles/
|-- dishes/
|-- flavours/
|-- ingredients/
|-- locations/
`-- misc/
I’m trying to categorize the degrees of freedom we have concerning our
application/ operational needs.
- An “ingredient” is equivalent to a operational component (i.e.
apache2, nginx, squid…) or an application. Anything we configure or
deploy.
- A “flavour” is a special characteristic that can be
activated/deactivated on a set of servers.
- A “dish” are things you can define as a set of “ingredients” and
"flavours". This are i.e. the different types of servers we manage.
- A “location” will be a group of dishes, i.e. if you create a small
group of server to deliver a functionality for testing /developing or a
set of web-servers.
- “misc” for those thing I forgot about.
We just figured out, that chef doesn’t support this approach.
So my question will be: What will you do to manage this kind of
scenario? I don’t want to work on a directory with way over 100
configuration files, do you see another way to manage this?
Thanks a lot for you help
I've recently grown to more than 50 roles and have also realised that it
would be good to stuff them into subdirs or classes of roles that make
sense.
On 29 October 2010 14:09, Christian Requena crekev@googlemail.com wrote:
Hola!
I'm evaluating the introduction of Chef in an environment with about 40
different types of server, delivering about 70 different applications
which are running partly different configurations. In other words .. a
big mess ... or should I say, a big challenge? 
Any way, we're trying to figure out how to define roles for our
colourful environment. We came up with this:
roles/
|-- dishes/
|-- flavours/
|-- ingredients/
|-- locations/
`-- misc/
I'm trying to categorize the degrees of freedom we have concerning our
application/ operational needs.
- An "ingredient" is equivalent to a operational component (i.e.
apache2, nginx, squid...) or an application. Anything we configure or
deploy.
- A "flavour" is a special characteristic that can be
activated/deactivated on a set of servers.
- A "dish" are things you can define as a set of "ingredients" and
"flavours". This are i.e. the different types of servers we manage.
- A "location" will be a group of dishes, i.e. if you create a small
group of server to deliver a functionality for testing /developing or a
set of web-servers.
- "misc" for those thing I forgot about.
We just figured out, that chef doesn't support this approach.
So my question will be: What will you do to manage this kind of
scenario? I don't want to work on a directory with way over 100
configuration files, do you see another way to manage this?
Thanks a lot for you help
A word of advice, implementing configuration management is a really good
time to revisit all the legacy decisions that created complexity in your
environment. Don't waste the opportunity just redescribing what exists,
revisit the why it exists and move to simplify and standardize.
On Fri, Oct 29, 2010 at 3:09 PM, Christian Requena crekev@googlemail.comwrote:
Hola!
I'm evaluating the introduction of Chef in an environment with about 40
different types of server, delivering about 70 different applications
which are running partly different configurations. In other words .. a
big mess ... or should I say, a big challenge?
Any way, we're trying to figure out how to define roles for our
colourful environment. We came up with this:
roles/
|-- dishes/
|-- flavours/
|-- ingredients/
|-- locations/
`-- misc/
I'm trying to categorize the degrees of freedom we have concerning our
application/ operational needs.
- An "ingredient" is equivalent to a operational component (i.e.
apache2, nginx, squid...) or an application. Anything we configure or
deploy.
- A "flavour" is a special characteristic that can be
activated/deactivated on a set of servers.
- A "dish" are things you can define as a set of "ingredients" and
"flavours". This are i.e. the different types of servers we manage.
- A "location" will be a group of dishes, i.e. if you create a small
group of server to deliver a functionality for testing /developing or a
set of web-servers.
- "misc" for those thing I forgot about.
We just figured out, that chef doesn't support this approach.
So my question will be: What will you do to manage this kind of
scenario? I don't want to work on a directory with way over 100
configuration files, do you see another way to manage this?
Thanks a lot for you help
you can prefix the roles and toss em in subdirs for now (just to organzie the way things look)
i.e. prod_webserver role -> roles/prod/prod_webserver.rb
this is what we have done to help organize roles. Not ideal, but workable
On Oct 29, 2010, at 2:21 PM, AJ Christensen wrote:
I've recently grown to more than 50 roles and have also realised that it would be good to stuff them into subdirs or classes of roles that make sense.
On 29 October 2010 14:09, Christian Requena crekev@googlemail.com wrote:
Hola!
I'm evaluating the introduction of Chef in an environment with about 40
different types of server, delivering about 70 different applications
which are running partly different configurations. In other words .. a
big mess ... or should I say, a big challenge?
Any way, we're trying to figure out how to define roles for our
colourful environment. We came up with this:
roles/
|-- dishes/
|-- flavours/
|-- ingredients/
|-- locations/
`-- misc/
I'm trying to categorize the degrees of freedom we have concerning our
application/ operational needs.
- An "ingredient" is equivalent to a operational component (i.e.
apache2, nginx, squid...) or an application. Anything we configure or
deploy.
- A "flavour" is a special characteristic that can be
activated/deactivated on a set of servers.
- A "dish" are things you can define as a set of "ingredients" and
"flavours". This are i.e. the different types of servers we manage.
- A "location" will be a group of dishes, i.e. if you create a small
group of server to deliver a functionality for testing /developing or a
set of web-servers.
- "misc" for those thing I forgot about.
We just figured out, that chef doesn't support this approach.
So my question will be: What will you do to manage this kind of
scenario? I don't want to work on a directory with way over 100
configuration files, do you see another way to manage this?
Thanks a lot for you help
I would like to subscribe to what Andrew said.
When the configuration of servers and applications grows organically
at a company, it often ends-up with myriads of gratuitous software
combinations and versions (just like evolution and life :-).
40-50 roles sounds like too many roles to me.
Just like code needs refactoring and cleanup from time to time, so
does configuration management.
2010/10/29 Andrew Shafer andrew@cloudscaling.com:
A word of advice, implementing configuration management is a really good
time to revisit all the legacy decisions that created complexity in your
environment. Don't waste the opportunity just redescribing what exists,
revisit the why it exists and move to simplify and standardize.
On Fri, Oct 29, 2010 at 3:09 PM, Christian Requena crekev@googlemail.com
wrote:
Hola!
I'm evaluating the introduction of Chef in an environment with about 40
different types of server, delivering about 70 different applications
which are running partly different configurations. In other words .. a
big mess ... or should I say, a big challenge?
Any way, we're trying to figure out how to define roles for our
colourful environment. We came up with this:
roles/
|-- dishes/
|-- flavours/
|-- ingredients/
|-- locations/
`-- misc/
I'm trying to categorize the degrees of freedom we have concerning our
application/ operational needs.
- An "ingredient" is equivalent to a operational component (i.e.
apache2, nginx, squid...) or an application. Anything we configure or
deploy.
- A "flavour" is a special characteristic that can be
activated/deactivated on a set of servers.
- A "dish" are things you can define as a set of "ingredients" and
"flavours". This are i.e. the different types of servers we manage.
- A "location" will be a group of dishes, i.e. if you create a small
group of server to deliver a functionality for testing /developing or a
set of web-servers.
- "misc" for those thing I forgot about.
We just figured out, that chef doesn't support this approach.
So my question will be: What will you do to manage this kind of
scenario? I don't want to work on a directory with way over 100
configuration files, do you see another way to manage this?
Thanks a lot for you help
You can certainly adapt the existing tools to work this way. 'knife
role from file' takes a path to the role, so you should be able to get
that done easily. Integrating that with rake should be similarly
simple.
Adam
On Fri, Oct 29, 2010 at 2:09 PM, Christian Requena
crekev@googlemail.com wrote:
Hola!
I'm evaluating the introduction of Chef in an environment with about 40
different types of server, delivering about 70 different applications
which are running partly different configurations. In other words .. a
big mess ... or should I say, a big challenge?
Any way, we're trying to figure out how to define roles for our
colourful environment. We came up with this:
roles/
|-- dishes/
|-- flavours/
|-- ingredients/
|-- locations/
`-- misc/
I'm trying to categorize the degrees of freedom we have concerning our
application/ operational needs.
- An "ingredient" is equivalent to a operational component (i.e.
apache2, nginx, squid...) or an application. Anything we configure or
deploy.
- A "flavour" is a special characteristic that can be
activated/deactivated on a set of servers.
- A "dish" are things you can define as a set of "ingredients" and
"flavours". This are i.e. the different types of servers we manage.
- A "location" will be a group of dishes, i.e. if you create a small
group of server to deliver a functionality for testing /developing or a
set of web-servers.
- "misc" for those thing I forgot about.
We just figured out, that chef doesn't support this approach.
So my question will be: What will you do to manage this kind of
scenario? I don't want to work on a directory with way over 100
configuration files, do you see another way to manage this?
Thanks a lot for you help
--
Opscode, Inc.
Adam Jacob, CTO
T: (206) 508-7449 E: adam@opscode.com
Yo,
On 29 October 2010 14:29, Miquel Torres tobami@googlemail.com wrote:
I would like to subscribe to what Andrew said.
When the configuration of servers and applications grows organically
at a company, it often ends-up with myriads of gratuitous software
combinations and versions (just like evolution and life :-).
40-50 roles sounds like too many roles to me.
Depends on the infrastructure really! You can't immediately go to the
smallest, simplest set of roles, no? Takes time.
For a long way, smashing roles together via run_list was the only way to get
correct attribute stacking behavior (i.e. totally neglecting Cookbook
attributes). Since this has been fixed, I've been shifting mostly back to
default attributes in new cookbooks. We've probably just got legacy stuff
hanging around =)
I've ended up having 12-15 "lump" roles that massage together a smaller
subset of bits (via run_list) who individually (generally) have default
attributes that aren't exposed in the Cookbook attributes.
Just like code needs refactoring and cleanup from time to time, so
does configuration management.
F'sho
2010/10/29 Andrew Shafer andrew@cloudscaling.com:
A word of advice, implementing configuration management is a really good
time to revisit all the legacy decisions that created complexity in your
environment. Don't waste the opportunity just redescribing what exists,
revisit the why it exists and move to simplify and standardize.
On Fri, Oct 29, 2010 at 3:09 PM, Christian Requena <
crekev@googlemail.com>
wrote:
Hola!
I'm evaluating the introduction of Chef in an environment with about 40
different types of server, delivering about 70 different applications
which are running partly different configurations. In other words .. a
big mess ... or should I say, a big challenge?
Any way, we're trying to figure out how to define roles for our
colourful environment. We came up with this:
roles/
|-- dishes/
|-- flavours/
|-- ingredients/
|-- locations/
`-- misc/
I'm trying to categorize the degrees of freedom we have concerning our
application/ operational needs.
- An "ingredient" is equivalent to a operational component (i.e.
apache2, nginx, squid...) or an application. Anything we configure or
deploy.
- A "flavour" is a special characteristic that can be
activated/deactivated on a set of servers.
- A "dish" are things you can define as a set of "ingredients" and
"flavours". This are i.e. the different types of servers we manage.
- A "location" will be a group of dishes, i.e. if you create a small
group of server to deliver a functionality for testing /developing or a
set of web-servers.
- "misc" for those thing I forgot about.
We just figured out, that chef doesn't support this approach.
So my question will be: What will you do to manage this kind of
scenario? I don't want to work on a directory with way over 100
configuration files, do you see another way to manage this?
Thanks a lot for you help
yes, but how do you tell chef to use the roles in the subdirectories?
On 10/29/2010 11:26 PM, Jesse Nelson wrote:
you can prefix the roles and toss em in subdirs for now (just to
organzie the way things look)
i.e. prod_webserver role -> roles/prod/prod_webserver.rb
this is what we have done to help organize roles. Not ideal, but workable
On Oct 29, 2010, at 2:21 PM, AJ Christensen wrote:
I've recently grown to more than 50 roles and have also realised that
it would be good to stuff them into subdirs or classes of roles that
make sense.
On 29 October 2010 14:09, Christian Requena <crekev@googlemail.com
mailto:crekev@googlemail.com> wrote:
Hola!
I'm evaluating the introduction of Chef in an environment with
about 40
different types of server, delivering about 70 different applications
which are running partly different configurations. In other words
.. a
big mess ... or should I say, a big challenge? ;)
Any way, we're trying to figure out how to define roles for our
colourful environment. We came up with this:
roles/
|-- dishes/
|-- flavours/
|-- ingredients/
|-- locations/
`-- misc/
I'm trying to categorize the degrees of freedom we have
concerning our
application/ operational needs.
* An "ingredient" is equivalent to a operational component (i.e.
apache2, nginx, squid...) or an application. Anything we configure or
deploy.
* A "flavour" is a special characteristic that can be
activated/deactivated on a set of servers.
* A "dish" are things you can define as a set of "ingredients" and
"flavours". This are i.e. the different types of servers we manage.
* A "location" will be a group of dishes, i.e. if you create a small
group of server to deliver a functionality for testing
/developing or a
set of web-servers.
* "misc" for those thing I forgot about.
We just figured out, that chef doesn't support this approach.
So my question will be: What will you do to manage this kind of
scenario? I don't want to work on a directory with way over 100
configuration files, do you see another way to manage this?
Thanks a lot for you help
Ok, I understand what you mean. But what have you done to optimize your
config/deployment? I could go extreme an say, I'll write 3 mighty roles
1 for DEV, 1 for Q&A and 1 for production, but then I'll have such
mighty/complex things ... scary.
On 10/30/2010 12:15 AM, AJ Christensen wrote:
Yo,
On 29 October 2010 14:29, Miquel Torres <tobami@googlemail.com
mailto:tobami@googlemail.com> wrote:
I would like to subscribe to what Andrew said.
When the configuration of servers and applications grows organically
at a company, it often ends-up with myriads of gratuitous software
combinations and versions (just like evolution and life :-).
40-50 roles sounds like too many roles to me.
Depends on the infrastructure really! You can't immediately go to the
smallest, simplest set of roles, no? Takes time.
For a long way, smashing roles together via run_list was the only way
to get correct attribute stacking behavior (i.e. totally neglecting
Cookbook attributes). Since this has been fixed, I've been shifting
mostly back to default attributes in new cookbooks. We've probably
just got legacy stuff hanging around =)
I've ended up having 12-15 "lump" roles that massage together a
smaller subset of bits (via run_list) who individually (generally)
have default attributes that aren't exposed in the Cookbook attributes.
Just like code needs refactoring and cleanup from time to time, so
does configuration management.
F'sho
2010/10/29 Andrew Shafer <andrew@cloudscaling.com
<mailto:andrew@cloudscaling.com>>:
>
> A word of advice, implementing configuration management is a
really good
> time to revisit all the legacy decisions that created complexity
in your
> environment. Don't waste the opportunity just redescribing what
exists,
> revisit the why it exists and move to simplify and standardize.
>
>
> On Fri, Oct 29, 2010 at 3:09 PM, Christian Requena
<crekev@googlemail.com <mailto:crekev@googlemail.com>>
> wrote:
>>
>> Hola!
>>
>> I'm evaluating the introduction of Chef in an environment with
about 40
>> different types of server, delivering about 70 different
applications
>> which are running partly different configurations. In other
words .. a
>> big mess ... or should I say, a big challenge? ;)
>>
>> Any way, we're trying to figure out how to define roles for our
>> colourful environment. We came up with this:
>>
>> roles/
>> |-- dishes/
>> |-- flavours/
>> |-- ingredients/
>> |-- locations/
>> `-- misc/
>>
>> I'm trying to categorize the degrees of freedom we have
concerning our
>> application/ operational needs.
>>
>> * An "ingredient" is equivalent to a operational component (i.e.
>> apache2, nginx, squid...) or an application. Anything we
configure or
>> deploy.
>> * A "flavour" is a special characteristic that can be
>> activated/deactivated on a set of servers.
>> * A "dish" are things you can define as a set of "ingredients" and
>> "flavours". This are i.e. the different types of servers we manage.
>> * A "location" will be a group of dishes, i.e. if you create a
small
>> group of server to deliver a functionality for testing
/developing or a
>> set of web-servers.
>> * "misc" for those thing I forgot about.
>>
>> We just figured out, that chef doesn't support this approach.
>>
>> So my question will be: What will you do to manage this kind of
>> scenario? I don't want to work on a directory with way over 100
>> configuration files, do you see another way to manage this?
>>
>> Thanks a lot for you help
>
>
Actually we have a pretty standardized application stack (java stuff),
but we have a lot of applications and till now "we" like to use
dedicated server groups for different sets of applications.
Of course you are right, I'm still redescribing what we have. But this
is already a big thing. Until now we've being working with a very
ineffective approach and I can't just bring a totally new thing in.
Please don't forget, I'm evaluating the use of Chef in my company. I
want to change things, but first I must achieve a good balance between
efficiency and usability to reach acceptance.
On 10/29/2010 11:22 PM, Andrew Shafer wrote:
A word of advice, implementing configuration management is a really
good time to revisit all the legacy decisions that created complexity
in your environment. Don't waste the opportunity just redescribing
what exists, revisit the why it exists and move to simplify and
standardize.
On Fri, Oct 29, 2010 at 3:09 PM, Christian Requena
<crekev@googlemail.com mailto:crekev@googlemail.com> wrote:
Hola!
I'm evaluating the introduction of Chef in an environment with
about 40
different types of server, delivering about 70 different applications
which are running partly different configurations. In other words .. a
big mess ... or should I say, a big challenge? ;)
Any way, we're trying to figure out how to define roles for our
colourful environment. We came up with this:
roles/
|-- dishes/
|-- flavours/
|-- ingredients/
|-- locations/
`-- misc/
I'm trying to categorize the degrees of freedom we have concerning our
application/ operational needs.
* An "ingredient" is equivalent to a operational component (i.e.
apache2, nginx, squid...) or an application. Anything we configure or
deploy.
* A "flavour" is a special characteristic that can be
activated/deactivated on a set of servers.
* A "dish" are things you can define as a set of "ingredients" and
"flavours". This are i.e. the different types of servers we manage.
* A "location" will be a group of dishes, i.e. if you create a small
group of server to deliver a functionality for testing /developing
or a
set of web-servers.
* "misc" for those thing I forgot about.
We just figured out, that chef doesn't support this approach.
So my question will be: What will you do to manage this kind of
scenario? I don't want to work on a directory with way over 100
configuration files, do you see another way to manage this?
Thanks a lot for you help
Hi Christian!
On Sat, Oct 30, 2010 at 8:42 AM, Christian Requena
crekev@googlemail.com wrote:
Actually we have a pretty standardized application stack (java stuff), but
we have a lot of applications and till now "we" like to use dedicated server
groups for different sets of applications.
Are your applications largely similar? What would the purpose of the
various roles be?
I advise looking into data bags. They're arbitrary stores of JSON data
that you can use to describe the differences in your application
stacks, and then write recipes to process the data to deploy the
applications.
http://wiki.opscode.com/display/chef/Data+Bags
We do this with our own "application" and "database" cookbooks we
publish on the Cookbooks site. Examples are in the README of the
cookbook.
http://cookbooks.opscode.com/cookbooks/application
--
Opscode, Inc
Joshua Timberman, Technical Evangelist
IRC, Skype, Twitter, Github: jtimberman