Conflict between using environments to manage cookbook releases and to manage application environments


#1

Dear Chefs,

I am flummoxed by my conflicting needs for Chef environments. I currently
have 2 environments:

_default
QA: cookbooks that I am testing, I know exactly which nodes are in here
and they can tolerate a service restart or two. No important nodes belong
to this environment

PROD: Only stable, tested cookbook versions here, the vast majority of
nodes belong to this environment

on the other hand, my organization has 4, count 'em, 4
application_environments

PROD - production
QA - user-facing tests
TS - integration tests
DV - development servers

All of these servers should be using stable, tested cookbooks as my devs
don’t want to be disrupted by a misbehaving cookbook. Thus all of them
belong to PROD chef_environment, besides the couple that I am using to test
cookbooks at a given moment.

I currently manage application_environments with data bags that look like
this

apps/esb.json
{
“pr”: {
“database_schema”: “production_schema”
},
“qa”: {
“database_schema”: “qa_schema”
}
. . . .
}

additionally, I add a top-level attribute to each node “app_env” that
indicates which application_environment a node belongs to

Some cookbooks, such as the haproxy cookbook, treat chef_environments like
application_environments

I would love to treat chef_environments like application_environments but
that would completely break my workflow. How are others dealing w/ the
issues I have brought up here?


#2

On Tue, May 1, 2012 at 1:47 AM, Bryan Berry bryan.berry@gmail.com wrote:

I would love to treat chef_environments like application_environments but
that would completely break my workflow. How are others dealing w/ the
issues I have brought up here?

Why would it break your workflow?

You have a single policy, “stable”, which needs to get set in each of
your “stable” environments. What is driving change in the chef server?
Is it all manual, or are you doing something like using Jenkins to
update Chef?

That said, there would be nothing wrong with treating the Chef
environments exactly as you do now, and just having a small alteration
to how you deal with application_enviornments: rather than a single
data bag with many keys, I would do something like:

apps_esb: { <-- data bag
"pr": { … }, # <-- data bag items
"qa": { … },
}

Even this I think is actually less than ideal for application
environment modeling. One step further:

app_versions: {
“foo”: { “pr”: “12345”,
“qa”: “123456”},
“bar”: { "pr: “12345”,
“qa”: “123456”}
}

app_foo: {
“12345”: {
# everything related to this release of app_foo, data wise
},
“12346”: {
}
}

I would take it a step further and say you should have app_versions
moved out of a data bag an in to a top level chef environment, with
it’s cookbook policy being basically identical, and the versions for
every app set as environment attributes. Keep the application-release
specific values local to the application data itself.

Best,
Adam


Opscode, Inc.
Adam Jacob, Chief Customer Officer
T: (206) 619-7151 E: adam@opscode.com


#3

Hi,

We have a very similar requirements here but the way we represent it
in chef is with different environments. We do not allow the _default
environment and instead have the following set

  • production
  • staging (still needed as some of the infrastructure is still part of
    a manual non-chef managed release process)
  • uat
  • development
  • ci (aka integration tests)
  • desktop (i.e our desktop managed machines)
  • chef_dev (i.e. when developing cookbooks)

For each of these environments we have flags indicating the
datacenter/location of the environment and the equivalent of a
"stable" flag. If the “stable” flag is set then we require that the
environment explicitly lists a version of a cookbook and that the
cookbook version is frozen. This workflow is enforced by a recipe that
included at the start of every chef run. You can see a snippet that
illustrates the idea at [1].

[1] https://gist.github.com/2048310

On Tue, May 1, 2012 at 6:47 PM, Bryan Berry bryan.berry@gmail.com wrote:

Dear Chefs,

I am flummoxed by my conflicting needs for Chef environments. I currently
have 2 environments:

_default
QA: cookbooks that I am testing, I know exactly which nodes are in here
and they can tolerate a service restart or two. No important nodes belong to
this environment

PROD: Only stable, tested cookbook versions here, the vast majority of nodes
belong to this environment

on the other hand, my organization has 4, count 'em, 4
application_environments

PROD - production
QA - user-facing tests
TS - integration tests
DV - development servers

All of these servers should be using stable, tested cookbooks as my devs
don’t want to be disrupted by a misbehaving cookbook. Thus all of them
belong to PROD chef_environment, besides the couple that I am using to test
cookbooks at a given moment.

I currently manage application_environments with data bags that look like
this

apps/esb.json
{
“pr”: {
“database_schema”: “production_schema”
},
“qa”: {
“database_schema”: “qa_schema”
}
. . . .
}

additionally, I add a top-level attribute to each node “app_env” that
indicates which application_environment a node belongs to

Some cookbooks, such as the haproxy cookbook, treat chef_environments like
application_environments

I would love to treat chef_environments like application_environments but
that would completely break my workflow. How are others dealing w/ the
issues I have brought up here?


Cheers,

Peter Donald


#4

Peter,

That’s an awesome assertion to have as part of your run_list – may be
stealing this one for ourselves!

–AJ

On 2 May 2012 11:43, Peter Donald peter@realityforge.org wrote:

Hi,

We have a very similar requirements here but the way we represent it
in chef is with different environments. We do not allow the _default
environment and instead have the following set

  • production
  • staging (still needed as some of the infrastructure is still part of
    a manual non-chef managed release process)
  • uat
  • development
  • ci (aka integration tests)
  • desktop (i.e our desktop managed machines)
  • chef_dev (i.e. when developing cookbooks)

For each of these environments we have flags indicating the
datacenter/location of the environment and the equivalent of a
"stable" flag. If the “stable” flag is set then we require that the
environment explicitly lists a version of a cookbook and that the
cookbook version is frozen. This workflow is enforced by a recipe that
included at the start of every chef run. You can see a snippet that
illustrates the idea at [1].

[1] https://gist.github.com/2048310

On Tue, May 1, 2012 at 6:47 PM, Bryan Berry bryan.berry@gmail.com wrote:

Dear Chefs,

I am flummoxed by my conflicting needs for Chef environments. I currently
have 2 environments:

_default
QA: cookbooks that I am testing, I know exactly which nodes are in here
and they can tolerate a service restart or two. No important nodes belong to
this environment

PROD: Only stable, tested cookbook versions here, the vast majority of nodes
belong to this environment

on the other hand, my organization has 4, count 'em, 4
application_environments

PROD - production
QA - user-facing tests
TS - integration tests
DV - development servers

All of these servers should be using stable, tested cookbooks as my devs
don’t want to be disrupted by a misbehaving cookbook. Thus all of them
belong to PROD chef_environment, besides the couple that I am using to test
cookbooks at a given moment.

I currently manage application_environments with data bags that look like
this

apps/esb.json
{
“pr”: {
“database_schema”: “production_schema”
},
“qa”: {
“database_schema”: “qa_schema”
}
. . . .
}

additionally, I add a top-level attribute to each node “app_env” that
indicates which application_environment a node belongs to

Some cookbooks, such as the haproxy cookbook, treat chef_environments like
application_environments

I would love to treat chef_environments like application_environments but
that would completely break my workflow. How are others dealing w/ the
issues I have brought up here?


Cheers,

Peter Donald


#5

gotta say I really like Peter’s solution, will have to look at it in more
detail

To answer Adam’s, question I use manual upload for my cookbooks

The big cause of the instability in my cookbooks is that I am essentially
learning these applications at the same time that I write cookbooks for
them, not the ideal situation. I am also under a lot of pressure to set up
a quite complicated architecture in a very short period of time.
Consequently, many of my initial configuration choices are less than ideal.
Going back and fixing those choices w/out breaking running apps is a
challenge. The obvious answer here is to implement automated cookbook
testing w/ minitest/cucumber and toft/vagrant. Once my cookbooks are more
stable and tested, i should be able to move many of those attributes that I
have in data_bags over to environments. The only drawback is that I will
have a long duplicated list of cookbook version constraints in each
environment.

On Wed, May 2, 2012 at 2:02 AM, AJ Christensen aj@junglist.gen.nz wrote:

Peter,

That’s an awesome assertion to have as part of your run_list – may be
stealing this one for ourselves!

–AJ

On 2 May 2012 11:43, Peter Donald peter@realityforge.org wrote:

Hi,

We have a very similar requirements here but the way we represent it
in chef is with different environments. We do not allow the _default
environment and instead have the following set

  • production
  • staging (still needed as some of the infrastructure is still part of
    a manual non-chef managed release process)
  • uat
  • development
  • ci (aka integration tests)
  • desktop (i.e our desktop managed machines)
  • chef_dev (i.e. when developing cookbooks)

For each of these environments we have flags indicating the
datacenter/location of the environment and the equivalent of a
"stable" flag. If the “stable” flag is set then we require that the
environment explicitly lists a version of a cookbook and that the
cookbook version is frozen. This workflow is enforced by a recipe that
included at the start of every chef run. You can see a snippet that
illustrates the idea at [1].

[1] https://gist.github.com/2048310

On Tue, May 1, 2012 at 6:47 PM, Bryan Berry bryan.berry@gmail.com
wrote:

Dear Chefs,

I am flummoxed by my conflicting needs for Chef environments. I
currently

have 2 environments:

_default
QA: cookbooks that I am testing, I know exactly which nodes are in
here

and they can tolerate a service restart or two. No important nodes
belong to

this environment

PROD: Only stable, tested cookbook versions here, the vast majority of
nodes

belong to this environment

on the other hand, my organization has 4, count 'em, 4
application_environments

PROD - production
QA - user-facing tests
TS - integration tests
DV - development servers

All of these servers should be using stable, tested cookbooks as my devs
don’t want to be disrupted by a misbehaving cookbook. Thus all of them
belong to PROD chef_environment, besides the couple that I am using to
test

cookbooks at a given moment.

I currently manage application_environments with data bags that look
like

this

apps/esb.json
{
“pr”: {
“database_schema”: “production_schema”
},
“qa”: {
“database_schema”: “qa_schema”
}
. . . .
}

additionally, I add a top-level attribute to each node “app_env” that
indicates which application_environment a node belongs to

Some cookbooks, such as the haproxy cookbook, treat chef_environments
like

application_environments

I would love to treat chef_environments like application_environments
but

that would completely break my workflow. How are others dealing w/ the
issues I have brought up here?


Cheers,

Peter Donald


#6

Hi,
We are using environments in a way that more matches our business, and how we currently think about things. So we currently have 60+ environments.

These take the form of:-

_prod
_test
_dev

Where , may equal, SAP, or Documentum etc…

    Each of the projects may/may not have a prod, test or dev environment, though strictly speaking that should be a <project>_prod, <project>_test, or <project>_dev environment.  This does nice things like orders your environments by project, and for each project it is easy to see if you have a prod environment or just dev etc. Also you have a nice drop down in the chef console, such that you can select your environment and only browse the nodes in that environment.  This is a nice easy win for us.  Makes the chef web console actually useful, as we use it "read only".

   We plan to use environments in roles, but to be honest, so far we have not done so, execept in a few cases.  It is a work in progress, but for now, it gives us a nice easy business view of the environments that make sense.  Of course, we have to handle issues like separation of concerns and SOX compliance, so this is a precursor to doing this type of activity.

Almost forgot. - The above all refers to our “visible” network. For development of cookbooks we use the following process:-

  1. each sysadmin has vmware workstation, locally with 16Gb ram on workstation.

  2. each sysadmin has their own chef server, chef developer, and a redhat instance, all running in separate vm’s inside local vmware workstation.

  3. All testing is done here. Builds, etc. Allows snapshots, upgrades, trial/error, do whatever you like, it is yours and only yours.#

  4. We have a vmware domain, where we deploy “new” builds as a final check, this onto vmware ESX server.

  5. Deploy the new code to the “prodcution” chef server(ie the one at the start of the email).

            This keeps development separate from the prod/dev/test environments.    A bit of a different viewpoint from most.
    
    
                            Hope this helps.    We can expand on naming conventions in another thread, if that helps anyone, and maybe cover how we maintain install logs if that helps anyone.
    
    
                                                           Sc0tt...
    

From: Bryan Berry [bryan.berry@gmail.com]
Sent: 03 May 2012 08:19
To: chef@lists.opscode.com
Subject: [chef] Re: Re: Re: conflict between using environments to manage cookbook releases and to manage application environments

gotta say I really like Peter’s solution, will have to look at it in more detail

To answer Adam’s, question I use manual upload for my cookbooks

The big cause of the instability in my cookbooks is that I am essentially learning these applications at the same time that I write cookbooks for them, not the ideal situation. I am also under a lot of pressure to set up a quite complicated architecture in a very short period of time. Consequently, many of my initial configuration choices are less than ideal. Going back and fixing those choices w/out breaking running apps is a challenge. The obvious answer here is to implement automated cookbook testing w/ minitest/cucumber and toft/vagrant. Once my cookbooks are more stable and tested, i should be able to move many of those attributes that I have in data_bags over to environments. The only drawback is that I will have a long duplicated list of cookbook version constraints in each environment.

On Wed, May 2, 2012 at 2:02 AM, AJ Christensen <aj@junglist.gen.nzmailto:aj@junglist.gen.nz> wrote:
Peter,

That’s an awesome assertion to have as part of your run_list – may be
stealing this one for ourselves!

–AJ

On 2 May 2012 11:43, Peter Donald <peter@realityforge.orgmailto:peter@realityforge.org> wrote:

Hi,

We have a very similar requirements here but the way we represent it
in chef is with different environments. We do not allow the _default
environment and instead have the following set

  • production
  • staging (still needed as some of the infrastructure is still part of
    a manual non-chef managed release process)
  • uat
  • development
  • ci (aka integration tests)
  • desktop (i.e our desktop managed machines)
  • chef_dev (i.e. when developing cookbooks)

For each of these environments we have flags indicating the
datacenter/location of the environment and the equivalent of a
"stable" flag. If the “stable” flag is set then we require that the
environment explicitly lists a version of a cookbook and that the
cookbook version is frozen. This workflow is enforced by a recipe that
included at the start of every chef run. You can see a snippet that
illustrates the idea at [1].

[1] https://gist.github.com/2048310

On Tue, May 1, 2012 at 6:47 PM, Bryan Berry <bryan.berry@gmail.commailto:bryan.berry@gmail.com> wrote:

Dear Chefs,

I am flummoxed by my conflicting needs for Chef environments. I currently
have 2 environments:

_default
QA: cookbooks that I am testing, I know exactly which nodes are in here
and they can tolerate a service restart or two. No important nodes belong to
this environment

PROD: Only stable, tested cookbook versions here, the vast majority of nodes
belong to this environment

on the other hand, my organization has 4, count 'em, 4
application_environments

PROD - production
QA - user-facing tests
TS - integration tests
DV - development servers

All of these servers should be using stable, tested cookbooks as my devs
don’t want to be disrupted by a misbehaving cookbook. Thus all of them
belong to PROD chef_environment, besides the couple that I am using to test
cookbooks at a given moment.

I currently manage application_environments with data bags that look like
this

apps/esb.json
{
“pr”: {
“database_schema”: “production_schema”
},
“qa”: {
“database_schema”: “qa_schema”
}
. . . .
}

additionally, I add a top-level attribute to each node “app_env” that
indicates which application_environment a node belongs to

Some cookbooks, such as the haproxy cookbook, treat chef_environments like
application_environments

I would love to treat chef_environments like application_environments but
that would completely break my workflow. How are others dealing w/ the
issues I have brought up here?


Cheers,

Peter Donald


#7

Nothing wrong with manual uploading of cookbooks - the important step
is that you are taking the cookbook policy and promoting it through to
other environments, essentially.

Adam

On Wed, May 2, 2012 at 10:19 PM, Bryan Berry bryan.berry@gmail.com wrote:

gotta say I really like Peter’s solution, will have to look at it in more
detail

To answer Adam’s, question I use manual upload for my cookbooks

The big cause of the instability in my cookbooks is that I am essentially
learning these applications at the same time that I write cookbooks for
them, not the ideal situation. I am also under a lot of pressure to set up a
quite complicated architecture in a very short period of time.
Consequently, many of my initial configuration choices are less than ideal.
Going back and fixing those choices w/out breaking running apps is a
challenge. The obvious answer here is to implement automated cookbook
testing w/ minitest/cucumber and toft/vagrant. Once my cookbooks are more
stable and tested, i should be able to move many of those attributes that I
have in data_bags over to environments. The only drawback is that I will
have a long duplicated list of cookbook version constraints in each
environment.

On Wed, May 2, 2012 at 2:02 AM, AJ Christensen aj@junglist.gen.nz wrote:

Peter,

That’s an awesome assertion to have as part of your run_list – may be
stealing this one for ourselves!

–AJ

On 2 May 2012 11:43, Peter Donald peter@realityforge.org wrote:

Hi,

We have a very similar requirements here but the way we represent it
in chef is with different environments. We do not allow the _default
environment and instead have the following set

  • production
  • staging (still needed as some of the infrastructure is still part of
    a manual non-chef managed release process)
  • uat
  • development
  • ci (aka integration tests)
  • desktop (i.e our desktop managed machines)
  • chef_dev (i.e. when developing cookbooks)

For each of these environments we have flags indicating the
datacenter/location of the environment and the equivalent of a
"stable" flag. If the “stable” flag is set then we require that the
environment explicitly lists a version of a cookbook and that the
cookbook version is frozen. This workflow is enforced by a recipe that
included at the start of every chef run. You can see a snippet that
illustrates the idea at [1].

[1] https://gist.github.com/2048310

On Tue, May 1, 2012 at 6:47 PM, Bryan Berry bryan.berry@gmail.com
wrote:

Dear Chefs,

I am flummoxed by my conflicting needs for Chef environments. I
currently
have 2 environments:

_default
QA: cookbooks that I am testing, I know exactly which nodes are in
here
and they can tolerate a service restart or two. No important nodes
belong to
this environment

PROD: Only stable, tested cookbook versions here, the vast majority of
nodes
belong to this environment

on the other hand, my organization has 4, count 'em, 4
application_environments

PROD - production
QA - user-facing tests
TS - integration tests
DV - development servers

All of these servers should be using stable, tested cookbooks as my
devs
don’t want to be disrupted by a misbehaving cookbook. Thus all of them
belong to PROD chef_environment, besides the couple that I am using to
test
cookbooks at a given moment.

I currently manage application_environments with data bags that look
like
this

apps/esb.json
{
“pr”: {
“database_schema”: “production_schema”
},
“qa”: {
“database_schema”: “qa_schema”
}
. . . .
}

additionally, I add a top-level attribute to each node “app_env” that
indicates which application_environment a node belongs to

Some cookbooks, such as the haproxy cookbook, treat chef_environments
like
application_environments

I would love to treat chef_environments like application_environments
but
that would completely break my workflow. How are others dealing w/ the
issues I have brought up here?


Cheers,

Peter Donald


Opscode, Inc.
Adam Jacob, Chief Customer Officer
T: (206) 619-7151 E: adam@opscode.com


#8

we use a mix of the approaches described here. We have multiple projects
and each of them have multiple environments (dev, qa, staging, uat, prod),
chef environments are treated as first class entities for isolation and
customization. Most of the scaffolding is done automagically by enforcing
conventions(like environments will be -, and
instances will be ---<instance #>), all cookbooks
inside the prod and uat are version constrained and frozen, dev and qa does
not have any restriction, while staging and qa has version constrained
cookbooks but not frozen (and upstream of prod). Nagios/graphite and other
systems exploit the naming conventions to generate appropriate service
groupings . In all cases we have the app environment (be it RoR or
python/django or LAMP or tomcat/spring) is different from chef environments
, but they are derived automatically from the chef environment. This let
the devs work without necessarily tied down to the chef environments, while
also eliminates manually managing multiple environments.

All our cookbooks along with the community contributed ones goes through
CI, where foodcritic based lint checks, integration tests (against centos
5/6 & ubuntu 10/11), and the whole environment (chef+nodes+apps) is tested
(for nodes with empty runlist, nodes that are in _default, nodes having
recipes directly assigned in runlist etc). Individual stages of cookbook
testing is modeled as individual pipelines, there are multiple custom
foodcritic rules (like checks for supports directive in metadata ) that
ensures cookbooks are eligible for downstream CI stages.

From my experience, comparing individual techniques in isolation is not
very helpful, in stead if we can discuss on end to end workflows, involving
change management, qa, safety nates and integration with other services is
something we should discuss while comparing implications of individual
techniques.

On Thu, May 3, 2012 at 9:23 PM, Adam Jacob adam@opscode.com wrote:

Nothing wrong with manual uploading of cookbooks - the important step
is that you are taking the cookbook policy and promoting it through to
other environments, essentially.

Adam

On Wed, May 2, 2012 at 10:19 PM, Bryan Berry bryan.berry@gmail.com
wrote:

gotta say I really like Peter’s solution, will have to look at it in more
detail

To answer Adam’s, question I use manual upload for my cookbooks

The big cause of the instability in my cookbooks is that I am essentially
learning these applications at the same time that I write cookbooks for
them, not the ideal situation. I am also under a lot of pressure to set
up a
quite complicated architecture in a very short period of time.
Consequently, many of my initial configuration choices are less than
ideal.
Going back and fixing those choices w/out breaking running apps is a
challenge. The obvious answer here is to implement automated cookbook
testing w/ minitest/cucumber and toft/vagrant. Once my cookbooks are more
stable and tested, i should be able to move many of those attributes
that I
have in data_bags over to environments. The only drawback is that I will
have a long duplicated list of cookbook version constraints in each
environment.

On Wed, May 2, 2012 at 2:02 AM, AJ Christensen aj@junglist.gen.nz
wrote:

Peter,

That’s an awesome assertion to have as part of your run_list – may be
stealing this one for ourselves!

–AJ

On 2 May 2012 11:43, Peter Donald peter@realityforge.org wrote:

Hi,

We have a very similar requirements here but the way we represent it
in chef is with different environments. We do not allow the _default
environment and instead have the following set

  • production
  • staging (still needed as some of the infrastructure is still part of
    a manual non-chef managed release process)
  • uat
  • development
  • ci (aka integration tests)
  • desktop (i.e our desktop managed machines)
  • chef_dev (i.e. when developing cookbooks)

For each of these environments we have flags indicating the
datacenter/location of the environment and the equivalent of a
"stable" flag. If the “stable” flag is set then we require that the
environment explicitly lists a version of a cookbook and that the
cookbook version is frozen. This workflow is enforced by a recipe that
included at the start of every chef run. You can see a snippet that
illustrates the idea at [1].

[1] https://gist.github.com/2048310

On Tue, May 1, 2012 at 6:47 PM, Bryan Berry bryan.berry@gmail.com
wrote:

Dear Chefs,

I am flummoxed by my conflicting needs for Chef environments. I
currently
have 2 environments:

_default
QA: cookbooks that I am testing, I know exactly which nodes are in
here
and they can tolerate a service restart or two. No important nodes
belong to
this environment

PROD: Only stable, tested cookbook versions here, the vast majority
of

nodes
belong to this environment

on the other hand, my organization has 4, count 'em, 4
application_environments

PROD - production
QA - user-facing tests
TS - integration tests
DV - development servers

All of these servers should be using stable, tested cookbooks as my
devs
don’t want to be disrupted by a misbehaving cookbook. Thus all of
them

belong to PROD chef_environment, besides the couple that I am using
to

test
cookbooks at a given moment.

I currently manage application_environments with data bags that look
like
this

apps/esb.json
{
“pr”: {
“database_schema”: “production_schema”
},
“qa”: {
“database_schema”: “qa_schema”
}
. . . .
}

additionally, I add a top-level attribute to each node "app_env"
that

indicates which application_environment a node belongs to

Some cookbooks, such as the haproxy cookbook, treat chef_environments
like
application_environments

I would love to treat chef_environments like application_environments
but
that would completely break my workflow. How are others dealing w/
the

issues I have brought up here?


Cheers,

Peter Donald


Opscode, Inc.
Adam Jacob, Chief Customer Officer
T: (206) 619-7151 E: adam@opscode.com


#9

You have five environments.

Regards,

–AJ

On 1 May 2012 20:47, Bryan Berry bryan.berry@gmail.com wrote:

Dear Chefs,

I am flummoxed by my conflicting needs for Chef environments. I currently
have 2 environments:

_default
QA: cookbooks that I am testing, I know exactly which nodes are in here
and they can tolerate a service restart or two. No important nodes belong to
this environment

PROD: Only stable, tested cookbook versions here, the vast majority of nodes
belong to this environment

on the other hand, my organization has 4, count 'em, 4
application_environments

PROD - production
QA - user-facing tests
TS - integration tests
DV - development servers

All of these servers should be using stable, tested cookbooks as my devs
don’t want to be disrupted by a misbehaving cookbook. Thus all of them
belong to PROD chef_environment, besides the couple that I am using to test
cookbooks at a given moment.

I currently manage application_environments with data bags that look like
this

apps/esb.json
{
“pr”: {
“database_schema”: “production_schema”
},
“qa”: {
“database_schema”: “qa_schema”
}
. . . .
}

additionally, I add a top-level attribute to each node “app_env” that
indicates which application_environment a node belongs to

Some cookbooks, such as the haproxy cookbook, treat chef_environments like
application_environments

I would love to treat chef_environments like application_environments but
that would completely break my workflow. How are others dealing w/ the
issues I have brought up here?


#10

We started using chef before chef environments existed. We use an attribute – “scope” – that represented dev, ref, prod, etc. Search queries generally always include “AND scope:#{node[:scope]}”* Since 0.10 came out, we’ve been using environments primarily to control cookbook versions. We do a lot of canary deploys and using environments to control search queries, for example, would make this more difficult (I think). We primarily treat an environment as a “release.”

*We wrote a search wrapper similar to https://github.com/infochimps-labs/ironfan-pantry/tree/master/cookbooks/silverware that includes scope and a few other attributes automatically.


#11
  • Bryan Berry [2012-05-01]:

Some cookbooks, such as the haproxy cookbook, treat chef_environments
like application_environments

I would love to treat chef_environments like application_environments
but that would completely break my workflow. How are others dealing w/
the issues I have brought up here?

I hate this mixup of chef and application environments, maybe I just
haven’t seen the light yet. Anyways, we use the opscode platform and
introduced a second organization for cookbook tests. Big PITA and I
would really love to see a clean solution.

cheers,

Nikolay


“It’s all part of my Can’t-Do approach to life.” Wally


#12

I would like to see chef_environments renamed to something like “zones” and
only manage cook versions and no other attributes, and then see
environments as a place to hold values specific to your
application_environments

On Tue, May 1, 2012 at 11:06 AM, Nikolay Sturm opscode@erisiandiscord.dewrote:

  • Bryan Berry [2012-05-01]:

Some cookbooks, such as the haproxy cookbook, treat chef_environments
like application_environments

I would love to treat chef_environments like application_environments
but that would completely break my workflow. How are others dealing w/
the issues I have brought up here?

I hate this mixup of chef and application environments, maybe I just
haven’t seen the light yet. Anyways, we use the opscode platform and
introduced a second organization for cookbook tests. Big PITA and I
would really love to see a clean solution.

cheers,

Nikolay


“It’s all part of my Can’t-Do approach to life.” Wally


#13

Think of them as isolated data centers. I can understand you don’t want your production mixing with your in development setup. But your 5-phase pipeline sounds a bit overkill.

Your devs have a reasonable level of forgiveness to each other when someone pushes code to master with a bug, yes? Push a fix and all is well. I strongly recommend you get a culture that lets you have a single master/prod and many short lived one-off feature branches/envs.

The more phases there are between dev and prod, the more differences there will be, the more likely something that seems to work in dev will not work in prod.


josephholsten.com

On 2012-05-01, at 09:17, Bryan Berry wrote:

I would like to see chef_environments renamed to something like “zones” and only manage cook versions and no other attributes, and then see environments as a place to hold values specific to your application_environments

On Tue, May 1, 2012 at 11:06 AM, Nikolay Sturm opscode@erisiandiscord.de wrote:

  • Bryan Berry [2012-05-01]:

Some cookbooks, such as the haproxy cookbook, treat chef_environments
like application_environments

I would love to treat chef_environments like application_environments
but that would completely break my workflow. How are others dealing w/
the issues I have brought up here?

I hate this mixup of chef and application environments, maybe I just
haven’t seen the light yet. Anyways, we use the opscode platform and
introduced a second organization for cookbook tests. Big PITA and I
would really love to see a clean solution.

cheers,

Nikolay


“It’s all part of my Can’t-Do approach to life.” Wally


#14

hey, pls let me clarify that the 4 different environments was the decision
of the project’s software architect, not my own. I have to live with it and
support it regardless of how I feel about it.

I agree with your notes on culture and on version control, but these are
things that I have tried to affect with little success so far.

I practice what you recommend in terms of environments by having only two
Chef environments QA and PROD

basically stable and unstable.

On Tue, May 1, 2012 at 1:09 PM, Joseph Holsten joseph@josephholsten.comwrote:

Think of them as isolated data centers. I can understand you don’t want
your production mixing with your in development setup. But your 5-phase
pipeline sounds a bit overkill.

Your devs have a reasonable level of forgiveness to each other when
someone pushes code to master with a bug, yes? Push a fix and all is well.
I strongly recommend you get a culture that lets you have a single
master/prod and many short lived one-off feature branches/envs.

The more phases there are between dev and prod, the more differences there
will be, the more likely something that seems to work in dev will not work
in prod.


josephholsten.com

On 2012-05-01, at 09:17, Bryan Berry wrote:

I would like to see chef_environments renamed to something like "zones"
and only manage cook versions and no other attributes, and then see
environments as a place to hold values specific to your
application_environments

On Tue, May 1, 2012 at 11:06 AM, Nikolay Sturm <
opscode@erisiandiscord.de> wrote:

  • Bryan Berry [2012-05-01]:

Some cookbooks, such as the haproxy cookbook, treat chef_environments
like application_environments

I would love to treat chef_environments like application_environments
but that would completely break my workflow. How are others dealing w/
the issues I have brought up here?

I hate this mixup of chef and application environments, maybe I just
haven’t seen the light yet. Anyways, we use the opscode platform and
introduced a second organization for cookbook tests. Big PITA and I
would really love to see a clean solution.

cheers,

Nikolay


“It’s all part of my Can’t-Do approach to life.” Wally


#15

Bryan’s scenario is fairly normal for an company with multiple teams and projects that have a bit of crossover in their codebases. In our company we also have to support multiple ‘production like’ environments for application support and users who want to try ‘what if’ scenarios without impacting production data.

From: Bryan Berry [mailto:bryan.berry@gmail.com]
Sent: Tuesday, May 01, 2012 4:15 AM
To: chef@lists.opscode.com
Subject: [chef] Re: Re: conflict between using environments to manage cookbook releases and to manage application environments

hey, pls let me clarify that the 4 different environments was the decision of the project’s software architect, not my own. I have to live with it and support it regardless of how I feel about it.

I agree with your notes on culture and on version control, but these are things that I have tried to affect with little success so far.

I practice what you recommend in terms of environments by having only two Chef environments QA and PROD

basically stable and unstable.

On Tue, May 1, 2012 at 1:09 PM, Joseph Holsten <joseph@josephholsten.commailto:joseph@josephholsten.com> wrote:
Think of them as isolated data centers. I can understand you don’t want your production mixing with your in development setup. But your 5-phase pipeline sounds a bit overkill.

Your devs have a reasonable level of forgiveness to each other when someone pushes code to master with a bug, yes? Push a fix and all is well. I strongly recommend you get a culture that lets you have a single master/prod and many short lived one-off feature branches/envs.

The more phases there are between dev and prod, the more differences there will be, the more likely something that seems to work in dev will not work in prod.


josephholsten.comhttp://josephholsten.com

On 2012-05-01, at 09:17, Bryan Berry wrote:

I would like to see chef_environments renamed to something like “zones” and only manage cook versions and no other attributes, and then see environments as a place to hold values specific to your application_environments

On Tue, May 1, 2012 at 11:06 AM, Nikolay Sturm <opscode@erisiandiscord.demailto:opscode@erisiandiscord.de> wrote:

  • Bryan Berry [2012-05-01]:

Some cookbooks, such as the haproxy cookbook, treat chef_environments
like application_environments

I would love to treat chef_environments like application_environments
but that would completely break my workflow. How are others dealing w/
the issues I have brought up here?

I hate this mixup of chef and application environments, maybe I just
haven’t seen the light yet. Anyways, we use the opscode platform and
introduced a second organization for cookbook tests. Big PITA and I
would really love to see a clean solution.

cheers,

Nikolay


“It’s all part of my Can’t-Do approach to life.” Wally