I want to implement env_attributes() in Chef roles

DV1 · November 14, 2013, 7:02pm

Hey all,

I’m trying to come up with an efficient way to promote changes to our
applications one environment at a time. Say I have 4 prod environments, and
many roles in each environment, some for database, some for app server,
etc. My task is to promote a new version of database to each environment in
a specific role only, one environment at a time (to avoid downtime).

If I update the database version attribute in the role, it’ll update all
environments. If I update the database version attribute in the
environment, it’ll update all databases in that environment, not only the
one I’m targeting.

I want to have ability to specify something like the following in the role:

env_attributes(
“env-1” => {
“database” => {
“version” => “1.2.3”
}
},
“env-2” => {
“database” => {
“verison” => “1.2.1”
}
}
)

My current solution is to use env_run_lists() to do this, but it’s
time-consuming as I have to create a new recipe each time I want to upgrade
to new version.

Does that make sense? Would anyone know how I can go about implementing
this role functionality?

–
Best regards, Dmitriy V.

zuazo_onddo · November 14, 2013, 8:49pm

Hello DV,

On Thu, 14 Nov 2013 11:02:06 -0800
DV vindimy@gmail.com wrote:

[...]

I want to have ability to specify something like the following in the
role:

env_attributes(
"env-1" => {
"database" => {
"version" => "1.2.3"
}
},
"env-2" => {
"database" => {
"verison" => "1.2.1"
}
}
)

My current solution is to use env_run_lists() to do this, but it's
time-consuming as I have to create a new recipe each time I want to
upgrade to new version.

Does that make sense? Would anyone know how I can go about
implementing this role functionality?

An old ticket related to this topic:

https://tickets.opscode.com/browse/CHEF-3441

I recommend you read the comments below. It seems that it would not be
advisable to use this feature in case of adding it to the core.

I started the implementation of this some time ago. It's actually quite
simple. But I didn't continue with it because it seemed that there was
no interest.

Regards,

--
Xabier de Zuazo Oteiza
IT System Administrator - Onddo Labs S.L.
www.onddo.com

Public Key = http://www.onddo.com/xabier_zuazo.pub
Key Fingerprint = 8EFA 5B17 7275 5F1F 42B2 26B4 8E18 8B67 9DE1 9468

Morgan_Blackthorne · November 14, 2013, 9:10pm

Something I heard at the Summit was that folks are thinking of deprecating
environment run lists.

--
~~ StormeRider ~~

"Every world needs its heroes [...] They inspire us to be better than we
are. And they protect from the darkness that's just around the corner."

(from Smallville Season 6x1: "Zod")

On why I hate the phrase "that's so lame"... http://bit.ly/Ps3uSS

On Thu, Nov 14, 2013 at 12:49 PM, Xabier de Zuazo xabier@onddo.com wrote:

Hello DV,

On Thu, 14 Nov 2013 11:02:06 -0800
DV vindimy@gmail.com wrote:

[...]

I want to have ability to specify something like the following in the
role:

env_attributes(
"env-1" => {
"database" => {
"version" => "1.2.3"
}
},
"env-2" => {
"database" => {
"verison" => "1.2.1"
}
}
)

My current solution is to use env_run_lists() to do this, but it's
time-consuming as I have to create a new recipe each time I want to
upgrade to new version.

Does that make sense? Would anyone know how I can go about
implementing this role functionality?

An old ticket related to this topic:

https://tickets.opscode.com/browse/CHEF-3441

I recommend you read the comments below. It seems that it would not be
advisable to use this feature in case of adding it to the core.

I started the implementation of this some time ago. It's actually quite
simple. But I didn't continue with it because it seemed that there was
no interest.

Regards,

--
Xabier de Zuazo Oteiza
IT System Administrator - Onddo Labs S.L.
www.onddo.com

Public Key = http://www.onddo.com/xabier_zuazo.pub
Key Fingerprint = 8EFA 5B17 7275 5F1F 42B2 26B4 8E18 8B67 9DE1 9468

DV1 · November 14, 2013, 9:24pm

That doesn't make sense... There's a well-defined need for having
role+environment specific attributes. Most organizations have multiple prod
environments and want to roll out upgrades one environment at a time.

I think the method of having a switch/if statement in recipe or using
environment run lists is too burdensome... Plus I want my users to stop
writing one-off recipes. Having env attributes would make it super easy to
get the big picture by looking at just one role.

Xabier, if you have any code from your attempt you can share or point me to
the right direction, I'd like to give it a try!

On Thu, Nov 14, 2013 at 1:10 PM, Morgan Blackthorne
stormerider@gmail.comwrote:

Something I heard at the Summit was that folks are thinking of deprecating
environment run lists.

--
~~ StormeRider ~~

"Every world needs its heroes [...] They inspire us to be better than we
are. And they protect from the darkness that's just around the corner."

(from Smallville Season 6x1: "Zod")

On why I hate the phrase "that's so lame"... http://bit.ly/Ps3uSS

On Thu, Nov 14, 2013 at 12:49 PM, Xabier de Zuazo xabier@onddo.comwrote:

Hello DV,

On Thu, 14 Nov 2013 11:02:06 -0800
DV vindimy@gmail.com wrote:

[...]

I want to have ability to specify something like the following in the
role:

env_attributes(
"env-1" => {
"database" => {
"version" => "1.2.3"
}
},
"env-2" => {
"database" => {
"verison" => "1.2.1"
}
}
)

My current solution is to use env_run_lists() to do this, but it's
time-consuming as I have to create a new recipe each time I want to
upgrade to new version.

Does that make sense? Would anyone know how I can go about
implementing this role functionality?

An old ticket related to this topic:

https://tickets.opscode.com/browse/CHEF-3441

I recommend you read the comments below. It seems that it would not be
advisable to use this feature in case of adding it to the core.

I started the implementation of this some time ago. It's actually quite
simple. But I didn't continue with it because it seemed that there was
no interest.

Regards,

--
Xabier de Zuazo Oteiza
IT System Administrator - Onddo Labs S.L.
www.onddo.com

Public Key = http://www.onddo.com/xabier_zuazo.pub
Key Fingerprint = 8EFA 5B17 7275 5F1F 42B2 26B4 8E18 8B67 9DE1 9468

--
Best regards, Dmitriy V.

Morgan_Blackthorne · November 14, 2013, 9:45pm

Well, like you said, env run lists are a bit of a burden, and with people
writing more application cookbooks than roles, it's easier to put the logic
into a recipe (and is currently a best practice since roles aren't
versionable).

Until roles are versionable, which was loudly voted for at the Chef 12
discussion, I doubt you'll see any official traction on this front.
Currently less is more when it comes to roles.

--
~~ StormeRider ~~

"Every world needs its heroes [...] They inspire us to be better than we
are. And they protect from the darkness that's just around the corner."

(from Smallville Season 6x1: "Zod")

On why I hate the phrase "that's so lame"... http://bit.ly/Ps3uSS

On Thu, Nov 14, 2013 at 1:24 PM, DV vindimy@gmail.com wrote:

That doesn't make sense... There's a well-defined need for having
role+environment specific attributes. Most organizations have multiple prod
environments and want to roll out upgrades one environment at a time.

I think the method of having a switch/if statement in recipe or using
environment run lists is too burdensome... Plus I want my users to stop
writing one-off recipes. Having env attributes would make it super easy to
get the big picture by looking at just one role.

Xabier, if you have any code from your attempt you can share or point me
to the right direction, I'd like to give it a try!

On Thu, Nov 14, 2013 at 1:10 PM, Morgan Blackthorne <stormerider@gmail.com

wrote:

Something I heard at the Summit was that folks are thinking of
deprecating environment run lists.

--
~~ StormeRider ~~

"Every world needs its heroes [...] They inspire us to be better than we
are. And they protect from the darkness that's just around the corner."

(from Smallville Season 6x1: "Zod")

On why I hate the phrase "that's so lame"... http://bit.ly/Ps3uSS

On Thu, Nov 14, 2013 at 12:49 PM, Xabier de Zuazo xabier@onddo.comwrote:

Hello DV,

On Thu, 14 Nov 2013 11:02:06 -0800
DV vindimy@gmail.com wrote:

[...]

I want to have ability to specify something like the following in the
role:

env_attributes(
"env-1" => {
"database" => {
"version" => "1.2.3"
}
},
"env-2" => {
"database" => {
"verison" => "1.2.1"
}
}
)

My current solution is to use env_run_lists() to do this, but it's
time-consuming as I have to create a new recipe each time I want to
upgrade to new version.

Does that make sense? Would anyone know how I can go about
implementing this role functionality?

An old ticket related to this topic:

https://tickets.opscode.com/browse/CHEF-3441

I recommend you read the comments below. It seems that it would not be
advisable to use this feature in case of adding it to the core.

I started the implementation of this some time ago. It's actually quite
simple. But I didn't continue with it because it seemed that there was
no interest.

Regards,

--
Xabier de Zuazo Oteiza
IT System Administrator - Onddo Labs S.L.
www.onddo.com

Public Key = http://www.onddo.com/xabier_zuazo.pub
Key Fingerprint = 8EFA 5B17 7275 5F1F 42B2 26B4 8E18 8B67 9DE1 9468

--
Best regards, Dmitriy V.

zuazo_onddo · November 15, 2013, 6:30pm

Hi DV,

On Thu, 14 Nov 2013 13:24:57 -0800
DV vindimy@gmail.com wrote:

[...]

Xabier, if you have any code from your attempt you can share or point
me to the right direction, I'd like to give it a try!

I honestly don't remember how far I get. I rebased this against the
las Chef 10 stable, but this code is more than a year old and is
not tested, it may even be incomplete. Used the wrong way can destroy
the space time continuum.

If you want to test this with Chef 11, you will also need to patch
erchef:

github.com

chef-boneyard/chef_db/blob/master/src/chef_otto.erl#L290


      
          cleanup_couch_node_record({_}=R) ->
              N1 = ej:delete({<<"_rev">>}, ej:delete({<<"_id">>}, R)),
              FixedKeys = [{<<"chef_environment">>, <<"_default">>},
                           {<<"default">>, ?EMPTY_EJSON_HASH},
                           {<<"normal">>, ?EMPTY_EJSON_HASH},
                           {<<"override">>, ?EMPTY_EJSON_HASH},
                           {<<"automatic">>, ?EMPTY_EJSON_HASH},
                           {<<"run_list">>, []}],
              lists:foldl(fun({Key, Default}, Node) ->
                                  add_if_not_found(Node, Key, Default)
                          end, N1, FixedKeys).
          
          cleanup_couch_role_record(R) when is_list(R) ->
              cleanup_couch_role_record({R});
          cleanup_couch_role_record({_}=R) ->
              R1 = ej:delete({<<"_rev">>}, ej:delete({<<"_id">>}, R)),
              %% FIXME 1: what about chef_type and json_class?
              %% FIXME 2: refactor to DRY up role/node code
              FixedKeys = [{<<"description">>, <<"">>},
                           {<<"default_attributes">>, ?EMPTY_EJSON_HASH},
                           {<<"override_attributes">>, ?EMPTY_EJSON_HASH},

github.com

chef-boneyard/chef_wm/blob/master/src/chef_wm_malformed.erl#L122


      
              error_envelope([<<"Field '", Key/binary, "' missing">>]);
          
          %% Exact and string_match failures
          malformed_request_message(#ej_invalid{type=Type,
                                                key=Key
                                               }, _Req, _State) when Type =:= exact ;
                                                                     Type =:= string_match ->
              % TODO msg is provided here - we should use it if it is.
              error_envelope([<<"Field '", Key/binary, "' invalid">>]);
          
          %% Things that should be hashes, but aren't.
          malformed_request_message(#ej_invalid{type=json_type,
                                                key=Key
                                               }, _Req, _State) when Key =:= <<"override_attributes">> ;
                                                                     Key =:= <<"default_attributes">> ;
                                                                     Key =:= <<"normal">> ;
                                                                     Key =:= <<"default">> ;
                                                                     Key =:= <<"override">> ;
                                                                     Key =:= <<"automatic">> ;
                                                                     Key =:= <<"cookbook_versions">> ->
              error_envelope([<<"Field '", Key/binary, "' is not a hash">>]);

github.com

chef-boneyard/chef_objects/blob/master/src/chef_role.erl#L70


      
                   list/2
                   ]).
          -ifdef(TEST).
          -compile(export_all).
          -endif.
          
          -include("chef_types.hrl").
          
          -behaviour(chef_object).
          
          -define(DEFAULT_FIELD_VALUES,
                  [
                   {<<"json_class">>, <<"Chef::Role">>},
                   {<<"chef_type">>, <<"role">>},
                   {<<"default_attributes">>,  ?EMPTY_EJSON_HASH},
                   {<<"override_attributes">>,?EMPTY_EJSON_HASH},
                   {<<"run_list">>, []},
                   {<<"env_run_lists">>, ?EMPTY_EJSON_HASH}
                  ]).
          
          -define(VALIDATION_CONSTRAINTS,

[...]

Regards,

--
Xabier de Zuazo Oteiza
IT System Administrator - Onddo Labs S.L.
www.onddo.com

Public Key = http://www.onddo.com/xabier_zuazo.pub
Key Fingerprint = 8EFA 5B17 7275 5F1F 42B2 26B4 8E18 8B67 9DE1 9468

DV1 · November 18, 2013, 2:51am

Thanks Xabier! Unfortunately I don't know much about Chef to be able to
patch it.. If I end up implementing this sort of feature, I'll go the route
of writing a special recipe that looks at a special place containing
environments and attributes. Then it overrides attributes with the ones it
found, if applicable. Finally, I'll place that special recipe first thing
in the runlist.

On Fri, Nov 15, 2013 at 10:30 AM, Xabier de Zuazo xabier@onddo.com wrote:

Hi DV,

On Thu, 14 Nov 2013 13:24:57 -0800
DV vindimy@gmail.com wrote:

[...]

Xabier, if you have any code from your attempt you can share or point
me to the right direction, I'd like to give it a try!

GitHub - zuazo-forks/chef at 10.28.2-CHEF-3441

I honestly don't remember how far I get. I rebased this against the
las Chef 10 stable, but this code is more than a year old and is
not tested, it may even be incomplete. Used the wrong way can destroy
the space time continuum.

If you want to test this with Chef 11, you will also need to patch
erchef:

chef_db/src/chef_otto.erl at master · chef-boneyard/chef_db · GitHub

chef_wm/src/chef_wm_malformed.erl at master · chef-boneyard/chef_wm · GitHub
chef_objects/src/chef_role.erl at master · chef-boneyard/chef_objects · GitHub
[...]

Regards,

--
Xabier de Zuazo Oteiza
IT System Administrator - Onddo Labs S.L.
www.onddo.com

Public Key = http://www.onddo.com/xabier_zuazo.pub
Key Fingerprint = 8EFA 5B17 7275 5F1F 42B2 26B4 8E18 8B67 9DE1 9468

--
Best regards, Dmitriy V.

Topic		Replies	Views
Chef Attributes Chef Infra (archive)	1	214	December 14, 2011
Creating attributes that are tied to Environment and Roles Chef Infra (archive)	2	270	December 22, 2011
Iterating through chef environment attributes Chef Infra (archive)	1	1466	October 3, 2017
Chef 11 Attribute Changes -- Computed Attributes Edition Chef Infra (archive)	12	566	November 9, 2012
Question about chef environments and attributes Chef Infra (archive)	1	288	October 11, 2013

I want to implement env_attributes() in Chef roles

-- Xabier de Zuazo Oteiza IT System Administrator - Onddo Labs S.L. www.onddo.com

Public Key = http://www.onddo.com/xabier_zuazo.pub Key Fingerprint = 8EFA 5B17 7275 5F1F 42B2 26B4 8E18 8B67 9DE1 9468

-- Xabier de Zuazo Oteiza IT System Administrator - Onddo Labs S.L. www.onddo.com

Public Key = http://www.onddo.com/xabier_zuazo.pub Key Fingerprint = 8EFA 5B17 7275 5F1F 42B2 26B4 8E18 8B67 9DE1 9468

-- Xabier de Zuazo Oteiza IT System Administrator - Onddo Labs S.L. www.onddo.com

Public Key = http://www.onddo.com/xabier_zuazo.pub Key Fingerprint = 8EFA 5B17 7275 5F1F 42B2 26B4 8E18 8B67 9DE1 9468

-- Xabier de Zuazo Oteiza IT System Administrator - Onddo Labs S.L. www.onddo.com

Public Key = http://www.onddo.com/xabier_zuazo.pub Key Fingerprint = 8EFA 5B17 7275 5F1F 42B2 26B4 8E18 8B67 9DE1 9468

-- Xabier de Zuazo Oteiza IT System Administrator - Onddo Labs S.L. www.onddo.com

Public Key = http://www.onddo.com/xabier_zuazo.pub Key Fingerprint = 8EFA 5B17 7275 5F1F 42B2 26B4 8E18 8B67 9DE1 9468

-- Xabier de Zuazo Oteiza IT System Administrator - Onddo Labs S.L. www.onddo.com

Public Key = http://www.onddo.com/xabier_zuazo.pub Key Fingerprint = 8EFA 5B17 7275 5F1F 42B2 26B4 8E18 8B67 9DE1 9468

Related Topics

--
Xabier de Zuazo Oteiza
IT System Administrator - Onddo Labs S.L.
www.onddo.com

Public Key = http://www.onddo.com/xabier_zuazo.pub
Key Fingerprint = 8EFA 5B17 7275 5F1F 42B2 26B4 8E18 8B67 9DE1 9468

--
Xabier de Zuazo Oteiza
IT System Administrator - Onddo Labs S.L.
www.onddo.com

Public Key = http://www.onddo.com/xabier_zuazo.pub
Key Fingerprint = 8EFA 5B17 7275 5F1F 42B2 26B4 8E18 8B67 9DE1 9468

--
Xabier de Zuazo Oteiza
IT System Administrator - Onddo Labs S.L.
www.onddo.com

Public Key = http://www.onddo.com/xabier_zuazo.pub
Key Fingerprint = 8EFA 5B17 7275 5F1F 42B2 26B4 8E18 8B67 9DE1 9468

--
Xabier de Zuazo Oteiza
IT System Administrator - Onddo Labs S.L.
www.onddo.com

Public Key = http://www.onddo.com/xabier_zuazo.pub
Key Fingerprint = 8EFA 5B17 7275 5F1F 42B2 26B4 8E18 8B67 9DE1 9468

--
Xabier de Zuazo Oteiza
IT System Administrator - Onddo Labs S.L.
www.onddo.com

Public Key = http://www.onddo.com/xabier_zuazo.pub
Key Fingerprint = 8EFA 5B17 7275 5F1F 42B2 26B4 8E18 8B67 9DE1 9468

--
Xabier de Zuazo Oteiza
IT System Administrator - Onddo Labs S.L.
www.onddo.com

Public Key = http://www.onddo.com/xabier_zuazo.pub
Key Fingerprint = 8EFA 5B17 7275 5F1F 42B2 26B4 8E18 8B67 9DE1 9468