Chef Role and integration


#1

Hello,

we have several roles defined for different type of servers, like DB, web, rabbitmq, redis. and the integration is depend on roles, such as how the WEB server find the DB servers. in each role, there are cookbooks or sub roles in 3 layers, there are cookbooks for system, cookbooks for install tomcat/database, cookbooks to install war packages. But in some case, I just need to run the cookbooks for system or just need to deploy the war package, the easiest way is to change the run-list, but it changes the roles attribute in chef server which integration depend on. Any tips on my issue.

A typical role definition:
{
“name”: “hmweb”,
“chef_type”: “role”,
“json_class”: “Chef::Role”,
“description”: “for web application deployment, include hm-webapp”,
“override_attributes”: {
“aerohive_webapp”: {
“wars_array”: {
“hm-webapp.war”: {
“name”: “hm-webapp”,
“url”: “http://nexus-nms.aerohive.com/service/local/artifact/maven/redirect?r=hive-repository&g=com.aerohive.nms.web&a=hm-webapp&e=war&v=1.0-SNAPSHOT
},
“sysmgr-regional-webapp.war”: {
“name”: “sysmgr-regional-webapp”,
“url”: “http://nexus-nms.aerohive.com/service/local/artifact/maven/redirect?r=hive-repository&g=com.aerohive.nms&a=sysmgr-regional-webapp&e=war&v=1.0-SNAPSHOT
}
},
“service_log_enabled”: “true”
}
},
“run_list”: [
“role[_base]”,
“role[_webappbase]”,
“recipe[aerohive-webapp::deploy]”
]
}

Sincerely,
Mingfei Hua


#2

if you invoke chef-client with -o , the override option, it should not save
the node data back (iirc). you have to check no recipes or other code does
not use node.save explicitly ,

im curious why you dont want to run the complete run list (all the roles+
extra recipes), is performance (time) a concern ?
regards
ranjib

On Sat, Jun 28, 2014 at 11:20 PM, Mingfei Hua mfhua@aerohive.com wrote:

Hello,

we have several roles defined for different type of servers, like DB, web,
rabbitmq, redis. and the integration is depend on roles, such as how the
WEB server find the DB servers. in each role, there are cookbooks or sub
roles in 3 layers, there are cookbooks for system, cookbooks for install
tomcat/database, cookbooks to install war packages. But in some case, I
just need to run the cookbooks for system or just need to deploy the war
package, the easiest way is to change the run-list, but it changes the
roles attribute in chef server which integration depend on. Any tips on my
issue.

A typical role definition:

{

     "name": "hmweb",

     "chef_type": "role",

     "json_class": "Chef::Role",

     "description": "for web application deployment, include

hm-webapp",

     "override_attributes": {

               "aerohive_webapp": {

                        "wars_array": {

                                 "hm-webapp.war": {

                                           "name": "hm-webapp",

                                           "url": "

http://nexus-nms.aerohive.com/service/local/artifact/maven/redirect?r=hive-repository&g=com.aerohive.nms.web&a=hm-webapp&e=war&v=1.0-SNAPSHOT
"

                                 },

                                 "sysmgr-regional-webapp.war": {

                                           "name":

“sysmgr-regional-webapp”,

                                           "url": "

http://nexus-nms.aerohive.com/service/local/artifact/maven/redirect?r=hive-repository&g=com.aerohive.nms&a=sysmgr-regional-webapp&e=war&v=1.0-SNAPSHOT
"

                                 }

                        },

                        "service_log_enabled": "true"

               }

     },

     "run_list": [

         "role[_base]",

         "role[_webappbase]",

               "recipe[aerohive-webapp::deploy]"

     ]

}

Sincerely,

Mingfei Hua


#3

Hi Danjib,
If I use “-o” in chef-client, Not only the run list, but also the attribute “roles” in chef server be overwritten. The integration which depend on attribute “roles” may broken.

Regards,
Mingfei

From: Ranjib Dey [mailto:dey.ranjib@gmail.com]
Sent: 2014年6月29日 15:38
To: chef@lists.opscode.com
Subject: [chef] Re: Chef Role and integration

if you invoke chef-client with -o , the override option, it should not save the node data back (iirc). you have to check no recipes or other code does not use node.save explicitly ,

im curious why you dont want to run the complete run list (all the roles+ extra recipes), is performance (time) a concern ?
regards
ranjib

On Sat, Jun 28, 2014 at 11:20 PM, Mingfei Hua <mfhua@aerohive.commailto:mfhua@aerohive.com> wrote:

Hello,

we have several roles defined for different type of servers, like DB, web, rabbitmq, redis. and the integration is depend on roles, such as how the WEB server find the DB servers. in each role, there are cookbooks or sub roles in 3 layers, there are cookbooks for system, cookbooks for install tomcat/database, cookbooks to install war packages. But in some case, I just need to run the cookbooks for system or just need to deploy the war package, the easiest way is to change the run-list, but it changes the roles attribute in chef server which integration depend on. Any tips on my issue.

A typical role definition:
{
“name”: “hmweb”,
“chef_type”: “role”,
“json_class”: “Chef::Role”,
“description”: “for web application deployment, include hm-webapp”,
“override_attributes”: {
“aerohive_webapp”: {
“wars_array”: {
“hm-webapp.war”: {
“name”: “hm-webapp”,
“url”: “http://nexus-nms.aerohive.com/service/local/artifact/maven/redirect?r=hive-repository&g=com.aerohive.nms.web&a=hm-webapp&e=war&v=1.0-SNAPSHOT
},
“sysmgr-regional-webapp.war”: {
“name”: “sysmgr-regional-webapp”,
“url”: “http://nexus-nms.aerohive.com/service/local/artifact/maven/redirect?r=hive-repository&g=com.aerohive.nms&a=sysmgr-regional-webapp&e=war&v=1.0-SNAPSHOT
}
},
“service_log_enabled”: “true”
}
},
“run_list”: [
“role[_base]”,
“role[_webappbase]”,
“recipe[aerohive-webapp::deploy]”
]
}

Sincerely,
Mingfei Hua


#4

@Mingfei:

I have problems similar to yours. There seems to be a gap when it comes to fine-grained application life-cycle management. I am not sure if it is a feature/weakness in the Chariot design, or something that just has not yet been explored in depth by the community.

@Ranjib:

You ask ‘why you don’t want to run the complete run list’. I can think of various reasons:

  •      Running sub-sets of recipes when managing a test or user-acceptance environment.
    
  •      Doing emergency or unplanned maintenance/upgrades in a urgent but controlled way where the goal is to do the least amount of change.
    
  •      Off-cycle procedures which one might only want to run as planned event. For example: cleaning temp data or caches, or inflight upgrade of part of a system.
    

In the past I have read people saying that the above procedures are not what Chef is for and that I should introduce a further layer of technology such as Fabric, Rundeck, Opscode’s Push Jobs. I have issues with this idea. First of all it adds one more level of indirection, configuration, technology, language/DSL, which I can’t justify. Secondly, Chef provides a very solid/safe change/configuration management architecture, which I feel should be leveraged as much as possible.

An alternative might be to make a recipe smart enough to do certain things only when certain settings/attributes/flags are set. I think this is possible in some cases but not others. I think it’s rather inefficient/risky to have to re-release a cookbook (plus dependencies) just to handle a one-off case for example.

I tend to build some cookbooks to have recipes that represent basic execution blocks which I then can assemble into different sequences. One of these sequences will be the default one. The others will correspond to occasional use-cases. And finally the individual blocks can be used in an ad-hoc way if absolutely necessary.

I would love to hear about some experiences people have made with managing non-trivial medium sized applications with Chef.

Florian

From: Mingfei Hua [mailto:mfhua@aerohive.com]
Sent: 30 June 2014 08:59
To: chef@lists.opscode.com
Subject: [chef] RE: Re: Chef Role and integration

Hi Danjib,
If I use “-o” in chef-client, Not only the run list, but also the attribute “roles” in chef server be overwritten. The integration which depend on attribute “roles” may broken.

Regards,
Mingfei

From: Ranjib Dey [mailto:dey.ranjib@gmail.com]
Sent: 2014年6月29日 15:38
To: chef@lists.opscode.commailto:chef@lists.opscode.com
Subject: [chef] Re: Chef Role and integration

if you invoke chef-client with -o , the override option, it should not save the node data back (iirc). you have to check no recipes or other code does not use node.save explicitly ,

im curious why you dont want to run the complete run list (all the roles+ extra recipes), is performance (time) a concern ?
regards
ranjib

On Sat, Jun 28, 2014 at 11:20 PM, Mingfei Hua <mfhua@aerohive.commailto:mfhua@aerohive.com> wrote:

Hello,

we have several roles defined for different type of servers, like DB, web, rabbitmq, redis. and the integration is depend on roles, such as how the WEB server find the DB servers. in each role, there are cookbooks or sub roles in 3 layers, there are cookbooks for system, cookbooks for install tomcat/database, cookbooks to install war packages. But in some case, I just need to run the cookbooks for system or just need to deploy the war package, the easiest way is to change the run-list, but it changes the roles attribute in chef server which integration depend on. Any tips on my issue.

A typical role definition:
{
“name”: “hmweb”,
“chef_type”: “role”,
“json_class”: “Chef::Role”,
“description”: “for web application deployment, include hm-webapp”,
“override_attributes”: {
“aerohive_webapp”: {
“wars_array”: {
“hm-webapp.war”: {
“name”: “hm-webapp”,
“url”: “http://nexus-nms.aerohive.com/service/local/artifact/maven/redirect?r=hive-repository&g=com.aerohive.nms.web&a=hm-webapp&e=war&v=1.0-SNAPSHOT
},
“sysmgr-regional-webapp.war”: {
“name”: “sysmgr-regional-webapp”,
“url”: “http://nexus-nms.aerohive.com/service/local/artifact/maven/redirect?r=hive-repository&g=com.aerohive.nms&a=sysmgr-regional-webapp&e=war&v=1.0-SNAPSHOT
}
},
“service_log_enabled”: “true”
}
},
“run_list”: [
“role[_base]”,
“role[_webappbase]”,
“recipe[aerohive-webapp::deploy]”
]
}

Sincerely,
Mingfei Hua

LEGAL DISCLAIMER
This communication and any attached documents are strictly confidential and/or legally privileged and they may not be used or disclosed by someone who is not a named recipient. If you have received this electronic communication in error please notify the sender by replying to this electronic communication inserting the word “misdirected” as the subject and delete this communication from your system.


#5

@ Florian Hehlen,
I want to define run-level by clients attribute. For example, run-level 0 means just run the system base, run-level 1 means just install application and 2 means both. It can meet the requirement, but I need to change every cookbook to judge the run-level.

@Ranjib. My concerns on “why you don’t want to run the complete run list”.

  1.   The cookbook for application deployment is not well written,  it bring downtime even deploy the package already deployed.  So I don’t want to rerun it when I just add a system account which is in cookbooks belong to base.
    
  2.   More fast just run what you want.
    

From: Florian Hehlen [mailto:Florian.Hehlen@mri-group.com]
Sent: 2014年6月30日 21:36
To: chef@lists.opscode.com
Subject: [chef] RE: RE: Re: Chef Role and integration

@Mingfei:

I have problems similar to yours. There seems to be a gap when it comes to fine-grained application life-cycle management. I am not sure if it is a feature/weakness in the Chariot design, or something that just has not yet been explored in depth by the community.

@Ranjib:

You ask ‘why you don’t want to run the complete run list’. I can think of various reasons:

  •      Running sub-sets of recipes when managing a test or user-acceptance environment.
    
  •      Doing emergency or unplanned maintenance/upgrades in a urgent but controlled way where the goal is to do the least amount of change.
    
  •      Off-cycle procedures which one might only want to run as planned event. For example: cleaning temp data or caches, or inflight upgrade of part of a system.
    

In the past I have read people saying that the above procedures are not what Chef is for and that I should introduce a further layer of technology such as Fabric, Rundeck, Opscode’s Push Jobs. I have issues with this idea. First of all it adds one more level of indirection, configuration, technology, language/DSL, which I can’t justify. Secondly, Chef provides a very solid/safe change/configuration management architecture, which I feel should be leveraged as much as possible.

An alternative might be to make a recipe smart enough to do certain things only when certain settings/attributes/flags are set. I think this is possible in some cases but not others. I think it’s rather inefficient/risky to have to re-release a cookbook (plus dependencies) just to handle a one-off case for example.

I tend to build some cookbooks to have recipes that represent basic execution blocks which I then can assemble into different sequences. One of these sequences will be the default one. The others will correspond to occasional use-cases. And finally the individual blocks can be used in an ad-hoc way if absolutely necessary.

I would love to hear about some experiences people have made with managing non-trivial medium sized applications with Chef.

Florian

From: Mingfei Hua [mailto:mfhua@aerohive.com]
Sent: 30 June 2014 08:59
To: chef@lists.opscode.commailto:chef@lists.opscode.com
Subject: [chef] RE: Re: Chef Role and integration

Hi Danjib,
If I use “-o” in chef-client, Not only the run list, but also the attribute “roles” in chef server be overwritten. The integration which depend on attribute “roles” may broken.

Regards,
Mingfei

From: Ranjib Dey [mailto:dey.ranjib@gmail.com]
Sent: 2014年6月29日 15:38
To: chef@lists.opscode.commailto:chef@lists.opscode.com
Subject: [chef] Re: Chef Role and integration

if you invoke chef-client with -o , the override option, it should not save the node data back (iirc). you have to check no recipes or other code does not use node.save explicitly ,

im curious why you dont want to run the complete run list (all the roles+ extra recipes), is performance (time) a concern ?
regards
ranjib

On Sat, Jun 28, 2014 at 11:20 PM, Mingfei Hua <mfhua@aerohive.commailto:mfhua@aerohive.com> wrote:

Hello,

we have several roles defined for different type of servers, like DB, web, rabbitmq, redis. and the integration is depend on roles, such as how the WEB server find the DB servers. in each role, there are cookbooks or sub roles in 3 layers, there are cookbooks for system, cookbooks for install tomcat/database, cookbooks to install war packages. But in some case, I just need to run the cookbooks for system or just need to deploy the war package, the easiest way is to change the run-list, but it changes the roles attribute in chef server which integration depend on. Any tips on my issue.

A typical role definition:
{
“name”: “hmweb”,
“chef_type”: “role”,
“json_class”: “Chef::Role”,
“description”: “for web application deployment, include hm-webapp”,
“override_attributes”: {
“aerohive_webapp”: {
“wars_array”: {
“hm-webapp.war”: {
“name”: “hm-webapp”,
“url”: “http://nexus-nms.aerohive.com/service/local/artifact/maven/redirect?r=hive-repository&g=com.aerohive.nms.web&a=hm-webapp&e=war&v=1.0-SNAPSHOT
},
“sysmgr-regional-webapp.war”: {
“name”: “sysmgr-regional-webapp”,
“url”: “http://nexus-nms.aerohive.com/service/local/artifact/maven/redirect?r=hive-repository&g=com.aerohive.nms&a=sysmgr-regional-webapp&e=war&v=1.0-SNAPSHOT
}
},
“service_log_enabled”: “true”
}
},
“run_list”: [
“role[_base]”,
“role[_webappbase]”,
“recipe[aerohive-webapp::deploy]”
]
}

Sincerely,
Mingfei Hua

LEGAL DISCLAIMER
This communication and any attached documents are strictly confidential and/or legally privileged and they may not be used or disclosed by someone who is not a named recipient. If you have received this electronic communication in error please notify the sender by replying to this electronic communication inserting the word “misdirected” as the subject and delete this communication from your system.


#6

By attribute “roles” do you mean the attributes(override) in your role
you mentioned in your previous email.?
([aerohive_webapp][wars_array][hm-webapp.war][…])

If yes, I had a similar case and I used the -j option. (painful).
you can create a .json file and pass it to chef-client (chef-client -j
hotfix.json). In the json file you can mention only the roles/recipes
you want to update, as well as the attributes. It’s as good as editing
your role file, but helps for a quick fix.

let me know

Santhosh
On 6/30/2014 12:29 PM, Mingfei Hua wrote:

Hi Danjib,

If I use “-o” in chef-client, Not only the run list, but also the
attribute “roles” in chef server be overwritten. The integration
which depend on attribute “roles” may broken.

Regards,

Mingfei

*From:*Ranjib Dey [mailto:dey.ranjib@gmail.com]
Sent: 2014年6月29日15:38
To: chef@lists.opscode.com
Subject: [chef] Re: Chef Role and integration

if you invoke chef-client with -o , the override option, it should not
save the node data back (iirc). you have to check no recipes or other
code does not use node.save explicitly ,

im curious why you dont want to run the complete run list (all the
roles+ extra recipes), is performance (time) a concern ?

regards

ranjib

On Sat, Jun 28, 2014 at 11:20 PM, Mingfei Hua <mfhua@aerohive.com
mailto:mfhua@aerohive.com> wrote:

Hello,

we have several roles defined for different type of servers, like DB,
web, rabbitmq, redis. and the integration is depend on roles, such as
how the WEB server find the DB servers. in each role, there are
cookbooks or sub roles in 3 layers, there are cookbooks for system,
cookbooks for install tomcat/database, cookbooks to install war
packages. But in some case, I just need to run the cookbooks for
system or just need to deploy the war package, the easiest way is to
change the run-list, but it changes the roles attribute in chef server
which integration depend on. Any tips on my issue.

A typical role definition:

{

“name”: “hmweb”,

“chef_type”: “role”,

“json_class”: “Chef::Role”,

“description”: “for web application deployment, include hm-webapp”,

“override_attributes”: {

“aerohive_webapp”: {

“wars_array”: {

“hm-webapp.war”: {

“name”: “hm-webapp”,

“url”:
http://nexus-nms.aerohive.com/service/local/artifact/maven/redirect?r=hive-repository&g=com.aerohive.nms.web&a=hm-webapp&e=war&v=1.0-SNAPSHOT

},

“sysmgr-regional-webapp.war”: {

“name”: “sysmgr-regional-webapp”,

“url”:
http://nexus-nms.aerohive.com/service/local/artifact/maven/redirect?r=hive-repository&g=com.aerohive.nms&a=sysmgr-regional-webapp&e=war&v=1.0-SNAPSHOT

}

},

“service_log_enabled”: “true”

}

     },

“run_list”: [

“role[_base]”,

“role[_webappbase]”,

“recipe[aerohive-webapp::deploy]”

     ]

}

Sincerely,

Mingfei Hua


#7

Hi Santhosh,
There is attribute named “roles” for each node in chef server, it means which chef Roles it belongs to. This attribute changes automatically according to your run-list.

[cid:image001.png@01CF9529.80CCD8E0]

Regards,
Mingfei

From: Santhosh [mailto:santhoshsusbs@gmail.com]
Sent: 2014年7月1日 11:11
To: chef@lists.opscode.com
Subject: [chef] Re: RE: Re: Chef Role and integration

By attribute “roles” do you mean the attributes(override) in your role you mentioned in your previous email.? ([aerohive_webapp][wars_array][hm-webapp.war][…])

If yes, I had a similar case and I used the -j option. (painful).
you can create a .json file and pass it to chef-client (chef-client -j hotfix.json). In the json file you can mention only the roles/recipes you want to update, as well as the attributes. It’s as good as editing your role file, but helps for a quick fix.

let me know

Santhosh
On 6/30/2014 12:29 PM, Mingfei Hua wrote:
Hi Danjib,
If I use “-o” in chef-client, Not only the run list, but also the attribute “roles” in chef server be overwritten. The integration which depend on attribute “roles” may broken.

Regards,
Mingfei

From: Ranjib Dey [mailto:dey.ranjib@gmail.com]
Sent: 2014年6月29日 15:38
To: chef@lists.opscode.commailto:chef@lists.opscode.com
Subject: [chef] Re: Chef Role and integration

if you invoke chef-client with -o , the override option, it should not save the node data back (iirc). you have to check no recipes or other code does not use node.save explicitly ,

im curious why you dont want to run the complete run list (all the roles+ extra recipes), is performance (time) a concern ?
regards
ranjib

On Sat, Jun 28, 2014 at 11:20 PM, Mingfei Hua <mfhua@aerohive.commailto:mfhua@aerohive.com> wrote:

Hello,

we have several roles defined for different type of servers, like DB, web, rabbitmq, redis. and the integration is depend on roles, such as how the WEB server find the DB servers. in each role, there are cookbooks or sub roles in 3 layers, there are cookbooks for system, cookbooks for install tomcat/database, cookbooks to install war packages. But in some case, I just need to run the cookbooks for system or just need to deploy the war package, the easiest way is to change the run-list, but it changes the roles attribute in chef server which integration depend on. Any tips on my issue.

A typical role definition:
{
“name”: “hmweb”,
“chef_type”: “role”,
“json_class”: “Chef::Role”,
“description”: “for web application deployment, include hm-webapp”,
“override_attributes”: {
“aerohive_webapp”: {
“wars_array”: {
“hm-webapp.war”: {
“name”: “hm-webapp”,
“url”: “http://nexus-nms.aerohive.com/service/local/artifact/maven/redirect?r=hive-repository&g=com.aerohive.nms.web&a=hm-webapp&e=war&v=1.0-SNAPSHOT
},
“sysmgr-regional-webapp.war”: {
“name”: “sysmgr-regional-webapp”,
“url”: “http://nexus-nms.aerohive.com/service/local/artifact/maven/redirect?r=hive-repository&g=com.aerohive.nms&a=sysmgr-regional-webapp&e=war&v=1.0-SNAPSHOT
}
},
“service_log_enabled”: “true”
}
},
“run_list”: [
“role[_base]”,
“role[_webappbase]”,
“recipe[aerohive-webapp::deploy]”
]
}

Sincerely,
Mingfei Hua