What is the best way to install many sets of same environment with different properties in Chef?

Hi there,

We have a chef server and hundreds of servers. The same set of applications will be installed in these servers, i.e. 1 JBoss server, 3 RabbitMQ nodes for message queue cluster, 2 db nodes for db cluster.

And many sets of such environment will be installed, we are seeking the best way for this with chef.

The obstacle is that while installing, node A may need to know the IP and service username/password of node B, we need to provide the post-configuration property through pre-configure or provide property file such as JSON file as the chef command’s parameter. We consider below approaches:

Approach One:

  1.  Create roles such as JBoss_server, Rabbitmq_node, Rabbitmq_cluster, Mysql_node etc. Download and upload related cookbooks in our chef server. Define the run_list and needed property of these roles. E.g. while install mq cluster, we need to install mq_node1, then mq_node2, then provide node1’s ip for node2, register node2 into node1 as the cluster after installation.
    
  2.  Pickup enough servers, bootstrap chef-client for them from chef server with their SSH username/password. Created chef node such as set1-jboss, set1-mq1, set1-mqcluster, set1-mysqlnode1, set1-mysqlnode2, set2-jboss… etc.
    
  3.  Edit run_list of these chef nodes in chef server with according role.
    
  4.  SSH into these servers and run chef-client.
    
  5.  Repeat step 1 – step 4, create roles such as JBoss_server2, Rabbitmq_node2 and their run_list and needed properties etc. if another request comes to install another set of environment.
    

The installation request may come parallel – install two or more sets at the same time.

Approach Two:

  1.  Create cookbooks in chef server and bootstrap chef client for needed servers.
    
  2.  SSH into all these servers, create property file such as set1_jboss.json, set1_mq_node1.json, set1_mq_cluster.json etc. in each server. Run chef-client, specify the cookbook name and –j xxx.json to start installation.
    
  3.  Repeat step 2, create property file such as set2_jboss.json, set2_mq_node1.json… etc. Run chef-client with –j parameter and the json file in that server to start installation.
    

Also we may meet another issue: how to deal with the situation for the cluster configuration: install service in node1, install service in node2, then go back to node1 and configure with node2’s information? Go to node1 and run cookbook 1, then go to node2 and run cookbook 2, at last go back to node1 and run cookbook 3 to complete the cluster configuration? This seems difficult for us.

We are newbie in chef and not sure which way is the best way, or neither of them is.

Appreciate for any suggestion or guidance for these questions, we know that we throw too many questions in the post!

Thanks.

On May 1, 2015, at 9:04 AM, jeffty wantwatering@gmail.com wrote:

We are newbie in chef and not sure which way is the best way, or neither of them is.

The installation of the software should be kept separate from the operation of what is installed.

At installation time, it shouldn’t matter what the other servers are. You’re just installing software.

At operation time, each server should use the built-in chef discovery methods to find out what its servers are for the appropriate type — who is my JBoss server, who is/are my database server(s), etc….

--
Brad Knowles brad@shub-internet.org
LinkedIn Profile: http://tinyurl.com/y8kpxu

Got it and thanks Brad.

We'll separate the software installation and operation as you suggested.

And can you provide some samples about the built-in chef discovery method?
We are interested in that.

Thanks a lot.

Jeff

2015-05-01 22:42 GMT+08:00 Brad Knowles brad@shub-internet.org:

On May 1, 2015, at 9:04 AM, jeffty wantwatering@gmail.com wrote:

We are newbie in chef and not sure which way is the best way, or neither
of them is.

The installation of the software should be kept separate from the
operation of what is installed.

At installation time, it shouldn’t matter what the other servers are.
You’re just installing software.

At operation time, each server should use the built-in chef discovery
methods to find out what its servers are for the appropriate type — who is
my JBoss server, who is/are my database server(s), etc….

--
Brad Knowles brad@shub-internet.org
LinkedIn Profile: http://tinyurl.com/y8kpxu

On May 1, 2015, at 10:18 AM, jeffty wantwatering@gmail.com wrote:

And can you provide some samples about the built-in chef discovery method? We are interested in that.

You’re looking for “search”. I would start at https://docs.chef.io/chef_search.html.

The exact way you would use this can differ, depending on your cookbooks and how you want to use them. You could have your clients search for all nodes with a given role (e.g., role == database server), and as the servers of the different types are brought up they will register themselves using chef. Or, you could have the clients search a databag, and as the servers come up and become operational, they put their information in the appropriate databag.

There’s a number of different ways you can solve this problem with search, depending on your specific requirements.

--
Brad Knowles brad@shub-internet.org
LinkedIn Profile: http://tinyurl.com/y8kpxu

a) capture the order of nodes you need to provision (i.e. mq -> mysql ->
jboss), we'll need this later
b) for all of this rypes, declare roles. separate the installation and
cofiguration in distinct recipes/sub-recipe if need be. i.r
recipe[jboss::install], recipe[jboss::configure], Installation should not
need discovery, and can be parellelized
c) provision all the nodes with the recipes, those who does not need
discovery can be provisioned with roles directly.
d) ::configure sub recipes will require chef search for sub recipes (i'll
recommend using partial search, and use roles as search predicate instead
of attributes). apply them in your intended order by directly assigning the
roles. you can use knife-ssh, ansible of blender for that bit.

This will allow you to write individual recipes clutter free, independently
testable. roles being an aggregation of both install, and configure recipe,
will always ensure both recipes are applied on every node , every time.
i.e. the initial bootstrap recipe is a subset of node's final run-list.

we wrote blender[1] & blender-chef[2] to deal with such scenarios. It will
allow you to capture the whole workflow as a knife plugin :slight_smile: . like knife jboss environment create

[1]GitHub - PagerDuty/blender: A modular orchestration engine
[2]GitHub - PagerDuty/blender-chef: Chef search based host discovery for blender.

On Fri, May 1, 2015 at 7:04 AM, jeffty wantwatering@gmail.com wrote:

Hi there,

We have a chef server and hundreds of servers. The same set of
applications will be installed in these servers, i.e. 1 JBoss server, 3
RabbitMQ nodes for message queue cluster, 2 db nodes for db cluster.

And many sets of such environment will be installed, we are seeking the
best way for this with chef.

The obstacle is that while installing, node A may need to know the IP and
service username/password of node B, we need to provide the
post-configuration property through pre-configure or provide property file
such as JSON file as the chef command’s parameter. We consider below
approaches:

Approach One:

  1.  Create roles such as JBoss_server, Rabbitmq_node,
    

Rabbitmq_cluster, Mysql_node etc. Download and upload related cookbooks in
our chef server. Define the run_list and needed property of these roles.
E.g. while install mq cluster, we need to install mq_node1, then mq_node2,
then provide node1’s ip for node2, register node2 into node1 as the cluster
after installation.

  1.  Pickup enough servers, bootstrap chef-client for them from chef
    

server with their SSH username/password. Created chef node such as
set1-jboss, set1-mq1, set1-mqcluster, set1-mysqlnode1, set1-mysqlnode2,
set2-jboss… etc.

  1.  Edit run_list of these chef nodes in chef server with according
    

role.

  1.  SSH into these servers and run chef-client.
    
  2.  Repeat step 1 – step 4, create roles such as JBoss_server2,
    

Rabbitmq_node2 and their run_list and needed properties etc. if another
request comes to install another set of environment.

The installation request may come parallel -- install two or more sets at
the same time.

Approach Two:

  1.  Create cookbooks in chef server and bootstrap chef client for
    

needed servers.

  1.  SSH into all these servers, create property file such as
    

set1_jboss.json, set1_mq_node1.json, set1_mq_cluster.json etc. in each
server. Run chef-client, specify the cookbook name and –j xxx.json to start
installation.

  1.  Repeat step 2, create property file such as set2_jboss.json,
    

set2_mq_node1.json… etc. Run chef-client with –j parameter and the json
file in that server to start installation.

Also we may meet another issue: how to deal with the situation for the
cluster configuration: install service in node1, install service in node2,
then go back to node1 and configure with node2’s information? Go to node1
and run cookbook 1, then go to node2 and run cookbook 2, at last go back to
node1 and run cookbook 3 to complete the cluster configuration? This seems
difficult for us.

We are newbie in chef and not sure which way is the best way, or neither
of them is.

Appreciate for any suggestion or guidance for these questions, we know
that we throw too many questions in the post!

Thanks.

Thanks Dey!

Will look through the blenderJ

2015-05-01 23:53 GMT+08:00 Ranjib Dey dey.ranjib@gmail.com:

a) capture the order of nodes you need to provision (i.e. mq -> mysql ->
jboss), we'll need this later
b) for all of this rypes, declare roles. separate the installation and
cofiguration in distinct recipes/sub-recipe if need be. i.r
recipe[jboss::install], recipe[jboss::configure], Installation should not
need discovery, and can be parellelized
c) provision all the nodes with the recipes, those who does not need
discovery can be provisioned with roles directly.
d) ::configure sub recipes will require chef search for sub recipes (i'll
recommend using partial search, and use roles as search predicate instead
of attributes). apply them in your intended order by directly assigning the
roles. you can use knife-ssh, ansible of blender for that bit.

This will allow you to write individual recipes clutter free,
independently testable. roles being an aggregation of both install, and
configure recipe, will always ensure both recipes are applied on every node
, every time. i.e. the initial bootstrap recipe is a subset of node's final
run-list.

we wrote blender[1] & blender-chef[2] to deal with such scenarios. It will
allow you to capture the whole workflow as a knife plugin :slight_smile: . like knife jboss environment create

[1]GitHub - PagerDuty/blender: A modular orchestration engine
[2]GitHub - PagerDuty/blender-chef: Chef search based host discovery for blender.

On Fri, May 1, 2015 at 7:04 AM, jeffty wantwatering@gmail.com wrote:

Hi there,

We have a chef server and hundreds of servers. The same set of
applications will be installed in these servers, i.e. 1 JBoss server, 3
RabbitMQ nodes for message queue cluster, 2 db nodes for db cluster.

And many sets of such environment will be installed, we are seeking the
best way for this with chef.

The obstacle is that while installing, node A may need to know the IP and
service username/password of node B, we need to provide the
post-configuration property through pre-configure or provide property file
such as JSON file as the chef command’s parameter. We consider below
approaches:

Approach One:

  1.  Create roles such as JBoss_server, Rabbitmq_node,
    

Rabbitmq_cluster, Mysql_node etc. Download and upload related cookbooks in
our chef server. Define the run_list and needed property of these roles.
E.g. while install mq cluster, we need to install mq_node1, then mq_node2,
then provide node1’s ip for node2, register node2 into node1 as the cluster
after installation.

  1.  Pickup enough servers, bootstrap chef-client for them from chef
    

server with their SSH username/password. Created chef node such as
set1-jboss, set1-mq1, set1-mqcluster, set1-mysqlnode1, set1-mysqlnode2,
set2-jboss… etc.

  1.  Edit run_list of these chef nodes in chef server with according
    

role.

  1.  SSH into these servers and run chef-client.
    
  2.  Repeat step 1 – step 4, create roles such as JBoss_server2,
    

Rabbitmq_node2 and their run_list and needed properties etc. if another
request comes to install another set of environment.

The installation request may come parallel -- install two or more sets at
the same time.

Approach Two:

  1.  Create cookbooks in chef server and bootstrap chef client for
    

needed servers.

  1.  SSH into all these servers, create property file such as
    

set1_jboss.json, set1_mq_node1.json, set1_mq_cluster.json etc. in each
server. Run chef-client, specify the cookbook name and –j xxx.json to start
installation.

  1.  Repeat step 2, create property file such as set2_jboss.json,
    

set2_mq_node1.json… etc. Run chef-client with –j parameter and the json
file in that server to start installation.

Also we may meet another issue: how to deal with the situation for the
cluster configuration: install service in node1, install service in node2,
then go back to node1 and configure with node2’s information? Go to node1
and run cookbook 1, then go to node2 and run cookbook 2, at last go back to
node1 and run cookbook 3 to complete the cluster configuration? This seems
difficult for us.

We are newbie in chef and not sure which way is the best way, or neither
of them is.

Appreciate for any suggestion or guidance for these questions, we know
that we throw too many questions in the post!

Thanks.