Chef-client run on large number of nodes


#1

I have my own chef server installed and I am going to add some 1000 of nodes to
my server.
I wonder if chef-client run on all the nodes after a specified period of time
repeatedly it may result in lower performance of chef server.
Can I automate the chef-client run only after I update & save the run-list on
server. I mean only when I will update the run-list my chef server should
notify the nodes to run chef-client at its end.


#2

On Tuesday, October 1, 2013 at 4:57 AM, Developers upma gupta wrote:

I have my own chef server installed and I am going to add some 1000 of nodes to
my server.
I wonder if chef-client run on all the nodes after a specified period of time
repeatedly it may result in lower performance of chef server.
Can I automate the chef-client run only after I update & save the run-list on
server. I mean only when I will update the run-list my chef server should
notify the nodes to run chef-client at its end.

A few things:

  • Performance is generally going to depend on the request rate, not the aggregate total. Having all your nodes check in every 15 or 30 minutes with a random splay looks a lot different than 100s of nodes running chef all at once (because you updated some data and triggered a chef run on all of them).
  • There’s nothing built-in to chef to automatically trigger a chef run from the server (at this time), nor does the server know what nodes will be affected by a given data change, so you have to use SSH or some other tool to trigger chef runs.
  • Chef 11 server should be able to handle that number of nodes just fine. Keep an eye on it, and if you have any specific problems, post them to this list to ask for tuning advice.


Daniel DeLeo


#3

http://lists.opscode.com/sympa/info/chef
https://wiki.opscode.com/display/chef/Mailing+Lists

On Oct 1, 2013, at 12:28 PM, Daniel DeLeo dan@kallistec.com wrote:

On Tuesday, October 1, 2013 at 4:57 AM, Developers upma gupta wrote:

I have my own chef server installed and I am going to add some 1000 of nodes to
my server.
I wonder if chef-client run on all the nodes after a specified period of time
repeatedly it may result in lower performance of chef server.
Can I automate the chef-client run only after I update & save the run-list on
server. I mean only when I will update the run-list my chef server should
notify the nodes to run chef-client at its end.
A few things:

  • Performance is generally going to depend on the request rate, not the aggregate total. Having all your nodes check in every 15 or 30 minutes with a random splay looks a lot different than 100s of nodes running chef all at once (because you updated some data and triggered a chef run on all of them).
  • There’s nothing built-in to chef to automatically trigger a chef run from the server (at this time), nor does the server know what nodes will be affected by a given data change, so you have to use SSH or some other tool to trigger chef runs.
  • Chef 11 server should be able to handle that number of nodes just fine. Keep an eye on it, and if you have any specific problems, post them to this list to ask for tuning advice.


Daniel DeLeo