All -
Our Chef Server web interface is behaving oddly. When refreshing a list of nodes or roles or whatever it sometimes returns an error page. Usually a refresh of the page returns the proper results but not always.
The error just says “Error An error occurred, please try again”.
Is there some configuration setting or parameter we can/should tweak?
The server hasn’t been restarted in about 200 days - I have no idea if this is an issue or not.
Any ideas?
Thanks
Mark
On Wednesday, November 19, 2014 at 5:47 AM, Mark Nichols wrote:
All -
Our Chef Server web interface is behaving oddly. When refreshing a list of nodes or roles or whatever it sometimes returns an error page. Usually a refresh of the page returns the proper results but not always.
The error just says “Error An error occurred, please try again”.
Is there some configuration setting or parameter we can/should tweak?
The server hasn’t been restarted in about 200 days - I have no idea if this is an issue or not.
Any ideas?
Thanks
Mark
Check the server logs with chef-server-ctl tail. Nginx should be showing a 5XX response and the corresponding request to erchef should show errors in the logs.
--
Daniel DeLeo
Daniel,
When we run private-chef-ctl tail it dumps a bunch of stuff and keeps on running. By filtering for nginx via grep we see this:
tail: cannot open /var/log/opscode/couchdb/compact-20140916011701.log' for reading: Too many open files tail: cannot open /var/log/opscode/couchdb/compact-20140725111701.log' for reading: Too many open files
tail: cannot open /var/log/opscode/couchdb/compact-20141114071701.log' for reading: Too many open files tail: cannot open /var/log/opscode/couchdb/compact-20140903151701.log' for reading: Too many open files
tail: cannot open /var/log/opscode/couchdb/compact-20140902011701.log' for reading: Too many open files tail: cannot open /var/log/opscode/couchdb/compact-20140729051701.log' for reading: Too many open files
Looks like the file limit is set to 1000000 which seems like a lot.
We’re looking at active PIDS now to see what we can figure out. Any ideas?
Thanks,
Mark
On Nov 19, 2014, at 10:52 AM, Daniel DeLeo dan@kallistec.com wrote:
On Wednesday, November 19, 2014 at 5:47 AM, Mark Nichols wrote:
All -
Our Chef Server web interface is behaving oddly. When refreshing a list of nodes or roles or whatever it sometimes returns an error page. Usually a refresh of the page returns the proper results but not always.
The error just says “Error An error occurred, please try again”.
Is there some configuration setting or parameter we can/should tweak?
The server hasn’t been restarted in about 200 days - I have no idea if this is an issue or not.
Any ideas?
Thanks
MarkCheck the server logs with
chef-server-ctl tail. Nginx should be showing a 5XX response and the corresponding request to erchef should show errors in the logs.--
Daniel DeLeo
On Thursday, November 20, 2014 at 7:09 AM, Mark Nichols wrote:
Daniel,
When we run
private-chef-ctl tailit dumps a bunch of stuff and keeps on running. By filtering for nginx via grep we see this:tail: cannot open
/var/log/opscode/couchdb/compact-20140916011701.log' for reading: Too many open files tail: cannot open/var/log/opscode/couchdb/compact-20140725111701.log' for reading: Too many open files
tail: cannot open/var/log/opscode/couchdb/compact-20141114071701.log' for reading: Too many open files tail: cannot open/var/log/opscode/couchdb/compact-20140903151701.log' for reading: Too many open files
tail: cannot open/var/log/opscode/couchdb/compact-20140902011701.log' for reading: Too many open files tail: cannot open/var/log/opscode/couchdb/compact-20140729051701.log' for reading: Too many open filesLooks like the file limit is set to 1000000 which seems like a lot.
We’re looking at active PIDS now to see what we can figure out. Any ideas?
Thanks,
Mark
Is this Enterprise Chef 11.x?
In any case, you should be able to bounce CouchDB (chef-server-ctl has a command for this) to resolve the issue. Depending on your load profile, you might need to do this periodically.
--
Daniel DeLeo