Search sometimes returns no result

Hi,

I have an issue where search in recipes sometimes returns no result.

There is no exception in solr.log

Here is the relevant part of the logs:

[gilles@chefserver] ~ $ grep ‘role:monitoring’ /var/log/chef/solr.log
| sort | uniq -c
5 INFO: [] webapp=/solr path=/select
params={indent=off&start=0&q=role:monitoring+AND+app_environment:production&wt=ruby&fq=%2BX_CHEF_database_CHEF_X:chef+%2BX_CHEF_type_CHEF_X:node&rows=20}
hits=1 status=0 QTime=1
1 INFO: [] webapp=/solr path=/select
params={indent=off&start=0&q=role:monitoring+AND+app_environment:production&wt=ruby&fq=%2BX_CHEF_database_CHEF_X:chef+%2BX_CHEF_type_CHEF_X:node&rows=20}
hits=1 status=0 QTime=2
1 INFO: [] webapp=/solr path=/select
params={indent=off&start=0&q=role:monitoring+AND+app_environment:production&wt=ruby&fq=%2BX_CHEF_database_CHEF_X:chef+%2BX_CHEF_type_CHEF_X:node&rows=20}
hits=1 status=0 QTime=45
4 INFO: [] webapp=/solr path=/select
params={sort=X_CHEF_id_CHEF_X+asc&indent=off&start=0&q=role:monitoring+AND+app_environment:production&wt=ruby&fq=%2BX_CHEF_database_CHEF_X:chef+%2BX_CHEF_type_CHEF_X:node&rows=1000}
hits=0 status=0 QTime=0
21 INFO: [] webapp=/solr path=/select
params={sort=X_CHEF_id_CHEF_X+asc&indent=off&start=0&q=role:monitoring+AND+app_environment:production&wt=ruby&fq=%2BX_CHEF_database_CHEF_X:chef+%2BX_CHEF_type_CHEF_X:node&rows=1000}
hits=0 status=0 QTime=1
1 INFO: [] webapp=/solr path=/select
params={sort=X_CHEF_id_CHEF_X+asc&indent=off&start=0&q=role:monitoring+AND+app_environment:production&wt=ruby&fq=%2BX_CHEF_database_CHEF_X:chef+%2BX_CHEF_type_CHEF_X:node&rows=1000}
hits=0 status=0 QTime=2
142 INFO: [] webapp=/solr path=/select
params={sort=X_CHEF_id_CHEF_X+asc&indent=off&start=0&q=role:monitoring+AND+app_environment:production&wt=ruby&fq=%2BX_CHEF_database_CHEF_X:chef+%2BX_CHEF_type_CHEF_X:node&rows=1000}
hits=1 status=0 QTime=0
531 INFO: [] webapp=/solr path=/select
params={sort=X_CHEF_id_CHEF_X+asc&indent=off&start=0&q=role:monitoring+AND+app_environment:production&wt=ruby&fq=%2BX_CHEF_database_CHEF_X:chef+%2BX_CHEF_type_CHEF_X:node&rows=1000}
hits=1 status=0 QTime=1
109 INFO: [] webapp=/solr path=/select
params={sort=X_CHEF_id_CHEF_X+asc&indent=off&start=0&q=role:monitoring+AND+app_environment:production&wt=ruby&fq=%2BX_CHEF_database_CHEF_X:chef+%2BX_CHEF_type_CHEF_X:node&rows=1000}
hits=1 status=0 QTime=2

Sorry for the long lines.

The first thing to notice is that not all the queries are the same,
most of them have the extra ‘sort=X_CHEF_id_CHEF_X+asc’ param, even
thought the same recipes are running on all nodes and I have
chef-client 0.9.6 installed on all machines

And as you can see sometimes the query has hits=1, sometimes (rarely), hits=0

Nobody touched the chef server conf, no cookbook upload, no role
upload… nothing from a user.

Nothing shows up in solr-indexer either.

I’m running out of ideas, if anyone has one, I take.

Thanks a lot