CHEF 11 : delay before databag update in search


#1

Hello all,

It seems that there’s a kind of slow update when modifying data in a databag and searching for this particular value through “knife search” or “Chef::Search::Query”

For example, if I look for a particular URL in the databag test

Ø knife search test url:www.google.com

Then change this URL via chef-webui to : www.yahoo.com.

The following search with knife or Chef::Search::Query will return “0 items found” for a few minutes.

Ø knife search test url:www.yahoo.com

When this little delay has passed, the result will be fine.

Any idea on why this occurs ? Thanks in advance.

Regards,

Philippe Bérard
Administrateur Systèmes

Tel : +33 (0)1 39 23 31 17
Mob : +33 (0)6 01 27 87 86
Fax : +33 (0)1 39 55 47 56

58, Rue Pottier
78150 Le Chesnay
http://www.jalios.com www.jalios.com

JALIOS

P Afin de contribuer au respect de l’environnement, merci de n’imprimer ce message qu’en cas de nécessité.


#2

Hi Philippe,

On Feb 19, 2013, at 8:18 AM, Philippe Bérard wrote:

It seems that there’s a kind of slow update when modifying data in a databag and searching for this particular value through “knife search” or “Chef::Search::Query”

Chef uses Solr to index objects for search. The Lucene index makes newly updated data available for search queries after a commit. This setup is not new to Chef 11, however, the default tuning of the solr/lucene commit interval is now set at 60 seconds. On average, you will need to wait 30 seconds before an object update is visible via search. You can tune this in /etc/chef-server/chef-server.rb like this:

chef_solr['commit_interval'] = 60000 # in ms

For very small deployments, you can probably turn this down quite a bit.

  • seth

#3

If you guys brought Solr up to version 4, the new soft commit feature
lets you get document visibility latency down to a second.

Michael Della Bitta


Appinions
18 East 41st Street, 2nd Floor
New York, NY 10017-6271

www.appinions.com

Where Influence Isn’t a Game

On Wed, Feb 20, 2013 at 2:12 PM, Seth Falcon seth@opscode.com wrote:

Hi Philippe,

On Feb 19, 2013, at 8:18 AM, Philippe Bérard wrote:

It seems that there’s a kind of slow update when modifying data in a databag and searching for this particular value through “knife search” or “Chef::Search::Query”

Chef uses Solr to index objects for search. The Lucene index makes newly updated data available for search queries after a commit. This setup is not new to Chef 11, however, the default tuning of the solr/lucene commit interval is now set at 60 seconds. On average, you will need to wait 30 seconds before an object update is visible via search. You can tune this in /etc/chef-server/chef-server.rb like this:

chef_solr['commit_interval'] = 60000 # in ms

For very small deployments, you can probably turn this down quite a bit.

  • seth

#4

Thank you all for these important informations

Regards,

– Philippe Bérard

-----Message d’origine-----
De : Michael Della Bitta [mailto:michael.della.bitta@appinions.com]
Envoyé : mercredi 20 février 2013 20:47
À : chef@lists.opscode.com
Objet : [chef] Re: Re: CHEF 11 : delay before databag update in search

If you guys brought Solr up to version 4, the new soft commit feature lets
you get document visibility latency down to a second.

Michael Della Bitta


Appinions
18 East 41st Street, 2nd Floor
New York, NY 10017-6271

www.appinions.com

Where Influence Isn’t a Game

On Wed, Feb 20, 2013 at 2:12 PM, Seth Falcon seth@opscode.com wrote:

Hi Philippe,

On Feb 19, 2013, at 8:18 AM, Philippe Bérard wrote:

It seems that there’s a kind of slow update when modifying data in a
databag and searching for this particular value through “knife search” or
“Chef::Search::Query”

Chef uses Solr to index objects for search. The Lucene index makes newly
updated data available for search queries after a commit. This setup is not
new to Chef 11, however, the default tuning of the solr/lucene commit
interval is now set at 60 seconds. On average, you will need to wait 30
seconds before an object update is visible via search. You can tune this in
/etc/chef-server/chef-server.rb like this:

chef_solr['commit_interval'] = 60000 # in ms

For very small deployments, you can probably turn this down quite a bit.

  • seth