The indexer isn't indexing

Chef Infra (archive)
1

Hi,

I’m having some trouble with an installation of Chef 0.7.14 from
Opscode’s repository on Debian 5.0.3.

Chef itself has started correctly, and clients are able to connect to
run recipes from it, however the search index doesn’t seem to be
getting populated.

When browsing the web interface, and selecting the Search tab, I see
an empty page which the documentation suggests is caused by the index
not having been created yet. Also when I use Ferret directly it isn’t
returning any results:

infrastructure:/srv/chef/search_index# irb
irb(main):001:0> require ‘ferret’
=> true
irb(main):002:0> index = Ferret::Index::Index.new(:path => “.”)
=> #<Ferret::Index::Index:0x7fddf1df0008 @mon_entering_queue=[],
@key=nil, @mon_count=0, @writer=nil, @default_input_field=:id,
@dir=#Ferret::Store::FSDirectory:0x7fddf1deffb8, @mon_owner=nil,
@qp=nil, @searcher=nil, @default_field=:, @close_dir=true,
@auto_flush=false, @mon_waiting_queue=[], @open=true,
@options={:lock_retry_time=>2, :default_field=>:,
:dir=>#Ferret::Store::FSDirectory:0x7fddf1deffb8, :path=>".",
:analyzer=>#Ferret::Analysis::StandardAnalyzer:0x7fddf1defe78},
@id_field=:id, @reader=nil>
irb(main):003:0> index.size
=> 0
irb(main):004:0> index.search("*")
=> #<struct Ferret::Search::TopDocs total_hits=0, hits=[],
max_score=0.0, searcher=#Ferret::Search::Searcher:0x7fddf3ee2ea0>

The indexer service appears to be running, and the log files claim to
be indexing things, as shown by the sample below:

[Wed, 27 Jan 2010 16:43:23 +0000] INFO: Headers
{“destination”=>"/queue/chef/index", “content-type”=>“text/plain;
charset=UTF-8”, “content-length”=>“7452”}
[Wed, 27 Jan 2010 16:43:23 +0000] INFO: Indexed object from
/queue/chef/index in 0.107933 seconds

I’m not really sure where else to look to debug this any further now.
If anyone can point me in the right direction, I’d be very
appreciative.

Cheers,

Jon Wood

2

Further to this, it appears that it’s specifically the indexer failing

  • messages are getting to the stompserver when I update a Node, but
    the indexer doesn’t seem to be picking them up.

2010/1/27 Jon Wood jon@blankpad.net:

Hi,

I’m having some trouble with an installation of Chef 0.7.14 from
Opscode’s repository on Debian 5.0.3.

Chef itself has started correctly, and clients are able to connect to
run recipes from it, however the search index doesn’t seem to be
getting populated.

When browsing the web interface, and selecting the Search tab, I see
an empty page which the documentation suggests is caused by the index
not having been created yet. Also when I use Ferret directly it isn’t
returning any results:

infrastructure:/srv/chef/search_index# irb
irb(main):001:0> require ‘ferret’
=> true
irb(main):002:0> index = Ferret::Index::Index.new(:path => “.”)
=> #<Ferret::Index::Index:0x7fddf1df0008 @mon_entering_queue=[],
@key=nil, @mon_count=0, @writer=nil, @default_input_field=:id,
@dir=#Ferret::Store::FSDirectory:0x7fddf1deffb8, @mon_owner=nil,
@qp=nil, @searcher=nil, @default_field=:, @close_dir=true,
@auto_flush=false, @mon_waiting_queue=[], @open=true,
@options={:lock_retry_time=>2, :default_field=>:,
:dir=>#Ferret::Store::FSDirectory:0x7fddf1deffb8, :path=>".",
:analyzer=>#Ferret::Analysis::StandardAnalyzer:0x7fddf1defe78},
@id_field=:id, @reader=nil>
irb(main):003:0> index.size
=> 0
irb(main):004:0> index.search("*")
=> #<struct Ferret::Search::TopDocs total_hits=0, hits=[],
max_score=0.0, searcher=#Ferret::Search::Searcher:0x7fddf3ee2ea0>

The indexer service appears to be running, and the log files claim to
be indexing things, as shown by the sample below:

[Wed, 27 Jan 2010 16:43:23 +0000] INFO: Headers
{“destination”=>"/queue/chef/index", “content-type”=>“text/plain;
charset=UTF-8”, “content-length”=>“7452”}
[Wed, 27 Jan 2010 16:43:23 +0000] INFO: Indexed object from
/queue/chef/index in 0.107933 seconds

I’m not really sure where else to look to debug this any further now.
If anyone can point me in the right direction, I’d be very
appreciative.

Cheers,

Jon Wood

3

Hi Jon,

On the chance that you are still having this issue, there is a issue
with .7 that where ferret will occasionally corrupt the indexes.

The easiest thing to do in this case is to delete the indexes, since you
mention debian I am going to assume you are using .deb’s.

  1. turn off chef-indexer (/etc/init.d/chef-indexer stop)
  2. rm /var/lib/chef/indexes/* (I believe that’s the path, rubygem
    installs it’s in /srv/chef/indexes)
  3. start chef-indexer (/etc/init.d/chef-indexer start)

Unfortunately that does mean all your nodes will need to do a
chef-client run either on it’s normal interval or by manual intervention
before it will be re-indexed again.

This issue is fixed for .8 as it uses solr instead… and doesn’t
corrupt randomly and easily

Scott

On 1/27/10 10:31 AM, Jon Wood wrote:

Further to this, it appears that it’s specifically the indexer failing

  • messages are getting to the stompserver when I update a Node, but
    the indexer doesn’t seem to be picking them up.

2010/1/27 Jon Wood jon@blankpad.net:

Hi,

I’m having some trouble with an installation of Chef 0.7.14 from
Opscode’s repository on Debian 5.0.3.

Chef itself has started correctly, and clients are able to connect to
run recipes from it, however the search index doesn’t seem to be
getting populated.

When browsing the web interface, and selecting the Search tab, I see
an empty page which the documentation suggests is caused by the index
not having been created yet. Also when I use Ferret directly it isn’t
returning any results:

infrastructure:/srv/chef/search_index# irb
irb(main):001:0> require ‘ferret’
=> true
irb(main):002:0> index = Ferret::Index::Index.new(:path => “.”)
=> #<Ferret::Index::Index:0x7fddf1df0008 @mon_entering_queue=[],
@key=nil, @mon_count=0, @writer=nil, @default_input_field=:id,
@dir=#Ferret::Store::FSDirectory:0x7fddf1deffb8, @mon_owner=nil,
@qp=nil, @searcher=nil, @default_field=:, @close_dir=true,
@auto_flush=false, @mon_waiting_queue=[], @open=true,
@options={:lock_retry_time=>2, :default_field=>:,
:dir=>#Ferret::Store::FSDirectory:0x7fddf1deffb8, :path=>".",
:analyzer=>#Ferret::Analysis::StandardAnalyzer:0x7fddf1defe78},
@id_field=:id, @reader=nil>
irb(main):003:0> index.size
=> 0
irb(main):004:0> index.search("*")
=> #<struct Ferret::Search::TopDocs total_hits=0, hits=[],
max_score=0.0, searcher=#Ferret::Search::Searcher:0x7fddf3ee2ea0>

The indexer service appears to be running, and the log files claim to
be indexing things, as shown by the sample below:

[Wed, 27 Jan 2010 16:43:23 +0000] INFO: Headers
{“destination”=>"/queue/chef/index", “content-type”=>“text/plain;
charset=UTF-8”, “content-length”=>“7452”}
[Wed, 27 Jan 2010 16:43:23 +0000] INFO: Indexed object from
/queue/chef/index in 0.107933 seconds

I’m not really sure where else to look to debug this any further now.
If anyone can point me in the right direction, I’d be very
appreciative.

Cheers,

Jon Wood

!DSPAM:4b60868a94711804284693!