Chef Server 12.4.1 search matching attributes it shouldn't


#1

Hi,

All of our servers have an attribute indicating their datacenter:

default[“facility”] = “DC1”

We do that so we can restrict searches to a single DC:

knife search node “facility:DC1”

That’s worked very well for several years. Recently, we added a new cookbook that created this attribute on its nodes:

default[“flume_relay”][“kafka_sink”][“facility”] = “DC1”

Now servers with that cookbook who have the root “facility” attribute set to “DC2”,“DC3”, etc are now being returned incorrectly in facility:DC1 searches because of having the attribute [“flume_relay”][“kafka_sink”][“facility”] = “DC1”.

Historically we’ve expected nested attributes not to greedy match like that (you have to do something like flume_relay_kafka_sink_facility to search for a nested attribute).

Any advice is greatly appreciated, because this could cause a lot of issues for us.

-J


#2

Unfortunately this is expected and “correct” behavior. When we expand nested attributes in to the index, it generates all suffixes of the key path. So ["flume_relay"]["kafka_sink"]["facility"] is expanded to flume_relay_kafka_sink_facility, kafka_sink_facility, facility. The usual fix for this is to do a second filtering in your Ruby code to drop false positives:

nodes = []
search(:node, "facility:#{whatever}") do |n|
  nodes << n if n["flume_relay"]["kafka_sink"]["facility"] == whatever
end

This is frustrating but it would be a major compat break to change it so we haven’t wanted to tackle it (yet).


#3

Hey Noah,

Thanks for explaining the behavior. I understand the hesitation to introduce a breaking change. What about adding a prefix operator (^) that indicates an exact attribute match?

Something like:

knife search node “^facility:DC1”

knife search node “^facility:DC1”

-J


#4

That wouldn’t be possible with how we currently store the search index. Lucene is built to searching text data in a fixed facet schema rather than nested facets like we have. To make it work we actually proactively expand things so the index data can’t tell the difference between the two keys. Lossy transform :frowning: