Hello,
Will reindexing my Chef server cause any recipes that depend on Search
to receive incomplete results if they run before the rebuild is
complete?
-J
Yes if, for example, you have a search based load balancer it will
pull results that will be incomplete while re-indexing is being run.
In scenarios I know that a partial or incomplete result would be worse
than maintaing current state. I have in the past put checks on results
to be in some bounds in an only_if on some resource.
On Wed, Feb 22, 2012 at 6:16 PM, Jason J. W. Williams
jasonjwwilliams@gmail.com wrote:
Hello,
Will reindexing my Chef server cause any recipes that depend on Search
to receive incomplete results if they run before the rebuild is
complete?-J
On Wed, Feb 22, 2012 at 7:57 PM, Jesse Nelson spheromak@gmail.com wrote:
Yes if, for example, you have a search based load balancer it will
pull results that will be incomplete while re-indexing is being run.
In scenarios I know that a partial or incomplete result would be worse
than maintaing current state. I have in the past put checks on results
to be in some bounds in an only_if on some resource.
That's exactly our scenario (HAProxy). Is there way to disable
chef-server while the re-indexing is going on that won't disrupt the
indexing?
-J
Couple things I can think of:
You can block server's API port with iptables. Causing node client
runs to fail, but not change anything.
Add to all nodes runlist in the front a recipe that simply ends the
run with node.exit. Till you're done with your upgrade of course.
Knife ssh disable all your clients 
I am sure there are more!
On Wed, Feb 22, 2012 at 7:03 PM, Jason J. W. Williams
jasonjwwilliams@gmail.com wrote:
On Wed, Feb 22, 2012 at 7:57 PM, Jesse Nelson spheromak@gmail.com wrote:
Yes if, for example, you have a search based load balancer it will
pull results that will be incomplete while re-indexing is being run.
In scenarios I know that a partial or incomplete result would be worse
than maintaing current state. I have in the past put checks on results
to be in some bounds in an only_if on some resource.That's exactly our scenario (HAProxy). Is there way to disable
chef-server while the re-indexing is going on that won't disrupt the
indexing?-J
Alright...thought of the iptables option...was hoping for something a
little more built-in. Thank you.
-J
On Wednesday, February 22, 2012, Jesse Nelson spheromak@gmail.com wrote:
Couple things I can think of:
You can block server's API port with iptables. Causing node client
runs to fail, but not change anything.Add to all nodes runlist in the front a recipe that simply ends the
run with node.exit. Till you're done with your upgrade of course.Knife ssh disable all your clients
I am sure there are more!
On Wed, Feb 22, 2012 at 7:03 PM, Jason J. W. Williams
jasonjwwilliams@gmail.com wrote:On Wed, Feb 22, 2012 at 7:57 PM, Jesse Nelson spheromak@gmail.com
wrote:Yes if, for example, you have a search based load balancer it will
pull results that will be incomplete while re-indexing is being run.
In scenarios I know that a partial or incomplete result would be worse
than maintaing current state. I have in the past put checks on results
to be in some bounds in an only_if on some resource.That's exactly our scenario (HAProxy). Is there way to disable
chef-server while the re-indexing is going on that won't disrupt the
indexing?-J
I would argue for putting guards in your recipe around how many
results you have returned, and perhaps going so far as to cache
earlier results to use in this case.
Adam
On Wed, Feb 22, 2012 at 7:35 PM, Jason J. W. Williams
jasonjwwilliams@gmail.com wrote:
Alright...thought of the iptables option...was hoping for something a little
more built-in. Thank you.-J
On Wednesday, February 22, 2012, Jesse Nelson spheromak@gmail.com wrote:
Couple things I can think of:
You can block server's API port with iptables. Causing node client
runs to fail, but not change anything.Add to all nodes runlist in the front a recipe that simply ends the
run with node.exit. Till you're done with your upgrade of course.Knife ssh disable all your clients
I am sure there are more!
On Wed, Feb 22, 2012 at 7:03 PM, Jason J. W. Williams
jasonjwwilliams@gmail.com wrote:On Wed, Feb 22, 2012 at 7:57 PM, Jesse Nelson spheromak@gmail.com
wrote:Yes if, for example, you have a search based load balancer it will
pull results that will be incomplete while re-indexing is being run.
In scenarios I know that a partial or incomplete result would be worse
than maintaing current state. I have in the past put checks on results
to be in some bounds in an only_if on some resource.That's exactly our scenario (HAProxy). Is there way to disable
chef-server while the re-indexing is going on that won't disrupt the
indexing?-J
--
Opscode, Inc.
Adam Jacob, Chief Customer Officer
T: (206) 619-7151 E: adam@opscode.com
Hi Adam,
Those are all pretty brittle options. Layering customized caching all over the place introduces a lot of opportunities for bugs and unanticipated failure modes The ability to tell the Chef server to stop serving new clients would be very helpful here. Or the ability to trigger a reindex from the command line on the server itself without chef-server running. Heck even Chef building the new index in parallel to the existing index and then swapping the new index into battery would be great. Thanks for all of your help.
-J
Sent via iPhone
Is your email Premiere?
On Feb 24, 2012, at 10:42, Adam Jacob adam@opscode.com wrote:
I would argue for putting guards in your recipe around how many
results you have returned, and perhaps going so far as to cache
earlier results to use in this case.Adam
On Wed, Feb 22, 2012 at 7:35 PM, Jason J. W. Williams
jasonjwwilliams@gmail.com wrote:Alright...thought of the iptables option...was hoping for something a little
more built-in. Thank you.-J
On Wednesday, February 22, 2012, Jesse Nelson spheromak@gmail.com wrote:
Couple things I can think of:
You can block server's API port with iptables. Causing node client
runs to fail, but not change anything.Add to all nodes runlist in the front a recipe that simply ends the
run with node.exit. Till you're done with your upgrade of course.Knife ssh disable all your clients
I am sure there are more!
On Wed, Feb 22, 2012 at 7:03 PM, Jason J. W. Williams
jasonjwwilliams@gmail.com wrote:On Wed, Feb 22, 2012 at 7:57 PM, Jesse Nelson spheromak@gmail.com
wrote:Yes if, for example, you have a search based load balancer it will
pull results that will be incomplete while re-indexing is being run.
In scenarios I know that a partial or incomplete result would be worse
than maintaing current state. I have in the past put checks on results
to be in some bounds in an only_if on some resource.That's exactly our scenario (HAProxy). Is there way to disable
chef-server while the re-indexing is going on that won't disrupt the
indexing?-J
--
Opscode, Inc.
Adam Jacob, Chief Customer Officer
T: (206) 619-7151 E: adam@opscode.com
For sure. Didn't mean to imply it wasn't a big deal, simply that if I
had to fix that problem that way today, that would probably be the
(not super ideal) approach I would take.
Adam
On Fri, Feb 24, 2012 at 10:05 AM, Jason J. W. Williams
jasonjwwilliams@gmail.com wrote:
Hi Adam,
Those are all pretty brittle options. Layering customized caching all over the place introduces a lot of opportunities for bugs and unanticipated failure modes The ability to tell the Chef server to stop serving new clients would be very helpful here. Or the ability to trigger a reindex from the command line on the server itself without chef-server running. Heck even Chef building the new index in parallel to the existing index and then swapping the new index into battery would be great. Thanks for all of your help.
-J
Sent via iPhone
Is your email Premiere?
On Feb 24, 2012, at 10:42, Adam Jacob adam@opscode.com wrote:
I would argue for putting guards in your recipe around how many
results you have returned, and perhaps going so far as to cache
earlier results to use in this case.Adam
On Wed, Feb 22, 2012 at 7:35 PM, Jason J. W. Williams
jasonjwwilliams@gmail.com wrote:Alright...thought of the iptables option...was hoping for something a little
more built-in. Thank you.-J
On Wednesday, February 22, 2012, Jesse Nelson spheromak@gmail.com wrote:
Couple things I can think of:
You can block server's API port with iptables. Causing node client
runs to fail, but not change anything.Add to all nodes runlist in the front a recipe that simply ends the
run with node.exit. Till you're done with your upgrade of course.Knife ssh disable all your clients
I am sure there are more!
On Wed, Feb 22, 2012 at 7:03 PM, Jason J. W. Williams
jasonjwwilliams@gmail.com wrote:On Wed, Feb 22, 2012 at 7:57 PM, Jesse Nelson spheromak@gmail.com
wrote:Yes if, for example, you have a search based load balancer it will
pull results that will be incomplete while re-indexing is being run.
In scenarios I know that a partial or incomplete result would be worse
than maintaing current state. I have in the past put checks on results
to be in some bounds in an only_if on some resource.That's exactly our scenario (HAProxy). Is there way to disable
chef-server while the re-indexing is going on that won't disrupt the
indexing?-J
--
Opscode, Inc.
Adam Jacob, Chief Customer Officer
T: (206) 619-7151 E: adam@opscode.com
--
Opscode, Inc.
Adam Jacob, Chief Customer Officer
T: (206) 619-7151 E: adam@opscode.com
This is one reason why a SSL proxy in front of the chef server is handy.
The chef server service only listens on localhost.
I just stop apache (or nginx) when I want clients to not have access for
any reason (search index rebuild
or other maintenance or oh fsck I shouldn't have uploaded that cookbook
moment).
On Fri, Feb 24, 2012 at 10:14 AM, Adam Jacob adam@opscode.com wrote:
For sure. Didn't mean to imply it wasn't a big deal, simply that if I
had to fix that problem that way today, that would probably be the
(not super ideal) approach I would take.Adam
On Fri, Feb 24, 2012 at 10:05 AM, Jason J. W. Williams
jasonjwwilliams@gmail.com wrote:Hi Adam,
Those are all pretty brittle options. Layering customized caching all
over the place introduces a lot of opportunities for bugs and unanticipated
failure modes The ability to tell the Chef server to stop serving new
clients would be very helpful here. Or the ability to trigger a reindex
from the command line on the server itself without chef-server running.
Heck even Chef building the new index in parallel to the existing index and
then swapping the new index into battery would be great. Thanks for all of
your help.-J
Sent via iPhone
Is your email Premiere?
On Feb 24, 2012, at 10:42, Adam Jacob adam@opscode.com wrote:
I would argue for putting guards in your recipe around how many
results you have returned, and perhaps going so far as to cache
earlier results to use in this case.Adam
On Wed, Feb 22, 2012 at 7:35 PM, Jason J. W. Williams
jasonjwwilliams@gmail.com wrote:Alright...thought of the iptables option...was hoping for something a
little
more built-in. Thank you.-J
On Wednesday, February 22, 2012, Jesse Nelson spheromak@gmail.com
wrote:Couple things I can think of:
You can block server's API port with iptables. Causing node client
runs to fail, but not change anything.Add to all nodes runlist in the front a recipe that simply ends the
run with node.exit. Till you're done with your upgrade of course.Knife ssh disable all your clients
I am sure there are more!
On Wed, Feb 22, 2012 at 7:03 PM, Jason J. W. Williams
jasonjwwilliams@gmail.com wrote:On Wed, Feb 22, 2012 at 7:57 PM, Jesse Nelson spheromak@gmail.com
wrote:Yes if, for example, you have a search based load balancer it will
pull results that will be incomplete while re-indexing is being run.
In scenarios I know that a partial or incomplete result would be
worse
than maintaing current state. I have in the past put checks on
results
to be in some bounds in an only_if on some resource.That's exactly our scenario (HAProxy). Is there way to disable
chef-server while the re-indexing is going on that won't disrupt the
indexing?-J
--
Opscode, Inc.
Adam Jacob, Chief Customer Officer
T: (206) 619-7151 E: adam@opscode.com--
Opscode, Inc.
Adam Jacob, Chief Customer Officer
T: (206) 619-7151 E: adam@opscode.com
Hi Peter,
That's not a bad idea. I'll take a look at dropping HAProxy in front.
-J
On Fri, Feb 24, 2012 at 1:32 PM, Peter Struijk peter@delftblues.net wrote:
This is one reason why a SSL proxy in front of the chef server is handy. The
chef server service only listens on localhost.
I just stop apache (or nginx) when I want clients to not have access for any
reason (search index rebuild
or other maintenance or oh fsck I shouldn't have uploaded that cookbook
moment).On Fri, Feb 24, 2012 at 10:14 AM, Adam Jacob adam@opscode.com wrote:
For sure. Didn't mean to imply it wasn't a big deal, simply that if I
had to fix that problem that way today, that would probably be the
(not super ideal) approach I would take.Adam
On Fri, Feb 24, 2012 at 10:05 AM, Jason J. W. Williams
jasonjwwilliams@gmail.com wrote:Hi Adam,
Those are all pretty brittle options. Layering customized caching all
over the place introduces a lot of opportunities for bugs and unanticipated
failure modes The ability to tell the Chef server to stop serving new
clients would be very helpful here. Or the ability to trigger a reindex from
the command line on the server itself without chef-server running. Heck even
Chef building the new index in parallel to the existing index and then
swapping the new index into battery would be great. Thanks for all of your
help.-J
Sent via iPhone
Is your email Premiere?
On Feb 24, 2012, at 10:42, Adam Jacob adam@opscode.com wrote:
I would argue for putting guards in your recipe around how many
results you have returned, and perhaps going so far as to cache
earlier results to use in this case.Adam
On Wed, Feb 22, 2012 at 7:35 PM, Jason J. W. Williams
jasonjwwilliams@gmail.com wrote:Alright...thought of the iptables option...was hoping for something a
little
more built-in. Thank you.-J
On Wednesday, February 22, 2012, Jesse Nelson spheromak@gmail.com
wrote:Couple things I can think of:
You can block server's API port with iptables. Causing node client
runs to fail, but not change anything.Add to all nodes runlist in the front a recipe that simply ends the
run with node.exit. Till you're done with your upgrade of course.Knife ssh disable all your clients
I am sure there are more!
On Wed, Feb 22, 2012 at 7:03 PM, Jason J. W. Williams
jasonjwwilliams@gmail.com wrote:On Wed, Feb 22, 2012 at 7:57 PM, Jesse Nelson spheromak@gmail.com
wrote:Yes if, for example, you have a search based load balancer it will
pull results that will be incomplete while re-indexing is being
run.
In scenarios I know that a partial or incomplete result would be
worse
than maintaing current state. I have in the past put checks on
results
to be in some bounds in an only_if on some resource.That's exactly our scenario (HAProxy). Is there way to disable
chef-server while the re-indexing is going on that won't disrupt the
indexing?-J
--
Opscode, Inc.
Adam Jacob, Chief Customer Officer
T: (206) 619-7151 E: adam@opscode.com--
Opscode, Inc.
Adam Jacob, Chief Customer Officer
T: (206) 619-7151 E: adam@opscode.com