How to know if a chef-client run is occuring

Michael_Hart · August 21, 2013, 4:22pm

Is there a definitive way of querying the chef server to see if a chef-client run is occurring on a node? We’ve noticed that a “knife status” will return a timestamp of “382528 hours ago”, or however many hours you are away from epoch, and but it’s not entirely consistent and using that in code feels like a bit of a hack. Ideally I’d like an API to return true or false if a chef-client run is occurring. Thoughts?

cheers
mike

–
Michael Hart
Arctic Wolf Networks
M: 226.388.4773

Ranjib · August 21, 2013, 4:42pm

you mean in-flight chef runs? chef server cant say that, Chef 11 (may be
10.16 onward too) introduced file fcntl based locks while chef run is
underway, to prevent concurrent chef runs, you can use knife ssh against
the node in question and check if the lock is present, this will be a near
real time indicator of any on going chef run.
default location of the lock file is
Chef::Config[:file_cache_path]/chef-client-running.pid.

github.com

chef/chef/blob/main/lib/chef/config.rb#L138


      
          #
          # Author:: Adam Jacob (<adam@chef.io>)
          # Author:: Christopher Brown (<cb@chef.io>)
          # Author:: AJ Christensen (<aj@chef.io>)
          # Author:: Mark Mzyk (<mmzyk@chef.io>)
          # Author:: Kyle Goodwin (<kgoodwin@primerevenue.com>)
          # Copyright:: Copyright (c) Chef Software Inc.
          # License:: Apache License, Version 2.0
          #
          # Licensed under the Apache License, Version 2.0 (the "License");
          # you may not use this file except in compliance with the License.
          # You may obtain a copy of the License at
          #
          #     http://www.apache.org/licenses/LICENSE-2.0
          #
          # Unless required by applicable law or agreed to in writing, software
          # distributed under the License is distributed on an "AS IS" BASIS,
          # WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
          # See the License for the specific language governing permissions and
          # limitations under the License.
          
          require_relative "log"
          require "chef-config/logger"
          
          # DI our logger into ChefConfig before we load the config. Some defaults are
          # auto-detected, and this emits log messages on some systems, all of which will
          # occur at require-time. So we need to set the logger first.
          ChefConfig.logger = Chef::Log
          
          require "chef-config/config"
          require "chef-utils" unless defined?(ChefUtils::CANARY)
          require_relative "platform/query_helpers"
          
          # Ohai::Config defines its own log_level and log_location. When loaded, it will
          # override the default ChefConfig::Config values. We save them here before
          # loading ohai/config so that we can override them again inside Chef::Config.
          #
          # REMOVEME once these configurables are removed from the top level of Ohai.
          LOG_LEVEL = ChefConfig::Config[:log_level] unless defined? LOG_LEVEL
          LOG_LOCATION = ChefConfig::Config[:log_location] unless defined? LOG_LOCATION
          
          # Load the ohai config into the chef config. We can't have an empty ohai
          # configuration context because `ohai.plugins_path << some_path` won't work,
          # and providing default ohai config values here isn't DRY.
          require "ohai/config"
          
          class Chef
            Config = ChefConfig::Config
          
            # We re-open ChefConfig::Config to add additional settings. Generally,
            # everything should go in chef-config so it's shared with whoever uses that.
            # We make exceptions to that rule when:
            # * The functionality isn't likely to be useful outside of Chef
            # * The functionality makes use of a dependency we don't want to add to chef-config
            class Config
          
              default :event_loggers do
                evt_loggers = []
                if ChefUtils.windows?
                  evt_loggers << :win_evt
                end
                evt_loggers
              end
          
              # Override the default values that were set by Ohai.
              #
              # REMOVEME once these configurables are removed from the top level of Ohai.
              default :log_level, LOG_LEVEL
              default :log_location, LOG_LOCATION
          
              # Ohai::Config[:log_level] is deprecated and warns when set. Unfortunately,
              # there is no way to distinguish between setting log_level and setting
              # Ohai::Config[:log_level]. Since log_level and log_location are used by
              # chef-client and other tools (e.g., knife), we will mute the warnings here
              # by redefining the config_attr_writer to not warn for these options.
              #
              # REMOVEME once the warnings for these configurables are removed from Ohai.
              %i{log_level log_location}.each do |option|
                config_attr_writer option do |value|
                  value
                end
              end
          
            end
          end

github.com

chef/chef/blob/main/lib/chef/run_lock.rb#L45


      
          # is modifying the system at a time.
          class RunLock
            include Chef::Mixin::CreatePath
          
            attr_reader :runlock
            attr_reader :mutex
            attr_reader :runlock_file
          
            # Create a new instance of RunLock
            # === Arguments
            # * :lockfile::: the full path to the lockfile.
            def initialize(lockfile)
              @runlock_file = lockfile
              @runlock = nil
              @mutex = nil
              @runpid = nil
            end
          
            # Acquire the system-wide lock. Will block indefinitely if another process
            # already has the lock and Chef::Config[:run_lock_timeout] is
            # not set. Otherwise will block for Chef::Config[:run_lock_timeout]

On Wed, Aug 21, 2013 at 9:22 AM, Michael Hart
michael.hart@arcticwolf.comwrote:

Is there a definitive way of querying the chef server to see if a
chef-client run is occurring on a node? We've noticed that a "knife status"
will return a timestamp of "382528 hours ago", or however many hours you
are away from epoch, and but it's not entirely consistent and using that in
code feels like a bit of a hack. Ideally I'd like an API to return true or
false if a chef-client run is occurring. Thoughts?

cheers
mike

--
Michael Hart
Arctic Wolf Networks
M: 226.388.4773

kallistec · August 21, 2013, 4:43pm

On Wednesday, August 21, 2013 at 9:22 AM, Michael Hart wrote:

Is there a definitive way of querying the chef server to see if a chef-client run is occurring on a node? We've noticed that a "knife status" will return a timestamp of "382528 hours ago", or however many hours you are away from epoch, and but it's not entirely consistent and using that in code feels like a bit of a hack. Ideally I'd like an API to return true or false if a chef-client run is occurring. Thoughts?

Chef client communicates over HTTP, which is a stateless protocol, so there's no robust way for the server to know anything other than the last time a client made a request.

In Enterprise Chef (née Hosted and Private Chef), upcoming updates will include a node run history reporting feature that emulates the ability to track running clients by having them check in at the beginning and end of a run. How much of this makes it into the open source version and when is an open question at this point, but you could use a custom event dispatcher to track the state of clients in a similar way by integrating with a different system.

cheers
mike

--
Michael Hart
Arctic Wolf Networks
M: 226.388.4773

--
Daniel DeLeo

Ranjib · August 21, 2013, 4:45pm

btw, knife status uses a node attribute called 'ohai_time', which is an
automatic attribute, provided by ohai. You can use that if you want to
check the last successfull chef run. Also, you can use the event handler
for real time chef run progress.

On Wed, Aug 21, 2013 at 9:42 AM, Ranjib Dey dey.ranjib@gmail.com wrote:

you mean in-flight chef runs? chef server cant say that, Chef 11 (may be
10.16 onward too) introduced file fcntl based locks while chef run is
underway, to prevent concurrent chef runs, you can use knife ssh against
the node in question and check if the lock is present, this will be a near
real time indicator of any on going chef run.
default location of the lock file is
Chef::Config[:file_cache_path]/chef-client-running.pid.

chef/lib/chef/config.rb at main · chef/chef · GitHub
chef/lib/chef/run_lock.rb at main · chef/chef · GitHub

On Wed, Aug 21, 2013 at 9:22 AM, Michael Hart <michael.hart@arcticwolf.com

wrote:

Is there a definitive way of querying the chef server to see if a
chef-client run is occurring on a node? We've noticed that a "knife status"
will return a timestamp of "382528 hours ago", or however many hours you
are away from epoch, and but it's not entirely consistent and using that in
code feels like a bit of a hack. Ideally I'd like an API to return true or
false if a chef-client run is occurring. Thoughts?

cheers
mike

--
Michael Hart
Arctic Wolf Networks
M: 226.388.4773

Michael_Hart · August 22, 2013, 12:45am

Thanks Daniel, the feature in Enterprise Chef sounds interesting. Do you know the timeline for this feature’s release in Enterprise Chef?

cheers
mike

–
Michael Hart
Arctic Wolf Networks
M: 226.388.4773

On 2013-08-21, at 12:43 PM, Daniel DeLeo <dan@kallistec.com mailto:dan@kallistec.com> wrote:

On Wednesday, August 21, 2013 at 9:22 AM, Michael Hart wrote:

Is there a definitive way of querying the chef server to see if a chef-client run is occurring on a node? We’ve noticed that a “knife status” will return a timestamp of “382528 hours ago”, or however many hours you are away from epoch, and but it’s not entirely consistent and using that in code feels like a bit of a hack. Ideally I’d like an API to return true or false if a chef-client run is occurring. Thoughts?
Chef client communicates over HTTP, which is a stateless protocol, so there’s no robust way for the server to know anything other than the last time a client made a request.

In Enterprise Chef (née Hosted and Private Chef), upcoming updates will include a node run history reporting feature that emulates the ability to track running clients by having them check in at the beginning and end of a run. How much of this makes it into the open source version and when is an open question at this point, but you could use a custom event dispatcher to track the state of clients in a similar way by integrating with a different system.

cheers
mike

–
Michael Hart
Arctic Wolf Networks
M: 226.388.4773

–
Daniel DeLeo

kallistec · August 22, 2013, 3:02pm

On Wednesday, August 21, 2013 at 5:45 PM, Michael Hart wrote:

Thanks Daniel, the feature in Enterprise Chef sounds interesting. Do you know the timeline for this feature's release in Enterprise Chef?

cheers
mike

We're looking at a release some time in Q4 this year. The core code is there, but we have a lot of performance and other testing to do before we ship it to everyone.

--
Daniel DeLeo

Topic		Replies	Views
Chef : the definitive date? Chef Infra (archive)	16	445	October 21, 2013
How could I know from the chef server side that whether the running of a chef-client is done? Chef Infra (archive)	3	269	January 16, 2010
Client locking Chef Infra (archive)	2	351	August 12, 2014
Chef tutorial Run Chef Periodically Chef Infra (archive)	0	703	January 4, 2017
Node status tracking time stamp Chef Infra (archive)	6	1975	February 4, 2014

How to know if a chef-client run is occuring

Related topics