# Knife monitoring

**URL:** <https://discourse.chef.io/t/knife-monitoring/5339>\
**Category:** Chef Infra (archive)\
**Created:** [May 23, 2014, 12:02pm UTC](https://discourse.chef.io/t/knife-monitoring/5339 "2014-05-23T12:02:35Z")\
**Posts on this page:** 8\
**Page:** 1

<div class="post-metadata">

**Author:** ![Gregory\_Patmore](https://sea2.discourse-cdn.com/flex016/user_avatar/discourse.chef.io/gregory_patmore/32/46_2.png) [@Gregory\_Patmore](https://discourse.chef.io/u/Gregory_Patmore)\
**Post date:** [May 23, 2014, 12:02pm UTC](https://discourse.chef.io/t/knife-monitoring/5339/1 "2014-05-23T12:02:35Z")

</div>

Hey all,

As our chef repos grow and becomes more collaborative, a need to track activity on the chef server is rising in priority for me.

We’ve wrapped the knife util to track activity, but can’t enforce that people will use the wrapped util. Is there any way anyone is doing this? I’d like to implement tracking at the chef server side. Any plugin or advice would be appreciated.

Gregory Patmore

---

<div class="post-metadata">

**Author:** ![Mike\_Splain](https://sea2.discourse-cdn.com/flex016/user_avatar/discourse.chef.io/mike_splain/32/323_2.png) [@Mike\_Splain](https://discourse.chef.io/u/Mike_Splain)\
**Post date:** [May 23, 2014, 12:08pm UTC](https://discourse.chef.io/t/knife-monitoring/5339/2 "2014-05-23T12:08:04Z")

</div>

We tried to tackle that problem and had similar issues. Instead, we're  
going the automated route, only Jenkins can make changes to chef: automated  
pushes of cookbooks, environments, roles, etc. that tracking and history  
are built into Jenkins and only occur after a proper merge has been done in  
GitHub.

I know that's not exactly what you're asking but just something to think  
about.

Good luck!

Mike

On Friday, May 23, 2014, Gregory Patmore [gregorypatmore@gmail.com](mailto:gregorypatmore@gmail.com) wrote:

> Hey all,
> 
> As our chef repos grow and becomes more collaborative, a need to track  
> activity on the chef server is rising in priority for me.
> 
> We've wrapped the knife util to track activity, but can't enforce that  
> people will use the wrapped util. Is there any way anyone is doing this?  
> I'd like to implement tracking at the chef server side. Any plugin or  
> advice would be appreciated.
> 
> Gregory Patmore

---

<div class="post-metadata">

**Author:** ![jeffbyrnes](https://sea2.discourse-cdn.com/flex016/user_avatar/discourse.chef.io/jeffbyrnes/32/1478_2.png) [@jeffbyrnes](https://discourse.chef.io/u/jeffbyrnes)\
**Post date:** [May 23, 2014, 1:38pm UTC](https://discourse.chef.io/t/knife-monitoring/5339/3 "2014-05-23T13:38:27Z")

</div>

We went a similar route, though we aren’t using Jenkins, but instead, have “chefhooks” bit running on a worker server. It will apply any change that’s pushed to our git repo, so it’s very trusting, but since it’s so easy to update environments, data bags, etc., nobody does it using knife (with the exception of encrypted data bags, which must be edited via knife).

We don’t, however, handle cookbooks that way; those are all driven by a Berkshelf workflow that relies on Travis to test things using Test Kitchen & the EC2 driver.

–  
Jeff Byrnes  
@berkleebassist  
Operations Engineer  
EverTrue  
704.516.4628

On May 23, 2014 at 8:08:04 AM, Mike Splain ([mike.splain@gmail.com](mailto:mike.splain@gmail.com)) wrote:

We tried to tackle that problem and had similar issues. Instead, we’re going the automated route, only Jenkins can make changes to chef: automated pushes of cookbooks, environments, roles, etc. that tracking and history are built into Jenkins and only occur after a proper merge has been done in GitHub.

I know that’s not exactly what you’re asking but just something to think about.

Good luck!

Mike

On Friday, May 23, 2014, Gregory Patmore [gregorypatmore@gmail.com](mailto:gregorypatmore@gmail.com) wrote:  
Hey all,

As our chef repos grow and becomes more collaborative, a need to track activity on the chef server is rising in priority for me.

We’ve wrapped the knife util to track activity, but can’t enforce that people will use the wrapped util. Is there any way anyone is doing this? I’d like to implement tracking at the chef server side. Any plugin or advice would be appreciated.

Gregory Patmore

---

<div class="post-metadata">

**Author:** ![Sander\_Botman](https://sea2.discourse-cdn.com/flex016/user_avatar/discourse.chef.io/sander_botman/32/318_2.png) [@Sander\_Botman](https://discourse.chef.io/u/Sander_Botman)\
**Post date:** [May 23, 2014, 3:29pm UTC](https://discourse.chef.io/t/knife-monitoring/5339/4 "2014-05-23T15:29:01Z")

</div>

Hi Gregory,

We were running into the same problem and wrote also some knife wrappers, but indeed this is not solving anything.  
Then we wrote some tooling that would scan the nginx log on who/what has changed something and pull the information on the backend out of chef and commit it in git.  
But this was very chatty, error prone and not scalable… (but worked for some months)

> **[Chef Monitoring](https://www.cupfighter.net/2014/02/chef-monitoring)**
>
> At Schuberg Philis we have a certain way of working which gives every engineer a lot of freedom. That amount of freedom can only be provided and maintained with the right tool-set and auditing capabilities. Auditability and accountability are the...

Then we started with the development of chef-guard (written in Go)  
We use the chef-guard between erchef and bookshelf on the frontend and communicate from the guard to the backend.  
It scans who is doing what and commit every change (plain json) into github, so you will endup with a repo with all your change information.  
It integrates with foodcritic and rubocop and can scan every cookbook upload on syntax/style and even block uploads that do not comply. (if you want)  
Idea is to even check your uploads against already uploaded cookbooks and block you if you have made changes but not updated the version.  
And we can disabled the --force overwrite on frozen cookbooks, because what’s the use of freezing if you can do that 😉

Next week [svanharmelen@schubergphilis.com](mailto:svanharmelen@schubergphilis.com) who wrote the tool will opensource it on github, so you can develop on it yourself.  
We have been running with this in production since Chef-Conf 2014 and squashed all the minor bugs…  
So keep in contact and we will help you out on making your environment 100% …

Kind regards,  
Sander Botman

Schuberg Philis  
Boeingavenue 271  
1119 PD Schiphol-Rijk  
[http://www.schubergphilis.com](http://www.schubergphilis.com)

-----Original Message-----  
From: Gregory Patmore [[mailto:gregorypatmore@gmail.com](mailto:gregorypatmore@gmail.com)]  
Sent: vrijdag 23 mei 2014 14:03  
To: chef  
Subject: [chef] Knife monitoring

Hey all,

As our chef repos grow and becomes more collaborative, a need to track activity on the chef server is rising in priority for me.

We’ve wrapped the knife util to track activity, but can’t enforce that people will use the wrapped util. Is there any way anyone is doing this? I’d like to implement tracking at the chef server side. Any plugin or advice would be appreciated.

Gregory Patmore

---

<div class="post-metadata">

**Author:** ![jdunn](https://sea2.discourse-cdn.com/flex016/user_avatar/discourse.chef.io/jdunn/32/1800_2.png) [@jdunn](https://discourse.chef.io/u/jdunn)\
**Post date:** [May 23, 2014, 7:00pm UTC](https://discourse.chef.io/t/knife-monitoring/5339/5 "2014-05-23T19:00:11Z")

</div>

Check out Schuberg Phillis's chef-monitor, a way that you can tap the  
RabbitMQ bus in Chef Server to grab events of interest.

[https://www.cupfighter.net/2014/02/chef-monitoring](https://www.cupfighter.net/2014/02/chef-monitoring)

Chef Actions (a for-pay addon to EC that we announced at ChefConf and  
that will ship soon) has a similar architecture, though, of course,  
many more features.

- Julian

On Fri, May 23, 2014 at 8:02 AM, Gregory Patmore  
[gregorypatmore@gmail.com](mailto:gregorypatmore@gmail.com) wrote:

> Hey all,
> 
> As our chef repos grow and becomes more collaborative, a need to track activity on the chef server is rising in priority for me.
> 
> We've wrapped the knife util to track activity, but can't enforce that people will use the wrapped util. Is there any way anyone is doing this? I'd like to implement tracking at the chef server side. Any plugin or advice would be appreciated.
> 
> Gregory Patmore

--  
[Julian C. Dunn [jdunn@aquezada.com](mailto:jdunn@aquezada.com) \* Sorry, I'm ]  
[WWW: [http://www.aquezada.com/staff/julian](http://www.aquezada.com/staff/julian) \* only Web 1.0 ]  
[gopher://sdf.org/1/users/keymaker/ \* compliant!]  
[PGP: 91B3 7A9D 683C 7C16 715F 442C 6065 D533 FDC2 05B9]

---

<div class="post-metadata">

**Author:** ![Tristan\_Keen](https://avatars.discourse-cdn.com/v4/letter/t/8edcca/32.png) [@Tristan\_Keen](https://discourse.chef.io/u/Tristan_Keen)\
**Post date:** [May 23, 2014, 7:14pm UTC](https://discourse.chef.io/t/knife-monitoring/5339/6 "2014-05-23T19:14:13Z")

</div>

We're looking into how to do something similar in using git to be the  
primary record of state (including audit-ability & changelog), i.e.  
cookbooks in separate repos, and a chef-data repo for databags,  
environments, etc. (we don't use roles).

But storing nodes troubles me - a lot of tools & cookbooks expect to  
store state in the "normal" priority attributes, which are persisted in  
Chef Server's node database. Transient state is generated by cookbooks at  
the other priority levels, which can be used to implement Search and other  
data publishing/metrics activity. All that would get blitzed if we store  
all the nodes in the same chef-data repo, and did a blanket upload of all  
nodes (along with databags & environments) from the git repo whenever  
someone pushed a single new node/databag/whatever.

Any suggestions on how do to cope with this?

Tristan.

On 23 May 2014 14:38, Jeff Byrnes [jeff@evertrue.com](mailto:jeff@evertrue.com) wrote:

> We went a similar route, though we aren’t using Jenkins, but instead, have  
> “chefhooks” bit running on a worker server. It will apply any change that’s  
> pushed to our git repo, so it’s very trusting, but since it’s so easy to  
> update environments, data bags, etc., nobody does it using knife (with the  
> exception of encrypted data bags, which must be edited via knife).
> 
> We don’t, however, handle cookbooks that way; those are all driven by a  
> Berkshelf workflow that relies on Travis to test things using Test Kitchen  
> & the EC2 driver.
> 
> --  
> Jeff Byrnes  
> @berkleebassist [http://twitter.com/berkleebassist](http://twitter.com/berkleebassist)  
> Operations Engineer  
> EverTrue [http://www.evertrue.com/](http://www.evertrue.com/)  
> 704.516.4628
> 
> On May 23, 2014 at 8:08:04 AM, Mike Splain ([mike.splain@gmail.com](mailto:mike.splain@gmail.com)) wrote:
> 
> We tried to tackle that problem and had similar issues. Instead, we're  
> going the automated route, only Jenkins can make changes to chef: automated  
> pushes of cookbooks, environments, roles, etc. that tracking and history  
> are built into Jenkins and only occur after a proper merge has been done in  
> GitHub.
> 
> I know that's not exactly what you're asking but just something to think  
> about.
> 
> Good luck!
> 
> Mike
> 
> On Friday, May 23, 2014, Gregory Patmore [gregorypatmore@gmail.com](mailto:gregorypatmore@gmail.com) wrote:
> 
> > Hey all,
> > 
> > As our chef repos grow and becomes more collaborative, a need to track  
> > activity on the chef server is rising in priority for me.
> > 
> > We've wrapped the knife util to track activity, but can't enforce that  
> > people will use the wrapped util. Is there any way anyone is doing this?  
> > I'd like to implement tracking at the chef server side. Any plugin or  
> > advice would be appreciated.
> > 
> > Gregory Patmore

---

<div class="post-metadata">

**Author:** ![Gregory\_Patmore](https://sea2.discourse-cdn.com/flex016/user_avatar/discourse.chef.io/gregory_patmore/32/46_2.png) [@Gregory\_Patmore](https://discourse.chef.io/u/Gregory_Patmore)\
**Post date:** [May 24, 2014, 5:23pm UTC](https://discourse.chef.io/t/knife-monitoring/5339/7 "2014-05-24T17:23:49Z")

</div>

we have a private github server, so the issue isn't as critical for us since we have control over access to the server.

Revision control is relatively simple on the source side, and as long as you can guarantee that anyone with a knife account will no go off reservation it will suffice, but as the team, infrastructure, and various environments grow it becomes more of an exercise in faith then a reliable system (this is just my experience).

I've been pondering the same questions you have lately. Employing tools like chef-zero and other chef helpers require local access to node information, which, as you point out, can be represented incompletely.

The only idea I've had that I haven't dismissed is tracking the node objects in a different repo then your cookbooks, but even that doesn't quite cut it if I understand you're meaning.

Sent from my iPhone

> On May 23, 2014, at 3:14 PM, Tristan Keen [tristan.keen@gmail.com](mailto:tristan.keen@gmail.com) wrote:
> 
> We're looking into how to do something similar in using git to be the primary record of state (including audit-ability & changelog), i.e. cookbooks in separate repos, and a chef-data repo for databags, environments, etc. (we don't use roles).
> 
> But storing nodes troubles me - a lot of tools & cookbooks expect to store state in the "normal" priority attributes, which are persisted in Chef Server's node database. Transient state is generated by cookbooks at the other priority levels, which can be used to implement Search and other data publishing/metrics activity. All that would get blitzed if we store all the nodes in the same chef-data repo, and did a blanket upload of all nodes (along with databags & environments) from the git repo whenever someone pushed a single new node/databag/whatever.
> 
> Any suggestions on how do to cope with this?
> 
> Tristan.
> 
> > On 23 May 2014 14:38, Jeff Byrnes [jeff@evertrue.com](mailto:jeff@evertrue.com) wrote:  
> > We went a similar route, though we aren’t using Jenkins, but instead, have “chefhooks” bit running on a worker server. It will apply any change that’s pushed to our git repo, so it’s very trusting, but since it’s so easy to update environments, data bags, etc., nobody does it using knife (with the exception of encrypted data bags, which must be edited via knife).
> > 
> > We don’t, however, handle cookbooks that way; those are all driven by a Berkshelf workflow that relies on Travis to test things using Test Kitchen & the EC2 driver.
> > 
> > --  
> > Jeff Byrnes  
> > @berkleebassist  
> > Operations Engineer  
> > EverTrue  
> > 704.516.4628
> > 
> > > On May 23, 2014 at 8:08:04 AM, Mike Splain ([mike.splain@gmail.com](mailto:mike.splain@gmail.com)) wrote:
> > > 
> > > We tried to tackle that problem and had similar issues. Instead, we're going the automated route, only Jenkins can make changes to chef: automated pushes of cookbooks, environments, roles, etc. that tracking and history are built into Jenkins and only occur after a proper merge has been done in GitHub.
> > > 
> > > I know that's not exactly what you're asking but just something to think about.
> > > 
> > > Good luck!
> > > 
> > > Mike
> > > 
> > > > On Friday, May 23, 2014, Gregory Patmore [gregorypatmore@gmail.com](mailto:gregorypatmore@gmail.com) wrote:  
> > > > Hey all,
> > > > 
> > > > As our chef repos grow and becomes more collaborative, a need to track activity on the chef server is rising in priority for me.
> > > > 
> > > > We've wrapped the knife util to track activity, but can't enforce that people will use the wrapped util. Is there any way anyone is doing this? I'd like to implement tracking at the chef server side. Any plugin or advice would be appreciated.
> > > > 
> > > > Gregory Patmore

---

<div class="post-metadata">

**Author:** ![Gregory\_Patmore](https://sea2.discourse-cdn.com/flex016/user_avatar/discourse.chef.io/gregory_patmore/32/46_2.png) [@Gregory\_Patmore](https://discourse.chef.io/u/Gregory_Patmore)\
**Post date:** [May 24, 2014, 5:31pm UTC](https://discourse.chef.io/t/knife-monitoring/5339/8 "2014-05-24T17:31:10Z")

</div>

Thanks Julian,

This is more in line with what I'm after. Tracking changes in 4 different environments with up to 20 knife-ers across 3 other teams besides my own is getting to be a cumbersome obligation. It forces me to be somewhat inflexible on my production environment, and with multiple dev processes happening in the non prod environments it's starting to turn into the Wild Wild West.

Luckily I caught it but almost re-introduced the heart bleed bug when an uninformed DBA trying to push through OpenSSL upgrades by way of a dependency for Postgres.

I want to be the voice of DevOps philosophy, sourcing config management from all tech staff, but without strict change control and issue forensics on the chef side, I'm forced to either throttle back software production too much or become the Operations Ogre defending his bridge lol.

Sent from my iPhone

> On May 23, 2014, at 3:00 PM, "Julian C. Dunn" [jdunn@aquezada.com](mailto:jdunn@aquezada.com) wrote:
> 
> Check out Schuberg Phillis's chef-monitor, a way that you can tap the  
> RabbitMQ bus in Chef Server to grab events of interest.
> 
> [https://www.cupfighter.net/2014/02/chef-monitoring](https://www.cupfighter.net/2014/02/chef-monitoring)
> 
> Chef Actions (a for-pay addon to EC that we announced at ChefConf and  
> that will ship soon) has a similar architecture, though, of course,  
> many more features.
> 
> - Julian
> 
> On Fri, May 23, 2014 at 8:02 AM, Gregory Patmore  
> [gregorypatmore@gmail.com](mailto:gregorypatmore@gmail.com) wrote:
> 
> > Hey all,
> > 
> > As our chef repos grow and becomes more collaborative, a need to track activity on the chef server is rising in priority for me.
> > 
> > We've wrapped the knife util to track activity, but can't enforce that people will use the wrapped util. Is there any way anyone is doing this? I'd like to implement tracking at the chef server side. Any plugin or advice would be appreciated.
> > 
> > Gregory Patmore
> 
> --  
> [Julian C. Dunn [jdunn@aquezada.com](mailto:jdunn@aquezada.com) \* Sorry, I'm ]  
> [WWW: [http://www.aquezada.com/staff/julian](http://www.aquezada.com/staff/julian) \* only Web 1.0 ]  
> [gopher://sdf.org/1/users/keymaker/ \* compliant!]  
> [PGP: 91B3 7A9D 683C 7C16 715F 442C 6065 D533 FDC2 05B9]