# Automatic Node Configuration - "Failed to authenticate"

**URL:** <https://discourse.chef.io/t/automatic-node-configuration-failed-to-authenticate/5972>\
**Category:** Chef Infra (archive)\
**Created:** [November 24, 2014, 8:45pm UTC](https://discourse.chef.io/t/automatic-node-configuration-failed-to-authenticate/5972 "2014-11-24T20:45:33Z")\
**Posts on this page:** 5\
**Page:** 1

<div class="post-metadata">

**Author:** ![Douglas\_Garstang](https://avatars.discourse-cdn.com/v4/letter/d/c2a13f/32.png) [@Douglas\_Garstang](https://discourse.chef.io/u/Douglas_Garstang)\
**Post date:** [November 24, 2014, 8:45pm UTC](https://discourse.chef.io/t/automatic-node-configuration-failed-to-authenticate/5972/1 "2014-11-24T20:45:33Z")

</div>

I’m trying to automate the addition of chef nodes to the chef server.  
Cloud-init is running a script that downloads an admin user’s pem file from  
an S3 bucket, generates a new node’s json file and adds the node with the  
knife command, ie:

knife node from file /tmp/somenewnode.json -c /tmp/knife.rb

That works fine. I can see the new node on the chef server. After that,  
another script downloaded from S3 is creating /etc/chef/client.rb file for  
the new node, like so:

ssl\_verify\_mode :verify\_peer  
log\_level :info  
log\_location STDOUT  
chef\_server\_url ‘[https://chef01.prod.slicetest.com:443](https://chef01.prod.slicetest.com:443)’  
validation\_client\_name ‘chef-validator’  
validation\_key  
’/etc/chef/validation\_keys/validator-chef01.prod.slicetest.com.pem’

When I run the chef-client I get a message:

## Authentication Error:

Failed to authenticate to the chef server (http 401).

## Server Response:

Invalid signature for user or client ‘chef-validator’

What am I missing here? I’ve confirmed the validator key is correct. I’m  
making sure to remove both the node and the client from the chef server  
before running (as I know that having an existing client cert on the server  
will break it). Is this something to do with the trusted\_certs thing? How  
is that supposed to work?

Doug.

–  
Regards,

Douglas Garstang  
[http://www.linkedin.com/in/garstang](http://www.linkedin.com/in/garstang)  
Email: [doug.garstang@gmail.com](mailto:doug.garstang@gmail.com)  
Cell: +1-805-340-5627

---

<div class="post-metadata">

**Author:** ![kallistec](https://sea2.discourse-cdn.com/flex016/user_avatar/discourse.chef.io/kallistec/32/23_2.png) [@kallistec](https://discourse.chef.io/u/kallistec)\
**Post date:** [November 24, 2014, 8:56pm UTC](https://discourse.chef.io/t/automatic-node-configuration-failed-to-authenticate/5972/2 "2014-11-24T20:56:06Z")

</div>

On Monday, November 24, 2014 at 12:45 PM, Douglas Garstang wrote:

> ## Authentication Error:
> 
> Failed to authenticate to the chef server (http 401).
> 
> ## Server Response:
> 
> Invalid signature for user or client 'chef-validator'
> 
> What am I missing here? I've confirmed the validator key is correct. I'm making sure to remove both the node and the client from the chef server before running (as I know that having an existing client cert on the server will break it). Is this something to do with the trusted\_certs thing? How is that supposed to work?
> 
> Doug.

Either the key is wrong, the validator username is wrong, or your clock is skewed by more than 15 minutes from the server (though the error message should normally indicate the last one).

trusted\_certs has nothing to do with Chef’s application level authentication. The only thing it does is add certificates to the root CA bundle OpenSSL uses to check certificate validity for SSL/TLS connections. The point of this is to make it simpler to use chef-client with SSL peer verification (cert checking) enabled if you use self-signed certs (which is many/most people running their own servers).

I don’t know if it works anymore but I wrote this knife plugin a while ago to check client keys: [https://github.com/danielsdeleo/knife-plugins/blob/master/key\_check.rb](https://github.com/danielsdeleo/knife-plugins/blob/master/key_check.rb) You run it from a working admin account and it compares the private key you have (which contains the public key) to the public key you can download from the server. You can accomplish the same with various openssl commands.

--  
Daniel DeLeo

---

<div class="post-metadata">

**Author:** ![Douglas\_Garstang](https://avatars.discourse-cdn.com/v4/letter/d/c2a13f/32.png) [@Douglas\_Garstang](https://discourse.chef.io/u/Douglas_Garstang)\
**Post date:** [November 24, 2014, 9:03pm UTC](https://discourse.chef.io/t/automatic-node-configuration-failed-to-authenticate/5972/3 "2014-11-24T21:03:02Z")

</div>

Thanks Daniel.

Can you confirm that the private key on the chef server at  
/etc/chef-server/chef-validator.pem is what I should be using as a  
validation key on the client side?

Douglas.

On Mon, Nov 24, 2014 at 12:56 PM, Daniel DeLeo [dan@kallistec.com](mailto:dan@kallistec.com) wrote:

> On Monday, November 24, 2014 at 12:45 PM, Douglas Garstang wrote:
> 
> > ## Authentication Error:
> > 
> > Failed to authenticate to the chef server (http 401).
> > 
> > ## Server Response:
> > 
> > Invalid signature for user or client 'chef-validator'
> > 
> > What am I missing here? I've confirmed the validator key is correct. I'm  
> > making sure to remove both the node and the client from the chef server  
> > before running (as I know that having an existing client cert on the server  
> > will break it). Is this something to do with the trusted\_certs thing? How  
> > is that supposed to work?
> > 
> > Doug.
> 
> Either the key is wrong, the validator username is wrong, or your clock is  
> skewed by more than 15 minutes from the server (though the error message  
> should normally indicate the last one).
> 
> trusted\_certs has nothing to do with Chef’s application level  
> authentication. The only thing it does is add certificates to the root CA  
> bundle OpenSSL uses to check certificate validity for SSL/TLS connections.  
> The point of this is to make it simpler to use chef-client with SSL peer  
> verification (cert checking) enabled if you use self-signed certs (which is  
> many/most people running their own servers).
> 
> I don’t know if it works anymore but I wrote this knife plugin a while ago  
> to check client keys:  
> [https://github.com/danielsdeleo/knife-plugins/blob/master/key\_check.rb](https://github.com/danielsdeleo/knife-plugins/blob/master/key_check.rb)  
> You run it from a working admin account and it compares the private key you  
> have (which contains the public key) to the public key you can download  
> from the server. You can accomplish the same with various openssl commands.
> 
> --  
> Daniel DeLeo

--  
Regards,

Douglas Garstang  
[http://www.linkedin.com/in/garstang](http://www.linkedin.com/in/garstang)  
Email: [doug.garstang@gmail.com](mailto:doug.garstang@gmail.com)  
Cell: +1-805-340-5627

---

<div class="post-metadata">

**Author:** ![Douglas\_Garstang](https://avatars.discourse-cdn.com/v4/letter/d/c2a13f/32.png) [@Douglas\_Garstang](https://discourse.chef.io/u/Douglas_Garstang)\
**Post date:** [November 24, 2014, 9:08pm UTC](https://discourse.chef.io/t/automatic-node-configuration-failed-to-authenticate/5972/4 "2014-11-24T21:08:21Z")

</div>

I'm horribly confused. I just got it to work, and the validation key I used  
is _NOT_ what's on the server at /etc/chef-server/chef-validator.pem.

Doug.

On Mon, Nov 24, 2014 at 1:03 PM, Douglas Garstang [doug.garstang@gmail.com](mailto:doug.garstang@gmail.com)  
wrote:

> Thanks Daniel.
> 
> Can you confirm that the private key on the chef server at  
> /etc/chef-server/chef-validator.pem is what I should be using as a  
> validation key on the client side?
> 
> Douglas.
> 
> On Mon, Nov 24, 2014 at 12:56 PM, Daniel DeLeo [dan@kallistec.com](mailto:dan@kallistec.com) wrote:
> 
> > On Monday, November 24, 2014 at 12:45 PM, Douglas Garstang wrote:
> > 
> > > ## Authentication Error:
> > > 
> > > Failed to authenticate to the chef server (http 401).
> > > 
> > > ## Server Response:
> > > 
> > > Invalid signature for user or client 'chef-validator'
> > > 
> > > What am I missing here? I've confirmed the validator key is correct.  
> > > I'm making sure to remove both the node and the client from the chef server  
> > > before running (as I know that having an existing client cert on the server  
> > > will break it). Is this something to do with the trusted\_certs thing? How  
> > > is that supposed to work?
> > > 
> > > Doug.
> > 
> > Either the key is wrong, the validator username is wrong, or your clock  
> > is skewed by more than 15 minutes from the server (though the error message  
> > should normally indicate the last one).
> > 
> > trusted\_certs has nothing to do with Chef’s application level  
> > authentication. The only thing it does is add certificates to the root CA  
> > bundle OpenSSL uses to check certificate validity for SSL/TLS connections.  
> > The point of this is to make it simpler to use chef-client with SSL peer  
> > verification (cert checking) enabled if you use self-signed certs (which is  
> > many/most people running their own servers).
> > 
> > I don’t know if it works anymore but I wrote this knife plugin a while  
> > ago to check client keys:  
> > [https://github.com/danielsdeleo/knife-plugins/blob/master/key\_check.rb](https://github.com/danielsdeleo/knife-plugins/blob/master/key_check.rb)  
> > You run it from a working admin account and it compares the private key you  
> > have (which contains the public key) to the public key you can download  
> > from the server. You can accomplish the same with various openssl commands.
> > 
> > --  
> > Daniel DeLeo
> 
> --  
> Regards,
> 
> Douglas Garstang  
> [http://www.linkedin.com/in/garstang](http://www.linkedin.com/in/garstang)  
> Email: [doug.garstang@gmail.com](mailto:doug.garstang@gmail.com)  
> Cell: +1-805-340-5627

--  
Regards,

Douglas Garstang  
[http://www.linkedin.com/in/garstang](http://www.linkedin.com/in/garstang)  
Email: [doug.garstang@gmail.com](mailto:doug.garstang@gmail.com)  
Cell: +1-805-340-5627

---

<div class="post-metadata">

**Author:** ![kallistec](https://sea2.discourse-cdn.com/flex016/user_avatar/discourse.chef.io/kallistec/32/23_2.png) [@kallistec](https://discourse.chef.io/u/kallistec)\
**Post date:** [November 24, 2014, 10:43pm UTC](https://discourse.chef.io/t/automatic-node-configuration-failed-to-authenticate/5972/5 "2014-11-24T22:43:10Z")

</div>

On Monday, November 24, 2014 at 1:08 PM, Douglas Garstang wrote:

> I'm horribly confused. I just got it to work, and the validation key I used is _NOT_ what's on the server at /etc/chef-server/chef-validator.pem.
> 
> Doug.

If you run `knife client reregister VALIDATOR_NAME` (or the equivalent HTTP API calls via some other means) then a new key is generated for the validator and the old one becomes invalid. I don’t think there’s a way that this can be fixed automatically, since the server doesn’t keep a copy of the private key.

My personal opinion is that the validator mechanism for creating new clients/nodes is overly error-prone and should only be used when you absolutely must have “self bootstrapping” behavior, for example when PXE booting or using AWS autoscaling and you’ve baked the credentials into your OS image. From your previous email, it looks like you’re creating node data from a script/program you’re writing yourself. If it’s an option for you, you might want to do something like the following:

knife client create -u ADMIN\_USER -k ADMIN\_KEY -f NEW\_NODE\_KEY.pem NEW\_NODE\_NAME  
knife node create -u NEW\_NODE\_NAME -k NEW\_NODE\_KEY.pem # or other knife command/API tool

Then you need to copy the NEW\_NODE\_KEY.pem to the new machine and configure it with the correct NEW\_NODE\_NAME. The downside to this approach is that you need to protect NEW\_NODE\_KEY.pem from being disclosed to anyone/anything that shouldn’t have it, I _think_ you can achieve this for s3 via IAM but I’m not 100% sure.

HTH,

--  
Daniel DeLeo