All nodes returning "an empty run list." on bootstrap

tadeu · August 31, 2018, 12:36pm

Hi guys,

I'm a new with Chef bootstrap and I'm facing some issues regarding bootstraping my nodes.

Every node that I did a knife bootstrap in the end returns "[2018-08-31T09:27:10-03:00] WARN: Node has an empty run list"

This is how I'm executing my boostrap:

knife bootstrap -x root --sudo -P -r "recipe[bootstrap]" --environment qa

I don't know what I'm missing here, could someone please send some light?

Thanks in advanced!!

Larryc · August 31, 2018, 12:44pm

It might be you need single quotes instead of double quotes Here is an example from the docs

knife bootstrap 192.0.2.0 -P vanilla -x root -r 'recipe[apt],recipe[xfs],recipe[vim]'

tadeu · August 31, 2018, 1:04pm

Hi @Larryc,

Same error =(

[tadeu@hp-notebook chef-repo]$ knife bootstrap hambrosia.qa.company.intranet -x root --sudo -P -r 'recipe[bootstrap]'--environment qa
Doing old-style registration with the validation key at /home/tadeu/.chef/validation.pem...
Delete your validation key in order to use your user credentials instead

Connecting to hambrosia.qa.company.intranet
root@hambrosia.qa.company.intranet's password:
hambrosia.qa.company.intranet Loaded plugins: fastestmirror
hambrosia.qa.company.intranet Loading mirror speeds from cached hostfile
base | 3.6 kB 00:00
centosplus | 2.9 kB 00:00
chef | 3.6 kB 00:00
extras | 2.9 kB 00:00
pacotes-dev-company | 3.6 kB 00:00
third-party | 2.9 kB 00:00
updates | 2.9 kB 00:00
(1/2): pacotes-dev-company/primary_db | 62 kB 00:00
(2/2): updates/primary_db | 1.5 MB 00:00
hambrosia.qa.company.intranet Resolving Dependencies
hambrosia.qa.company.intranet --> Running transaction check
hambrosia.qa.company.intranet ---> Package chef.x86_64 0:13.10.0-1.el7 will be installed
hambrosia.qa.company.intranet --> Finished Dependency Resolution
hambrosia.qa.company.intranet
hambrosia.qa.company.intranet Dependencies Resolved
hambrosia.qa.company.intranet
hambrosia.qa.company.intranet ================================================================================
hambrosia.qa.company.intranet Package Arch Version Repository Size
hambrosia.qa.company.intranet ================================================================================
hambrosia.qa.company.intranet Installing:
hambrosia.qa.company.intranet chef x86_64 13.10.0-1.el7 chef 49 M
hambrosia.qa.company.intranet
hambrosia.qa.company.intranet Transaction Summary
hambrosia.qa.company.intranet ================================================================================
hambrosia.qa.company.intranet Install 1 Package
hambrosia.qa.company.intranet
hambrosia.qa.company.intranet Total download size: 49 M
hambrosia.qa.company.intranet Installed size: 149 M
hambrosia.qa.company.intranet Downloading packages:
chef-13.10.0-1.el7.x86_64.rpm | 49 MB 00:00
hambrosia.qa.company.intranet Running transaction check
hambrosia.qa.company.intranet Running transaction test
hambrosia.qa.company.intranet Transaction test succeeded
hambrosia.qa.company.intranet Running transaction
Installing : chef-13.10.0-1.el7.x86_64 1/1
hambrosia.qa.company.intranet Thank you for installing Chef!
Verifying : chef-13.10.0-1.el7.x86_64 1/1
hambrosia.qa.company.intranet
hambrosia.qa.company.intranet Installed:
hambrosia.qa.company.intranet chef.x86_64 0:13.10.0-1.el7
hambrosia.qa.company.intranet
hambrosia.qa.company.intranet Complete!
hambrosia.qa.company.intranet Starting the first Chef Client run...
hambrosia.qa.company.intranet Starting Chef Client, version 13.10.0
hambrosia.qa.company.intranet Creating a new client identity for hambrosia.qa.company.intranet using the validator key.
hambrosia.qa.company.intranet resolving cookbooks for run list:
hambrosia.qa.company.intranet Synchronizing Cookbooks:
hambrosia.qa.company.intranet Installing Cookbook Gems:
hambrosia.qa.company.intranet Compiling Cookbooks...
hambrosia.qa.company.intranet [2018-08-31T10:01:07-03:00] WARN: Node hambrosia.qa.company.intranet has an empty run list.
hambrosia.qa.company.intranet Converging 0 resources
hambrosia.qa.company.intranet
hambrosia.qa.company.intranet Running handlers:
hambrosia.qa.company.intranet Running handlers complete
hambrosia.qa.company.intranet Chef Client finished, 0/0 resources updated in 03 seconds

Larryc · August 31, 2018, 1:16pm

Do you see a runlist if you do

knife node show hambrosia.qa.company.intranet

tadeu · August 31, 2018, 1:27pm

Nope:

[tadeu@hp-notebook chef-repo]$ knife node show hambrosia.qa.company.intranet
Node Name: hambrosia.qa.company.intranet
Environment: _default
FQDN: hambrosia.qa.company.intranet
IP: 10.150.230.38
Run List:
Roles:
Recipes:
Platform: centos 7.5.1804
Tags:

Larryc · August 31, 2018, 1:34pm

Is your recipe you are setting, already on the chef server?

knife cookbook list

tadeu · August 31, 2018, 1:36pm

Yes,

[tadeu@hp-notebook roles]$ knife cookbook list
bootstrap 0.1.9

Larryc · August 31, 2018, 1:48pm

Can you make sure you have a space between the runlist and the the environment. I noticed in that log you posted that there is no space between them. Which would also make sense because your environment is still set to _default in that node show you posted.

knife bootstrap hambrosia.qa.company.intranet -x root --sudo -P -r 'recipe[bootstrap]' --environment qa

tadeu · August 31, 2018, 2:06pm

That makes sense. Thanks @Larryc.

Now I'm getting 401

[tadeu@hp-notebook chef-repo]$ knife bootstrap hambrosia.qa.company.intranet -x root --sudo -P -r 'recipe[bootstrap]' --environment qa
Doing old-style registration with the validation key at /home/tadeu/.chef/validation.pem...
Delete your validation key in order to use your user credentials instead

Connecting to hambrosia.qa.company.intranet
root@hambrosia.qa.company.intranet's password:
hambrosia.qa.company.intranet Loaded plugins: fastestmirror
hambrosia.qa.company.intranet Loading mirror speeds from cached hostfile
hambrosia.qa.company.intranet Resolving Dependencies
hambrosia.qa.company.intranet --> Running transaction check
hambrosia.qa.company.intranet ---> Package chef.x86_64 0:13.10.0-1.el7 will be installed
hambrosia.qa.company.intranet --> Finished Dependency Resolution
hambrosia.qa.company.intranet
hambrosia.qa.company.intranet Dependencies Resolved
hambrosia.qa.company.intranet
hambrosia.qa.company.intranet ================================================================================
hambrosia.qa.company.intranet Package Arch Version Repository Size
hambrosia.qa.company.intranet ================================================================================
hambrosia.qa.company.intranet Installing:
hambrosia.qa.company.intranet chef x86_64 13.10.0-1.el7 chef 49 M
hambrosia.qa.company.intranet
hambrosia.qa.company.intranet Transaction Summary
hambrosia.qa.company.intranet ================================================================================
hambrosia.qa.company.intranet Install 1 Package
hambrosia.qa.company.intranet
hambrosia.qa.company.intranet Total download size: 49 M
hambrosia.qa.company.intranet Installed size: 149 M
hambrosia.qa.company.intranet Downloading packages:
chef-13.10.0-1.el7.x86_64.rpm | 49 MB 00:01
hambrosia.qa.company.intranet Running transaction check
hambrosia.qa.company.intranet Running transaction test
hambrosia.qa.company.intranet Transaction test succeeded
hambrosia.qa.company.intranet Running transaction
hambrosia.qa.company.intranet Warning: RPMDB altered outside of yum.
Installing : chef-13.10.0-1.el7.x86_64 1/1
hambrosia.qa.company.intranet Thank you for installing Chef!
Verifying : chef-13.10.0-1.el7.x86_64 1/1
hambrosia.qa.company.intranet
hambrosia.qa.company.intranet Installed:
hambrosia.qa.company.intranet chef.x86_64 0:13.10.0-1.el7
hambrosia.qa.company.intranet
hambrosia.qa.company.intranet Complete!
hambrosia.qa.company.intranet Starting the first Chef Client run...
hambrosia.qa.company.intranet Starting Chef Client, version 13.10.0
hambrosia.qa.company.intranet
hambrosia.qa.company.intranet ================================================================================
hambrosia.qa.company.intranet Chef encountered an error attempting to load the node data for "hambrosia.qa.company.intranet"
hambrosia.qa.company.intranet ================================================================================
hambrosia.qa.company.intranet
hambrosia.qa.company.intranet Authentication Error:
hambrosia.qa.company.intranet ---------------------
hambrosia.qa.company.intranet Failed to authenticate to the chef server (http 401).
hambrosia.qa.company.intranet
hambrosia.qa.company.intranet Server Response:
hambrosia.qa.company.intranet ----------------
hambrosia.qa.company.intranet Failed to authenticate as 'hambrosia.qa.company.intranet'. Ensure that your node_name and client key are correct.
hambrosia.qa.company.intranet
hambrosia.qa.company.intranet Relevant Config Settings:
hambrosia.qa.company.intranet -------------------------
hambrosia.qa.company.intranet chef_server_url "https://hze01.qa.company.intranet/organizations/company"
hambrosia.qa.company.intranet node_name "hambrosia.qa.company.intranet"
hambrosia.qa.company.intranet client_key "/etc/chef/client.pem"
hambrosia.qa.company.intranet
hambrosia.qa.company.intranet If these settings are correct, your client_key may be invalid, or
hambrosia.qa.company.intranet you may have a chef user with the same client name as this node.
hambrosia.qa.company.intranet
hambrosia.qa.company.intranet System Info:
hambrosia.qa.company.intranet ------------
hambrosia.qa.company.intranet chef_version=13.10.0
hambrosia.qa.company.intranet ruby=ruby 2.4.4p296 (2018-03-28 revision 63013) [x86_64-linux]
hambrosia.qa.company.intranet program_name=chef-client worker: ppid=3230;start=11:04:14;
hambrosia.qa.company.intranet executable=/opt/chef/bin/chef-client
hambrosia.qa.company.intranet
hambrosia.qa.company.intranet
hambrosia.qa.company.intranet Running handlers:
hambrosia.qa.company.intranet [2018-08-31T11:04:17-03:00] ERROR: Running exception handlers
hambrosia.qa.company.intranet Running handlers complete
hambrosia.qa.company.intranet [2018-08-31T11:04:17-03:00] ERROR: Exception handlers complete
hambrosia.qa.company.intranet Chef Client failed. 0 resources updated in 02 seconds
hambrosia.qa.company.intranet [2018-08-31T11:04:17-03:00] FATAL: Stacktrace dumped to /var/chef/cache/chef-stacktrace.out
hambrosia.qa.company.intranet [2018-08-31T11:04:17-03:00] FATAL: Please provide the contents of the stacktrace.out file if you file a bug report
hambrosia.qa.company.intranet [2018-08-31T11:04:17-03:00] ERROR: 401 "Unauthorized"
hambrosia.qa.company.intranet [2018-08-31T11:04:17-03:00] FATAL: Chef::Exceptions::ChildConvergeError: Chef run process exited unsuccessfully (exit code 1)

Larryc · August 31, 2018, 2:21pm

I would make sure you can ssh into that server with the root/pw combo you are providing.

tadeu · August 31, 2018, 2:24pm

[tadeu@hp-notebook chef-repo]$ ssh -l root hambrosia.qa.company.intranet
root@hambrosia.qa.company.intranet's password:
Last login: Fri Aug 31 11:21:08 2018 from 10.140.80.209
[root@hambrosia ~]#

Larryc · August 31, 2018, 2:27pm

On the node run chef-client as root and see what that outputs.

tadeu · August 31, 2018, 2:53pm

[root@hambrosia etc]# chef-client
Starting Chef Client, version 13.10.0

================================================================================
Chef encountered an error attempting to load the node data for "hambrosia.qa.company.intranet"

Authentication Error:

Failed to authenticate to the chef server (http 401).

Server Response:

Failed to authenticate as 'hambrosia.qa.company.intranet'. Ensure that your node_name and client key are correct.

Relevant Config Settings:

chef_server_url "https://hze01.qa.company.intranet/organizations/company"
node_name "hambrosia.qa.company.intranet"
client_key "/etc/chef/client.pem"

If these settings are correct, your client_key may be invalid, or
you may have a chef user with the same client name as this node.

System Info:

chef_version=13.10.0
ruby=ruby 2.4.4p296 (2018-03-28 revision 63013) [x86_64-linux]
program_name=chef-client worker: ppid=18692;start=11:53:05;
executable=/opt/chef/bin/chef-client

Running handlers:
[2018-08-31T11:53:08-03:00] ERROR: Running exception handlers
Running handlers complete
[2018-08-31T11:53:08-03:00] ERROR: Exception handlers complete
Chef Client failed. 0 resources updated in 03 seconds
[2018-08-31T11:53:08-03:00] FATAL: Stacktrace dumped to /var/chef/cache/chef-stacktrace.out
[2018-08-31T11:53:08-03:00] FATAL: Please provide the contents of the stacktrace.out file if you file a bug report
[2018-08-31T11:53:08-03:00] ERROR: 401 "Unauthorized"
[2018-08-31T11:53:08-03:00] FATAL: Chef::Exceptions::ChildConvergeError: Chef run process exited unsuccessfully (exit code 1)

Larryc · August 31, 2018, 3:29pm

I would delete the node from the chef server and try bootstrapping it again

knife node delete hambrosia.qa.company.intranet

and then run the bootstrap again

you might want to delete the /etc/chef directory before you do it.

tadeu · August 31, 2018, 4:10pm

I was doing this

Larryc · August 31, 2018, 4:20pm

It just appears that your client.rb on the node is messed up, its weird that it was able to bootstrap but now it cannot communicate to the chef server.

tadeu · August 31, 2018, 4:32pm

I'm trying to manager two servers, one from 'QA' and one for 'PROD" and I'm starting facing issues when I try to do bootstrao the nodes with these two environments.

tadeu · August 31, 2018, 4:34pm

[tadeu@hp-notebook bootstrap]$ which chg
/usr/bin/chg

[tadeu@hp-notebook bootstrap]$ cat /usr/bin/chg
#!/bin/bash
case $1 in
prod|qa)
unlink ~/.chef; ln -s ~/.chef_$1 ~/.chef;;
--help|-h)
echo -e "Uso:\n\t$(basename $0) [prod|qa]";;
*)
echo -e "Environment:\n\t$(ls -lh ~/.chef | cut -d_ -f2-)";;
esac

Larryc · August 31, 2018, 4:44pm

Are you switching out .chef directories because you are using 2 different chef servers? If you are you should check out knife block it will help you manage multiple knife configs.

Topic		Replies	Views
Empty run list when bootstrapping windows nodes Chef Infra (archive)	0	360	September 6, 2012
Empty run list after failed run (Windows) Chef Infra (archive)	3	1312	February 23, 2015
Changing runlist of an active node Chef Infra (archive)	1	308	December 3, 2012
Strange error - knife ssh "name:node_name" "pwd" -x root -P password returns FATAL: No nodes returned from search! Chef Infra (archive)	0	436	November 5, 2014
Bootstrapping first node: Failed to authenticate to the chef server (http 401) Chef Infra (archive)	1	2562	October 12, 2016