All nodes returning "an empty run list." on bootstrap

Hi guys,

I'm a new with Chef bootstrap and I'm facing some issues regarding bootstraping my nodes.

Every node that I did a knife bootstrap in the end returns "[2018-08-31T09:27:10-03:00] WARN: Node has an empty run list"

This is how I'm executing my boostrap:

knife bootstrap -x root --sudo -P -r "recipe[bootstrap]" --environment qa

I don't know what I'm missing here, could someone please send some light?

Thanks in advanced!!

It might be you need single quotes instead of double quotes Here is an example from the docs

knife bootstrap 192.0.2.0 -P vanilla -x root -r 'recipe[apt],recipe[xfs],recipe[vim]'

Hi @Larryc,

Same error =(

[tadeu@hp-notebook chef-repo]$ knife bootstrap hambrosia.qa.company.intranet -x root --sudo -P -r 'recipe[bootstrap]'--environment qa
Doing old-style registration with the validation key at /home/tadeu/.chef/validation.pem...
Delete your validation key in order to use your user credentials instead

Connecting to hambrosia.qa.company.intranet
root@hambrosia.qa.company.intranet's password:
hambrosia.qa.company.intranet Loaded plugins: fastestmirror
hambrosia.qa.company.intranet Loading mirror speeds from cached hostfile
base | 3.6 kB 00:00
centosplus | 2.9 kB 00:00
chef | 3.6 kB 00:00
extras | 2.9 kB 00:00
pacotes-dev-company | 3.6 kB 00:00
third-party | 2.9 kB 00:00
updates | 2.9 kB 00:00
(1/2): pacotes-dev-company/primary_db | 62 kB 00:00
(2/2): updates/primary_db | 1.5 MB 00:00
hambrosia.qa.company.intranet Resolving Dependencies
hambrosia.qa.company.intranet --> Running transaction check
hambrosia.qa.company.intranet ---> Package chef.x86_64 0:13.10.0-1.el7 will be installed
hambrosia.qa.company.intranet --> Finished Dependency Resolution
hambrosia.qa.company.intranet
hambrosia.qa.company.intranet Dependencies Resolved
hambrosia.qa.company.intranet
hambrosia.qa.company.intranet ================================================================================
hambrosia.qa.company.intranet Package Arch Version Repository Size
hambrosia.qa.company.intranet ================================================================================
hambrosia.qa.company.intranet Installing:
hambrosia.qa.company.intranet chef x86_64 13.10.0-1.el7 chef 49 M
hambrosia.qa.company.intranet
hambrosia.qa.company.intranet Transaction Summary
hambrosia.qa.company.intranet ================================================================================
hambrosia.qa.company.intranet Install 1 Package
hambrosia.qa.company.intranet
hambrosia.qa.company.intranet Total download size: 49 M
hambrosia.qa.company.intranet Installed size: 149 M
hambrosia.qa.company.intranet Downloading packages:
chef-13.10.0-1.el7.x86_64.rpm | 49 MB 00:00
hambrosia.qa.company.intranet Running transaction check
hambrosia.qa.company.intranet Running transaction test
hambrosia.qa.company.intranet Transaction test succeeded
hambrosia.qa.company.intranet Running transaction
Installing : chef-13.10.0-1.el7.x86_64 1/1
hambrosia.qa.company.intranet Thank you for installing Chef!
Verifying : chef-13.10.0-1.el7.x86_64 1/1
hambrosia.qa.company.intranet
hambrosia.qa.company.intranet Installed:
hambrosia.qa.company.intranet chef.x86_64 0:13.10.0-1.el7
hambrosia.qa.company.intranet
hambrosia.qa.company.intranet Complete!
hambrosia.qa.company.intranet Starting the first Chef Client run...
hambrosia.qa.company.intranet Starting Chef Client, version 13.10.0
hambrosia.qa.company.intranet Creating a new client identity for hambrosia.qa.company.intranet using the validator key.
hambrosia.qa.company.intranet resolving cookbooks for run list:
hambrosia.qa.company.intranet Synchronizing Cookbooks:
hambrosia.qa.company.intranet Installing Cookbook Gems:
hambrosia.qa.company.intranet Compiling Cookbooks...
hambrosia.qa.company.intranet [2018-08-31T10:01:07-03:00] WARN: Node hambrosia.qa.company.intranet has an empty run list.
hambrosia.qa.company.intranet Converging 0 resources
hambrosia.qa.company.intranet
hambrosia.qa.company.intranet Running handlers:
hambrosia.qa.company.intranet Running handlers complete
hambrosia.qa.company.intranet Chef Client finished, 0/0 resources updated in 03 seconds

Do you see a runlist if you do

knife node show hambrosia.qa.company.intranet

Nope:

[tadeu@hp-notebook chef-repo]$ knife node show hambrosia.qa.company.intranet
Node Name: hambrosia.qa.company.intranet
Environment: _default
FQDN: hambrosia.qa.company.intranet
IP: 10.150.230.38
Run List:
Roles:
Recipes:
Platform: centos 7.5.1804
Tags:

Is your recipe you are setting, already on the chef server?

knife cookbook list

Yes,

[tadeu@hp-notebook roles]$ knife cookbook list
bootstrap 0.1.9

Can you make sure you have a space between the runlist and the the environment. I noticed in that log you posted that there is no space between them. Which would also make sense because your environment is still set to _default in that node show you posted.

knife bootstrap hambrosia.qa.company.intranet -x root --sudo -P -r 'recipe[bootstrap]' --environment qa

That makes sense. Thanks @Larryc.

Now I'm getting 401

[tadeu@hp-notebook chef-repo]$ knife bootstrap hambrosia.qa.company.intranet -x root --sudo -P -r 'recipe[bootstrap]' --environment qa
Doing old-style registration with the validation key at /home/tadeu/.chef/validation.pem...
Delete your validation key in order to use your user credentials instead

Connecting to hambrosia.qa.company.intranet
root@hambrosia.qa.company.intranet's password:
hambrosia.qa.company.intranet Loaded plugins: fastestmirror
hambrosia.qa.company.intranet Loading mirror speeds from cached hostfile
hambrosia.qa.company.intranet Resolving Dependencies
hambrosia.qa.company.intranet --> Running transaction check
hambrosia.qa.company.intranet ---> Package chef.x86_64 0:13.10.0-1.el7 will be installed
hambrosia.qa.company.intranet --> Finished Dependency Resolution
hambrosia.qa.company.intranet
hambrosia.qa.company.intranet Dependencies Resolved
hambrosia.qa.company.intranet
hambrosia.qa.company.intranet ================================================================================
hambrosia.qa.company.intranet Package Arch Version Repository Size
hambrosia.qa.company.intranet ================================================================================
hambrosia.qa.company.intranet Installing:
hambrosia.qa.company.intranet chef x86_64 13.10.0-1.el7 chef 49 M
hambrosia.qa.company.intranet
hambrosia.qa.company.intranet Transaction Summary
hambrosia.qa.company.intranet ================================================================================
hambrosia.qa.company.intranet Install 1 Package
hambrosia.qa.company.intranet
hambrosia.qa.company.intranet Total download size: 49 M
hambrosia.qa.company.intranet Installed size: 149 M
hambrosia.qa.company.intranet Downloading packages:
chef-13.10.0-1.el7.x86_64.rpm | 49 MB 00:01
hambrosia.qa.company.intranet Running transaction check
hambrosia.qa.company.intranet Running transaction test
hambrosia.qa.company.intranet Transaction test succeeded
hambrosia.qa.company.intranet Running transaction
hambrosia.qa.company.intranet Warning: RPMDB altered outside of yum.
Installing : chef-13.10.0-1.el7.x86_64 1/1
hambrosia.qa.company.intranet Thank you for installing Chef!
Verifying : chef-13.10.0-1.el7.x86_64 1/1
hambrosia.qa.company.intranet
hambrosia.qa.company.intranet Installed:
hambrosia.qa.company.intranet chef.x86_64 0:13.10.0-1.el7
hambrosia.qa.company.intranet
hambrosia.qa.company.intranet Complete!
hambrosia.qa.company.intranet Starting the first Chef Client run...
hambrosia.qa.company.intranet Starting Chef Client, version 13.10.0
hambrosia.qa.company.intranet
hambrosia.qa.company.intranet ================================================================================
hambrosia.qa.company.intranet Chef encountered an error attempting to load the node data for "hambrosia.qa.company.intranet"
hambrosia.qa.company.intranet ================================================================================
hambrosia.qa.company.intranet
hambrosia.qa.company.intranet Authentication Error:
hambrosia.qa.company.intranet ---------------------
hambrosia.qa.company.intranet Failed to authenticate to the chef server (http 401).
hambrosia.qa.company.intranet
hambrosia.qa.company.intranet Server Response:
hambrosia.qa.company.intranet ----------------
hambrosia.qa.company.intranet Failed to authenticate as 'hambrosia.qa.company.intranet'. Ensure that your node_name and client key are correct.
hambrosia.qa.company.intranet
hambrosia.qa.company.intranet Relevant Config Settings:
hambrosia.qa.company.intranet -------------------------
hambrosia.qa.company.intranet chef_server_url "https://hze01.qa.company.intranet/organizations/company"
hambrosia.qa.company.intranet node_name "hambrosia.qa.company.intranet"
hambrosia.qa.company.intranet client_key "/etc/chef/client.pem"
hambrosia.qa.company.intranet
hambrosia.qa.company.intranet If these settings are correct, your client_key may be invalid, or
hambrosia.qa.company.intranet you may have a chef user with the same client name as this node.
hambrosia.qa.company.intranet
hambrosia.qa.company.intranet System Info:
hambrosia.qa.company.intranet ------------
hambrosia.qa.company.intranet chef_version=13.10.0
hambrosia.qa.company.intranet ruby=ruby 2.4.4p296 (2018-03-28 revision 63013) [x86_64-linux]
hambrosia.qa.company.intranet program_name=chef-client worker: ppid=3230;start=11:04:14;
hambrosia.qa.company.intranet executable=/opt/chef/bin/chef-client
hambrosia.qa.company.intranet
hambrosia.qa.company.intranet
hambrosia.qa.company.intranet Running handlers:
hambrosia.qa.company.intranet [2018-08-31T11:04:17-03:00] ERROR: Running exception handlers
hambrosia.qa.company.intranet Running handlers complete
hambrosia.qa.company.intranet [2018-08-31T11:04:17-03:00] ERROR: Exception handlers complete
hambrosia.qa.company.intranet Chef Client failed. 0 resources updated in 02 seconds
hambrosia.qa.company.intranet [2018-08-31T11:04:17-03:00] FATAL: Stacktrace dumped to /var/chef/cache/chef-stacktrace.out
hambrosia.qa.company.intranet [2018-08-31T11:04:17-03:00] FATAL: Please provide the contents of the stacktrace.out file if you file a bug report
hambrosia.qa.company.intranet [2018-08-31T11:04:17-03:00] ERROR: 401 "Unauthorized"
hambrosia.qa.company.intranet [2018-08-31T11:04:17-03:00] FATAL: Chef::Exceptions::ChildConvergeError: Chef run process exited unsuccessfully (exit code 1)

I would make sure you can ssh into that server with the root/pw combo you are providing.

[tadeu@hp-notebook chef-repo]$ ssh -l root hambrosia.qa.company.intranet
root@hambrosia.qa.company.intranet's password:
Last login: Fri Aug 31 11:21:08 2018 from 10.140.80.209
[root@hambrosia ~]#

On the node run chef-client as root and see what that outputs.

[root@hambrosia etc]# chef-client
Starting Chef Client, version 13.10.0

================================================================================
Chef encountered an error attempting to load the node data for "hambrosia.qa.company.intranet"

Authentication Error:

Failed to authenticate to the chef server (http 401).

Server Response:

Failed to authenticate as 'hambrosia.qa.company.intranet'. Ensure that your node_name and client key are correct.

Relevant Config Settings:

chef_server_url "https://hze01.qa.company.intranet/organizations/company"
node_name "hambrosia.qa.company.intranet"
client_key "/etc/chef/client.pem"

If these settings are correct, your client_key may be invalid, or
you may have a chef user with the same client name as this node.

System Info:

chef_version=13.10.0
ruby=ruby 2.4.4p296 (2018-03-28 revision 63013) [x86_64-linux]
program_name=chef-client worker: ppid=18692;start=11:53:05;
executable=/opt/chef/bin/chef-client

Running handlers:
[2018-08-31T11:53:08-03:00] ERROR: Running exception handlers
Running handlers complete
[2018-08-31T11:53:08-03:00] ERROR: Exception handlers complete
Chef Client failed. 0 resources updated in 03 seconds
[2018-08-31T11:53:08-03:00] FATAL: Stacktrace dumped to /var/chef/cache/chef-stacktrace.out
[2018-08-31T11:53:08-03:00] FATAL: Please provide the contents of the stacktrace.out file if you file a bug report
[2018-08-31T11:53:08-03:00] ERROR: 401 "Unauthorized"
[2018-08-31T11:53:08-03:00] FATAL: Chef::Exceptions::ChildConvergeError: Chef run process exited unsuccessfully (exit code 1)

I would delete the node from the chef server and try bootstrapping it again

knife node delete hambrosia.qa.company.intranet

and then run the bootstrap again

you might want to delete the /etc/chef directory before you do it.

I was doing this :frowning:

It just appears that your client.rb on the node is messed up, its weird that it was able to bootstrap but now it cannot communicate to the chef server.

I'm trying to manager two servers, one from 'QA' and one for 'PROD" and I'm starting facing issues when I try to do bootstrao the nodes with these two environments.

[tadeu@hp-notebook bootstrap]$ which chg
/usr/bin/chg

[tadeu@hp-notebook bootstrap]$ cat /usr/bin/chg
#!/bin/bash
case $1 in
prod|qa)
unlink ~/.chef; ln -s ~/.chef_$1 ~/.chef;;
--help|-h)
echo -e "Uso:\n\t$(basename $0) [prod|qa]";;
*)
echo -e "Environment:\n\t$(ls -lh ~/.chef | cut -d_ -f2-)";;
esac

Are you switching out .chef directories because you are using 2 different chef servers? If you are you should check out knife block it will help you manage multiple knife configs.