Setting up a continuous integration system with Chef... (not going so well so far)

Chef Infra (archive)
1

I posted about this yesterday, I'm going to post again - but this time with
links to code.

Here is the code:

My goal is to be able to install java, oracle DB, jenkins server, install
jenkins slaves, create jobs, update jobs with Chef Solo on a system of 15
servers. 1 server will be the jenkins master and the other 14 will be
slaves.

I'm starting my recipe with java and jenkins installs.

I'm guessing I'm doing several things wrong:

I ended up configuring the java attributes in nodes/redstone.json as I
couldn't get them to work inside my recipe. It seems to me it would be
better to do this in my recipe as this will not be node specific, I want it
global.

Here is what happens with the current code (even with include_recipe
"java::default"
as suggested by Christopher Armstrong)

Any help, suggestions, critiques would be greatly appreciated. Links to
real-life projects would be great, almost everything I see is completely
trivial or just doc on attributes for recipes. I'd love to see a real life
project I could download and execute.

I have to get this going soon or my boss will pull the plug :frowning:

Thanks!
phil

here is the output for running the current code:

philswenson@zoso:~/dev/chef/myfirstcookbook (master)$ knife solo cook
root@redstone
WARNING: solo.rb found, but since knife-solo v0.3.0 it is not used any more
WARNING: Please read the upgrade instructions:
https://github.com/matschaffer/knife-solo/wiki/Upgrading-to-0.3.0
Running Chef on redstone...
Checking Chef version...
Installing Berkshelf cookbooks to
'/Users/philswenson/.berkshelf/knife-solo/35c024dd80a9ab482eadf86b98a86fb90c9521d7'...
Using myfirstcookbook (0.1.0)
Using jenkins (2.0.2)
Using git (3.1.0)
Using java (1.20.0)
Using apt (2.3.8)
Using runit (1.5.10)
Using build-essential (2.0.0)
Using yum (3.1.4)
Using yum-epel (0.3.4)
Using dmg (2.2.0)
Using windows (1.30.0)
Using chef_handler (1.1.5)
Uploading the kitchen...
cannot delete non-empty directory: main
WARNING: Local role_path './roles' does not exist
WARNING: Local data_bag_path './data_bags' does not exist
WARNING: Local environment_path './environments' does not exist
Generating solo config...
Running Chef...
Starting Chef Client, version 11.10.4
[2014-03-14T19:07:47-04:00] WARN: unable to detect ip6address
Compiling Cookbooks...
HI PHIL HI PHIL

================================================================================
Recipe Compile Error in /root/chef-solo/cookbooks-3/main/recipes/default.rb

NoMethodError

undefined method `' for nil:NilClass

Cookbook Trace:

/root/chef-solo/cookbooks-2/java/recipes/oracle.rb:38:in from_file' /root/chef-solo/cookbooks-2/java/recipes/default.rb:22:in from_file'
/root/chef-solo/cookbooks-3/main/recipes/default.rb:8:in `from_file'

Relevant File Content:

/root/chef-solo/cookbooks-2/java/recipes/oracle.rb:

31:
32: case node['java']['jdk_version'].to_s
33: when "6"
34: tarball_url = node['java']['jdk']['6'][arch]['url']
35: tarball_checksum = node['java']['jdk']['6'][arch]['checksum']
36: bin_cmds = node['java']['jdk']['6']['bin_cmds']
37: when "7"
38>> tarball_url = node['java']['jdk']['7'][arch]['url']
39: tarball_checksum = node['java']['jdk']['7'][arch]['checksum']
40: bin_cmds = node['java']['jdk']['7']['bin_cmds']
41: end
42:
43: if tarball_url =~ /example.com/
44: Chef::Application.fatal!("You must change the download link to your
private repository. You can no longer download java directly from
http://download.oracle.com without a web broswer")
45: end
46:
47: include_recipe "java::set_java_home"

Running handlers:
[2014-03-14T19:07:47-04:00] ERROR: Running exception handlers
Running handlers complete

[2014-03-14T19:07:47-04:00] ERROR: Exception handlers complete
[2014-03-14T19:07:47-04:00] FATAL: Stacktrace dumped to
/var/chef/cache/chef-stacktrace.out
Chef Client failed. 0 resources updated in 0.279135717 seconds
[2014-03-14T19:07:47-04:00] ERROR: undefined method `' for nil:NilClass
[2014-03-14T19:07:47-04:00] FATAL: Chef::Exceptions::ChildConvergeError:
Chef run process exited unsuccessfully (exit code 1)
ERROR: RuntimeError: chef-solo failed. See output above.
philswenson@zoso:~/dev/chef/myfirstcookbook (master)$

2

Hi Phil,

On Fri, Mar 14, 2014 at 11:24 PM, phil swenson phil.swenson@gmail.comwrote:

I posted about this yesterday, I'm going to post again - but this time
with links to code.

Here is the code:

GitHub - pswenson/JenkinsChefRecipe

Code always helps, so thanks for linking that up...

My goal is to be able to install java, oracle DB, jenkins server, install
jenkins slaves, create jobs, update jobs with Chef Solo on a system of 15
servers. 1 server will be the jenkins master and the other 14 will be
slaves.

I'm starting my recipe with java and jenkins installs.

I'm guessing I'm doing several things wrong:

I ended up configuring the java attributes in nodes/redstone.json as I
couldn't get them to work inside my recipe. It seems to me it would be
better to do this in my recipe as this will not be node specific, I want it
global.

Here is what happens with the current code (even with include_recipe
"java::default"
as suggested by Christopher Armstrong)

Any help, suggestions, critiques would be greatly appreciated. Links to
real-life projects would be great, almost everything I see is completely
trivial or just doc on attributes for recipes. I'd love to see a real life
project I could download and execute.

Good cookbooks typically use the README file to document the attributes you
may want to manipulate - that's certainly true for the java coobkook (see
https://github.com/socrata-cookbooks/java/blob/master/README.md ). As an
example of putting it all together, you might want to look at this cookbook
( GitHub - zts/cooking-with-jenkins: A spike to explore use of Jenkins for Chef cookbook testing ) I wrote to experiment with
Jenkins configuration. It hasn't been updated to use the latest version of
the Jenkins cookbook, but it will show you how to make use of some
community cookbooks to achieve your goal.

With that said, let's look at the problem you're having...

here is the output for running the current code:

philswenson@zoso:~/dev/chef/myfirstcookbook (master)$ knife solo cook
root@redstone
WARNING: solo.rb found, but since knife-solo v0.3.0 it is not used any more
WARNING: Please read the upgrade instructions:
Upgrading to 0.3.0 · matschaffer/knife-solo Wiki · GitHub
Running Chef on redstone...
Checking Chef version...
Installing Berkshelf cookbooks to
'/Users/philswenson/.berkshelf/knife-solo/35c024dd80a9ab482eadf86b98a86fb90c9521d7'...
Using myfirstcookbook (0.1.0)
Using jenkins (2.0.2)
Using git (3.1.0)
Using java (1.20.0)
Using apt (2.3.8)
Using runit (1.5.10)
Using build-essential (2.0.0)
Using yum (3.1.4)
Using yum-epel (0.3.4)
Using dmg (2.2.0)
Using windows (1.30.0)
Using chef_handler (1.1.5)
Uploading the kitchen...
cannot delete non-empty directory: main
WARNING: Local role_path './roles' does not exist
WARNING: Local data_bag_path './data_bags' does not exist
WARNING: Local environment_path './environments' does not exist
Generating solo config...
Running Chef...
Starting Chef Client, version 11.10.4
[2014-03-14T19:07:47-04:00] WARN: unable to detect ip6address
Compiling Cookbooks...
HI PHIL HI PHIL

================================================================================
Recipe Compile Error in /root/chef-solo/cookbooks-3/main/recipes/default.rb

================================================================================

NoMethodError

undefined method `' for nil:NilClass

Cookbook Trace:

/root/chef-solo/cookbooks-2/java/recipes/oracle.rb:38:in from_file' /root/chef-solo/cookbooks-2/java/recipes/default.rb:22:in from_file'
/root/chef-solo/cookbooks-3/main/recipes/default.rb:8:in `from_file'

Relevant File Content:

/root/chef-solo/cookbooks-2/java/recipes/oracle.rb:

31:
32: case node['java']['jdk_version'].to_s
33: when "6"
34: tarball_url = node['java']['jdk']['6'][arch]['url']
35: tarball_checksum = node['java']['jdk']['6'][arch]['checksum']
36: bin_cmds = node['java']['jdk']['6']['bin_cmds']
37: when "7"
38>> tarball_url = node['java']['jdk']['7'][arch]['url']
39: tarball_checksum = node['java']['jdk']['7'][arch]['checksum']
40: bin_cmds = node['java']['jdk']['7']['bin_cmds']
41: end
42:
43: if tarball_url =~ /example.com/
44: Chef::Application.fatal!("You must change the download link to
your private repository. You can no longer download java directly from
http://download.oracle.com without a web broswer")
45: end
46:
47: include_recipe "java::set_java_home"

The error "undefined method `' for nil:NilClass" means that some code has
tried to look up a key - "item['key']" on an item that is set to nil (which
doesn't know how to look up keys. The error doesn't tell us exactly which
item was nil - it technically could be "node", but it almost certainly
isn't, so we can guess that it's one of the chain of attributes - 'java'
-> 'jdk' -> '7' -> 'arch'.

If the 'java' attribute hasn't been set, node['java'] will equal nil. That
means that node['java']['jdk'] would equal nil['jdk'], which would cause
the error you saw. You do set the 'java' attribute in your node file, so
that's probably not the problem here. Let's look at the next one. If
node['java']['jdk'] is unset, that will equal nil - and
node['java']['jdk']['7'] will cause the error.

The java cookbook's attributes file should be setting that attribute, so
why isn't this working? Daniel DeLeo explained this in your previous
thread - most likely, you haven't put "depends 'java'" in your cookbook's
metadata.rb. If you were using chef-client/server, this would cause an
error as soon as you tried to include_recipe "java" - because you're using
chef-solo, the include_recipe succeeds while the cookbook hasn't been
properly loaded.

As Daniel noted, the solution is to add the dependency to your metadata.rb

Checking your code to confirm that, I see a metadata.rb in the root of your
git repository - but that repository doesn't look like a cookbook (I guess
it's your chef-repo for use with "knife solo"). You don't need a
metadata.rb in your chef-repo (it won't do anything), but you do need one
in every cookbook.

I'd suggest installing the Foodcritic tool ( http://www.foodcritic.io/ ),
which you can do with "gem install foodcritic". It checks your Chef code
for a bunch of different things, suggesting improvements for any problems
it finds. Let's check yours:

$ pwd
/Users/zts/code/JenkinsChefRecipe
$ foodcritic cookbooks
FC011: Missing README in markdown format: cookbooks/main/README.md:1
FC031: Cookbook without metadata file: cookbooks/main/metadata.rb:1
FC045: Consider setting cookbook name in metadata:
cookbooks/main/metadata.rb:1

You don't need to fix everything it tells you, but you should make sure you
understand each issue before choosing to ignore it. The main problem it
reports here is that the cookbook "main" didn't have a metadata file.
Let's move the one from the root of your repository into cookbooks/main,
and try again:

$ mv metadata.rb cookbooks/main/
$ foodcritic cookbooks
FC007: Ensure recipe dependencies are reflected in cookbook metadata:
cookbooks/main/recipes/default.rb:8
FC007: Ensure recipe dependencies are reflected in cookbook metadata:
cookbooks/main/recipes/default.rb:16
FC008: Generated cookbook metadata needs updating:
cookbooks/main/metadata.rb:3
FC011: Missing README in markdown format: cookbooks/main/README.md:1

Now we see that your recipe "main::default" depends on two cookbooks that
haven't been declared in your metadata.rb. Fix that up, then the
attributes from the 'java' and 'jenkins' cookbooks will be properly loaded,
and you should get a little bit further. You should also update the other
fields in metadata.rb (especially the cookbook name) to match your cookbook.

Hope that helps,

Zac

3

I just wanted to say that this reply is amazing. The way people in our community step up to help each other is truly delightful.

--
Daniel DeLeo

On Saturday, March 15, 2014 at 4:39 AM, Zac Stevens wrote:

Hi Phil,

On Fri, Mar 14, 2014 at 11:24 PM, phil swenson <phil.swenson@gmail.com (mailto:phil.swenson@gmail.com)> wrote:

I posted about this yesterday, I’m going to post again - but this time with links to code.

Here is the code:

GitHub - pswenson/JenkinsChefRecipe

Code always helps, so thanks for linking that up...

My goal is to be able to install java, oracle DB, jenkins server, install jenkins slaves, create jobs, update jobs with Chef Solo on a system of 15 servers. 1 server will be the jenkins master and the other 14 will be slaves.

I’m starting my recipe with java and jenkins installs.

I’m guessing I’m doing several things wrong:

I ended up configuring the java attributes in nodes/redstone.json as I couldn’t get them to work inside my recipe. It seems to me it would be better to do this in my recipe as this will not be node specific, I want it global.

Here is what happens with the current code (even with include_recipe "java::default”
as suggested by Christopher Armstrong)

Any help, suggestions, critiques would be greatly appreciated. Links to real-life projects would be great, almost everything I see is completely trivial or just doc on attributes for recipes. I’d love to see a real life project I could download and execute.

Good cookbooks typically use the README file to document the attributes you may want to manipulate - that's certainly true for the java coobkook (see https://github.com/socrata-cookbooks/java/blob/master/README.md ). As an example of putting it all together, you might want to look at this cookbook ( GitHub - zts/cooking-with-jenkins: A spike to explore use of Jenkins for Chef cookbook testing ) I wrote to experiment with Jenkins configuration. It hasn't been updated to use the latest version of the Jenkins cookbook, but it will show you how to make use of some community cookbooks to achieve your goal.

With that said, let's look at the problem you're having...

here is the output for running the current code:

philswenson@zoso:~/dev/chef/myfirstcookbook (master)$ knife solo cook root@redstone
WARNING: solo.rb found, but since knife-solo v0.3.0 it is not used any more
WARNING: Please read the upgrade instructions: Upgrading to 0.3.0 · matschaffer/knife-solo Wiki · GitHub
Running Chef on redstone...
Checking Chef version...
Installing Berkshelf cookbooks to '/Users/philswenson/.berkshelf/knife-solo/35c024dd80a9ab482eadf86b98a86fb90c9521d7'...
Using myfirstcookbook (0.1.0)
Using jenkins (2.0.2)
Using git (3.1.0)
Using java (1.20.0)
Using apt (2.3.8)
Using runit (1.5.10)
Using build-essential (2.0.0)
Using yum (3.1.4)
Using yum-epel (0.3.4)
Using dmg (2.2.0)
Using windows (1.30.0)
Using chef_handler (1.1.5)
Uploading the kitchen...
cannot delete non-empty directory: main
WARNING: Local role_path './roles' does not exist
WARNING: Local data_bag_path './data_bags' does not exist
WARNING: Local environment_path './environments' does not exist
Generating solo config...
Running Chef...
Starting Chef Client, version 11.10.4
[2014-03-14T19:07:47-04:00] WARN: unable to detect ip6address
Compiling Cookbooks...
HI PHIL HI PHIL

================================================================================
Recipe Compile Error in /root/chef-solo/cookbooks-3/main/recipes/default.rb

NoMethodError

undefined method `' for nil:NilClass

Cookbook Trace:

/root/chef-solo/cookbooks-2/java/recipes/oracle.rb:38:in from_file' /root/chef-solo/cookbooks-2/java/recipes/default.rb:22:in from_file'
/root/chef-solo/cookbooks-3/main/recipes/default.rb:8:in `from_file'

Relevant File Content:

/root/chef-solo/cookbooks-2/java/recipes/oracle.rb:

31:
32: case node['java']['jdk_version'].to_s
33: when "6"
34: tarball_url = node['java']['jdk']['6'][arch]['url']
35: tarball_checksum = node['java']['jdk']['6'][arch]['checksum']
36: bin_cmds = node['java']['jdk']['6']['bin_cmds']
37: when "7"
38>> tarball_url = node['java']['jdk']['7'][arch]['url']
39: tarball_checksum = node['java']['jdk']['7'][arch]['checksum']
40: bin_cmds = node['java']['jdk']['7']['bin_cmds']
41: end
42:
43: if tarball_url =~ /example.com/ (http://example.com/)
44: Chef::Application.fatal!("You must change the download link to your private repository. You can no longer download java directly from http://download.oracle.com without a web broswer")
45: end
46:
47: include_recipe "java::set_java_home"

The error "undefined method `' for nil:NilClass" means that some code has tried to look up a key - "item['key']" on an item that is set to nil (which doesn't know how to look up keys. The error doesn't tell us exactly which item was nil - it technically could be "node", but it almost certainly isn't, so we can guess that it's one of the chain of attributes - 'java' -> 'jdk' -> '7' -> 'arch'.

If the 'java' attribute hasn't been set, node['java'] will equal nil. That means that node['java']['jdk'] would equal nil['jdk'], which would cause the error you saw. You do set the 'java' attribute in your node file, so that's probably not the problem here. Let's look at the next one. If node['java']['jdk'] is unset, that will equal nil - and node['java']['jdk']['7'] will cause the error.

The java cookbook's attributes file should be setting that attribute, so why isn't this working? Daniel DeLeo explained this in your previous thread - most likely, you haven't put "depends 'java'" in your cookbook's metadata.rb. If you were using chef-client/server, this would cause an error as soon as you tried to include_recipe "java" - because you're using chef-solo, the include_recipe succeeds while the cookbook hasn't been properly loaded.

As Daniel noted, the solution is to add the dependency to your metadata.rb

Checking your code to confirm that, I see a metadata.rb in the root of your git repository - but that repository doesn't look like a cookbook (I guess it's your chef-repo for use with "knife solo"). You don't need a metadata.rb in your chef-repo (it won't do anything), but you do need one in every cookbook.

I'd suggest installing the Foodcritic tool ( http://www.foodcritic.io/ ), which you can do with "gem install foodcritic". It checks your Chef code for a bunch of different things, suggesting improvements for any problems it finds. Let's check yours:

$ pwd
/Users/zts/code/JenkinsChefRecipe
$ foodcritic cookbooks
FC011: Missing README in markdown format: cookbooks/main/README.md:1 (http://README.md:1)
FC031: Cookbook without metadata file: cookbooks/main/metadata.rb:1
FC045: Consider setting cookbook name in metadata: cookbooks/main/metadata.rb:1

You don't need to fix everything it tells you, but you should make sure you understand each issue before choosing to ignore it. The main problem it reports here is that the cookbook "main" didn't have a metadata file. Let's move the one from the root of your repository into cookbooks/main, and try again:

$ mv metadata.rb cookbooks/main/
$ foodcritic cookbooks
FC007: Ensure recipe dependencies are reflected in cookbook metadata: cookbooks/main/recipes/default.rb:8
FC007: Ensure recipe dependencies are reflected in cookbook metadata: cookbooks/main/recipes/default.rb:16
FC008: Generated cookbook metadata needs updating: cookbooks/main/metadata.rb:3
FC011: Missing README in markdown format: cookbooks/main/README.md:1 (http://README.md:1)

Now we see that your recipe "main::default" depends on two cookbooks that haven't been declared in your metadata.rb. Fix that up, then the attributes from the 'java' and 'jenkins' cookbooks will be properly loaded, and you should get a little bit further. You should also update the other fields in metadata.rb (especially the cookbook name) to match your cookbook.

Hope that helps,

Zac

4

Awesome response Zac. While we are in the same subject, we have the following cookbook at RightScale that we use to configure a CI server with Chef for Chef: GitHub - rightscale-cookbooks/rs-cookbooks_ci: Chef cookbook for RightScale's Cookbook CI Platform

—
Kannan.
On Mar 15, 2014, at 8:51 AM, Daniel DeLeo dan@kallistec.com wrote:

I just wanted to say that this reply is amazing. The way people in our community step up to help each other is truly delightful.

--
Daniel DeLeo

On Saturday, March 15, 2014 at 4:39 AM, Zac Stevens wrote:

Hi Phil,

On Fri, Mar 14, 2014 at 11:24 PM, phil swenson <phil.swenson@gmail.com (mailto:phil.swenson@gmail.com)> wrote:

I posted about this yesterday, I’m going to post again - but this time with links to code.

Here is the code:

GitHub - pswenson/JenkinsChefRecipe

Code always helps, so thanks for linking that up...

My goal is to be able to install java, oracle DB, jenkins server, install jenkins slaves, create jobs, update jobs with Chef Solo on a system of 15 servers. 1 server will be the jenkins master and the other 14 will be slaves.

I’m starting my recipe with java and jenkins installs.

I’m guessing I’m doing several things wrong:

I ended up configuring the java attributes in nodes/redstone.json as I couldn’t get them to work inside my recipe. It seems to me it would be better to do this in my recipe as this will not be node specific, I want it global.

Here is what happens with the current code (even with include_recipe "java::default”
as suggested by Christopher Armstrong)

Any help, suggestions, critiques would be greatly appreciated. Links to real-life projects would be great, almost everything I see is completely trivial or just doc on attributes for recipes. I’d love to see a real life project I could download and execute.

Good cookbooks typically use the README file to document the attributes you may want to manipulate - that's certainly true for the java coobkook (see https://github.com/socrata-cookbooks/java/blob/master/README.md ). As an example of putting it all together, you might want to look at this cookbook ( GitHub - zts/cooking-with-jenkins: A spike to explore use of Jenkins for Chef cookbook testing ) I wrote to experiment with Jenkins configuration. It hasn't been updated to use the latest version of the Jenkins cookbook, but it will show you how to make use of some community cookbooks to achieve your goal.

With that said, let's look at the problem you're having...

here is the output for running the current code:

philswenson@zoso:~/dev/chef/myfirstcookbook (master)$ knife solo cook root@redstone
WARNING: solo.rb found, but since knife-solo v0.3.0 it is not used any more
WARNING: Please read the upgrade instructions: Upgrading to 0.3.0 · matschaffer/knife-solo Wiki · GitHub
Running Chef on redstone...
Checking Chef version...
Installing Berkshelf cookbooks to '/Users/philswenson/.berkshelf/knife-solo/35c024dd80a9ab482eadf86b98a86fb90c9521d7'...
Using myfirstcookbook (0.1.0)
Using jenkins (2.0.2)
Using git (3.1.0)
Using java (1.20.0)
Using apt (2.3.8)
Using runit (1.5.10)
Using build-essential (2.0.0)
Using yum (3.1.4)
Using yum-epel (0.3.4)
Using dmg (2.2.0)
Using windows (1.30.0)
Using chef_handler (1.1.5)
Uploading the kitchen...
cannot delete non-empty directory: main
WARNING: Local role_path './roles' does not exist
WARNING: Local data_bag_path './data_bags' does not exist
WARNING: Local environment_path './environments' does not exist
Generating solo config...
Running Chef...
Starting Chef Client, version 11.10.4
[2014-03-14T19:07:47-04:00] WARN: unable to detect ip6address
Compiling Cookbooks...
HI PHIL HI PHIL

================================================================================
Recipe Compile Error in /root/chef-solo/cookbooks-3/main/recipes/default.rb

NoMethodError

undefined method `' for nil:NilClass

Cookbook Trace:

/root/chef-solo/cookbooks-2/java/recipes/oracle.rb:38:in from_file' /root/chef-solo/cookbooks-2/java/recipes/default.rb:22:in from_file'
/root/chef-solo/cookbooks-3/main/recipes/default.rb:8:in `from_file'

Relevant File Content:

/root/chef-solo/cookbooks-2/java/recipes/oracle.rb:

31:
32: case node['java']['jdk_version'].to_s
33: when "6"
34: tarball_url = node['java']['jdk']['6'][arch]['url']
35: tarball_checksum = node['java']['jdk']['6'][arch]['checksum']
36: bin_cmds = node['java']['jdk']['6']['bin_cmds']
37: when "7"
38>> tarball_url = node['java']['jdk']['7'][arch]['url']
39: tarball_checksum = node['java']['jdk']['7'][arch]['checksum']
40: bin_cmds = node['java']['jdk']['7']['bin_cmds']
41: end
42:
43: if tarball_url =~ /example.com/ (http://example.com/)
44: Chef::Application.fatal!("You must change the download link to your private repository. You can no longer download java directly from http://download.oracle.com without a web broswer")
45: end
46:
47: include_recipe "java::set_java_home"

The error "undefined method `' for nil:NilClass" means that some code has tried to look up a key - "item['key']" on an item that is set to nil (which doesn't know how to look up keys. The error doesn't tell us exactly which item was nil - it technically could be "node", but it almost certainly isn't, so we can guess that it's one of the chain of attributes - 'java' -> 'jdk' -> '7' -> 'arch'.

If the 'java' attribute hasn't been set, node['java'] will equal nil. That means that node['java']['jdk'] would equal nil['jdk'], which would cause the error you saw. You do set the 'java' attribute in your node file, so that's probably not the problem here. Let's look at the next one. If node['java']['jdk'] is unset, that will equal nil - and node['java']['jdk']['7'] will cause the error.

The java cookbook's attributes file should be setting that attribute, so why isn't this working? Daniel DeLeo explained this in your previous thread - most likely, you haven't put "depends 'java'" in your cookbook's metadata.rb. If you were using chef-client/server, this would cause an error as soon as you tried to include_recipe "java" - because you're using chef-solo, the include_recipe succeeds while the cookbook hasn't been properly loaded.

As Daniel noted, the solution is to add the dependency to your metadata.rb

Checking your code to confirm that, I see a metadata.rb in the root of your git repository - but that repository doesn't look like a cookbook (I guess it's your chef-repo for use with "knife solo"). You don't need a metadata.rb in your chef-repo (it won't do anything), but you do need one in every cookbook.

I'd suggest installing the Foodcritic tool ( http://www.foodcritic.io/ ), which you can do with "gem install foodcritic". It checks your Chef code for a bunch of different things, suggesting improvements for any problems it finds. Let's check yours:

$ pwd
/Users/zts/code/JenkinsChefRecipe
$ foodcritic cookbooks
FC011: Missing README in markdown format: cookbooks/main/README.md:1 (http://README.md:1)
FC031: Cookbook without metadata file: cookbooks/main/metadata.rb:1
FC045: Consider setting cookbook name in metadata: cookbooks/main/metadata.rb:1

You don't need to fix everything it tells you, but you should make sure you understand each issue before choosing to ignore it. The main problem it reports here is that the cookbook "main" didn't have a metadata file. Let's move the one from the root of your repository into cookbooks/main, and try again:

$ mv metadata.rb cookbooks/main/
$ foodcritic cookbooks
FC007: Ensure recipe dependencies are reflected in cookbook metadata: cookbooks/main/recipes/default.rb:8
FC007: Ensure recipe dependencies are reflected in cookbook metadata: cookbooks/main/recipes/default.rb:16
FC008: Generated cookbook metadata needs updating: cookbooks/main/metadata.rb:3
FC011: Missing README in markdown format: cookbooks/main/README.md:1 (http://README.md:1)

Now we see that your recipe "main::default" depends on two cookbooks that haven't been declared in your metadata.rb. Fix that up, then the attributes from the 'java' and 'jenkins' cookbooks will be properly loaded, and you should get a little bit further. You should also update the other fields in metadata.rb (especially the cookbook name) to match your cookbook.

Hope that helps,

Zac

5

Moreover this would have been much more clear if Phil used chef-zero
instead of chef-solo, because chef zero is able to apply the same
constraints check of chef server.
We moved in that direction and we are very satisfied in this moment. But I
notice that a lot of people still uses chef-solo and I would be curious to
know why nobody speaks about chef-zero that is really amazing from my point
of view.
Cheers,
Marco
Il 15/mar/2014 12:39 "Zac Stevens" zts@cryptocracy.com ha scritto:

Hi Phil,

On Fri, Mar 14, 2014 at 11:24 PM, phil swenson phil.swenson@gmail.comwrote:

I posted about this yesterday, I'm going to post again - but this time
with links to code.

Here is the code:

GitHub - pswenson/JenkinsChefRecipe

Code always helps, so thanks for linking that up...

My goal is to be able to install java, oracle DB, jenkins server, install
jenkins slaves, create jobs, update jobs with Chef Solo on a system of 15
servers. 1 server will be the jenkins master and the other 14 will be
slaves.

I'm starting my recipe with java and jenkins installs.

I'm guessing I'm doing several things wrong:

I ended up configuring the java attributes in nodes/redstone.json as I
couldn't get them to work inside my recipe. It seems to me it would be
better to do this in my recipe as this will not be node specific, I want it
global.

Here is what happens with the current code (even with include_recipe
"java::default"
as suggested by Christopher Armstrong)

Any help, suggestions, critiques would be greatly appreciated. Links to
real-life projects would be great, almost everything I see is completely
trivial or just doc on attributes for recipes. I'd love to see a real life
project I could download and execute.

Good cookbooks typically use the README file to document the attributes
you may want to manipulate - that's certainly true for the java coobkook
(see https://github.com/socrata-cookbooks/java/blob/master/README.md ).
As an example of putting it all together, you might want to look at this
cookbook ( GitHub - zts/cooking-with-jenkins: A spike to explore use of Jenkins for Chef cookbook testing ) I wrote to
experiment with Jenkins configuration. It hasn't been updated to use the
latest version of the Jenkins cookbook, but it will show you how to make
use of some community cookbooks to achieve your goal.

With that said, let's look at the problem you're having...

here is the output for running the current code:

philswenson@zoso:~/dev/chef/myfirstcookbook (master)$ knife solo cook
root@redstone
WARNING: solo.rb found, but since knife-solo v0.3.0 it is not used any
more
WARNING: Please read the upgrade instructions:
Upgrading to 0.3.0 · matschaffer/knife-solo Wiki · GitHub
Running Chef on redstone...
Checking Chef version...
Installing Berkshelf cookbooks to
'/Users/philswenson/.berkshelf/knife-solo/35c024dd80a9ab482eadf86b98a86fb90c9521d7'...
Using myfirstcookbook (0.1.0)
Using jenkins (2.0.2)
Using git (3.1.0)
Using java (1.20.0)
Using apt (2.3.8)
Using runit (1.5.10)
Using build-essential (2.0.0)
Using yum (3.1.4)
Using yum-epel (0.3.4)
Using dmg (2.2.0)
Using windows (1.30.0)
Using chef_handler (1.1.5)
Uploading the kitchen...
cannot delete non-empty directory: main
WARNING: Local role_path './roles' does not exist
WARNING: Local data_bag_path './data_bags' does not exist
WARNING: Local environment_path './environments' does not exist
Generating solo config...
Running Chef...
Starting Chef Client, version 11.10.4
[2014-03-14T19:07:47-04:00] WARN: unable to detect ip6address
Compiling Cookbooks...
HI PHIL HI PHIL

================================================================================
Recipe Compile Error in
/root/chef-solo/cookbooks-3/main/recipes/default.rb

================================================================================

NoMethodError

undefined method `' for nil:NilClass

Cookbook Trace:

/root/chef-solo/cookbooks-2/java/recipes/oracle.rb:38:in from_file' /root/chef-solo/cookbooks-2/java/recipes/default.rb:22:in from_file'
/root/chef-solo/cookbooks-3/main/recipes/default.rb:8:in `from_file'

Relevant File Content:

/root/chef-solo/cookbooks-2/java/recipes/oracle.rb:

31:
32: case node['java']['jdk_version'].to_s
33: when "6"
34: tarball_url = node['java']['jdk']['6'][arch]['url']
35: tarball_checksum = node['java']['jdk']['6'][arch]['checksum']
36: bin_cmds = node['java']['jdk']['6']['bin_cmds']
37: when "7"
38>> tarball_url = node['java']['jdk']['7'][arch]['url']
39: tarball_checksum = node['java']['jdk']['7'][arch]['checksum']
40: bin_cmds = node['java']['jdk']['7']['bin_cmds']
41: end
42:
43: if tarball_url =~ /example.com/
44: Chef::Application.fatal!("You must change the download link to
your private repository. You can no longer download java directly from
http://download.oracle.com without a web broswer")
45: end
46:
47: include_recipe "java::set_java_home"

The error "undefined method `' for nil:NilClass" means that some code
has tried to look up a key - "item['key']" on an item that is set to nil
(which doesn't know how to look up keys. The error doesn't tell us exactly
which item was nil - it technically could be "node", but it almost
certainly isn't, so we can guess that it's one of the chain of attributes -
'java' -> 'jdk' -> '7' -> 'arch'.

If the 'java' attribute hasn't been set, node['java'] will equal nil.
That means that node['java']['jdk'] would equal nil['jdk'], which would
cause the error you saw. You do set the 'java' attribute in your node
file, so that's probably not the problem here. Let's look at the next one.
If node['java']['jdk'] is unset, that will equal nil - and
node['java']['jdk']['7'] will cause the error.

The java cookbook's attributes file should be setting that attribute, so
why isn't this working? Daniel DeLeo explained this in your previous
thread - most likely, you haven't put "depends 'java'" in your cookbook's
metadata.rb. If you were using chef-client/server, this would cause an
error as soon as you tried to include_recipe "java" - because you're using
chef-solo, the include_recipe succeeds while the cookbook hasn't been
properly loaded.

As Daniel noted, the solution is to add the dependency to your metadata.rb

Checking your code to confirm that, I see a metadata.rb in the root of
your git repository - but that repository doesn't look like a cookbook (I
guess it's your chef-repo for use with "knife solo"). You don't need a
metadata.rb in your chef-repo (it won't do anything), but you do need one
in every cookbook.

I'd suggest installing the Foodcritic tool ( http://www.foodcritic.io/ ),
which you can do with "gem install foodcritic". It checks your Chef code
for a bunch of different things, suggesting improvements for any problems
it finds. Let's check yours:

$ pwd
/Users/zts/code/JenkinsChefRecipe
$ foodcritic cookbooks
FC011: Missing README in markdown format: cookbooks/main/README.md:1
FC031: Cookbook without metadata file: cookbooks/main/metadata.rb:1
FC045: Consider setting cookbook name in metadata:
cookbooks/main/metadata.rb:1

You don't need to fix everything it tells you, but you should make sure
you understand each issue before choosing to ignore it. The main problem
it reports here is that the cookbook "main" didn't have a metadata file.
Let's move the one from the root of your repository into cookbooks/main,
and try again:

$ mv metadata.rb cookbooks/main/
$ foodcritic cookbooks
FC007: Ensure recipe dependencies are reflected in cookbook metadata:
cookbooks/main/recipes/default.rb:8
FC007: Ensure recipe dependencies are reflected in cookbook metadata:
cookbooks/main/recipes/default.rb:16
FC008: Generated cookbook metadata needs updating:
cookbooks/main/metadata.rb:3
FC011: Missing README in markdown format: cookbooks/main/README.md:1

Now we see that your recipe "main::default" depends on two cookbooks that
haven't been declared in your metadata.rb. Fix that up, then the
attributes from the 'java' and 'jenkins' cookbooks will be properly loaded,
and you should get a little bit further. You should also update the other
fields in metadata.rb (especially the cookbook name) to match your cookbook.

Hope that helps,

Zac

6

Great reply Zac! Thanks for taking the time!

I made the suggested change to metadata.rb and now get this error (I think
this becomes a specific question about the java recipe):
gist:c71326962190c0661da7 · GitHub]

I think I'm going to manually install Java for now, and move on the
Jenkins. Thanks for your jenkins example!

I'll definitely have to check out chef zero as well.

On Sat, Mar 15, 2014 at 5:39 AM, Zac Stevens zts@cryptocracy.com wrote:

Hi Phil,

On Fri, Mar 14, 2014 at 11:24 PM, phil swenson phil.swenson@gmail.comwrote:

I posted about this yesterday, I'm going to post again - but this time
with links to code.

Here is the code:

GitHub - pswenson/JenkinsChefRecipe

Code always helps, so thanks for linking that up...

My goal is to be able to install java, oracle DB, jenkins server, install
jenkins slaves, create jobs, update jobs with Chef Solo on a system of 15
servers. 1 server will be the jenkins master and the other 14 will be
slaves.

I'm starting my recipe with java and jenkins installs.

I'm guessing I'm doing several things wrong:

I ended up configuring the java attributes in nodes/redstone.json as I
couldn't get them to work inside my recipe. It seems to me it would be
better to do this in my recipe as this will not be node specific, I want it
global.

Here is what happens with the current code (even with include_recipe
"java::default"
as suggested by Christopher Armstrong)

Any help, suggestions, critiques would be greatly appreciated. Links to
real-life projects would be great, almost everything I see is completely
trivial or just doc on attributes for recipes. I'd love to see a real life
project I could download and execute.

Good cookbooks typically use the README file to document the attributes
you may want to manipulate - that's certainly true for the java coobkook
(see https://github.com/socrata-cookbooks/java/blob/master/README.md ).
As an example of putting it all together, you might want to look at this
cookbook ( GitHub - zts/cooking-with-jenkins: A spike to explore use of Jenkins for Chef cookbook testing ) I wrote to
experiment with Jenkins configuration. It hasn't been updated to use the
latest version of the Jenkins cookbook, but it will show you how to make
use of some community cookbooks to achieve your goal.

With that said, let's look at the problem you're having...

here is the output for running the current code:

philswenson@zoso:~/dev/chef/myfirstcookbook (master)$ knife solo cook
root@redstone
WARNING: solo.rb found, but since knife-solo v0.3.0 it is not used any
more
WARNING: Please read the upgrade instructions:
Upgrading to 0.3.0 · matschaffer/knife-solo Wiki · GitHub
Running Chef on redstone...
Checking Chef version...
Installing Berkshelf cookbooks to
'/Users/philswenson/.berkshelf/knife-solo/35c024dd80a9ab482eadf86b98a86fb90c9521d7'...
Using myfirstcookbook (0.1.0)
Using jenkins (2.0.2)
Using git (3.1.0)
Using java (1.20.0)
Using apt (2.3.8)
Using runit (1.5.10)
Using build-essential (2.0.0)
Using yum (3.1.4)
Using yum-epel (0.3.4)
Using dmg (2.2.0)
Using windows (1.30.0)
Using chef_handler (1.1.5)
Uploading the kitchen...
cannot delete non-empty directory: main
WARNING: Local role_path './roles' does not exist
WARNING: Local data_bag_path './data_bags' does not exist
WARNING: Local environment_path './environments' does not exist
Generating solo config...
Running Chef...
Starting Chef Client, version 11.10.4
[2014-03-14T19:07:47-04:00] WARN: unable to detect ip6address
Compiling Cookbooks...
HI PHIL HI PHIL

================================================================================
Recipe Compile Error in
/root/chef-solo/cookbooks-3/main/recipes/default.rb

================================================================================

NoMethodError

undefined method `' for nil:NilClass

Cookbook Trace:

/root/chef-solo/cookbooks-2/java/recipes/oracle.rb:38:in from_file' /root/chef-solo/cookbooks-2/java/recipes/default.rb:22:in from_file'
/root/chef-solo/cookbooks-3/main/recipes/default.rb:8:in `from_file'

Relevant File Content:

/root/chef-solo/cookbooks-2/java/recipes/oracle.rb:

31:
32: case node['java']['jdk_version'].to_s
33: when "6"
34: tarball_url = node['java']['jdk']['6'][arch]['url']
35: tarball_checksum = node['java']['jdk']['6'][arch]['checksum']
36: bin_cmds = node['java']['jdk']['6']['bin_cmds']
37: when "7"
38>> tarball_url = node['java']['jdk']['7'][arch]['url']
39: tarball_checksum = node['java']['jdk']['7'][arch]['checksum']
40: bin_cmds = node['java']['jdk']['7']['bin_cmds']
41: end
42:
43: if tarball_url =~ /example.com/
44: Chef::Application.fatal!("You must change the download link to
your private repository. You can no longer download java directly from
http://download.oracle.com without a web broswer")
45: end
46:
47: include_recipe "java::set_java_home"

The error "undefined method `' for nil:NilClass" means that some code
has tried to look up a key - "item['key']" on an item that is set to nil
(which doesn't know how to look up keys. The error doesn't tell us exactly
which item was nil - it technically could be "node", but it almost
certainly isn't, so we can guess that it's one of the chain of attributes -
'java' -> 'jdk' -> '7' -> 'arch'.

If the 'java' attribute hasn't been set, node['java'] will equal nil.
That means that node['java']['jdk'] would equal nil['jdk'], which would
cause the error you saw. You do set the 'java' attribute in your node
file, so that's probably not the problem here. Let's look at the next one.
If node['java']['jdk'] is unset, that will equal nil - and
node['java']['jdk']['7'] will cause the error.

The java cookbook's attributes file should be setting that attribute, so
why isn't this working? Daniel DeLeo explained this in your previous
thread - most likely, you haven't put "depends 'java'" in your cookbook's
metadata.rb. If you were using chef-client/server, this would cause an
error as soon as you tried to include_recipe "java" - because you're using
chef-solo, the include_recipe succeeds while the cookbook hasn't been
properly loaded.

As Daniel noted, the solution is to add the dependency to your metadata.rb

Checking your code to confirm that, I see a metadata.rb in the root of
your git repository - but that repository doesn't look like a cookbook (I
guess it's your chef-repo for use with "knife solo"). You don't need a
metadata.rb in your chef-repo (it won't do anything), but you do need one
in every cookbook.

I'd suggest installing the Foodcritic tool ( http://www.foodcritic.io/ ),
which you can do with "gem install foodcritic". It checks your Chef code
for a bunch of different things, suggesting improvements for any problems
it finds. Let's check yours:

$ pwd
/Users/zts/code/JenkinsChefRecipe
$ foodcritic cookbooks
FC011: Missing README in markdown format: cookbooks/main/README.md:1
FC031: Cookbook without metadata file: cookbooks/main/metadata.rb:1
FC045: Consider setting cookbook name in metadata:
cookbooks/main/metadata.rb:1

You don't need to fix everything it tells you, but you should make sure
you understand each issue before choosing to ignore it. The main problem
it reports here is that the cookbook "main" didn't have a metadata file.
Let's move the one from the root of your repository into cookbooks/main,
and try again:

$ mv metadata.rb cookbooks/main/
$ foodcritic cookbooks
FC007: Ensure recipe dependencies are reflected in cookbook metadata:
cookbooks/main/recipes/default.rb:8
FC007: Ensure recipe dependencies are reflected in cookbook metadata:
cookbooks/main/recipes/default.rb:16
FC008: Generated cookbook metadata needs updating:
cookbooks/main/metadata.rb:3
FC011: Missing README in markdown format: cookbooks/main/README.md:1

Now we see that your recipe "main::default" depends on two cookbooks that
haven't been declared in your metadata.rb. Fix that up, then the
attributes from the 'java' and 'jenkins' cookbooks will be properly loaded,
and you should get a little bit further. You should also update the other
fields in metadata.rb (especially the cookbook name) to match your cookbook.

Hope that helps,

Zac