Bootstrapping a Chef Node using a Java


#1

Hi everybody,

I’m developing a web service in Java on a system that needs to provision,
create, remove nodes.

I have this restriction that the web service must be developed in Java. This
web service must:

1 - create nodes on different Cloud environments (EC2, rackspace,
openstack, etc)
2 - Bootstrap these nodes with chef and one or 2 roles

How do you suggest creating this web service? I thought about doing system
calls to knife, but I don’t think this is a good idea. Is there any Java
specific library that is similar to knife, a library that I can provision
cloud nodes (like knife ec2) and bootstrap chef (life knife bootstrap)?

Thank you very much for the help

Daniel Cukier


#2

On Aug 2, 2011, at 3:38 PM, Daniel Cukier wrote:

Hi everybody,

I’m developing a web service in Java on a system that needs to provision, create, remove nodes.

I have this restriction that the web service must be developed in Java. This web service must:

1 - create nodes on different Cloud environments (EC2, rackspace, openstack, etc)
2 - Bootstrap these nodes with chef and one or 2 roles

How do you suggest creating this web service? I thought about doing system calls to knife, but I don’t think this is a good idea. Is there any Java specific library that is similar to knife, a library that I can provision cloud nodes (like knife ec2) and bootstrap chef (life knife bootstrap)?

As far as I know there is no Chef client binding in Java, but you can look at PyChef as an example of how to make one. Some naive Google pointed me at http://stackoverflow.com/questions/3630615/java-cloud-apis as a seemingly decent list of cloud interaction libraries in Java, but I make no claims of knowledge in that field. Even as a primarily Python guy I would still say Fog is basically unbeatable for this kind of task, so I would highly recommend you see if Fog under JRuby is an option. PyChef will probably not work under Jython as-is due to its use ctypes, but that could also be investigated.

–NOah


#3

Daniel,

Take a look at jclouds if you need provisioning in Java. From there, you can
either bake in the client in the AMI or shellout to run knife bootstrap.
Remember that knife bootstraps are nothing more than a shell script in an
ssh session. Take a look at any of the bootstrap templates to get a real
idea. All the client node registration is done by chef-client not knife in
bootstrap scenarios.

However you aren’t going to be able to really reimplement chef in Java and
running the client on JRuby isn’t possible.

You might be able to use jruby and pull in the chef gems for some API
interaction if need be.

In the end, Chef is written in ruby. Whatever awkward requirements are going
to have to accept that.

If that’s a nonstarter you can also look at creating custom Whirr scripts.

On Aug 2, 2011 6:38 PM, “Daniel Cukier” danicuki@gmail.com wrote:

Hi everybody,

I’m developing a web service in Java on a system that needs to provision,
create, remove nodes.

I have this restriction that the web service must be developed in Java.
This web service must:

1 - create nodes on different Cloud environments (EC2, rackspace,
openstack, etc)
2 - Bootstrap these nodes with chef and one or 2 roles

How do you suggest creating this web service? I thought about doing system
calls to knife, but I don’t think this is a good idea. Is there any Java
specific library that is similar to knife, a library that I can provision
cloud nodes (like knife ec2) and bootstrap chef (life knife bootstrap)?

Thank you very much for the help

Daniel Cukier


#4

Thanks John and Noah.

In fact, I did some tests and it won’t be possible to run chef inside the
JVM. My only option is to provision the machine using jcloud and then run
the bootstrap scripts via ssh.

John, where can I find the bootstrap templates you mentioned?

Thanks

On Tue, Aug 2, 2011 at 7:49 PM, John E. Vincent (lusis) <lusis.org+
chef-list@gmail.com> wrote:

Daniel,

Take a look at jclouds if you need provisioning in Java. From there, you
can either bake in the client in the AMI or shellout to run knife bootstrap.
Remember that knife bootstraps are nothing more than a shell script in an
ssh session. Take a look at any of the bootstrap templates to get a real
idea. All the client node registration is done by chef-client not knife in
bootstrap scenarios.

However you aren’t going to be able to really reimplement chef in Java and
running the client on JRuby isn’t possible.

You might be able to use jruby and pull in the chef gems for some API
interaction if need be.

In the end, Chef is written in ruby. Whatever awkward requirements are
going to have to accept that.

If that’s a nonstarter you can also look at creating custom Whirr scripts.

On Aug 2, 2011 6:38 PM, “Daniel Cukier” danicuki@gmail.com wrote:

Hi everybody,

I’m developing a web service in Java on a system that needs to provision,
create, remove nodes.

I have this restriction that the web service must be developed in Java.
This web service must:

1 - create nodes on different Cloud environments (EC2, rackspace,
openstack, etc)
2 - Bootstrap these nodes with chef and one or 2 roles

How do you suggest creating this web service? I thought about doing
system calls to knife, but I don’t think this is a good idea. Is there any
Java specific library that is similar to knife, a library that I can
provision cloud nodes (like knife ec2) and bootstrap chef (life knife
bootstrap)?

Thank you very much for the help

Daniel Cukier


#5

Here you go:

https://github.com/opscode/chef/tree/master/chef/lib/chef/knife/bootstrap

Here’s a custom one I use for Natty:

They really aren’t that hard to write. The main steps are:

  • Get chef client installed
  • Create /etc/chef/client.rb
  • Create a firstboot json file
  • Run chef-client

All you really need in first-boot.json is a run_list (mine are always
a single role plus a custom one-shot bootstrap recipe for a given
role). Chef client does all the work of registering itself with the
chef server.

On Wed, Aug 3, 2011 at 12:46 PM, Daniel Cukier danicuki@gmail.com wrote:

Thanks John and Noah.
In fact, I did some tests and it won’t be possible to run chef inside the
JVM. My only option is to provision the machine using jcloud and then run
the bootstrap scripts via ssh.
John, where can I find the bootstrap templates you mentioned?
Thanks

On Tue, Aug 2, 2011 at 7:49 PM, John E. Vincent (lusis)
lusis.org+chef-list@gmail.com wrote:

Daniel,

Take a look at jclouds if you need provisioning in Java. From there, you
can either bake in the client in the AMI or shellout to run knife bootstrap.
Remember that knife bootstraps are nothing more than a shell script in an
ssh session. Take a look at any of the bootstrap templates to get a real
idea. All the client node registration is done by chef-client not knife in
bootstrap scenarios.

However you aren’t going to be able to really reimplement chef in Java and
running the client on JRuby isn’t possible.

You might be able to use jruby and pull in the chef gems for some API
interaction if need be.

In the end, Chef is written in ruby. Whatever awkward requirements are
going to have to accept that.

If that’s a nonstarter you can also look at creating custom Whirr scripts.

On Aug 2, 2011 6:38 PM, “Daniel Cukier” danicuki@gmail.com wrote:

Hi everybody,

I’m developing a web service in Java on a system that needs to
provision, create, remove nodes.

I have this restriction that the web service must be developed in Java.
This web service must:

1 - create nodes on different Cloud environments (EC2, rackspace,
openstack, etc)
2 - Bootstrap these nodes with chef and one or 2 roles

How do you suggest creating this web service? I thought about doing
system calls to knife, but I don’t think this is a good idea. Is there any
Java specific library that is similar to knife, a library that I can
provision cloud nodes (like knife ec2) and bootstrap chef (life knife
bootstrap)?

Thank you very much for the help

Daniel Cukier


#6

Thanks John,

thats all I needed for now!

Now I’ll go through jcloud (to create nodes) and SSH access using Java (I’ll
try http://www.ganymed.ethz.ch/ssh2/). The rest is child candy!

All the best

Daniel Cukier

On Wed, Aug 3, 2011 at 2:10 PM, John E. Vincent (lusis) <lusis.org+
chef-list@gmail.com> wrote:

Here you go:

https://github.com/opscode/chef/tree/master/chef/lib/chef/knife/bootstrap

Here’s a custom one I use for Natty:

https://gist.github.com/c52bab560b3dcb3402b7

They really aren’t that hard to write. The main steps are:

  • Get chef client installed
  • Create /etc/chef/client.rb
  • Create a firstboot json file
  • Run chef-client

All you really need in first-boot.json is a run_list (mine are always
a single role plus a custom one-shot bootstrap recipe for a given
role). Chef client does all the work of registering itself with the
chef server.

On Wed, Aug 3, 2011 at 12:46 PM, Daniel Cukier danicuki@gmail.com wrote:

Thanks John and Noah.
In fact, I did some tests and it won’t be possible to run chef inside the
JVM. My only option is to provision the machine using jcloud and then run
the bootstrap scripts via ssh.
John, where can I find the bootstrap templates you mentioned?
Thanks

On Tue, Aug 2, 2011 at 7:49 PM, John E. Vincent (lusis)
lusis.org+chef-list@gmail.com wrote:

Daniel,

Take a look at jclouds if you need provisioning in Java. From there, you
can either bake in the client in the AMI or shellout to run knife
bootstrap.

Remember that knife bootstraps are nothing more than a shell script in
an

ssh session. Take a look at any of the bootstrap templates to get a real
idea. All the client node registration is done by chef-client not knife
in

bootstrap scenarios.

However you aren’t going to be able to really reimplement chef in Java
and

running the client on JRuby isn’t possible.

You might be able to use jruby and pull in the chef gems for some API
interaction if need be.

In the end, Chef is written in ruby. Whatever awkward requirements are
going to have to accept that.

If that’s a nonstarter you can also look at creating custom Whirr
scripts.

On Aug 2, 2011 6:38 PM, “Daniel Cukier” danicuki@gmail.com wrote:

Hi everybody,

I’m developing a web service in Java on a system that needs to
provision, create, remove nodes.

I have this restriction that the web service must be developed in
Java.

This web service must:

1 - create nodes on different Cloud environments (EC2, rackspace,
openstack, etc)
2 - Bootstrap these nodes with chef and one or 2 roles

How do you suggest creating this web service? I thought about doing
system calls to knife, but I don’t think this is a good idea. Is there
any

Java specific library that is similar to knife, a library that I can
provision cloud nodes (like knife ec2) and bootstrap chef (life knife
bootstrap)?

Thank you very much for the help

Daniel Cukier


#7

Check with the Jclouds guys on the SSH access. They have something
built in already I think.

Also if you’re using EC2, you can look at sticking some of the
information into user-data (just be aware of the security risks should
the box get compromised)

On Wed, Aug 3, 2011 at 1:14 PM, Daniel Cukier danicuki@gmail.com wrote:

Thanks John,
thats all I needed for now!
Now I’ll go through jcloud (to create nodes) and SSH access using Java (I’ll
try http://www.ganymed.ethz.ch/ssh2/). The rest is child candy!

All the best
Daniel Cukier
On Wed, Aug 3, 2011 at 2:10 PM, John E. Vincent (lusis)
lusis.org+chef-list@gmail.com wrote:

Here you go:

https://github.com/opscode/chef/tree/master/chef/lib/chef/knife/bootstrap

Here’s a custom one I use for Natty:

https://gist.github.com/c52bab560b3dcb3402b7

They really aren’t that hard to write. The main steps are:

  • Get chef client installed
  • Create /etc/chef/client.rb
  • Create a firstboot json file
  • Run chef-client

All you really need in first-boot.json is a run_list (mine are always
a single role plus a custom one-shot bootstrap recipe for a given
role). Chef client does all the work of registering itself with the
chef server.

On Wed, Aug 3, 2011 at 12:46 PM, Daniel Cukier danicuki@gmail.com wrote:

Thanks John and Noah.
In fact, I did some tests and it won’t be possible to run chef inside
the
JVM. My only option is to provision the machine using jcloud and then
run
the bootstrap scripts via ssh.
John, where can I find the bootstrap templates you mentioned?
Thanks

On Tue, Aug 2, 2011 at 7:49 PM, John E. Vincent (lusis)
lusis.org+chef-list@gmail.com wrote:

Daniel,

Take a look at jclouds if you need provisioning in Java. From there,
you
can either bake in the client in the AMI or shellout to run knife
bootstrap.
Remember that knife bootstraps are nothing more than a shell script in
an
ssh session. Take a look at any of the bootstrap templates to get a
real
idea. All the client node registration is done by chef-client not knife
in
bootstrap scenarios.

However you aren’t going to be able to really reimplement chef in Java
and
running the client on JRuby isn’t possible.

You might be able to use jruby and pull in the chef gems for some API
interaction if need be.

In the end, Chef is written in ruby. Whatever awkward requirements are
going to have to accept that.

If that’s a nonstarter you can also look at creating custom Whirr
scripts.

On Aug 2, 2011 6:38 PM, “Daniel Cukier” danicuki@gmail.com wrote:

Hi everybody,

I’m developing a web service in Java on a system that needs to
provision, create, remove nodes.

I have this restriction that the web service must be developed in
Java.
This web service must:

1 - create nodes on different Cloud environments (EC2, rackspace,
openstack, etc)
2 - Bootstrap these nodes with chef and one or 2 roles

How do you suggest creating this web service? I thought about doing
system calls to knife, but I don’t think this is a good idea. Is
there any
Java specific library that is similar to knife, a library that I can
provision cloud nodes (like knife ec2) and bootstrap chef (life knife
bootstrap)?

Thank you very much for the help

Daniel Cukier