Transactional recipe


#1

All,

I faced with complex relation between resources.
Let’s look at this:

remote_file copies a disk image
execute losetup binds the image with a device
execute mount attaches the device to the filesystem
directory creates folder on the mounted image.

This recipe is not reliable because of an exception in a middle of the
recipe
requires manual intervention to reset environment.

I.e. I should unmount the device because the first resource would write to
mounted .

I could wrap all these resources inside of one ruby_block and use a lot
nested begin/ensure
but then I lose any profit of the chef.

I want to preserve linear structure of the recipe and set rollback
callbacks on successfully executed resources.

Is there any standard way?


Daneel S. Yaitskov


#2

Hi Daneel.

It’s possible that Chef (or any other CM tool) may not be the best choice
in tooling to do this.

That being said, it totally is possible.

You’ll want to create a custom resource type for this, (use the LWRP DSL),
and implement the transactional logic in the provider. Always keep in mind
that Chef is about ensuring state. “Test for state, take action to repair*
IF NEEDED*”

The state you wish to ensure is “A directory exists inside an unmounted
image”.

To pull this off, you’ll need to mount the image, check for the directory,
repair if needed, then clean up after yourself.
You’ll need to do this on every Chef run.

Stating the problem like that, I’d personally choose focus on artifact
generation using classic procedural scripting out of band of Chef.

If you do choose to implement this with a Chef provider, you’d want to
start by thinking about the interface when using the custom resource.
Something like this:

somerecipe.rb

image_directory “example image” do
path '/srv/images/blackbox.img’
directory '/var/log/twiddlybits’
action :create
end

If you want to use the LWRP DSL, the above example will need to be in
cookbook named “image”,
If you’d rather use the full blown class interface (a so called HWRP), you
can ship it in a cookbook with any name.

Either way, you’ll want to implement the transactional logic in the
provider, and you’ll probably want to use mixlib::shellout to handle stdout.

Don’t forget to call new_resource.updated_by_last_action(true) if you have
to make the directory.

Good luck!

-s

On Mon, Nov 25, 2013 at 8:03 AM, Daneel Yaitskov
rtfm.rtfm.rtfm@gmail.comwrote:

All,

I faced with complex relation between resources.
Let’s look at this:

remote_file copies a disk image
execute losetup binds the image with a device
execute mount attaches the device to the filesystem
directory creates folder on the mounted image.

This recipe is not reliable because of an exception in a middle of the
recipe
requires manual intervention to reset environment.

I.e. I should unmount the device because the first resource would write to
mounted .

I could wrap all these resources inside of one ruby_block and use a lot
nested begin/ensure
but then I lose any profit of the chef.

I want to preserve linear structure of the recipe and set rollback
callbacks on successfully executed resources.

Is there any standard way?


Daneel S. Yaitskov


#3

On Monday, November 25, 2013 at 8:47 AM, Sean OMeara wrote:

Hi Daneel.

It’s possible that Chef (or any other CM tool) may not be the best choice in tooling to do this.

That being said, it totally is possible.

You’ll want to create a custom resource type for this, (use the LWRP DSL), and implement the transactional logic in the provider. Always keep in mind that Chef is about ensuring state. “Test for state, take action to repair IF NEEDED”

The state you wish to ensure is “A directory exists inside an unmounted image”.

To pull this off, you’ll need to mount the image, check for the directory, repair if needed, then clean up after yourself.
You’ll need to do this on every Chef run.

Stating the problem like that, I’d personally choose focus on artifact generation using classic procedural scripting out of band of Chef.

If you do choose to implement this with a Chef provider, you’d want to start by thinking about the interface when using the custom resource. Something like this:

somerecipe.rb

image_directory “example image” do
path '/srv/images/blackbox.img’
directory '/var/log/twiddlybits’
action :create
end

If you want to use the LWRP DSL, the above example will need to be in cookbook named “image”,
If you’d rather use the full blown class interface (a so called HWRP), you can ship it in a cookbook with any name.

Either way, you’ll want to implement the transactional logic in the provider, and you’ll probably want to use mixlib::shellout to handle stdout.

Don’t forget to call new_resource.updated_by_last_action(true) if you have to make the directory.

Good luck!

-s

This is all good advice. If it’s not possible for you to do in a reasonable amount of time, you should look at using not_if or only_if guards on your execute resources so that they’ll always do the right thing instead of chaining notifies together.

On Mon, Nov 25, 2013 at 8:03 AM, Daneel Yaitskov <rtfm.rtfm.rtfm@gmail.com (mailto:rtfm.rtfm.rtfm@gmail.com)> wrote:

All,

I faced with complex relation between resources.
Let’s look at this:

remote_file copies a disk image
execute losetup binds the image with a device
execute mount attaches the device to the filesystem
directory creates folder on the mounted image.

This recipe is not reliable because of an exception in a middle of the recipe
requires manual intervention to reset environment.

I.e. I should unmount the device because the first resource would write to mounted .

I could wrap all these resources inside of one ruby_block and use a lot nested begin/ensure
but then I lose any profit of the chef.

I want to preserve linear structure of the recipe and set rollback callbacks on successfully executed resources.

Is there any standard way?


Daneel S. Yaitskov


Daniel DeLeo


#4

Thank you for nice abstraction for the image dir resource.

I wasn’t able to coin it myself. Chef is too new for me. I had to keep a
lot in my head at time.
My minimal goal is to express all in chef somehow.

Chef state based model has one big plus such recipes enforce a developer to
do very adoptable
scrips. And the faw of this approach makes code bloat.

Look at my final solution based on standard resources. It’s a real crap.

disk = vm_cfg.disk.folder + ‘/’ + vm_cfg.disk.name
execute “cp #{ vm_cfg.disk.template } #{ disk }” do
not_if "[ -f #{ disk } ]"
end

execute “virsh shutdown #{ name }” do
only_if "virsh domstate #{ name } | grep -c running"
end

ruby_block “resize disk” do
block do
resize_raw_image(disk, vm_cfg.disk.size_mb)
end
not_if { File.size(disk) == vm_cfg.disk.size_mb * 1024 * 1024 }
end

execute “check loop is free” do
command "losetup -d /dev/loop0"
only_if "losetup -a | grep /dev/loop0"
end

execute “bind image with device” do
command "losetup /dev/loop0 #{ disk }"
not_if "losetup /dev/loop0 | grep #{disk}"
end

execute “clear kernel cache” do
command "partprobe /dev/loop0"
end

directory “/tmp/mount-vm-image”

if previous fail

execute “umount image” do
command "umount /dev/loop0p1"
only_if "df | grep /tmp/mount-vm-image"
end

execute “mount image” do
command "mount -t ext4 /dev/loop0p1 /tmp/mount-vm-image"
end

template “/tmp/mount-vm-image/etc/hostname” do
source "hostname.erb"
variables ({ :hostname => vm_fqdn(node.net, name) })
end

execute “unmount image” do
command "umount /dev/loop0p1"
end

directory “/tmp/mount-vm-image” do
action :delete
end

execute " free loop" do
command "losetup -d /dev/loop0"
end

On Mon, Nov 25, 2013 at 8:57 PM, Daniel DeLeo dan@kallistec.com wrote:

On Monday, November 25, 2013 at 8:47 AM, Sean OMeara wrote:

Hi Daneel.

It’s possible that Chef (or any other CM tool) may not be the best choice
in tooling to do this.

That being said, it totally is possible.

You’ll want to create a custom resource type for this, (use the LWRP DSL),
and implement the transactional logic in the provider. Always keep in mind
that Chef is about ensuring state. “Test for state, take action to repair*
IF NEEDED*”

The state you wish to ensure is “A directory exists inside an unmounted
image”.

To pull this off, you’ll need to mount the image, check for the directory,
repair if needed, then clean up after yourself.
You’ll need to do this on every Chef run.

Stating the problem like that, I’d personally choose focus on artifact
generation using classic procedural scripting out of band of Chef.

If you do choose to implement this with a Chef provider, you’d want to
start by thinking about the interface when using the custom resource.
Something like this:

somerecipe.rb

image_directory “example image” do
path '/srv/images/blackbox.img’
directory '/var/log/twiddlybits’
action :create
end

If you want to use the LWRP DSL, the above example will need to be in
cookbook named “image”,
If you’d rather use the full blown class interface (a so called HWRP), you
can ship it in a cookbook with any name.

Either way, you’ll want to implement the transactional logic in the
provider, and you’ll probably want to use mixlib::shellout to handle stdout.

Don’t forget to call new_resource.updated_by_last_action(true) if you have
to make the directory.

Good luck!

-s

This is all good advice. If it’s not possible for you to do in a
reasonable amount of time, you should look at using not_if or only_if
guards on your execute resources so that they’ll always do the right thing
instead of chaining notifies together.

On Mon, Nov 25, 2013 at 8:03 AM, Daneel Yaitskov <rtfm.rtfm.rtfm@gmail.com

wrote:

All,

I faced with complex relation between resources.
Let’s look at this:

remote_file copies a disk image
execute losetup binds the image with a device
execute mount attaches the device to the filesystem
directory creates folder on the mounted image.

This recipe is not reliable because of an exception in a middle of the
recipe
requires manual intervention to reset environment.

I.e. I should unmount the device because the first resource would write to
mounted .

I could wrap all these resources inside of one ruby_block and use a lot
nested begin/ensure
but then I lose any profit of the chef.

I want to preserve linear structure of the recipe and set rollback
callbacks on successfully executed resources.

Is there any standard way?


Daneel S. Yaitskov


Daniel DeLeo


Daneel S. Yaitskov