RE: Re: RE: Re: docker cookbook and idempotency

You are right, I am also using docker_containter resources. Didn’t want to ask about that because it’s basically the same question to begin with. I am having another problem with docker_container; when using linking, sometimes the link target is run only after the link source. That problem is exacerbated by using :delayed, of course, where you have even less control over the sequence of things.

Funny you suggest :immediate . I had the same thought you had, but in the end had to switch back from :immediate to :delayed. The problem with :immediate was that when I build a docker context from multiple individual resources, the notification was sent from the first resource, before the context was fully built or updated.

template “Dockerfile” do

notifies :build, “docker_image[myimage]”, :immediate

end

template “othercontextfile” do

notifies :build, “docker_image[myimage]”, :immediate

end

docker_image “myimage” do

action :build

end

Kevin Keane

The NetTech

http://www.4nettech.com

Our values: Privacy, Liberty, Justice

See https://www.4nettech.com/corp/the-nettech-values.html

-----Original message-----
From: Sean OMeara someara@chef.io
Sent: Tuesday 1st September 2015 13:22
To: chef@lists.opscode.com
Subject: [chef] Re: RE: Re: docker cookbook and idempotency

PS: you probably also want to use the :immediately notification timing.

I’m just going out on a limb here and assuming you’ll have a docker_container resource in your recipe coming up soon.

The default is :delayed, which will wait until the end of the chef-client run.

remote_directory “/tmp/mydockercontext” do

notifies :build, “docker_image[myimage]”, :immediately

end

docker_image “myimage” do

action :build_if_missing

end

docker_container ‘my_thang’ do

repo ‘myimage’

end

On Tue, Sep 1, 2015 at 4:25 PM, Kevin Keane Subscription <subscription@kkeane.com mailto:subscription@kkeane.com > wrote:

Yeah, I kinda thought that it would be difficult to implement idempotency.

I’m not sure the build_if_missing would work, because of resource ordering. If I understand you correctly, you are suggesting something like this:

docker_image "myimage do

action :build_if_missing

end

remote_directory “/tmp/mydockercontext” do

notifies :build, “docker_image[myimage]”

end

But with this order, the docker image would be built before the context is set up.

Reversing the order of the resources would break the notification, because you cannot notify resources that come later in the recipe. Or am I mistaken about that?

Kevin Keane

The NetTech

http://www.4nettech.com http://www.4nettech.com

Our values: Privacy, Liberty, Justice

See https://www.4nettech.com/corp/the-nettech-values.html

-----Original message-----
From: Sean OMeara <someara@chef.io mailto:someara@chef.io >
Sent: Tuesday 1st September 2015 1:09
To: chef@lists.opscode.com mailto:chef@lists.opscode.com
Subject: [chef] Re: docker cookbook and idempotency

Hi Kevin

Unfortunately, :build will perform work on every chef-client run. There isn’t really a way to “check” to see if the image contents are out of date given the inputs.

I’d have your build context notify a docker_image with the :build_if_missing action set.

-s

On Tue, Sep 1, 2015 at 12:21 AM, Kevin Keane Subscription <subscription@kkeane.com mailto:subscription@kkeane.com > wrote:

I love the docker cookbook - thanks to those who created and maintain it!

That said, there are a few things I’m trying to understand better. Specifically, how does idempotency work with the docker_image :build action? Does this rebuild the image on every chef run, or only when the build context has changed?

Currently, I am using a notification to rebuild the image when the build context has changed. But this is unsatisfactory because when the context is already up to date but the image is outdated or missing, it will not trigger a rebuild.

How do I solve this?

Kevin Keane

The NetTech

http://www.4nettech.com http://www.4nettech.com

Our values: Privacy, Liberty, Justice

See https://www.4nettech.com/corp/the-nettech-values.html

You still have full control over the sequencing... you just need to be
aware that they're queued up and will run at the end of the chef-client run.

It sounds like you're going to end up with a healthy amount of notification
/ subscription chaining.

context-1 notifies -> image
context-2 notifies -> image
context-3 notifies -> image

image notifies -> link_source

link_source notifies -> link_target_1
link_source notifies -> link_target_2
link_source notifies -> link_target_3

Flipping it around to "subscribes" might make more sense if these get
too unwieldy.

-s

On Wed, Sep 2, 2015 at 1:39 AM, Kevin Keane Subscription <
subscription@kkeane.com> wrote:

You are right, I am also using docker_containter resources. Didn't want
to ask about that because it's basically the same question to begin with. I
am having another problem with docker_container; when using linking,
sometimes the link target is run only after the link source. That problem
is exacerbated by using :delayed, of course, where you have even less
control over the sequence of things.

Funny you suggest :immediate . I had the same thought you had, but in the
end had to switch back from :immediate to :delayed. The problem with
:immediate was that when I build a docker context from multiple individual
resources, the notification was sent from the first resource, before the
context was fully built or updated.

template "Dockerfile" do

...

notifies :build, "docker_image[myimage]", :immediate

end

template "othercontextfile" do

...

notifies :build, "docker_image[myimage]", :immediate

end

docker_image "myimage" do

action :build

...

end

Kevin Keane

The NetTech

http://www.4nettech.com

Our values: Privacy, Liberty, Justice

See https://www.4nettech.com/corp/the-nettech-values.html

-----Original message-----
From: Sean OMeara someara@chef.io
Sent: Tuesday 1st September 2015 13:22
To: chef@lists.opscode.com
Subject: [chef] Re: RE: Re: docker cookbook and idempotency

PS: you probably also want to use the :immediately notification timing.

I'm just going out on a limb here and assuming you'll have a
docker_container resource in your recipe coming up soon.

The default is :delayed, which will wait until the end of the chef-client
run.

remote_directory "/tmp/mydockercontext" do

notifies :build, "docker_image[myimage]", :immediately

end

docker_image "myimage" do

action :build_if_missing

end

docker_container 'my_thang' do

repo 'myimage'

end

On Tue, Sep 1, 2015 at 4:25 PM, Kevin Keane Subscription <
subscription@kkeane.com> wrote:

Yeah, I kinda thought that it would be difficult to implement idempotency.

I'm not sure the build_if_missing would work, because of resource
ordering. If I understand you correctly, you are suggesting something
like this:

docker_image "myimage do

action :build_if_missing

end

remote_directory "/tmp/mydockercontext" do

notifies :build, "docker_image[myimage]"

end

But with this order, the docker image would be built before the context
is set up.

Reversing the order of the resources would break the notification,
because you cannot notify resources that come later in the recipe. Or am I
mistaken about that?

Kevin Keane

The NetTech

http://www.4nettech.com

Our values: Privacy, Liberty, Justice

See https://www.4nettech.com/corp/the-nettech-values.html

-----Original message-----
From: Sean OMeara someara@chef.io
Sent: Tuesday 1st September 2015 1:09
To: chef@lists.opscode.com
Subject: [chef] Re: docker cookbook and idempotency

Hi Kevin

Unfortunately, :build will perform work on every chef-client run. There
isn't really a way to "check" to see if the image contents are out of date
given the inputs.

I'd have your build context notify a docker_image with the
:build_if_missing action set.

-s

On Tue, Sep 1, 2015 at 12:21 AM, Kevin Keane Subscription <
subscription@kkeane.com> wrote:

I love the docker cookbook - thanks to those who created and maintain it!

That said, there are a few things I'm trying to understand better.
Specifically, how does idempotency work with the docker_image :build
action? Does this rebuild the image on every chef run, or only when the
build context has changed?

Currently, I am using a notification to rebuild the image when the
build context has changed. But this is unsatisfactory because when the
context is already up to date but the image is outdated or missing, it will
not trigger a rebuild.

How do I solve this?

Kevin Keane

The NetTech

http://www.4nettech.com

Our values: Privacy, Liberty, Justice

See https://www.4nettech.com/corp/the-nettech-values.html