5/14 Opscode Code Review


#1

It’s ChefConf week!

If you’re in the Bay Area, there’s a Chef Meetup tonight hosted by
James Sulinski:

Bryan Berry is organizing a hackday tomorrow at the hotel at 1pm in
the Irvine room:
http://lists.opscode.com/sympa/arc/chef/2012-04/msg00452.html

If you haven’t checked out Marc Paradise’s email about WhyRun, sit
down with a nice cup of tea and give it a read. We’re pretty excited
about where that’s going:
http://lists.opscode.com/sympa/arc/chef-dev/2012-05/msg00036.html

CHEF-3045: We lazy load a number of objects, like cookbook files and
templates. Access to these objects can time out if they’re store
somewhere like S3. Should we download these objects right away by
default? For long Chef runs this has the benefit of moving a number of
potential failures to the beginning of the run. Thoughts?

To merge:
CHEF-2389 - link provider is not idempotent when used with relative link (to)
CHEF-2627 - Knife SSH should return exit code based on whether or
not ssh command is successful or not
CHEF-3088 - Execute resource should accept command Arrays
CHEF-3092 - knife cookbook upload -a should batch uploads

Other:
CHEF-1735 - FreeBSD service provider cannot determine rc variable
name properly
Reopened - Needs a regression test
CHEF-2731 - knife cookbook install should have an option to use current branch
Reviewed - Needs further review, will re-review at the next meeting.
CHEF-3045 - Chef errors out with 403 when retrieving cookbook_file,
template resources on a very long Chef run
Reviewed - Should lazy loading be off by default?
CHEF-3085 - Make knife ssh cssh platform agnostic
Reopened - Use shellout, provide backward compatibility


Bryan McLellan | opscode | technical program manager, open source
© 206.607.7108 | (t) @btmspox | (b) http://blog.loftninjas.org


#2

On May 14, 2012, at 7:38 PM, Bryan McLellan wrote:

CHEF-3045: We lazy load a number of objects, like cookbook files and
templates. Access to these objects can time out if they’re store
somewhere like S3. Should we download these objects right away by
default? For long Chef runs this has the benefit of moving a number of
potential failures to the beginning of the run. Thoughts?

Is it possible to keep the default set the way it is, but to allow those who need it to switch the default to optimistic loading? If not, can we have a method of specifying which object(s) might need to be loaded early?

It seems to me that the current default is almost always the right choice, but as you have observed there are times when it may not work out so well. I’m just trying to help identify relatively easy ways where we can have our cake and eat it too. :wink:


Brad Knowles brad@shub-internet.org
LinkedIn Profile: http://tinyurl.com/y8kpxu


#3

On Monday, May 14, 2012 at 7:55 PM, Brad Knowles wrote:

On May 14, 2012, at 7:38 PM, Bryan McLellan wrote:

CHEF-3045: We lazy load a number of objects, like cookbook files and
templates. Access to these objects can time out if they’re store
somewhere like S3. Should we download these objects right away by
default? For long Chef runs this has the benefit of moving a number of
potential failures to the beginning of the run. Thoughts?

Is it possible to keep the default set the way it is, but to allow those who need it to switch the default to optimistic loading? If not, can we have a method of specifying which object(s) might need to be loaded early?

It seems to me that the current default is almost always the right choice, but as you have observed there are times when it may not work out so well. I’m just trying to help identify relatively easy ways where we can have our cake and eat it too. :wink:
For more background here, the issue is that with Hosted Chef, we distribute signed, time-limited URLs with an expiration of 60m to the client when computing its cookbook list. Andrea Campi did some work to make S3 storage an option for the OSS Chef server, so I believe this issue can also occur with OSS Chef server if you’re using that feature (I haven’t looked at that code too much outside of initial review).

If your client run takes longer than 60m, and you have a template or cookbook file resource that attempts to lazy load a file from the cookbook after that time, you get an error. Making eager loading of cookbook files and templates an option sort of solves the issue, in that you can configure eager loading after you’ve been bitten by this issue and found out that you need it, but this is a pretty bad user experience compared to never seeing the issue in the first place. I’ve also noticed that people new to chef seem to be confused by the lazy loading behavior (why is the template not transferred to the client?).

That said, I know there are some people relying on lazy loading of these files to avoid transferring lots of filespecificity-specific large binaries. So we’re looking for feedback to see if changing the default behavior would be a net win or loss for the overall user base.

It would also be possible to implement a URL refresh to solve the problem, but this would require some changes to Hosted Chef, so it would be unlikely we could implement it soon, given the amount of work we need to do on the Erlang port, why run, and reporting features.


Brad Knowles <brad@shub-internet.org (mailto:brad@shub-internet.org)>
LinkedIn Profile: http://tinyurl.com/y8kpxu


Dan DeLeo