http_proxy not work for recipe_url?

Chef Infra (archive)
1

I have a solo.rb file with the following in it:

json_attribs "https://u:p@server/path/node.json"
recipe_url "https://u:p@server/path/cookbooks.tar.gz"
http_proxy "http://proxy:3128"
https_proxy “http://proxy:3128”

chef-solo

seems to be using the proxy to fetch the node.json file but it seems to
be trying to connect directly to “server” to fetch the cookbooks.tar.gz
file.

Is this a known issue/limitation of the proxy support in chef? I would
have thought it would be generic for all HTTP activities.

From chef’s debug:

[Tue, 06 Aug 2013 13:56:00 -0400] DEBUG: using proxy:3128 for proxy
[Tue, 06 Aug 2013 13:56:00 -0400] DEBUG: Sending HTTP Request via GET to
server:443/path/node.json
[Tue, 06 Aug 2013 13:56:01 -0400] DEBUG: Streaming download from
https://u:p@server/path/node.json to tempfile
/tmp/chef-rest20130806-23887-14zy0ty-0
[Tue, 06 Aug 2013 13:56:01 -0400] DEBUG: Creating path
/var/tmp/chef-solo to extract recipes into
/usr/lib/ruby/1.8/net/http.rb:560:in `initialize’: Connection timed out

  • connect(2) (Errno::ETIMEDOUT)
    from /usr/lib/ruby/1.8/net/http.rb:560:in open' from /usr/lib/ruby/1.8/net/http.rb:560:inconnect’
    from /usr/lib/ruby/1.8/timeout.rb:53:in timeout' from /usr/lib/ruby/1.8/timeout.rb:101:intimeout’
    from /usr/lib/ruby/1.8/net/http.rb:560:in connect' from /usr/lib/ruby/1.8/net/http.rb:553:indo_start’
    from /usr/lib/ruby/1.8/net/http.rb:542:in start' from /usr/lib/ruby/1.8/open-uri.rb:242:inopen_http’
    from /usr/lib/ruby/1.8/open-uri.rb:616:in buffer_open' from /usr/lib/ruby/1.8/open-uri.rb:164:inopen_loop’
    from /usr/lib/ruby/1.8/open-uri.rb:162:in catch' from /usr/lib/ruby/1.8/open-uri.rb:162:inopen_loop’
    from /usr/lib/ruby/1.8/open-uri.rb:132:in open_uri' from /usr/lib/ruby/1.8/open-uri.rb:518:inopen’
    from /usr/lib/ruby/1.8/open-uri.rb:30:in open' from /usr/lib/ruby/gems/1.8/gems/chef-0.9.14/bin/../lib/chef/application/solo.rb:164:inreconfigure’
    from
    /usr/lib/ruby/gems/1.8/gems/chef-0.9.14/bin/…/lib/chef/application/solo.rb:163:in
    open' from /usr/lib/ruby/gems/1.8/gems/chef-0.9.14/bin/../lib/chef/application/solo.rb:163:inreconfigure’
    from
    /usr/lib/ruby/gems/1.8/gems/chef-0.9.14/bin/…/lib/chef/application.rb:60:in
    run' from /usr/lib/ruby/gems/1.8/gems/chef-0.9.14/bin/chef-solo:25 from /usr/bin/chef-solo:19:inload’
    from /usr/bin/chef-solo:19

Any ideas?

Cheers,
b.

2

On Thursday, September 5, 2013 at 1:39 PM, Brian J. Murrell wrote:

I have a solo.rb file with the following in it:

json_attribs "https://u:p@server/path/node.json"
recipe_url "https://u:p@server/path/cookbooks.tar.gz"
http_proxy "http://proxy:3128"
https_proxy "http://proxy:3128"

chef-solo

seems to be using the proxy to fetch the node.json file but it seems to
be trying to connect directly to "server" to fetch the cookbooks.tar.gz
file.

Is this a known issue/limitation of the proxy support in chef? I would
have thought it would be generic for all HTTP activities.

This feature is kinda hacked in to allow for simple bootstrapping cases, such as the old chef-server install via chef-solo. I'd consider it a bug.

Out of curiosity, why distribute cookbooks this way?

--
Daniel DeLeo

3

On 13-09-06 03:28 PM, Daniel DeLeo wrote:

This feature is kinda hacked in to allow for simple bootstrapping cases, such as the old chef-server install via chef-solo. I'd consider it a bug.

So it was not extended to fetching the cookbooks also, just the .json file?

Out of curiosity, why distribute cookbooks this way?

As opposed to what other way, assuming the cookbooks are sitting on an
HTTP server somewhere. Or maybe they are not. How else would you
suggest I distribute them if I don't want to go the whole chef-server route?

b.

4

On Friday, September 6, 2013 at 2:52 PM, Brian J. Murrell wrote:

On 13-09-06 03:28 PM, Daniel DeLeo wrote:

This feature is kinda hacked in to allow for simple bootstrapping cases, such as the old chef-server install via chef-solo. I'd consider it a bug.

So it was not extended to fetching the cookbooks also, just the .json file?
Yep. The json file gets fetched by Chef::REST so it picked up proxy support for free when that was added, whereas the recipe_url is opened by ruby's openuri, which doesn't support parodying (or relies on ENV vars for configuring them, I haven't looked).

If you're interested in patching, the one tricky bit to note is that openuri supports FTP, so you'd need to make sure that still worked. If you're super motivated, I think that part of the codebase could use some heavy refactoring, but a patch that just solves the problem at hand would be welcome, too.

Out of curiosity, why distribute cookbooks this way?

As opposed to what other way, assuming the cookbooks are sitting on an
HTTP server somewhere. Or maybe they are not. How else would you
suggest I distribute them if I don't want to go the whole chef-server route?

Honestly, I have no idea how chef-solo users distribute their cookbooks, I'd assumed most would use git and run chef-solo with some wrapper to fetch any updates before chef-solo starts.

Also, chef-server is pretty easy to install nowadays, what turns you off about it?

b.

--
Daniel DeLeo