Is there a way to get a cookbook with several very large files to
upload via knife without this happening:
Uploading testcookbook [0.5.0]
<-----Pauses here for about 30 mins while the cookbook uploads 90MB
ERROR: Failed to upload #Chef::CookbookVersion:0x10178fdf0 : 401 Unauthorized
{“error”:[“Failed to authenticate. Please synchronize the clock on
your client”]}
/Library/Ruby/Gems/1.8/gems/rest-client-1.6.1/lib/restclient/abstract_response.rb:48:in
return!': 401 Unauthorized (RestClient::Request::Unauthorized) from /Library/Ruby/Gems/1.8/gems/rest-client-1.6.1/lib/restclient/request.rb:220:inprocess_result’
from /Library/Ruby/Gems/1.8/gems/rest-client-1.6.1/lib/restclient/request.rb:169:in
transmit' from /System/Library/Frameworks/Ruby.framework/Versions/1.8/usr/lib/ruby/1.8/net/http.rb:543:instart’
from /Library/Ruby/Gems/1.8/gems/rest-client-1.6.1/lib/restclient/request.rb:166:in
transmit' from /Library/Ruby/Gems/1.8/gems/rest-client-1.6.1/lib/restclient/request.rb:60:inexecute’
from /Library/Ruby/Gems/1.8/gems/rest-client-1.6.1/lib/restclient/request.rb:31:in
execute' from /Library/Ruby/Gems/1.8/gems/rest-client-1.6.1/lib/restclient/resource.rb:72:input’
from /Library/Ruby/Gems/1.8/gems/chef-0.10.8/lib/chef/cookbook_uploader.rb:134:in
uploader_function_for' from /Library/Ruby/Gems/1.8/gems/chef-0.10.8/lib/chef/cookbook_uploader.rb:25:incall’
from /Library/Ruby/Gems/1.8/gems/chef-0.10.8/lib/chef/cookbook_uploader.rb:25:in
setup_worker_threads' from /Library/Ruby/Gems/1.8/gems/chef-0.10.8/lib/chef/cookbook_uploader.rb:24:inloop’
from /Library/Ruby/Gems/1.8/gems/chef-0.10.8/lib/chef/cookbook_uploader.rb:24:in
setup_worker_threads' from /Library/Ruby/Gems/1.8/gems/chef-0.10.8/lib/chef/cookbook_uploader.rb:23:ininitialize’
from /Library/Ruby/Gems/1.8/gems/chef-0.10.8/lib/chef/cookbook_uploader.rb:23:in
new' from /Library/Ruby/Gems/1.8/gems/chef-0.10.8/lib/chef/cookbook_uploader.rb:23:insetup_worker_threads’
from /System/Library/Frameworks/Ruby.framework/Versions/1.8/usr/lib/ruby/1.8/rubygems/custom_require.rb:31:in
map' from /Library/Ruby/Gems/1.8/gems/chef-0.10.8/lib/chef/cookbook_uploader.rb:22:ineach’
from /Library/Ruby/Gems/1.8/gems/chef-0.10.8/lib/chef/cookbook_uploader.rb:22:in
map' from /Library/Ruby/Gems/1.8/gems/chef-0.10.8/lib/chef/cookbook_uploader.rb:22:insetup_worker_threads’
from /Library/Ruby/Gems/1.8/gems/chef-0.10.8/lib/chef/cookbook_uploader.rb:69:in
upload_cookbook' from /Library/Ruby/Gems/1.8/gems/chef-0.10.8/lib/chef/knife/cookbook_upload.rb:170:inupload’
from /Library/Ruby/Gems/1.8/gems/chef-0.10.8/lib/chef/knife/cookbook_upload.rb:103:in
run' from /Library/Ruby/Gems/1.8/gems/chef-0.10.8/lib/chef/knife/cookbook_upload.rb:94:ineach’
from /Library/Ruby/Gems/1.8/gems/chef-0.10.8/lib/chef/knife/cookbook_upload.rb:94:in
run' from /Library/Ruby/Gems/1.8/gems/chef-0.10.8/lib/chef/knife.rb:391:inrun_with_pretty_exceptions’
from /Library/Ruby/Gems/1.8/gems/chef-0.10.8/lib/chef/knife.rb:166:in run' from /Library/Ruby/Gems/1.8/gems/chef-0.10.8/lib/chef/application/knife.rb:128:inrun’
from /Library/Ruby/Gems/1.8/gems/chef-0.10.8/bin/knife:25
from /usr/bin/knife:19:in `load’
from /usr/bin/knife:19
Ohai,
On Fri, Mar 23, 2012 at 6:54 PM, Jason J. W. Williams
jasonjwwilliams@gmail.com wrote:
Is there a way to get a cookbook with several very large files to
upload via knife without this happening:
Uploading testcookbook [0.5.0]
<-----Pauses here for about 30 mins while the cookbook uploads 90MB
ERROR: Failed to upload #Chef::CookbookVersion:0x10178fdf0 : 401 Unauthorized
{"error":["Failed to authenticate. Please synchronize the clock on
your client"]}
/Library/Ruby/Gems/1.8/gems/rest-client-1.6.1/lib/restclient/abstract_response.rb:48:in
`return!': 401 Unauthorized (RestClient::Request::Unauthorized)
Why are you putting several very large files in Chef? Surely there are
more efficient ways to manage these files? It is likely that such
files would take a long time to download when required by the client
and likewise time out.
--
Opscode, Inc
Joshua Timberman, Technical Program Manager
IRC, Skype, Twitter, Github: jtimberman
I think timberman was trying to direct you to something like this. For anything over a one or two meg I find it works much better.
http://wiki.opscode.com/display/chef/Resources#Resources-RemoteFile
Transfer a file only if the remote source has changed (uses http_request resource)
remote_file "/tmp/couch.png" do
source "http://couchdb.apache.org/img/sketch.png"
action :nothing
end
http_request "HEAD #{http://couchdb.apache.org/img/sketch.png}" do
message ""
url http://couchdb.apache.org/img/sketch.png
action :head
if File.exists?("/tmp/couch.png")
headers "If-Modified-Since" => File.mtime("/tmp/couch.png").httpdate
end
notifies :create, resources(:remote_file => "/tmp/couch.png"), :immediately
end
Joshua
On Mar 23, 2012, at 7:01 PM, Joshua Timberman joshua@opscode.com wrote:
Ohai,
On Fri, Mar 23, 2012 at 6:54 PM, Jason J. W. Williams
jasonjwwilliams@gmail.com wrote:
Is there a way to get a cookbook with several very large files to
upload via knife without this happening:
Uploading testcookbook [0.5.0]
<-----Pauses here for about 30 mins while the cookbook uploads 90MB
ERROR: Failed to upload #Chef::CookbookVersion:0x10178fdf0 : 401 Unauthorized
{"error":["Failed to authenticate. Please synchronize the clock on
your client"]}
/Library/Ruby/Gems/1.8/gems/rest-client-1.6.1/lib/restclient/abstract_response.rb:48:in
`return!': 401 Unauthorized (RestClient::Request::Unauthorized)
Why are you putting several very large files in Chef? Surely there are
more efficient ways to manage these files? It is likely that such
files would take a long time to download when required by the client
and likewise time out.
--
Opscode, Inc
Joshua Timberman, Technical Program Manager
IRC, Skype, Twitter, Github: jtimberman
Hi all, I am on debian
doing:
package “foo” do
action :install
end
works fine in most cases.
Some packages however (I read somewhere there are 8 of them?) seem to be critical/core packages and trying to uninstall them mean that the normal way of installing under the covers:
apt-get -y
just doesnt work. upstart is such a package as it uninstalls sysvinit.
Is there a standard way of automatically installing this using chef?
If I had to do a commandline version I would do this:
yes “Yes, do as I say!” | apt-get -y --force-yes upstart
But then this deviates somewhat from the nicely distribution-agnostic package chef dsl
Thanks
You could wrap the "options" attribute in a case statement?
http://wiki.opscode.com/display/chef/Resources#Resources-Package
Install Package with options
package "debian-archive-keyring" do
action :install
options "--force-yes"
end
On Mar 24, 2012, at 2:13 PM, Geoff Meakin geoffmeakin@aciddevelopments.co.uk wrote:
Hi all, I am on debian
doing:
package "foo" do
action :install
end
works fine in most cases.
Some packages however (I read somewhere there are 8 of them?) seem to be critical/core packages and trying to uninstall them mean that the normal way of installing under the covers:
apt-get -y
just doesnt work. upstart is such a package as it uninstalls sysvinit.
Is there a standard way of automatically installing this using chef?
If I had to do a commandline version I would do this:
yes "Yes, do as I say!" | apt-get -y --force-yes upstart
But then this deviates somewhat from the nicely distribution-agnostic package chef dsl
Thanks
It still demands interactivity from STDIN that the particular phrase is typed: "Yes, do as I say!" (It's a weird apt thing for packages marked essential)
I guess I'll have to use a bash resource instead, I just wondered if package could do it to make it distro independent.
Cheers
Geoff
On 24 Mar 2012, at 21:33, Joshua Miller wrote:
You could wrap the "options" attribute in a case statement?
http://wiki.opscode.com/display/chef/Resources#Resources-Package
Install Package with options
package "debian-archive-keyring" do
action :install
options "--force-yes"
end
On Mar 24, 2012, at 2:13 PM, Geoff Meakin geoffmeakin@aciddevelopments.co.uk wrote:
Hi all, I am on debian
doing:
package "foo" do
action :install
end
works fine in most cases.
Some packages however (I read somewhere there are 8 of them?) seem to be critical/core packages and trying to uninstall them mean that the normal way of installing under the covers:
apt-get -y
just doesnt work. upstart is such a package as it uninstalls sysvinit.
Is there a standard way of automatically installing this using chef?
If I had to do a commandline version I would do this:
yes "Yes, do as I say!" | apt-get -y --force-yes upstart
But then this deviates somewhat from the nicely distribution-agnostic package chef dsl
Thanks
The -y option should take care of that I believe.
options "--force-yes -Y"
On Mar 25, 2012, at 2:34 AM, Geoff Meakin geoffmeakin@aciddevelopments.co.uk wrote:
It still demands interactivity from STDIN that the particular phrase is typed: "Yes, do as I say!" (It's a weird apt thing for packages marked essential)
I guess I'll have to use a bash resource instead, I just wondered if package could do it to make it distro independent.
Cheers
Geoff
On 24 Mar 2012, at 21:33, Joshua Miller wrote:
You could wrap the "options" attribute in a case statement?
http://wiki.opscode.com/display/chef/Resources#Resources-Package
Install Package with options
package "debian-archive-keyring" do
action :install
options "--force-yes"
end
On Mar 24, 2012, at 2:13 PM, Geoff Meakin geoffmeakin@aciddevelopments.co.uk wrote:
Hi all, I am on debian
doing:
package "foo" do
action :install
end
works fine in most cases.
Some packages however (I read somewhere there are 8 of them?) seem to be critical/core packages and trying to uninstall them mean that the normal way of installing under the covers:
apt-get -y
just doesnt work. upstart is such a package as it uninstalls sysvinit.
Is there a standard way of automatically installing this using chef?
If I had to do a commandline version I would do this:
yes "Yes, do as I say!" | apt-get -y --force-yes upstart
But then this deviates somewhat from the nicely distribution-agnostic package chef dsl
Thanks
Hi,
From the page you just sent -->
-y, --yes, --assume-yes
Automatic yes to prompts. Assume "yes" as answer to all prompts and run non-interactively. If an undesirable situation, such as changing a held package or removing an essential package, occurs then apt-get will abort.
As you can see apt-get aborts and doesnt do the work on an "essential" package.
Cheers
Geoff
On 25 Mar 2012, at 14:49, Joshua Miller wrote:
The -y option should take care of that I believe.
http://linux.die.net/man/8/apt-get
options "--force-yes -Y"
On Mar 25, 2012, at 2:34 AM, Geoff Meakin geoffmeakin@aciddevelopments.co.uk wrote:
It still demands interactivity from STDIN that the particular phrase is typed: "Yes, do as I say!" (It's a weird apt thing for packages marked essential)
I guess I'll have to use a bash resource instead, I just wondered if package could do it to make it distro independent.
Cheers
Geoff
On 24 Mar 2012, at 21:33, Joshua Miller wrote:
You could wrap the "options" attribute in a case statement?
http://wiki.opscode.com/display/chef/Resources#Resources-Package
Install Package with options
package "debian-archive-keyring" do
action :install
options "--force-yes"
end
On Mar 24, 2012, at 2:13 PM, Geoff Meakin geoffmeakin@aciddevelopments.co.uk wrote:
Hi all, I am on debian
doing:
package "foo" do
action :install
end
works fine in most cases.
Some packages however (I read somewhere there are 8 of them?) seem to be critical/core packages and trying to uninstall them mean that the normal way of installing under the covers:
apt-get -y
just doesnt work. upstart is such a package as it uninstalls sysvinit.
Is there a standard way of automatically installing this using chef?
If I had to do a commandline version I would do this:
yes "Yes, do as I say!" | apt-get -y --force-yes upstart
But then this deviates somewhat from the nicely distribution-agnostic package chef dsl
Thanks
Why are you putting several very large files in Chef? Surely there are
more efficient ways to manage these files? It is likely that such
files would take a long time to download when required by the client
and likewise time out.
Because if we're going to have to run our own mirrors of the files,
we'd just as well have it in Chef rather than manage an internal
mirror that's only used by Chef anyway.
-J
On Sat, Mar 24, 2012 at 7:48 AM, Joshua Miller jassinpain@gmail.com wrote:
I think timberman was trying to direct you to something like this. For
anything over a one or two meg I find it works much better.
http://wiki.opscode.com/display/chef/Resources#Resources-RemoteFile
Yeah, we use that already quite a lot for packages we don't need to mirror
ourselves. These files need to be mirrored by us.
-J
On 27 March 2012 10:26, Jason J. W. Williams jasonjwwilliams@gmail.com wrote:
Why are you putting several very large files in Chef? Surely there are
more efficient ways to manage these files? It is likely that such
files would take a long time to download when required by the client
and likewise time out.
Because if we're going to have to run our own mirrors of the files,
we'd just as well have it in Chef rather than manage an internal
mirror that's only used by Chef anyway.
This is a false assertion.
The chef-server file server is not just a 'mirror of files'. You
should explicitly avoid loading and streaming large files with the
Signed Header authentication, as many cycles will be used to hash HTTP
headers and body (in my experience).
In your position, I would consider distributing these large files via
BitTorrent or HTTP, potentially from CDN-backed Object Storage such as
S3 depending on geographic proximity.
Warm Regards,
--AJ
-J
In your position, I would consider distributing these large files via
BitTorrent or HTTP, potentially from CDN-backed Object Storage such as
S3 depending on geographic proximity.
Thanks AJ. Since we don't have a choice given the current behavior
that's what we're converting to for this.
-J
