Apt package provider: a lot of return code 100 on apt-get

ringods · September 28, 2010, 1:06pm

Hello,

I just bootstrapped a new set of VMs using ‘knife’ (0.9.8). When my
cookbooks are executed, I get errors on the apt-get command of the
package provider. See the following gist for an example:

gist.github.com

https://gist.github.com/ringods/600926

client.log

[Tue, 28 Sep 2010 14:49:46 +0200] INFO: Installing package[build-essential] version 11.4build1
[Tue, 28 Sep 2010 14:49:58 +0200] ERROR: package[build-essential] (/var/cache/chef/cookbooks/build-essential/recipes/default.rb:23:in `from_file') had an error:
apt-get -q -y install build-essential=11.4build1 returned 100, expected 0
/usr/lib/ruby/1.8/chef/mixin/command.rb:184:in `handle_command_failures'
/usr/lib/ruby/1.8/chef/mixin/command.rb:131:in `run_command'
/usr/lib/ruby/1.8/chef/mixin/command.rb:199:in `run_command_with_systems_locale'
/usr/lib/ruby/1.8/chef/provider/package/apt.rb:68:in `install_package'
/usr/lib/ruby/1.8/chef/provider/package.rb:60:in `action_install'
/usr/lib/ruby/1.8/chef/runner.rb:51:in `send'
/usr/lib/ruby/1.8/chef/runner.rb:51:in `run_action'

This file has been truncated. show original

If I execute the apt-get command manually, the command succeeds
without any problem. At the following chef-client run, the cookbook
barfs at the next package. Is there any way to debug why the apt-get
lines fail via Chef, but succeed manually?

Ringo

danryan · September 28, 2010, 4:27pm

Hey all,

Just sending a notice to folks that the Chef bootstrap process doesn’t work on the new EC2 micro instances when running Ubuntu 10.04. It fails on the Java recipe, specifically the openjdk-6-jre-headless package. This is a known bug (see https://bugs.launchpad.net/ubuntu/+source/linux-ec2/+bug/634487/). The same bootstrap runs perfectly on an m1.small instance type. I lost a good day’s work on this, so I’m passing it along so others don’t bang their heads against this, too

-Dan Ryan

AJ_Christensen · September 28, 2010, 5:09pm

Yo,

On 28 September 2010 09:27, Dan Ryan scriptfu@gmail.com wrote:

Hey all,

Just sending a notice to folks that the Chef bootstrap process doesn't work
on the new EC2 micro instances when running Ubuntu 10.04. It fails on the
Java recipe, specifically the openjdk-6-jre-headless package. This is a
known bug (see
Bug #634487 “t1.micro instance hangs when installing java” : Bugs : linux-ec2 package : Ubuntu). The
same bootstrap runs perfectly on an m1.small instance type. I lost a good
day's work on this, so I'm passing it along so others don't bang their heads
against this, too

I'm surprised the t1.micro worked at all. In my limited testing thus far,
the CPU was almost always 99% stolen by hypervisor. Took something like 35
minutes to install emacs.

-Dan Ryan

ringods · September 30, 2010, 11:34am

On 28 September 2010 15:06, Ringo De Smet ringo.desmet@gmail.com wrote:

I just bootstrapped a new set of VMs using 'knife' (0.9.8). When my
cookbooks are executed, I get errors on the apt-get command of the
package provider. See the following gist for an example:

client.log · GitHub

If I execute the apt-get command manually, the command succeeds
without any problem. At the following chef-client run, the cookbook
barfs at the next package. Is there any way to debug why the apt-get
lines fail via Chef, but succeed manually?

I have a perfectly reproducible case. Bootstrapping new VMs have
apt-package install errors just after knife bootstrap. If I restart
chef-client on these nodes, all apt-get works fine.

I looked in the Knife wiki page, but couldn't find how I can get the
chef-client configured in debug logging via knife bootstrap. Since the
problem disappears after chef-client restart, I really want that first
run in debug logging. Any info on how to do this?

Ringo

kallistec · September 30, 2010, 3:19pm

You'll have to modify the bootstrap script. Knife supports templates,
including user-defined ones, for bootstrap scripts. So copy the
bootstrap script from the source:

http://github.com/opscode/chef/tree/master/chef/lib/chef/knife/bootstrap/

to ~/.chef/bootstrap/debug.erb and modify it to run chef-client with
-l debug. Then run knife bootstrap with -d debug

HTH,

Dan DeLeo

On Thu, Sep 30, 2010 at 4:34 AM, Ringo De Smet ringo.desmet@gmail.com wrote:

On 28 September 2010 15:06, Ringo De Smet ringo.desmet@gmail.com wrote:

I just bootstrapped a new set of VMs using 'knife' (0.9.8). When my
cookbooks are executed, I get errors on the apt-get command of the
package provider. See the following gist for an example:

client.log · GitHub

If I execute the apt-get command manually, the command succeeds
without any problem. At the following chef-client run, the cookbook
barfs at the next package. Is there any way to debug why the apt-get
lines fail via Chef, but succeed manually?

I have a perfectly reproducible case. Bootstrapping new VMs have
apt-package install errors just after knife bootstrap. If I restart
chef-client on these nodes, all apt-get works fine.

I looked in the Knife wiki page, but couldn't find how I can get the
chef-client configured in debug logging via knife bootstrap. Since the
problem disappears after chef-client restart, I really want that first
run in debug logging. Any info on how to do this?

Ringo

Bryan_McLellan · September 30, 2010, 4:09pm

On Tue, Sep 28, 2010 at 6:06 AM, Ringo De Smet ringo.desmet@gmail.com wrote:

I just bootstrapped a new set of VMs using 'knife' (0.9.8). When my
cookbooks are executed, I get errors on the apt-get command of the
package provider. See the following gist for an example:

client.log · GitHub

Are you using any repositories that would not be signed by the
upstream keys, such as a local mirror or additional local package
repository?

Bryan

AJ_Christensen · September 30, 2010, 4:35pm

On 30 September 2010 09:09, Bryan McLellan btm@loftninjas.org wrote:

On Tue, Sep 28, 2010 at 6:06 AM, Ringo De Smet ringo.desmet@gmail.com
wrote:

I just bootstrapped a new set of VMs using 'knife' (0.9.8). When my
cookbooks are executed, I get errors on the apt-get command of the
package provider. See the following gist for an example:

client.log · GitHub

Are you using any repositories that would not be signed by the
upstream keys, such as a local mirror or additional local package
repository?

Isn't error code 100 a general underlying-subsystem failure?

Half-configured packages (dpkg --configure -a) or half-fetched APT sources
(apt-get update), as well!

Regards,

AJ

Bryan

Bryan_McLellan · September 30, 2010, 5:15pm

On Thu, Sep 30, 2010 at 9:35 AM, AJ Christensen aj@junglist.gen.nz wrote:

Isn't error code 100 a general underlying-subsystem failure?
Half-configured packages (dpkg --configure -a) or half-fetched APT sources
(apt-get update), as well!

Yeah, a 404 on any sources.list entry will cause apt-get update to return 100.

A key issue will still return 0.

It's common to have apt-get fail on an initial chef run if someone is
modifying sources.list because you have to make sure it happens in the
right order; get everything into the sources.list, get the keys,
trigger an immediate apt-get update execute.

Bryan

ringods · October 1, 2010, 7:44am

Dan,

On 30 September 2010 17:19, Daniel DeLeo dan@kallistec.com wrote:

You'll have to modify the bootstrap script. Knife supports templates,
including user-defined ones, for bootstrap scripts. So copy the
bootstrap script from the source:

http://github.com/opscode/chef/tree/master/chef/lib/chef/knife/bootstrap/

to ~/.chef/bootstrap/debug.erb and modify it to run chef-client with
-l debug. Then run knife bootstrap with -d debug

I did this, but it is still not what I want. Using the -l option, the
knife bootstrap runs in debug mode, but at that point in time,
everything is still OK. At the end of the run, the bootstrap
daemonizes the chef-client, and that first run gives apt-update and
apt-get errors. However, I bootstrap using apt packages and the
/etc/chef/client.rb file has logging set to :info. How can I get the
logging set to :debug in that file at bootstrap time? I tried to set
the log_level via debconf-set-selections (I bootstrap the APT way),
but that isn't set in the initial client.rb file.

Any idea how to set log_level to :debug in the initial /etc/chef/client.rb file?

Ringo

ringods · October 1, 2010, 7:47am

Bryan,

On 30 September 2010 18:09, Bryan McLellan btm@loftninjas.org wrote:

Are you using any repositories that would not be signed by the
upstream keys, such as a local mirror or additional local package
repository?

No. The apt bootstrap process adds apt.opscode.com, but that is OK as
the key is also added. See my response to Dan's suggestion for a more
detailed description of where it fails: the first daemonized
chef-client run just after knife bootstrap!

Ringo

ringods · October 1, 2010, 7:53am

On 1 October 2010 09:44, Ringo De Smet ringo.desmet@gmail.com wrote:

I did this, but it is still not what I want. Using the -l option, the
knife bootstrap runs in debug mode, but at that point in time,
everything is still OK. At the end of the run, the bootstrap
daemonizes the chef-client, and that first run gives apt-update and
apt-get errors. However, I bootstrap using apt packages and the
/etc/chef/client.rb file has logging set to :info. How can I get the
logging set to :debug in that file at bootstrap time? I tried to set
the log_level via debconf-set-selections (I bootstrap the APT way),
but that isn't set in the initial client.rb file.

Here you can find the log file when chef-client is run the first time
in daemonized way:

gist.github.com

https://gist.github.com/ringods/605892

client.log

# Logfile created on Fri Oct 01 09:41:32 +0200 2010 by logger.rb/22285
[Fri, 01 Oct 2010 09:41:32 +0200] INFO: Daemonizing..
[Fri, 01 Oct 2010 09:41:32 +0200] INFO: Forked, in 1545. Priveleges: 0 0
[Fri, 01 Oct 2010 09:41:41 +0200] INFO: Starting Chef Run (Version 0.9.8)
[Fri, 01 Oct 2010 09:41:41 +0200] ERROR: Running exception handlers
[Fri, 01 Oct 2010 09:41:41 +0200] ERROR: Exception handlers complete
[Fri, 01 Oct 2010 09:41:41 +0200] ERROR: Re-raising exception: Chef::Exceptions::Exec - apt-get update returned 100, expected 0
/usr/lib/ruby/1.8/chef/mixin/command.rb:184:in `handle_command_failures'
  /usr/lib/ruby/1.8/chef/mixin/command.rb:131:in `run_command'
  /usr/lib/ruby/1.8/chef/provider/execute.rb:49:in `action_run'

This file has been truncated. show original

Even the "apt-get update" fails already, while chef install via apt
worked flawlessly during the bootstrap process...

Ringo

ringods · October 6, 2010, 11:27am

On 1 October 2010 09:44, Ringo De Smet ringo.desmet@gmail.com wrote:

Dan,

On 30 September 2010 17:19, Daniel DeLeo dan@kallistec.com wrote:

You'll have to modify the bootstrap script. Knife supports templates,
including user-defined ones, for bootstrap scripts. So copy the
bootstrap script from the source:

http://github.com/opscode/chef/tree/master/chef/lib/chef/knife/bootstrap/

to ~/.chef/bootstrap/debug.erb and modify it to run chef-client with
-l debug. Then run knife bootstrap with -d debug

I did this, but it is still not what I want. Using the -l option, the
knife bootstrap runs in debug mode, but at that point in time,
everything is still OK. At the end of the run, the bootstrap
daemonizes the chef-client, and that first run gives apt-update and
apt-get errors. However, I bootstrap using apt packages and the
/etc/chef/client.rb file has logging set to :info. How can I get the
logging set to :debug in that file at bootstrap time? I tried to set
the log_level via debconf-set-selections (I bootstrap the APT way),
but that isn't set in the initial client.rb file.

Any idea how to set log_level to :debug in the initial /etc/chef/client.rb file?

Anyone?

Ringo

Seth_Chisamore · October 7, 2010, 5:15am

Ringo,
Just follow Dan's instructions above to create a custom bootstrap ERB, then
set log_info to :debug vs :info.

So if you copied
http://github.com/opscode/chef/blob/master/chef/lib/chef/knife/bootstrap/ubuntu10.04-gems.erbto
~/.chef/bootstrap/debug.erb

you would change line 25 from:

log_level :info

to

log_level :debug

You would then call knife bootstrap with the flag
"--template-file ~/.chef/bootstrap/debug.erb"

More information about custom bootstrap templates can be found here:
Chef Support for Automation & DevOps | Chef (see "BOOTSTRAP SUB-COMMANDS"
section)
http://wiki.opscode.com/display/chef/Knife+Bootstrap

Seth

--
Opscode, Inc.
Seth Chisamore, Technical Evangelist
T: (404) 348-0505 E: schisamo@opscode.com
Twitter, IRC, Github: schisamo

On Wed, Oct 6, 2010 at 4:27 AM, Ringo De Smet ringo.desmet@gmail.comwrote:

On 1 October 2010 09:44, Ringo De Smet ringo.desmet@gmail.com wrote:

Dan,

On 30 September 2010 17:19, Daniel DeLeo dan@kallistec.com wrote:

You'll have to modify the bootstrap script. Knife supports templates,
including user-defined ones, for bootstrap scripts. So copy the
bootstrap script from the source:

http://github.com/opscode/chef/tree/master/chef/lib/chef/knife/bootstrap/

to ~/.chef/bootstrap/debug.erb and modify it to run chef-client with
-l debug. Then run knife bootstrap with -d debug

I did this, but it is still not what I want. Using the -l option, the
knife bootstrap runs in debug mode, but at that point in time,
everything is still OK. At the end of the run, the bootstrap
daemonizes the chef-client, and that first run gives apt-update and
apt-get errors. However, I bootstrap using apt packages and the
/etc/chef/client.rb file has logging set to :info. How can I get the
logging set to :debug in that file at bootstrap time? I tried to set
the log_level via debconf-set-selections (I bootstrap the APT way),
but that isn't set in the initial client.rb file.

Any idea how to set log_level to :debug in the initial
/etc/chef/client.rb file?

Anyone?

Ringo

ringods · October 8, 2010, 7:06am

Seth,

On 7 October 2010 07:15, Seth Chisamore schisamo@opscode.com wrote:

Ringo,
Just follow Dan's instructions above to create a custom bootstrap ERB, then
set log_info to :debug vs :info.
So if you copied
http://github.com/opscode/chef/blob/master/chef/lib/chef/knife/bootstrap/ubuntu10.04-gems.erb
to ~/.chef/bootstrap/debug.erb
you would change line 25 from:
log_level :info
to
log_level :debug
You would then call knife bootstrap with the flag
"--template-file ~/.chef/bootstrap/debug.erb"
More information about custom bootstrap templates can be found here:
Chef Support for Automation & DevOps | Chef (see "BOOTSTRAP SUB-COMMANDS"
section)
http://wiki.opscode.com/display/chef/Knife+Bootstrap

This is possible with the gem based bootstrap method, but you can't
override the log level with the apt based bootstrap method. I tried
the same way as setting the chef server, but to no avail?

File a new ticket?

Ringo

Topic		Replies	Views
Sun java install apt-get update Chef Infra (archive)	3	458	April 28, 2011
Apt_repository resource causes chef run failure Chef Infra Client	0	203	January 19, 2023
Err in apt cookbook Chef Infra (archive)	5	365	June 14, 2013
Apt cookbook bug Chef Infra (archive)	6	408	August 6, 2013
Apt-get update strategy Chef Infra (archive)	10	864	March 28, 2011

Apt package provider: a lot of return code 100 on apt-get

Related topics