Apt package provider: a lot of return code 100 on apt-get

Chef Infra (archive)
1

Hello,

I just bootstrapped a new set of VMs using ‘knife’ (0.9.8). When my
cookbooks are executed, I get errors on the apt-get command of the
package provider. See the following gist for an example:

If I execute the apt-get command manually, the command succeeds
without any problem. At the following chef-client run, the cookbook
barfs at the next package. Is there any way to debug why the apt-get
lines fail via Chef, but succeed manually?

Ringo

2

Hey all,

Just sending a notice to folks that the Chef bootstrap process doesn’t work on the new EC2 micro instances when running Ubuntu 10.04. It fails on the Java recipe, specifically the openjdk-6-jre-headless package. This is a known bug (see https://bugs.launchpad.net/ubuntu/+source/linux-ec2/+bug/634487/). The same bootstrap runs perfectly on an m1.small instance type. I lost a good day’s work on this, so I’m passing it along so others don’t bang their heads against this, too :slight_smile:

-Dan Ryan

3

Yo,

On 28 September 2010 09:27, Dan Ryan scriptfu@gmail.com wrote:

Hey all,

Just sending a notice to folks that the Chef bootstrap process doesn't work
on the new EC2 micro instances when running Ubuntu 10.04. It fails on the
Java recipe, specifically the openjdk-6-jre-headless package. This is a
known bug (see
Bug #634487 “t1.micro instance hangs when installing java” : Bugs : linux-ec2 package : Ubuntu). The
same bootstrap runs perfectly on an m1.small instance type. I lost a good
day's work on this, so I'm passing it along so others don't bang their heads
against this, too :slight_smile:

I'm surprised the t1.micro worked at all. In my limited testing thus far,
the CPU was almost always 99% stolen by hypervisor. Took something like 35
minutes to install emacs. :slight_smile:

-Dan Ryan

4

On 28 September 2010 15:06, Ringo De Smet ringo.desmet@gmail.com wrote:

I just bootstrapped a new set of VMs using 'knife' (0.9.8). When my
cookbooks are executed, I get errors on the apt-get command of the
package provider. See the following gist for an example:

client.log · GitHub

If I execute the apt-get command manually, the command succeeds
without any problem. At the following chef-client run, the cookbook
barfs at the next package. Is there any way to debug why the apt-get
lines fail via Chef, but succeed manually?

I have a perfectly reproducible case. Bootstrapping new VMs have
apt-package install errors just after knife bootstrap. If I restart
chef-client on these nodes, all apt-get works fine.

I looked in the Knife wiki page, but couldn't find how I can get the
chef-client configured in debug logging via knife bootstrap. Since the
problem disappears after chef-client restart, I really want that first
run in debug logging. Any info on how to do this?

Ringo

5

You'll have to modify the bootstrap script. Knife supports templates,
including user-defined ones, for bootstrap scripts. So copy the
bootstrap script from the source:

http://github.com/opscode/chef/tree/master/chef/lib/chef/knife/bootstrap/

to ~/.chef/bootstrap/debug.erb and modify it to run chef-client with
-l debug. Then run knife bootstrap with -d debug

HTH,

Dan DeLeo

On Thu, Sep 30, 2010 at 4:34 AM, Ringo De Smet ringo.desmet@gmail.com wrote:

On 28 September 2010 15:06, Ringo De Smet ringo.desmet@gmail.com wrote:

I just bootstrapped a new set of VMs using 'knife' (0.9.8). When my
cookbooks are executed, I get errors on the apt-get command of the
package provider. See the following gist for an example:

client.log · GitHub

If I execute the apt-get command manually, the command succeeds
without any problem. At the following chef-client run, the cookbook
barfs at the next package. Is there any way to debug why the apt-get
lines fail via Chef, but succeed manually?

I have a perfectly reproducible case. Bootstrapping new VMs have
apt-package install errors just after knife bootstrap. If I restart
chef-client on these nodes, all apt-get works fine.

I looked in the Knife wiki page, but couldn't find how I can get the
chef-client configured in debug logging via knife bootstrap. Since the
problem disappears after chef-client restart, I really want that first
run in debug logging. Any info on how to do this?

Ringo

6

On Tue, Sep 28, 2010 at 6:06 AM, Ringo De Smet ringo.desmet@gmail.com wrote:

I just bootstrapped a new set of VMs using 'knife' (0.9.8). When my
cookbooks are executed, I get errors on the apt-get command of the
package provider. See the following gist for an example:

client.log · GitHub

Are you using any repositories that would not be signed by the
upstream keys, such as a local mirror or additional local package
repository?

Bryan

7

On 30 September 2010 09:09, Bryan McLellan btm@loftninjas.org wrote:

On Tue, Sep 28, 2010 at 6:06 AM, Ringo De Smet ringo.desmet@gmail.com
wrote:

I just bootstrapped a new set of VMs using 'knife' (0.9.8). When my
cookbooks are executed, I get errors on the apt-get command of the
package provider. See the following gist for an example:

client.log · GitHub

Are you using any repositories that would not be signed by the
upstream keys, such as a local mirror or additional local package
repository?

Isn't error code 100 a general underlying-subsystem failure?

Half-configured packages (dpkg --configure -a) or half-fetched APT sources
(apt-get update), as well!

Regards,

AJ

Bryan

8

On Thu, Sep 30, 2010 at 9:35 AM, AJ Christensen aj@junglist.gen.nz wrote:

Isn't error code 100 a general underlying-subsystem failure?
Half-configured packages (dpkg --configure -a) or half-fetched APT sources
(apt-get update), as well!

Yeah, a 404 on any sources.list entry will cause apt-get update to return 100.

A key issue will still return 0.

It's common to have apt-get fail on an initial chef run if someone is
modifying sources.list because you have to make sure it happens in the
right order; get everything into the sources.list, get the keys,
trigger an immediate apt-get update execute.

Bryan

9

Dan,

On 30 September 2010 17:19, Daniel DeLeo dan@kallistec.com wrote:

You'll have to modify the bootstrap script. Knife supports templates,
including user-defined ones, for bootstrap scripts. So copy the
bootstrap script from the source:

http://github.com/opscode/chef/tree/master/chef/lib/chef/knife/bootstrap/

to ~/.chef/bootstrap/debug.erb and modify it to run chef-client with
-l debug. Then run knife bootstrap with -d debug

I did this, but it is still not what I want. Using the -l option, the
knife bootstrap runs in debug mode, but at that point in time,
everything is still OK. At the end of the run, the bootstrap
daemonizes the chef-client, and that first run gives apt-update and
apt-get errors. However, I bootstrap using apt packages and the
/etc/chef/client.rb file has logging set to :info. How can I get the
logging set to :debug in that file at bootstrap time? I tried to set
the log_level via debconf-set-selections (I bootstrap the APT way),
but that isn't set in the initial client.rb file.

Any idea how to set log_level to :debug in the initial /etc/chef/client.rb file?

Ringo

10

Bryan,

On 30 September 2010 18:09, Bryan McLellan btm@loftninjas.org wrote:

Are you using any repositories that would not be signed by the
upstream keys, such as a local mirror or additional local package
repository?

No. The apt bootstrap process adds apt.opscode.com, but that is OK as
the key is also added. See my response to Dan's suggestion for a more
detailed description of where it fails: the first daemonized
chef-client run just after knife bootstrap!

Ringo

11

On 1 October 2010 09:44, Ringo De Smet ringo.desmet@gmail.com wrote:

I did this, but it is still not what I want. Using the -l option, the
knife bootstrap runs in debug mode, but at that point in time,
everything is still OK. At the end of the run, the bootstrap
daemonizes the chef-client, and that first run gives apt-update and
apt-get errors. However, I bootstrap using apt packages and the
/etc/chef/client.rb file has logging set to :info. How can I get the
logging set to :debug in that file at bootstrap time? I tried to set
the log_level via debconf-set-selections (I bootstrap the APT way),
but that isn't set in the initial client.rb file.

Here you can find the log file when chef-client is run the first time
in daemonized way:

Even the "apt-get update" fails already, while chef install via apt
worked flawlessly during the bootstrap process...

Ringo

12

On 1 October 2010 09:44, Ringo De Smet ringo.desmet@gmail.com wrote:

Dan,

On 30 September 2010 17:19, Daniel DeLeo dan@kallistec.com wrote:

You'll have to modify the bootstrap script. Knife supports templates,
including user-defined ones, for bootstrap scripts. So copy the
bootstrap script from the source:

http://github.com/opscode/chef/tree/master/chef/lib/chef/knife/bootstrap/

to ~/.chef/bootstrap/debug.erb and modify it to run chef-client with
-l debug. Then run knife bootstrap with -d debug

I did this, but it is still not what I want. Using the -l option, the
knife bootstrap runs in debug mode, but at that point in time,
everything is still OK. At the end of the run, the bootstrap
daemonizes the chef-client, and that first run gives apt-update and
apt-get errors. However, I bootstrap using apt packages and the
/etc/chef/client.rb file has logging set to :info. How can I get the
logging set to :debug in that file at bootstrap time? I tried to set
the log_level via debconf-set-selections (I bootstrap the APT way),
but that isn't set in the initial client.rb file.

Any idea how to set log_level to :debug in the initial /etc/chef/client.rb file?

Anyone?

Ringo

13

Ringo,
Just follow Dan's instructions above to create a custom bootstrap ERB, then
set log_info to :debug vs :info.

So if you copied
http://github.com/opscode/chef/blob/master/chef/lib/chef/knife/bootstrap/ubuntu10.04-gems.erbto
~/.chef/bootstrap/debug.erb

you would change line 25 from:

log_level :info

to

log_level :debug

You would then call knife bootstrap with the flag
"--template-file ~/.chef/bootstrap/debug.erb"

More information about custom bootstrap templates can be found here:
Chef Support for Automation & DevOps | Chef (see "BOOTSTRAP SUB-COMMANDS"
section)
http://wiki.opscode.com/display/chef/Knife+Bootstrap

Seth

--
Opscode, Inc.
Seth Chisamore, Technical Evangelist
T: (404) 348-0505 E: schisamo@opscode.com
Twitter, IRC, Github: schisamo

On Wed, Oct 6, 2010 at 4:27 AM, Ringo De Smet ringo.desmet@gmail.comwrote:

On 1 October 2010 09:44, Ringo De Smet ringo.desmet@gmail.com wrote:

Dan,

On 30 September 2010 17:19, Daniel DeLeo dan@kallistec.com wrote:

You'll have to modify the bootstrap script. Knife supports templates,
including user-defined ones, for bootstrap scripts. So copy the
bootstrap script from the source:

http://github.com/opscode/chef/tree/master/chef/lib/chef/knife/bootstrap/

to ~/.chef/bootstrap/debug.erb and modify it to run chef-client with
-l debug. Then run knife bootstrap with -d debug

I did this, but it is still not what I want. Using the -l option, the
knife bootstrap runs in debug mode, but at that point in time,
everything is still OK. At the end of the run, the bootstrap
daemonizes the chef-client, and that first run gives apt-update and
apt-get errors. However, I bootstrap using apt packages and the
/etc/chef/client.rb file has logging set to :info. How can I get the
logging set to :debug in that file at bootstrap time? I tried to set
the log_level via debconf-set-selections (I bootstrap the APT way),
but that isn't set in the initial client.rb file.

Any idea how to set log_level to :debug in the initial
/etc/chef/client.rb file?

Anyone?

Ringo

14

Seth,

On 7 October 2010 07:15, Seth Chisamore schisamo@opscode.com wrote:

Ringo,
Just follow Dan's instructions above to create a custom bootstrap ERB, then
set log_info to :debug vs :info.
So if you copied
http://github.com/opscode/chef/blob/master/chef/lib/chef/knife/bootstrap/ubuntu10.04-gems.erb
to ~/.chef/bootstrap/debug.erb
you would change line 25 from:
log_level :info
to
log_level :debug
You would then call knife bootstrap with the flag
"--template-file ~/.chef/bootstrap/debug.erb"
More information about custom bootstrap templates can be found here:
Chef Support for Automation & DevOps | Chef (see "BOOTSTRAP SUB-COMMANDS"
section)
http://wiki.opscode.com/display/chef/Knife+Bootstrap

This is possible with the gem based bootstrap method, but you can't
override the log level with the apt based bootstrap method. I tried
the same way as setting the chef server, but to no avail?

File a new ticket?

Ringo