Services not starting when bootstrapping an EC2 instance


#1

Hi all -

I’m struggling to put my finger on what’s going wrong here.

I’m trying to bootstrap an ec2 instance, and even though I can ssh up
after the bootstrapping is complete and "sudo service start"
it, the:

service do
action :start
end

bit doesn’t start the service that the same recipe is deploying…

Any suggestions?


#2

Another clue (maybe) is if the instance is restarted, the service comes up.

When you bootstrap - you have to specify the “ubuntu” user, I’m not
sure if that’s the problem or not…

On Sat, Mar 16, 2013 at 1:27 AM, Maven User maven.2.user@gmail.com wrote:

Hi all -

I’m struggling to put my finger on what’s going wrong here.

I’m trying to bootstrap an ec2 instance, and even though I can ssh up
after the bootstrapping is complete and "sudo service start"
it, the:

service do
action :start
end

bit doesn’t start the service that the same recipe is deploying…

Any suggestions?


#3

Is there a problem with the service itself? Does the package
installation automatically start the service in a bad state?

Is it an init script, or upstart, systemd, etc?

Can you post the full chef debug logs.

Cheers,

AJ

On 16 March 2013 18:54, Maven User maven.2.user@gmail.com wrote:

Another clue (maybe) is if the instance is restarted, the service comes up.

When you bootstrap - you have to specify the “ubuntu” user, I’m not
sure if that’s the problem or not…

On Sat, Mar 16, 2013 at 1:27 AM, Maven User maven.2.user@gmail.com wrote:

Hi all -

I’m struggling to put my finger on what’s going wrong here.

I’m trying to bootstrap an ec2 instance, and even though I can ssh up
after the bootstrapping is complete and "sudo service start"
it, the:

service do
action :start
end

bit doesn’t start the service that the same recipe is deploying…

Any suggestions?


#4

Dood - thanks so much for the reply!

It’s a standard init script and the service doesn’t even have a log
written at this point.

Then I ssh up and run “sudo service start” and it starts/logs.

Is there some weirdness with ubuntu supplied ubuntu ami’s?

00:30:50.640 DEBUG: received packet nr 276 type 94 len 188
00:30:50.640 INFO: channel_data: 0 166b
00:30:50.640 10.10.1.24 [2013-03-16T05:29:40+00:00] INFO: Processing
link[/etc/init.d/config-service] action create
(/var/chef/cache/cookbooks/maven_artifact/providers/default.rb line
104)
00:30:50.673 DEBUG: read 116 bytes
00:30:50.675 DEBUG: received packet nr 277 type 94 len 92
00:30:50.675 INFO: channel_data: 0 76b
00:30:50.731 10.10.1.24 [2013-03-16T05:29:40+00:00] INFO:
link[/etc/init.d/config-service] created
00:30:50.732 DEBUG: read 196 bytes
00:30:50.732 DEBUG: received packet nr 278 type 94 len 172
00:30:50.732 INFO: channel_data: 0 149b
00:30:50.732 10.10.1.24 [2013-03-16T05:29:40+00:00] INFO: Processing
execute[update-rc] action run
(/var/chef/cache/cookbooks/maven_artifact/providers/default.rb line
115)
00:30:50.733 DEBUG: read 532 bytes
00:30:50.733 DEBUG: received packet nr 279 type 94 len 508
00:30:50.735 INFO: channel_data: 0 486b
00:30:50.736 10.10.1.24 Adding system startup for
/etc/init.d/config-service …
00:30:50.736 10.10.1.24 /etc/rc0.d/K20config-service ->
…/init.d/config-service
00:30:50.736 10.10.1.24 /etc/rc1.d/K20config-service ->
…/init.d/config-service
00:30:50.736 10.10.1.24 /etc/rc6.d/K20config-service ->
…/init.d/config-service
00:30:50.736 10.10.1.24 /etc/rc2.d/S20config-service ->
…/init.d/config-service
00:30:50.771 10.10.1.24 /etc/rc3.d/S20config-service ->
…/init.d/config-service
00:30:50.772 10.10.1.24 /etc/rc4.d/S20config-service ->
…/init.d/config-service
00:30:50.772 10.10.1.24 /etc/rc5.d/S20config-service ->
…/init.d/config-service
00:30:50.772 DEBUG: read 116 bytes
00:30:50.772 DEBUG: received packet nr 280 type 94 len 92
00:30:50.772 INFO: channel_data: 0 71b
00:30:50.772 10.10.1.24 [2013-03-16T05:29:40+00:00] INFO:
execute[update-rc] ran successfully
00:30:50.772 DEBUG: read 212 bytes
00:30:50.773 DEBUG: received packet nr 281 type 94 len 188
00:30:50.773 INFO: channel_data: 0 160b
00:30:50.773 10.10.1.24 [2013-03-16T05:29:40+00:00] INFO: Processing
template[/srv/config-service/conf/services.properties] action create
(chef-config-service::configservice line 45)
00:30:51.510 DEBUG: read 148 bytes
00:30:51.510 DEBUG: read 148 bytes
00:30:51.605 DEBUG: received packet nr 282 type 94 len 124
00:30:51.605 INFO: channel_data: 0 106b
00:30:51.606 10.10.1.24 [2013-03-16T05:29:40+00:00] INFO:
template[/srv/config-service/conf/services.properties] updated content
00:30:51.606 DEBUG: read 180 bytes
00:30:51.606 DEBUG: received packet nr 283 type 94 len 124
00:30:51.606 INFO: channel_data: 0 110b
00:30:51.607 10.10.1.24 [2013-03-16T05:29:40+00:00] INFO:
template[/srv/config-service/conf/services.properties] mode changed to
644
00:30:51.607 DEBUG: received packet nr 284 type 94 len 156
00:30:51.607 INFO: channel_data: 0 128b

***** At this point, the service, when deployed to a vm, this works
just peachy, for the ami, it just does owt *****

00:30:51.607 10.10.1.24 [2013-03-16T05:29:40+00:00] INFO: Processing
service[config-service] action start
(chef-config-service::configservice line 50)
00:30:51.608 DEBUG: read 148 bytes

00:30:51.678 DEBUG: received packet nr 285 type 94 len 124
00:30:51.678 INFO: channel_data: 0 110b

On Sat, Mar 16, 2013 at 2:00 AM, AJ Christensen aj@junglist.gen.nz wrote:

Is there a problem with the service itself? Does the package
installation automatically start the service in a bad state?

Is it an init script, or upstart, systemd, etc?

Can you post the full chef debug logs.

Cheers,

AJ

On 16 March 2013 18:54, Maven User maven.2.user@gmail.com wrote:

Another clue (maybe) is if the instance is restarted, the service comes up.

When you bootstrap - you have to specify the “ubuntu” user, I’m not
sure if that’s the problem or not…

On Sat, Mar 16, 2013 at 1:27 AM, Maven User maven.2.user@gmail.com wrote:

Hi all -

I’m struggling to put my finger on what’s going wrong here.

I’m trying to bootstrap an ec2 instance, and even though I can ssh up
after the bootstrapping is complete and "sudo service start"
it, the:

service do
action :start
end

bit doesn’t start the service that the same recipe is deploying…

Any suggestions?


#5

Any other suggestions? Totally has me wedged :-/

I’m cranking up the verbosity to the script, but there’s nothing
exciting coming out…

On Mon, Mar 18, 2013 at 10:49 AM, Maven User maven.2.user@gmail.com wrote:

Dood - thanks so much for the reply!

It’s a standard init script and the service doesn’t even have a log
written at this point.

Then I ssh up and run “sudo service start” and it starts/logs.

Is there some weirdness with ubuntu supplied ubuntu ami’s?

00:30:50.640 DEBUG: received packet nr 276 type 94 len 188
00:30:50.640 INFO: channel_data: 0 166b
00:30:50.640 10.10.1.24 [2013-03-16T05:29:40+00:00] INFO: Processing
link[/etc/init.d/config-service] action create
(/var/chef/cache/cookbooks/maven_artifact/providers/default.rb line
104)
00:30:50.673 DEBUG: read 116 bytes
00:30:50.675 DEBUG: received packet nr 277 type 94 len 92
00:30:50.675 INFO: channel_data: 0 76b
00:30:50.731 10.10.1.24 [2013-03-16T05:29:40+00:00] INFO:
link[/etc/init.d/config-service] created
00:30:50.732 DEBUG: read 196 bytes
00:30:50.732 DEBUG: received packet nr 278 type 94 len 172
00:30:50.732 INFO: channel_data: 0 149b
00:30:50.732 10.10.1.24 [2013-03-16T05:29:40+00:00] INFO: Processing
execute[update-rc] action run
(/var/chef/cache/cookbooks/maven_artifact/providers/default.rb line
115)
00:30:50.733 DEBUG: read 532 bytes
00:30:50.733 DEBUG: received packet nr 279 type 94 len 508
00:30:50.735 INFO: channel_data: 0 486b
00:30:50.736 10.10.1.24 Adding system startup for
/etc/init.d/config-service …
00:30:50.736 10.10.1.24 /etc/rc0.d/K20config-service ->
…/init.d/config-service
00:30:50.736 10.10.1.24 /etc/rc1.d/K20config-service ->
…/init.d/config-service
00:30:50.736 10.10.1.24 /etc/rc6.d/K20config-service ->
…/init.d/config-service
00:30:50.736 10.10.1.24 /etc/rc2.d/S20config-service ->
…/init.d/config-service
00:30:50.771 10.10.1.24 /etc/rc3.d/S20config-service ->
…/init.d/config-service
00:30:50.772 10.10.1.24 /etc/rc4.d/S20config-service ->
…/init.d/config-service
00:30:50.772 10.10.1.24 /etc/rc5.d/S20config-service ->
…/init.d/config-service
00:30:50.772 DEBUG: read 116 bytes
00:30:50.772 DEBUG: received packet nr 280 type 94 len 92
00:30:50.772 INFO: channel_data: 0 71b
00:30:50.772 10.10.1.24 [2013-03-16T05:29:40+00:00] INFO:
execute[update-rc] ran successfully
00:30:50.772 DEBUG: read 212 bytes
00:30:50.773 DEBUG: received packet nr 281 type 94 len 188
00:30:50.773 INFO: channel_data: 0 160b
00:30:50.773 10.10.1.24 [2013-03-16T05:29:40+00:00] INFO: Processing
template[/srv/config-service/conf/services.properties] action create
(chef-config-service::configservice line 45)
00:30:51.510 DEBUG: read 148 bytes
00:30:51.510 DEBUG: read 148 bytes
00:30:51.605 DEBUG: received packet nr 282 type 94 len 124
00:30:51.605 INFO: channel_data: 0 106b
00:30:51.606 10.10.1.24 [2013-03-16T05:29:40+00:00] INFO:
template[/srv/config-service/conf/services.properties] updated content
00:30:51.606 DEBUG: read 180 bytes
00:30:51.606 DEBUG: received packet nr 283 type 94 len 124
00:30:51.606 INFO: channel_data: 0 110b
00:30:51.607 10.10.1.24 [2013-03-16T05:29:40+00:00] INFO:
template[/srv/config-service/conf/services.properties] mode changed to
644
00:30:51.607 DEBUG: received packet nr 284 type 94 len 156
00:30:51.607 INFO: channel_data: 0 128b

***** At this point, the service, when deployed to a vm, this works
just peachy, for the ami, it just does owt *****

00:30:51.607 10.10.1.24 [2013-03-16T05:29:40+00:00] INFO: Processing
service[config-service] action start
(chef-config-service::configservice line 50)
00:30:51.608 DEBUG: read 148 bytes

00:30:51.678 DEBUG: received packet nr 285 type 94 len 124
00:30:51.678 INFO: channel_data: 0 110b

On Sat, Mar 16, 2013 at 2:00 AM, AJ Christensen aj@junglist.gen.nz wrote:

Is there a problem with the service itself? Does the package
installation automatically start the service in a bad state?

Is it an init script, or upstart, systemd, etc?

Can you post the full chef debug logs.

Cheers,

AJ

On 16 March 2013 18:54, Maven User maven.2.user@gmail.com wrote:

Another clue (maybe) is if the instance is restarted, the service comes up.

When you bootstrap - you have to specify the “ubuntu” user, I’m not
sure if that’s the problem or not…

On Sat, Mar 16, 2013 at 1:27 AM, Maven User maven.2.user@gmail.com wrote:

Hi all -

I’m struggling to put my finger on what’s going wrong here.

I’m trying to bootstrap an ec2 instance, and even though I can ssh up
after the bootstrapping is complete and "sudo service start"
it, the:

service do
action :start
end

bit doesn’t start the service that the same recipe is deploying…

Any suggestions?


#6

If you toss ‘set -x’ into the Init script you should be able to see
extremely verbose debugging from Bash itself, as the script is
executed.

It looks like you’re using debug logging for knife bootstrap – not
debug logging in the Chef run itself. The latter will be more
informative :wink:

–AJ

On 19 March 2013 09:18, Maven User maven.2.user@gmail.com wrote:

Any other suggestions? Totally has me wedged :-/

I’m cranking up the verbosity to the script, but there’s nothing
exciting coming out…

On Mon, Mar 18, 2013 at 10:49 AM, Maven User maven.2.user@gmail.com wrote:

Dood - thanks so much for the reply!

It’s a standard init script and the service doesn’t even have a log
written at this point.

Then I ssh up and run “sudo service start” and it starts/logs.

Is there some weirdness with ubuntu supplied ubuntu ami’s?

00:30:50.640 DEBUG: received packet nr 276 type 94 len 188
00:30:50.640 INFO: channel_data: 0 166b
00:30:50.640 10.10.1.24 [2013-03-16T05:29:40+00:00] INFO: Processing
link[/etc/init.d/config-service] action create
(/var/chef/cache/cookbooks/maven_artifact/providers/default.rb line
104)
00:30:50.673 DEBUG: read 116 bytes
00:30:50.675 DEBUG: received packet nr 277 type 94 len 92
00:30:50.675 INFO: channel_data: 0 76b
00:30:50.731 10.10.1.24 [2013-03-16T05:29:40+00:00] INFO:
link[/etc/init.d/config-service] created
00:30:50.732 DEBUG: read 196 bytes
00:30:50.732 DEBUG: received packet nr 278 type 94 len 172
00:30:50.732 INFO: channel_data: 0 149b
00:30:50.732 10.10.1.24 [2013-03-16T05:29:40+00:00] INFO: Processing
execute[update-rc] action run
(/var/chef/cache/cookbooks/maven_artifact/providers/default.rb line
115)
00:30:50.733 DEBUG: read 532 bytes
00:30:50.733 DEBUG: received packet nr 279 type 94 len 508
00:30:50.735 INFO: channel_data: 0 486b
00:30:50.736 10.10.1.24 Adding system startup for
/etc/init.d/config-service …
00:30:50.736 10.10.1.24 /etc/rc0.d/K20config-service ->
…/init.d/config-service
00:30:50.736 10.10.1.24 /etc/rc1.d/K20config-service ->
…/init.d/config-service
00:30:50.736 10.10.1.24 /etc/rc6.d/K20config-service ->
…/init.d/config-service
00:30:50.736 10.10.1.24 /etc/rc2.d/S20config-service ->
…/init.d/config-service
00:30:50.771 10.10.1.24 /etc/rc3.d/S20config-service ->
…/init.d/config-service
00:30:50.772 10.10.1.24 /etc/rc4.d/S20config-service ->
…/init.d/config-service
00:30:50.772 10.10.1.24 /etc/rc5.d/S20config-service ->
…/init.d/config-service
00:30:50.772 DEBUG: read 116 bytes
00:30:50.772 DEBUG: received packet nr 280 type 94 len 92
00:30:50.772 INFO: channel_data: 0 71b
00:30:50.772 10.10.1.24 [2013-03-16T05:29:40+00:00] INFO:
execute[update-rc] ran successfully
00:30:50.772 DEBUG: read 212 bytes
00:30:50.773 DEBUG: received packet nr 281 type 94 len 188
00:30:50.773 INFO: channel_data: 0 160b
00:30:50.773 10.10.1.24 [2013-03-16T05:29:40+00:00] INFO: Processing
template[/srv/config-service/conf/services.properties] action create
(chef-config-service::configservice line 45)
00:30:51.510 DEBUG: read 148 bytes
00:30:51.510 DEBUG: read 148 bytes
00:30:51.605 DEBUG: received packet nr 282 type 94 len 124
00:30:51.605 INFO: channel_data: 0 106b
00:30:51.606 10.10.1.24 [2013-03-16T05:29:40+00:00] INFO:
template[/srv/config-service/conf/services.properties] updated content
00:30:51.606 DEBUG: read 180 bytes
00:30:51.606 DEBUG: received packet nr 283 type 94 len 124
00:30:51.606 INFO: channel_data: 0 110b
00:30:51.607 10.10.1.24 [2013-03-16T05:29:40+00:00] INFO:
template[/srv/config-service/conf/services.properties] mode changed to
644
00:30:51.607 DEBUG: received packet nr 284 type 94 len 156
00:30:51.607 INFO: channel_data: 0 128b

***** At this point, the service, when deployed to a vm, this works
just peachy, for the ami, it just does owt *****

00:30:51.607 10.10.1.24 [2013-03-16T05:29:40+00:00] INFO: Processing
service[config-service] action start
(chef-config-service::configservice line 50)
00:30:51.608 DEBUG: read 148 bytes

00:30:51.678 DEBUG: received packet nr 285 type 94 len 124
00:30:51.678 INFO: channel_data: 0 110b

On Sat, Mar 16, 2013 at 2:00 AM, AJ Christensen aj@junglist.gen.nz wrote:

Is there a problem with the service itself? Does the package
installation automatically start the service in a bad state?

Is it an init script, or upstart, systemd, etc?

Can you post the full chef debug logs.

Cheers,

AJ

On 16 March 2013 18:54, Maven User maven.2.user@gmail.com wrote:

Another clue (maybe) is if the instance is restarted, the service comes up.

When you bootstrap - you have to specify the “ubuntu” user, I’m not
sure if that’s the problem or not…

On Sat, Mar 16, 2013 at 1:27 AM, Maven User maven.2.user@gmail.com wrote:

Hi all -

I’m struggling to put my finger on what’s going wrong here.

I’m trying to bootstrap an ec2 instance, and even though I can ssh up
after the bootstrapping is complete and "sudo service start"
it, the:

service do
action :start
end

bit doesn’t start the service that the same recipe is deploying…

Any suggestions?


#7

I have !#/bin/bash -x and it reports nothing.

It’s like the script is never even called :-/

On Mon, Mar 18, 2013 at 5:05 PM, AJ Christensen aj@junglist.gen.nz wrote:

If you toss ‘set -x’ into the Init script you should be able to see
extremely verbose debugging from Bash itself, as the script is
executed.

It looks like you’re using debug logging for knife bootstrap – not
debug logging in the Chef run itself. The latter will be more
informative :wink:

–AJ

On 19 March 2013 09:18, Maven User maven.2.user@gmail.com wrote:

Any other suggestions? Totally has me wedged :-/

I’m cranking up the verbosity to the script, but there’s nothing
exciting coming out…

On Mon, Mar 18, 2013 at 10:49 AM, Maven User maven.2.user@gmail.com wrote:

Dood - thanks so much for the reply!

It’s a standard init script and the service doesn’t even have a log
written at this point.

Then I ssh up and run “sudo service start” and it starts/logs.

Is there some weirdness with ubuntu supplied ubuntu ami’s?

00:30:50.640 DEBUG: received packet nr 276 type 94 len 188
00:30:50.640 INFO: channel_data: 0 166b
00:30:50.640 10.10.1.24 [2013-03-16T05:29:40+00:00] INFO: Processing
link[/etc/init.d/config-service] action create
(/var/chef/cache/cookbooks/maven_artifact/providers/default.rb line
104)
00:30:50.673 DEBUG: read 116 bytes
00:30:50.675 DEBUG: received packet nr 277 type 94 len 92
00:30:50.675 INFO: channel_data: 0 76b
00:30:50.731 10.10.1.24 [2013-03-16T05:29:40+00:00] INFO:
link[/etc/init.d/config-service] created
00:30:50.732 DEBUG: read 196 bytes
00:30:50.732 DEBUG: received packet nr 278 type 94 len 172
00:30:50.732 INFO: channel_data: 0 149b
00:30:50.732 10.10.1.24 [2013-03-16T05:29:40+00:00] INFO: Processing
execute[update-rc] action run
(/var/chef/cache/cookbooks/maven_artifact/providers/default.rb line
115)
00:30:50.733 DEBUG: read 532 bytes
00:30:50.733 DEBUG: received packet nr 279 type 94 len 508
00:30:50.735 INFO: channel_data: 0 486b
00:30:50.736 10.10.1.24 Adding system startup for
/etc/init.d/config-service …
00:30:50.736 10.10.1.24 /etc/rc0.d/K20config-service ->
…/init.d/config-service
00:30:50.736 10.10.1.24 /etc/rc1.d/K20config-service ->
…/init.d/config-service
00:30:50.736 10.10.1.24 /etc/rc6.d/K20config-service ->
…/init.d/config-service
00:30:50.736 10.10.1.24 /etc/rc2.d/S20config-service ->
…/init.d/config-service
00:30:50.771 10.10.1.24 /etc/rc3.d/S20config-service ->
…/init.d/config-service
00:30:50.772 10.10.1.24 /etc/rc4.d/S20config-service ->
…/init.d/config-service
00:30:50.772 10.10.1.24 /etc/rc5.d/S20config-service ->
…/init.d/config-service
00:30:50.772 DEBUG: read 116 bytes
00:30:50.772 DEBUG: received packet nr 280 type 94 len 92
00:30:50.772 INFO: channel_data: 0 71b
00:30:50.772 10.10.1.24 [2013-03-16T05:29:40+00:00] INFO:
execute[update-rc] ran successfully
00:30:50.772 DEBUG: read 212 bytes
00:30:50.773 DEBUG: received packet nr 281 type 94 len 188
00:30:50.773 INFO: channel_data: 0 160b
00:30:50.773 10.10.1.24 [2013-03-16T05:29:40+00:00] INFO: Processing
template[/srv/config-service/conf/services.properties] action create
(chef-config-service::configservice line 45)
00:30:51.510 DEBUG: read 148 bytes
00:30:51.510 DEBUG: read 148 bytes
00:30:51.605 DEBUG: received packet nr 282 type 94 len 124
00:30:51.605 INFO: channel_data: 0 106b
00:30:51.606 10.10.1.24 [2013-03-16T05:29:40+00:00] INFO:
template[/srv/config-service/conf/services.properties] updated content
00:30:51.606 DEBUG: read 180 bytes
00:30:51.606 DEBUG: received packet nr 283 type 94 len 124
00:30:51.606 INFO: channel_data: 0 110b
00:30:51.607 10.10.1.24 [2013-03-16T05:29:40+00:00] INFO:
template[/srv/config-service/conf/services.properties] mode changed to
644
00:30:51.607 DEBUG: received packet nr 284 type 94 len 156
00:30:51.607 INFO: channel_data: 0 128b

***** At this point, the service, when deployed to a vm, this works
just peachy, for the ami, it just does owt *****

00:30:51.607 10.10.1.24 [2013-03-16T05:29:40+00:00] INFO: Processing
service[config-service] action start
(chef-config-service::configservice line 50)
00:30:51.608 DEBUG: read 148 bytes

00:30:51.678 DEBUG: received packet nr 285 type 94 len 124
00:30:51.678 INFO: channel_data: 0 110b

On Sat, Mar 16, 2013 at 2:00 AM, AJ Christensen aj@junglist.gen.nz wrote:

Is there a problem with the service itself? Does the package
installation automatically start the service in a bad state?

Is it an init script, or upstart, systemd, etc?

Can you post the full chef debug logs.

Cheers,

AJ

On 16 March 2013 18:54, Maven User maven.2.user@gmail.com wrote:

Another clue (maybe) is if the instance is restarted, the service comes up.

When you bootstrap - you have to specify the “ubuntu” user, I’m not
sure if that’s the problem or not…

On Sat, Mar 16, 2013 at 1:27 AM, Maven User maven.2.user@gmail.com wrote:

Hi all -

I’m struggling to put my finger on what’s going wrong here.

I’m trying to bootstrap an ec2 instance, and even though I can ssh up
after the bootstrapping is complete and "sudo service start"
it, the:

service do
action :start
end

bit doesn’t start the service that the same recipe is deploying…

Any suggestions?


#8

If you use the debug logging of Chef you should be able to see it
shelling out to the service. Can you try setting your log level to
debug and posting the relevant logs (around the service action
firing?)

Maybe you haven’t specified that the service supports status, and it
is falling back to grepping for a process name?

–AJ

On 19 March 2013 10:26, Maven User maven.2.user@gmail.com wrote:

I have !#/bin/bash -x and it reports nothing.

It’s like the script is never even called :-/

On Mon, Mar 18, 2013 at 5:05 PM, AJ Christensen aj@junglist.gen.nz wrote:

If you toss ‘set -x’ into the Init script you should be able to see
extremely verbose debugging from Bash itself, as the script is
executed.

It looks like you’re using debug logging for knife bootstrap – not
debug logging in the Chef run itself. The latter will be more
informative :wink:

–AJ

On 19 March 2013 09:18, Maven User maven.2.user@gmail.com wrote:

Any other suggestions? Totally has me wedged :-/

I’m cranking up the verbosity to the script, but there’s nothing
exciting coming out…

On Mon, Mar 18, 2013 at 10:49 AM, Maven User maven.2.user@gmail.com wrote:

Dood - thanks so much for the reply!

It’s a standard init script and the service doesn’t even have a log
written at this point.

Then I ssh up and run “sudo service start” and it starts/logs.

Is there some weirdness with ubuntu supplied ubuntu ami’s?

00:30:50.640 DEBUG: received packet nr 276 type 94 len 188
00:30:50.640 INFO: channel_data: 0 166b
00:30:50.640 10.10.1.24 [2013-03-16T05:29:40+00:00] INFO: Processing
link[/etc/init.d/config-service] action create
(/var/chef/cache/cookbooks/maven_artifact/providers/default.rb line
104)
00:30:50.673 DEBUG: read 116 bytes
00:30:50.675 DEBUG: received packet nr 277 type 94 len 92
00:30:50.675 INFO: channel_data: 0 76b
00:30:50.731 10.10.1.24 [2013-03-16T05:29:40+00:00] INFO:
link[/etc/init.d/config-service] created
00:30:50.732 DEBUG: read 196 bytes
00:30:50.732 DEBUG: received packet nr 278 type 94 len 172
00:30:50.732 INFO: channel_data: 0 149b
00:30:50.732 10.10.1.24 [2013-03-16T05:29:40+00:00] INFO: Processing
execute[update-rc] action run
(/var/chef/cache/cookbooks/maven_artifact/providers/default.rb line
115)
00:30:50.733 DEBUG: read 532 bytes
00:30:50.733 DEBUG: received packet nr 279 type 94 len 508
00:30:50.735 INFO: channel_data: 0 486b
00:30:50.736 10.10.1.24 Adding system startup for
/etc/init.d/config-service …
00:30:50.736 10.10.1.24 /etc/rc0.d/K20config-service ->
…/init.d/config-service
00:30:50.736 10.10.1.24 /etc/rc1.d/K20config-service ->
…/init.d/config-service
00:30:50.736 10.10.1.24 /etc/rc6.d/K20config-service ->
…/init.d/config-service
00:30:50.736 10.10.1.24 /etc/rc2.d/S20config-service ->
…/init.d/config-service
00:30:50.771 10.10.1.24 /etc/rc3.d/S20config-service ->
…/init.d/config-service
00:30:50.772 10.10.1.24 /etc/rc4.d/S20config-service ->
…/init.d/config-service
00:30:50.772 10.10.1.24 /etc/rc5.d/S20config-service ->
…/init.d/config-service
00:30:50.772 DEBUG: read 116 bytes
00:30:50.772 DEBUG: received packet nr 280 type 94 len 92
00:30:50.772 INFO: channel_data: 0 71b
00:30:50.772 10.10.1.24 [2013-03-16T05:29:40+00:00] INFO:
execute[update-rc] ran successfully
00:30:50.772 DEBUG: read 212 bytes
00:30:50.773 DEBUG: received packet nr 281 type 94 len 188
00:30:50.773 INFO: channel_data: 0 160b
00:30:50.773 10.10.1.24 [2013-03-16T05:29:40+00:00] INFO: Processing
template[/srv/config-service/conf/services.properties] action create
(chef-config-service::configservice line 45)
00:30:51.510 DEBUG: read 148 bytes
00:30:51.510 DEBUG: read 148 bytes
00:30:51.605 DEBUG: received packet nr 282 type 94 len 124
00:30:51.605 INFO: channel_data: 0 106b
00:30:51.606 10.10.1.24 [2013-03-16T05:29:40+00:00] INFO:
template[/srv/config-service/conf/services.properties] updated content
00:30:51.606 DEBUG: read 180 bytes
00:30:51.606 DEBUG: received packet nr 283 type 94 len 124
00:30:51.606 INFO: channel_data: 0 110b
00:30:51.607 10.10.1.24 [2013-03-16T05:29:40+00:00] INFO:
template[/srv/config-service/conf/services.properties] mode changed to
644
00:30:51.607 DEBUG: received packet nr 284 type 94 len 156
00:30:51.607 INFO: channel_data: 0 128b

***** At this point, the service, when deployed to a vm, this works
just peachy, for the ami, it just does owt *****

00:30:51.607 10.10.1.24 [2013-03-16T05:29:40+00:00] INFO: Processing
service[config-service] action start
(chef-config-service::configservice line 50)
00:30:51.608 DEBUG: read 148 bytes

00:30:51.678 DEBUG: received packet nr 285 type 94 len 124
00:30:51.678 INFO: channel_data: 0 110b

On Sat, Mar 16, 2013 at 2:00 AM, AJ Christensen aj@junglist.gen.nz wrote:

Is there a problem with the service itself? Does the package
installation automatically start the service in a bad state?

Is it an init script, or upstart, systemd, etc?

Can you post the full chef debug logs.

Cheers,

AJ

On 16 March 2013 18:54, Maven User maven.2.user@gmail.com wrote:

Another clue (maybe) is if the instance is restarted, the service comes up.

When you bootstrap - you have to specify the “ubuntu” user, I’m not
sure if that’s the problem or not…

On Sat, Mar 16, 2013 at 1:27 AM, Maven User maven.2.user@gmail.com wrote:

Hi all -

I’m struggling to put my finger on what’s going wrong here.

I’m trying to bootstrap an ec2 instance, and even though I can ssh up
after the bootstrapping is complete and "sudo service start"
it, the:

service do
action :start
end

bit doesn’t start the service that the same recipe is deploying…

Any suggestions?


#9

I THINK I have this working…

The default “jetty” init script (which is what ours is based on), to
check status, uses “service check” instead of “service
status”.

So when chef checks, it was checking the wrong thing, and there fore
quietly failing…

On Mon, Mar 18, 2013 at 5:41 PM, AJ Christensen aj@junglist.gen.nz wrote:

If you use the debug logging of Chef you should be able to see it
shelling out to the service. Can you try setting your log level to
debug and posting the relevant logs (around the service action
firing?)

Maybe you haven’t specified that the service supports status, and it
is falling back to grepping for a process name?

–AJ

On 19 March 2013 10:26, Maven User maven.2.user@gmail.com wrote:

I have !#/bin/bash -x and it reports nothing.

It’s like the script is never even called :-/

On Mon, Mar 18, 2013 at 5:05 PM, AJ Christensen aj@junglist.gen.nz wrote:

If you toss ‘set -x’ into the Init script you should be able to see
extremely verbose debugging from Bash itself, as the script is
executed.

It looks like you’re using debug logging for knife bootstrap – not
debug logging in the Chef run itself. The latter will be more
informative :wink:

–AJ

On 19 March 2013 09:18, Maven User maven.2.user@gmail.com wrote:

Any other suggestions? Totally has me wedged :-/

I’m cranking up the verbosity to the script, but there’s nothing
exciting coming out…

On Mon, Mar 18, 2013 at 10:49 AM, Maven User maven.2.user@gmail.com wrote:

Dood - thanks so much for the reply!

It’s a standard init script and the service doesn’t even have a log
written at this point.

Then I ssh up and run “sudo service start” and it starts/logs.

Is there some weirdness with ubuntu supplied ubuntu ami’s?

00:30:50.640 DEBUG: received packet nr 276 type 94 len 188
00:30:50.640 INFO: channel_data: 0 166b
00:30:50.640 10.10.1.24 [2013-03-16T05:29:40+00:00] INFO: Processing
link[/etc/init.d/config-service] action create
(/var/chef/cache/cookbooks/maven_artifact/providers/default.rb line
104)
00:30:50.673 DEBUG: read 116 bytes
00:30:50.675 DEBUG: received packet nr 277 type 94 len 92
00:30:50.675 INFO: channel_data: 0 76b
00:30:50.731 10.10.1.24 [2013-03-16T05:29:40+00:00] INFO:
link[/etc/init.d/config-service] created
00:30:50.732 DEBUG: read 196 bytes
00:30:50.732 DEBUG: received packet nr 278 type 94 len 172
00:30:50.732 INFO: channel_data: 0 149b
00:30:50.732 10.10.1.24 [2013-03-16T05:29:40+00:00] INFO: Processing
execute[update-rc] action run
(/var/chef/cache/cookbooks/maven_artifact/providers/default.rb line
115)
00:30:50.733 DEBUG: read 532 bytes
00:30:50.733 DEBUG: received packet nr 279 type 94 len 508
00:30:50.735 INFO: channel_data: 0 486b
00:30:50.736 10.10.1.24 Adding system startup for
/etc/init.d/config-service …
00:30:50.736 10.10.1.24 /etc/rc0.d/K20config-service ->
…/init.d/config-service
00:30:50.736 10.10.1.24 /etc/rc1.d/K20config-service ->
…/init.d/config-service
00:30:50.736 10.10.1.24 /etc/rc6.d/K20config-service ->
…/init.d/config-service
00:30:50.736 10.10.1.24 /etc/rc2.d/S20config-service ->
…/init.d/config-service
00:30:50.771 10.10.1.24 /etc/rc3.d/S20config-service ->
…/init.d/config-service
00:30:50.772 10.10.1.24 /etc/rc4.d/S20config-service ->
…/init.d/config-service
00:30:50.772 10.10.1.24 /etc/rc5.d/S20config-service ->
…/init.d/config-service
00:30:50.772 DEBUG: read 116 bytes
00:30:50.772 DEBUG: received packet nr 280 type 94 len 92
00:30:50.772 INFO: channel_data: 0 71b
00:30:50.772 10.10.1.24 [2013-03-16T05:29:40+00:00] INFO:
execute[update-rc] ran successfully
00:30:50.772 DEBUG: read 212 bytes
00:30:50.773 DEBUG: received packet nr 281 type 94 len 188
00:30:50.773 INFO: channel_data: 0 160b
00:30:50.773 10.10.1.24 [2013-03-16T05:29:40+00:00] INFO: Processing
template[/srv/config-service/conf/services.properties] action create
(chef-config-service::configservice line 45)
00:30:51.510 DEBUG: read 148 bytes
00:30:51.510 DEBUG: read 148 bytes
00:30:51.605 DEBUG: received packet nr 282 type 94 len 124
00:30:51.605 INFO: channel_data: 0 106b
00:30:51.606 10.10.1.24 [2013-03-16T05:29:40+00:00] INFO:
template[/srv/config-service/conf/services.properties] updated content
00:30:51.606 DEBUG: read 180 bytes
00:30:51.606 DEBUG: received packet nr 283 type 94 len 124
00:30:51.606 INFO: channel_data: 0 110b
00:30:51.607 10.10.1.24 [2013-03-16T05:29:40+00:00] INFO:
template[/srv/config-service/conf/services.properties] mode changed to
644
00:30:51.607 DEBUG: received packet nr 284 type 94 len 156
00:30:51.607 INFO: channel_data: 0 128b

***** At this point, the service, when deployed to a vm, this works
just peachy, for the ami, it just does owt *****

00:30:51.607 10.10.1.24 [2013-03-16T05:29:40+00:00] INFO: Processing
service[config-service] action start
(chef-config-service::configservice line 50)
00:30:51.608 DEBUG: read 148 bytes

00:30:51.678 DEBUG: received packet nr 285 type 94 len 124
00:30:51.678 INFO: channel_data: 0 110b

On Sat, Mar 16, 2013 at 2:00 AM, AJ Christensen aj@junglist.gen.nz wrote:

Is there a problem with the service itself? Does the package
installation automatically start the service in a bad state?

Is it an init script, or upstart, systemd, etc?

Can you post the full chef debug logs.

Cheers,

AJ

On 16 March 2013 18:54, Maven User maven.2.user@gmail.com wrote:

Another clue (maybe) is if the instance is restarted, the service comes up.

When you bootstrap - you have to specify the “ubuntu” user, I’m not
sure if that’s the problem or not…

On Sat, Mar 16, 2013 at 1:27 AM, Maven User maven.2.user@gmail.com wrote:

Hi all -

I’m struggling to put my finger on what’s going wrong here.

I’m trying to bootstrap an ec2 instance, and even though I can ssh up
after the bootstrapping is complete and "sudo service start"
it, the:

service do
action :start
end

bit doesn’t start the service that the same recipe is deploying…

Any suggestions?