application_php: deploy_revision fails on attempt to use ~apache/.ssh/known_hosts

How can I debug SSH-related git commands performed by the
deploy_revision provider for an application deployed with the
application_php cookbook?

I have a cookbook that depends on application and application_php,
with simple default recipe like:

application "my_app" do
    path "/srv/my_app"
    owner node[:apache][:user]
    group node[:apache][:user]

    scm_provider Chef::Provider::Git
    repository "git@github.com:my_org/my_app.git"
    deploy_key "my_deployment_private_key_here"

    mod_php_apache2
end

It fails like this (see more at https://gist.github.com/pmocek/4733480):

[2013-02-07T19:08:16+00:00] FATAL: Mixlib::ShellOut::ShellCommandFailed: deploy_revision[my_app] (/tmp/vagrant-chef-1/chef-solo-1/cookbooks/application/providers/default.rb line 122) had an error: Mixlib::ShellOut::ShellCommandFailed: Expected process to exit with [0], but received '128'
---- Begin output of git ls-remote git@github.com:my_org/my_app.git HEAD ----
STDOUT: 
STDERR: Could not create directory '/var/www/.ssh'.
Failed to add the host to the list of known hosts (/var/www/.ssh/known_hosts).
Permission denied (publickey).
fatal: The remote end hung up unexpectedly
---- End output of git ls-remote git@github.com:my_org/my_app.git HEAD ----
Ran git ls-remote git@github.com:my_org/my_app.git HEAD returned 128

/var/www is the home directory of node[:apache][:user].

In addition to my application’s cookbook, I’m using the following,
managed by Berkshelf:

* apache2 (1.3.2)
* application (2.0.0)
* application_php (1.0.4)
* build-essential (1.3.0)
* mysql (2.0.2)
* openssl (1.0.0)
* php (1.1.0)
* xml (1.1.0)

Searching the application and application_php cookbooks, I don’t
find any matches on “ls-remote” or “known_hosts”. I haven noticed
anything in the docs for those cookbooks or for the deploy_version
provider about setting an SSH configuration directory. Web
searches for combinations of the applicable cookbooks and
directories have not turned up useful information. Searching this
list, I found a related thread “Could not create directory
’/nonexistent/.ssh’. (may be red herring)”
but it didn’t get
past a suggestion to “adjust the home directory or instruct SSH
where to store the known hosts in the event the default cannot be
written to,” and it seems to me that doing so would likely mask
either a bug in or deficiency of documentation for one of these
cookbooks.

References:


Phil Mocek
http://mocek.org/

First thing, don't run your deploy as the apache user, this would leave you in a situation where apache can modify the code which is a potential security nightmare. Just make a service user for this application and give it a real homedir :slight_smile:

--Noah

On Feb 7, 2013, at 11:57 AM, Phil Mocek wrote:

How can I debug SSH-related git commands performed by the
deploy_revision provider for an application deployed with the
application_php cookbook?

I have a cookbook that depends on application and application_php,
with simple default recipe like:

application "my_app" do
path "/srv/my_app"
owner node[:apache][:user]
group node[:apache][:user]

   scm_provider Chef::Provider::Git
   repository "git@github.com:my_org/my_app.git"
   deploy_key "my_deployment_private_key_here"

   mod_php_apache2

end

It fails like this (see more at https://gist.github.com/pmocek/4733480):

[2013-02-07T19:08:16+00:00] FATAL: Mixlib::ShellOut::ShellCommandFailed: deploy_revision[my_app] (/tmp/vagrant-chef-1/chef-solo-1/cookbooks/application/providers/default.rb line 122) had an error: Mixlib::ShellOut::ShellCommandFailed: Expected process to exit with [0], but received '128'
---- Begin output of git ls-remote git@github.com:my_org/my_app.git HEAD ----
STDOUT:
STDERR: Could not create directory '/var/www/.ssh'.
Failed to add the host to the list of known hosts (/var/www/.ssh/known_hosts).
Permission denied (publickey).
fatal: The remote end hung up unexpectedly
---- End output of git ls-remote git@github.com:my_org/my_app.git HEAD ----
Ran git ls-remote git@github.com:my_org/my_app.git HEAD returned 128

/var/www is the home directory of node[:apache][:user].

In addition to my application's cookbook, I'm using the following,
managed by Berkshelf:

  • apache2 (1.3.2)
  • application (2.0.0)
  • application_php (1.0.4)
  • build-essential (1.3.0)
  • mysql (2.0.2)
  • openssl (1.0.0)
  • php (1.1.0)
  • xml (1.1.0)

Searching the application and application_php cookbooks, I don't
find any matches on "ls-remote" or "known_hosts". I haven noticed
anything in the docs for those cookbooks or for the deploy_version
provider about setting an SSH configuration directory. Web
searches for combinations of the applicable cookbooks and
directories have not turned up useful information. Searching this
list, I found a related thread "Could not create directory
'/nonexistent/.ssh'. (may be red herring)"
but it didn't get
past a suggestion to "adjust the home directory or instruct SSH
where to store the known hosts in the event the default cannot be
written to," and it seems to me that doing so would likely mask
either a bug in or deficiency of documentation for one of these
cookbooks.

References:

--
Phil Mocek
http://mocek.org/

I haven noticed anything in the docs for those cookbooks or for the
deploy_version provider about setting an SSH configuration directory.

Looking at deploy_version was a good intuition.
This has nothing to do with the application cookbook; if you try to do the
same using the deploy resource, you will likely get the same error.
It might be easier to debug it that way, so try to get a simpler recipe
with deploy_version first.

As Noah suggested, don't use the apache user; that's a security no-no.
Create a new user instead, and make sure it has write access to its home
directory.
That should get you going.

Andrea

On Thu, Feb 7, 2013 at 8:57 PM, Phil Mocek phil-lists@mocek.org wrote:

How can I debug SSH-related git commands performed by the
deploy_revision provider for an application deployed with the
application_php cookbook?

I have a cookbook that depends on application and application_php,
with simple default recipe like:

application "my_app" do
    path "/srv/my_app"
    owner node[:apache][:user]
    group node[:apache][:user]

    scm_provider Chef::Provider::Git
    repository "git@github.com:my_org/my_app.git"
    deploy_key "my_deployment_private_key_here"

    mod_php_apache2
end

It fails like this (see more at https://gist.github.com/pmocek/4733480):

[2013-02-07T19:08:16+00:00] FATAL:

Mixlib::ShellOut::ShellCommandFailed: deploy_revision[my_app]
(/tmp/vagrant-chef-1/chef-solo-1/cookbooks/application/providers/default.rb
line 122) had an error: Mixlib::ShellOut::ShellCommandFailed: Expected
process to exit with [0], but received '128'
---- Begin output of git ls-remote git@github.com:my_org/my_app.git
HEAD ----
STDOUT:
STDERR: Could not create directory '/var/www/.ssh'.
Failed to add the host to the list of known hosts
(/var/www/.ssh/known_hosts).
Permission denied (publickey).
fatal: The remote end hung up unexpectedly
---- End output of git ls-remote git@github.com:my_org/my_app.git
HEAD ----
Ran git ls-remote git@github.com:my_org/my_app.git HEAD returned 128

/var/www is the home directory of node[:apache][:user].

In addition to my application's cookbook, I'm using the following,
managed by Berkshelf:

* apache2 (1.3.2)
* application (2.0.0)
* application_php (1.0.4)
* build-essential (1.3.0)
* mysql (2.0.2)
* openssl (1.0.0)
* php (1.1.0)
* xml (1.1.0)

Searching the application and application_php cookbooks, I don't
find any matches on "ls-remote" or "known_hosts". I haven noticed
anything in the docs for those cookbooks or for the deploy_version
provider about setting an SSH configuration directory. Web
searches for combinations of the applicable cookbooks and
directories have not turned up useful information. Searching this
list, I found a related thread "Could not create directory
'/nonexistent/.ssh'. (may be red herring)"
but it didn't get
past a suggestion to "adjust the home directory or instruct SSH
where to store the known hosts in the event the default cannot be
written to," and it seems to me that doing so would likely mask
either a bug in or deficiency of documentation for one of these
cookbooks.

References:

--
Phil Mocek
http://mocek.org/

In addition you can use the deploy_revision resource that is well documented over at http://docs.opscode.com/resource_deploy.html

It also has the example for php app as well.


@millisami
~ Sachin Sagar Rai
Ruby on Rails Developer
http://tfm.com.np
http://nepalonrails.com (http://nepalonrails.tumblr.com)
Sent with Sparrow (http://www.sparrowmailapp.com/?sig)

On Friday, February 8, 2013 at 1:42 AM, Phil Mocek wrote:

How can I debug SSH-related git commands performed by the
deploy_revision provider for an application deployed with the
application_php cookbook?

I have a cookbook that depends on application and application_php,
with simple default recipe like:

application "my_app" do
path "/srv/my_app"
owner node[:apache][:user]
group node[:apache][:user]

scm_provider Chef::Provider::Git
repository "git@github.com (mailto:git@github.com):my_org/my_app.git"
deploy_key "my_deployment_private_key_here"

mod_php_apache2
end

It fails like this (see more at https://gist.github.com/pmocek/4733480):

[2013-02-07T19:08:16+00:00] FATAL: Mixlib::ShellOut::ShellCommandFailed: deploy_revision[my_app] (/tmp/vagrant-chef-1/chef-solo-1/cookbooks/application/providers/default.rb line 122) had an error: Mixlib::ShellOut::ShellCommandFailed: Expected process to exit with [0], but received '128'
---- Begin output of git ls-remote git@github.com (mailto:git@github.com):my_org/my_app.git HEAD ----
STDOUT:
STDERR: Could not create directory '/var/www/.ssh'.
Failed to add the host to the list of known hosts (/var/www/.ssh/known_hosts).
Permission denied (publickey).
fatal: The remote end hung up unexpectedly
---- End output of git ls-remote git@github.com (mailto:git@github.com):my_org/my_app.git HEAD ----
Ran git ls-remote git@github.com (mailto:git@github.com):my_org/my_app.git HEAD returned 128

/var/www is the home directory of node[:apache][:user].

In addition to my application's cookbook, I'm using the following,
managed by Berkshelf:

  • apache2 (1.3.2)
  • application (2.0.0)
  • application_php (1.0.4)
  • build-essential (1.3.0)
  • mysql (2.0.2)
  • openssl (1.0.0)
  • php (1.1.0)
  • xml (1.1.0)

Searching the application and application_php cookbooks, I don't
find any matches on "ls-remote" or "known_hosts". I haven noticed
anything in the docs for those cookbooks or for the deploy_version
provider about setting an SSH configuration directory. Web
searches for combinations of the applicable cookbooks and
directories have not turned up useful information. Searching this
list, I found a related thread "Could not create directory
'/nonexistent/.ssh'. (may be red herring)"
but it didn't get
past a suggestion to "adjust the home directory or instruct SSH
where to store the known hosts in the event the default cannot be
written to," and it seems to me that doing so would likely mask
either a bug in or deficiency of documentation for one of these
cookbooks.

References:

--
Phil Mocek
http://mocek.org/

On Thu, Feb 07, 2013 at 11:57:55AM -0800, Phil Mocek wrote:

I have a cookbook that depends on application and application_php,
with simple default recipe like:

application "my_app" do
    path "/srv/my_app"
    owner node[:apache][:user]
    group node[:apache][:user]

    scm_provider Chef::Provider::Git
    repository "git@github.com:my_org/my_app.git"
    deploy_key "my_deployment_private_key_here"

    mod_php_apache2
end

It fails like this (see more at https://gist.github.com/pmocek/4733480):
[snip]

As several people noted, I was having the user as which Apache
runs deploy my application, and this is inadvisable. However,
root cause of my problem was a corrupted deployment key, which
caused SSH to attempt to update the known_hosts file for my
application's user (apache). Fixing the key resulted in
successful deployment of the application and completion of
convergence.

To avoid the potential trouble of which others warned, I added the
following user resource to my recipe along with a corresponding
default attribute:

user node[:my_app][:user] do
    action [ :create, :lock ]
    system true
    home "/home/#{name}"
    supports :manage_home => true
    shell "/bin/false"
end

And I updated the application resource to use that user:

application "my_app" do
    path "/srv/my_app"
    owner node[:my_app][:user]
    group node[:my_app][:user]

    scm_provider Chef::Provider::Git
    repository "git@github.com:my_org/my_app.git"
    deploy_key "my_deployment_private_key_here"

    mod_php_apache2
end

--
Phil Mocek
http://mocek.org/