Issue deploying with application cookbook - STDERR: fatal: reference is not a tree: af5c7ff84dd852cf1c0b5ca60ea92546a20dda01

Hi,

I have a few apps that are deploying using the application cookbook (_php and _ruby). They work just fine most of the time, but every now and then when bootstrapping a new node I get these weird errors.

This is the interesting part on the debug log of the chef run:

  • deploy_revision[resourcemanager] action deploy[2013-07-23T14:08:47+00:00] INFO: Processing deploy_revision[resourcemanager] action deploy (/var/chef/cache/cookbooks/application/providers/default.rb line 122)
    [2013-07-23T14:08:47+00:00] DEBUG: deploy_revision[resourcemanager] finding current git revision
    [2013-07-23T14:08:47+00:00] DEBUG: deploy_revision[resourcemanager] resolving remote reference
    af5c7ff84dd852cf1c0b5ca60ea92546a20dda01 refs/heads/develop
    [2013-07-23T14:08:50+00:00] DEBUG: deploy_revision[resourcemanager] not creating /opt/apps/resourcemanager because it already exists
    [2013-07-23T14:08:50+00:00] DEBUG: deploy_revision[resourcemanager] not creating /opt/apps/resourcemanager/shared because it already exists
    [2013-07-23T14:08:50+00:00] DEBUG: deploy_revision[resourcemanager] finding current git revision
    [2013-07-23T14:08:50+00:00] INFO: deploy_revision[resourcemanager] cloning repo git@bitbucket.org:zynkmobile/resourcemanager.git to /opt/apps/resourcemanager/shared/cached-copy
    Cloning into ‘/opt/apps/resourcemanager/shared/cached-copy’…

    • clone from git@bitbucket.org:zynkmobile/resourcemanager.git into /opt/apps/resourcemanager/shared/cached-copy[2013-07-23T14:08:55+00:00] WARN: Error on deploying /opt/apps/resourcemanager/releases/af5c7ff84dd852cf1c0b5ca60ea92546a20dda01: Expected process to exit with [0], but received ‘128’
      ---- Begin output of git checkout -b deploy af5c7ff84dd852cf1c0b5ca60ea92546a20dda01 ----
      STDOUT:
      STDERR: fatal: reference is not a tree: af5c7ff84dd852cf1c0b5ca60ea92546a20dda01
      ---- End output of git checkout -b deploy af5c7ff84dd852cf1c0b5ca60ea92546a20dda01 ----
      Ran git checkout -b deploy af5c7ff84dd852cf1c0b5ca60ea92546a20dda01 returned 128
      [2013-07-23T14:08:55+00:00] INFO: Removing failed deploy /opt/apps/resourcemanager/releases/af5c7ff84dd852cf1c0b5ca60ea92546a20dda01

    • remove failed deploy /opt/apps/resourcemanager/releases/af5c7ff84dd852cf1c0b5ca60ea92546a20dda01[2013-07-23T14:08:55+00:00] DEBUG: Creating directory /var/chef/cache/revision-deploys

As can be seen, it finds the Git revision to deploy, clones the directory, but then cannot find the revision. I poked on the cached-copy, and it doesn’t have the branch from which I’m deploying, neither local nor origin/branch, which would certainly explain the error. What I don’t understand is why the branch is not there, since it’s definitely in the repo (and was found before, via git ls I presume).

Is there something else that should be set on the application resource so that this error would not occur?

Cheers,

Cassiano Leal
http://cassianoleal.com
http://twitter.com/cassianoleal

On Tuesday, July 23, 2013 at 7:17 AM, Cassiano Leal wrote:

Hi,

I have a few apps that are deploying using the application cookbook (_php and _ruby). They work just fine most of the time, but every now and then when bootstrapping a new node I get these weird errors.

This is the interesting part on the debug log of the chef run:

  • deploy_revision[resourcemanager] action deploy[2013-07-23T14:08:47+00:00] INFO: Processing deploy_revision[resourcemanager] action deploy (/var/chef/cache/cookbooks/application/providers/default.rb line 122)
    [2013-07-23T14:08:47+00:00] DEBUG: deploy_revision[resourcemanager] finding current git revision
    [2013-07-23T14:08:47+00:00] DEBUG: deploy_revision[resourcemanager] resolving remote reference
    af5c7ff84dd852cf1c0b5ca60ea92546a20dda01 refs/heads/develop
    [2013-07-23T14:08:50+00:00] DEBUG: deploy_revision[resourcemanager] not creating /opt/apps/resourcemanager because it already exists
    [2013-07-23T14:08:50+00:00] DEBUG: deploy_revision[resourcemanager] not creating /opt/apps/resourcemanager/shared because it already exists
    [2013-07-23T14:08:50+00:00] DEBUG: deploy_revision[resourcemanager] finding current git revision
    [2013-07-23T14:08:50+00:00] INFO: deploy_revision[resourcemanager] cloning repo git@bitbucket.org (mailto:git@bitbucket.org):zynkmobile/resourcemanager.git to /opt/apps/resourcemanager/shared/cached-copy
    Cloning into '/opt/apps/resourcemanager/shared/cached-copy'...

    • clone from git@bitbucket.org (mailto:git@bitbucket.org):zynkmobile/resourcemanager.git into /opt/apps/resourcemanager/shared/cached-copy[2013-07-23T14:08:55+00:00] WARN: Error on deploying /opt/apps/resourcemanager/releases/af5c7ff84dd852cf1c0b5ca60ea92546a20dda01: Expected process to exit with [0], but received '128'
      ---- Begin output of git checkout -b deploy af5c7ff84dd852cf1c0b5ca60ea92546a20dda01 ----
      STDOUT:
      STDERR: fatal: reference is not a tree: af5c7ff84dd852cf1c0b5ca60ea92546a20dda01
      ---- End output of git checkout -b deploy af5c7ff84dd852cf1c0b5ca60ea92546a20dda01 ----
      Ran git checkout -b deploy af5c7ff84dd852cf1c0b5ca60ea92546a20dda01 returned 128
      [2013-07-23T14:08:55+00:00] INFO: Removing failed deploy /opt/apps/resourcemanager/releases/af5c7ff84dd852cf1c0b5ca60ea92546a20dda01

    • remove failed deploy /opt/apps/resourcemanager/releases/af5c7ff84dd852cf1c0b5ca60ea92546a20dda01[2013-07-23T14:08:55+00:00] DEBUG: Creating directory /var/chef/cache/revision-deploys

As can be seen, it finds the Git revision to deploy, clones the directory, but then cannot find the revision. I poked on the cached-copy, and it doesn’t have the branch from which I’m deploying, neither local nor origin/branch, which would certainly explain the error. What I don’t understand is why the branch is not there, since it’s definitely in the repo (and was found before, via git ls I presume).

Is there something else that should be set on the application resource so that this error would not occur?

Cheers,

Cassiano Leal
http://cassianoleal.com
http://twitter.com/cassianoleal

Is anyone deleting these branches, or force-pushing?

--
Daniel DeLeo

On July 23, 2013 at 12:58:31, Daniel DeLeo (dan@kallistec.com) wrote:

On Tuesday, July 23, 2013 at 7:17 AM, Cassiano Leal wrote:

Hi,

I have a few apps that are deploying using the application cookbook (_php and _ruby). They work just fine most of the time, but every now and then when bootstrapping a new node I get these weird errors.

This is the interesting part on the debug log of the chef run:

  • deploy_revision[resourcemanager] action deploy[2013-07-23T14:08:47+00:00] INFO: Processing deploy_revision[resourcemanager] action deploy (/var/chef/cache/cookbooks/application/providers/default.rb line 122)
    [2013-07-23T14:08:47+00:00] DEBUG: deploy_revision[resourcemanager] finding current git revision
    [2013-07-23T14:08:47+00:00] DEBUG: deploy_revision[resourcemanager] resolving remote reference
    af5c7ff84dd852cf1c0b5ca60ea92546a20dda01 refs/heads/develop
    [2013-07-23T14:08:50+00:00] DEBUG: deploy_revision[resourcemanager] not creating /opt/apps/resourcemanager because it already exists
    [2013-07-23T14:08:50+00:00] DEBUG: deploy_revision[resourcemanager] not creating /opt/apps/resourcemanager/shared because it already exists
    [2013-07-23T14:08:50+00:00] DEBUG: deploy_revision[resourcemanager] finding current git revision
    [2013-07-23T14:08:50+00:00] INFO: deploy_revision[resourcemanager] cloning repo git@bitbucket.org:zynkmobile/resourcemanager.git to /opt/apps/resourcemanager/shared/cached-copy
    Cloning into ‘/opt/apps/resourcemanager/shared/cached-copy’…

    • clone from git@bitbucket.org:zynkmobile/resourcemanager.git into /opt/apps/resourcemanager/shared/cached-copy[2013-07-23T14:08:55+00:00] WARN: Error on deploying /opt/apps/resourcemanager/releases/af5c7ff84dd852cf1c0b5ca60ea92546a20dda01: Expected process to exit with [0], but received ‘128’
      ---- Begin output of git checkout -b deploy af5c7ff84dd852cf1c0b5ca60ea92546a20dda01 ----
      STDOUT:
      STDERR: fatal: reference is not a tree: af5c7ff84dd852cf1c0b5ca60ea92546a20dda01
      ---- End output of git checkout -b deploy af5c7ff84dd852cf1c0b5ca60ea92546a20dda01 ----
      Ran git checkout -b deploy af5c7ff84dd852cf1c0b5ca60ea92546a20dda01 returned 128
      [2013-07-23T14:08:55+00:00] INFO: Removing failed deploy /opt/apps/resourcemanager/releases/af5c7ff84dd852cf1c0b5ca60ea92546a20dda01

    • remove failed deploy /opt/apps/resourcemanager/releases/af5c7ff84dd852cf1c0b5ca60ea92546a20dda01[2013-07-23T14:08:55+00:00] DEBUG: Creating directory /var/chef/cache/revision-deploys

As can be seen, it finds the Git revision to deploy, clones the directory, but then cannot find the revision. I poked on the cached-copy, and it doesn’t have the branch from which I’m deploying, neither local nor origin/branch, which would certainly explain the error. What I don’t understand is why the branch is not there, since it’s definitely in the repo (and was found before, via git ls I presume).

Is there something else that should be set on the application resource so that this error would not occur?

Cheers,

Cassiano Leal
http://cassianoleal.com
http://twitter.com/cassianoleal

Is anyone deleting these branches, or force-pushing?
I’m not sure I understand your question. The deploy is managed 100% by the application_ruby cookbook. We’re still stabilising some of the features, so we’re deploying from the develop branch.

I managed to work around the issue manually by doing (after letting it fail with the error above):

cd /opt/apps/resourcemanager/shared/cached-copy

export GIT_SSH=/opt/apps/resourcemanager/deploy-ssh-wrapper

git fetch origin develop

And then re-running chef-client. Why hasn’t this branch been pulled from the repo in the first place?


Daniel DeLeo

Cassiano Leal
http://cassianoleal.com
http://twitter.com/cassianoleal

On Tuesday, July 23, 2013 at 10:12 AM, Cassiano Leal wrote:

On July 23, 2013 at 12:58:31, Daniel DeLeo (dan@kallistec.com (mailto:dan@kallistec.com)) wrote:

On Tuesday, July 23, 2013 at 7:17 AM, Cassiano Leal wrote:

Hi,

I have a few apps that are deploying using the application cookbook (_php and _ruby). They work just fine most of the time, but every now and then when bootstrapping a new node I get these weird errors.

This is the interesting part on the debug log of the chef run:

  • deploy_revision[resourcemanager] action deploy[2013-07-23T14:08:47+00:00] INFO: Processing deploy_revision[resourcemanager] action deploy (/var/chef/cache/cookbooks/application/providers/default.rb line 122)
    [2013-07-23T14:08:47+00:00] DEBUG: deploy_revision[resourcemanager] finding current git revision
    [2013-07-23T14:08:47+00:00] DEBUG: deploy_revision[resourcemanager] resolving remote reference
    af5c7ff84dd852cf1c0b5ca60ea92546a20dda01 refs/heads/develop
    [2013-07-23T14:08:50+00:00] DEBUG: deploy_revision[resourcemanager] not creating /opt/apps/resourcemanager because it already exists
    [2013-07-23T14:08:50+00:00] DEBUG: deploy_revision[resourcemanager] not creating /opt/apps/resourcemanager/shared because it already exists
    [2013-07-23T14:08:50+00:00] DEBUG: deploy_revision[resourcemanager] finding current git revision
    [2013-07-23T14:08:50+00:00] INFO: deploy_revision[resourcemanager] cloning repo git@bitbucket.org (mailto:git@bitbucket.org):zynkmobile/resourcemanager.git to /opt/apps/resourcemanager/shared/cached-copy
    Cloning into '/opt/apps/resourcemanager/shared/cached-copy'...

    • clone from git@bitbucket.org (mailto:git@bitbucket.org):zynkmobile/resourcemanager.git into /opt/apps/resourcemanager/shared/cached-copy[2013-07-23T14:08:55+00:00] WARN: Error on deploying /opt/apps/resourcemanager/releases/af5c7ff84dd852cf1c0b5ca60ea92546a20dda01: Expected process to exit with [0], but received '128'
      ---- Begin output of git checkout -b deploy af5c7ff84dd852cf1c0b5ca60ea92546a20dda01 ----
      STDOUT:
      STDERR: fatal: reference is not a tree: af5c7ff84dd852cf1c0b5ca60ea92546a20dda01
      ---- End output of git checkout -b deploy af5c7ff84dd852cf1c0b5ca60ea92546a20dda01 ----
      Ran git checkout -b deploy af5c7ff84dd852cf1c0b5ca60ea92546a20dda01 returned 128
      [2013-07-23T14:08:55+00:00] INFO: Removing failed deploy /opt/apps/resourcemanager/releases/af5c7ff84dd852cf1c0b5ca60ea92546a20dda01

    • remove failed deploy /opt/apps/resourcemanager/releases/af5c7ff84dd852cf1c0b5ca60ea92546a20dda01[2013-07-23T14:08:55+00:00] DEBUG: Creating directory /var/chef/cache/revision-deploys

As can be seen, it finds the Git revision to deploy, clones the directory, but then cannot find the revision. I poked on the cached-copy, and it doesn’t have the branch from which I’m deploying, neither local nor origin/branch, which would certainly explain the error. What I don’t understand is why the branch is not there, since it’s definitely in the repo (and was found before, via git ls I presume).

Is there something else that should be set on the application resource so that this error would not occur?

Cheers,

Cassiano Leal
http://cassianoleal.com
http://twitter.com/cassianoleal

Is anyone deleting these branches, or force-pushing?
I’m not sure I understand your question. The deploy is managed 100% by the application_ruby cookbook. We’re still stabilising some of the features, so we’re deploying from the develop branch.

If you force push, commits may no longer be reachable from any branch/tag. This is an area of git that I haven't explored before, but it's described a little bit here:

http://stefan.saasen.me/articles/git-clone-in-haskell-from-the-bottom-up/

I managed to work around the issue manually by doing (after letting it fail with the error above):

cd /opt/apps/resourcemanager/shared/cached-copy
export GIT_SSH=/opt/apps/resourcemanager/deploy-ssh-wrapper
git fetch origin develop

And then re-running chef-client. Why hasn’t this branch been pulled from the repo in the first place?
When you do that, does it deploy the same reference it failed to deploy the first time? E.g, af5c7ff84dd852cf1c0b5ca60ea92546a20dda01 from your example?

--
Daniel DeLeo

On July 23, 2013 at 14:19:00, Daniel DeLeo (dan@kallistec.com) wrote:

On Tuesday, July 23, 2013 at 10:12 AM, Cassiano Leal wrote:

On July 23, 2013 at 12:58:31, Daniel DeLeo (dan@kallistec.com) wrote:

On Tuesday, July 23, 2013 at 7:17 AM, Cassiano Leal wrote:

Hi,

I have a few apps that are deploying using the application cookbook (_php and _ruby). They work just fine most of the time, but every now and then when bootstrapping a new node I get these weird errors.

This is the interesting part on the debug log of the chef run:

  • deploy_revision[resourcemanager] action deploy[2013-07-23T14:08:47+00:00] INFO: Processing deploy_revision[resourcemanager] action deploy (/var/chef/cache/cookbooks/application/providers/default.rb line 122)
    [2013-07-23T14:08:47+00:00] DEBUG: deploy_revision[resourcemanager] finding current git revision
    [2013-07-23T14:08:47+00:00] DEBUG: deploy_revision[resourcemanager] resolving remote reference
    af5c7ff84dd852cf1c0b5ca60ea92546a20dda01 refs/heads/develop
    [2013-07-23T14:08:50+00:00] DEBUG: deploy_revision[resourcemanager] not creating /opt/apps/resourcemanager because it already exists
    [2013-07-23T14:08:50+00:00] DEBUG: deploy_revision[resourcemanager] not creating /opt/apps/resourcemanager/shared because it already exists
    [2013-07-23T14:08:50+00:00] DEBUG: deploy_revision[resourcemanager] finding current git revision
    [2013-07-23T14:08:50+00:00] INFO: deploy_revision[resourcemanager] cloning repo git@bitbucket.org:zynkmobile/resourcemanager.git to /opt/apps/resourcemanager/shared/cached-copy
    Cloning into ‘/opt/apps/resourcemanager/shared/cached-copy’…

    • clone from git@bitbucket.org:zynkmobile/resourcemanager.git into /opt/apps/resourcemanager/shared/cached-copy[2013-07-23T14:08:55+00:00] WARN: Error on deploying /opt/apps/resourcemanager/releases/af5c7ff84dd852cf1c0b5ca60ea92546a20dda01: Expected process to exit with [0], but received ‘128’
      ---- Begin output of git checkout -b deploy af5c7ff84dd852cf1c0b5ca60ea92546a20dda01 ----
      STDOUT:
      STDERR: fatal: reference is not a tree: af5c7ff84dd852cf1c0b5ca60ea92546a20dda01
      ---- End output of git checkout -b deploy af5c7ff84dd852cf1c0b5ca60ea92546a20dda01 ----
      Ran git checkout -b deploy af5c7ff84dd852cf1c0b5ca60ea92546a20dda01 returned 128
      [2013-07-23T14:08:55+00:00] INFO: Removing failed deploy /opt/apps/resourcemanager/releases/af5c7ff84dd852cf1c0b5ca60ea92546a20dda01

    • remove failed deploy /opt/apps/resourcemanager/releases/af5c7ff84dd852cf1c0b5ca60ea92546a20dda01[2013-07-23T14:08:55+00:00] DEBUG: Creating directory /var/chef/cache/revision-deploys

As can be seen, it finds the Git revision to deploy, clones the directory, but then cannot find the revision. I poked on the cached-copy, and it doesn’t have the branch from which I’m deploying, neither local nor origin/branch, which would certainly explain the error. What I don’t understand is why the branch is not there, since it’s definitely in the repo (and was found before, via git ls I presume).

Is there something else that should be set on the application resource so that this error would not occur?

Cheers,

Cassiano Leal
http://cassianoleal.com
http://twitter.com/cassianoleal

Is anyone deleting these branches, or force-pushing?
I’m not sure I understand your question. The deploy is managed 100% by the application_ruby cookbook. We’re still stabilising some of the features, so we’re deploying from the develop branch.

If you force push, commits may no longer be reachable from any branch/tag. This is an area of git that I haven’t explored before, but it’s described a little bit here:

http://stefan.saasen.me/articles/git-clone-in-haskell-from-the-bottom-up/

I managed to work around the issue manually by doing (after letting it fail with the error above):

cd /opt/apps/resourcemanager/shared/cached-copy

export GIT_SSH=/opt/apps/resourcemanager/deploy-ssh-wrapper

git fetch origin develop

And then re-running chef-client. Why hasn’t this branch been pulled from the repo in the first place?

When you do that, does it deploy the same reference it failed to deploy the first time? E.g, af5c7ff84dd852cf1c0b5ca60ea92546a20dda01 from your example?

Yes:

$ ls -l /opt/apps/resourcemanager/releases/
total 4
drwxr-xr-x 15 resourcemanager apps 4096 Jul 23 15:24 af5c7ff84dd852cf1c0b5ca60ea92546a20dda01


Daniel DeLeo

Seems relevant to add — I’m referencing the branch name:

application_ruby “resourcemanager” do

revision ‘develop’

end

Could this be the source of my problems? If so, how do I fix it, or what should I be passing to ‘revision’ instead of the branch name?

Cheers,

Cassiano Leal
http://cassianoleal.com
http://twitter.com/cassianoleal

On July 23, 2013 at 14:21:03, Cassiano Leal (cassianoleal@gmail.com) wrote:

On July 23, 2013 at 14:19:00, Daniel DeLeo (dan@kallistec.com) wrote:

On Tuesday, July 23, 2013 at 10:12 AM, Cassiano Leal wrote:

On July 23, 2013 at 12:58:31, Daniel DeLeo (dan@kallistec.com) wrote:

On Tuesday, July 23, 2013 at 7:17 AM, Cassiano Leal wrote:

Hi,

I have a few apps that are deploying using the application cookbook (_php and _ruby). They work just fine most of the time, but every now and then when bootstrapping a new node I get these weird errors.

This is the interesting part on the debug log of the chef run:

  • deploy_revision[resourcemanager] action deploy[2013-07-23T14:08:47+00:00] INFO: Processing deploy_revision[resourcemanager] action deploy (/var/chef/cache/cookbooks/application/providers/default.rb line 122)
    [2013-07-23T14:08:47+00:00] DEBUG: deploy_revision[resourcemanager] finding current git revision
    [2013-07-23T14:08:47+00:00] DEBUG: deploy_revision[resourcemanager] resolving remote reference
    af5c7ff84dd852cf1c0b5ca60ea92546a20dda01 refs/heads/develop
    [2013-07-23T14:08:50+00:00] DEBUG: deploy_revision[resourcemanager] not creating /opt/apps/resourcemanager because it already exists
    [2013-07-23T14:08:50+00:00] DEBUG: deploy_revision[resourcemanager] not creating /opt/apps/resourcemanager/shared because it already exists
    [2013-07-23T14:08:50+00:00] DEBUG: deploy_revision[resourcemanager] finding current git revision
    [2013-07-23T14:08:50+00:00] INFO: deploy_revision[resourcemanager] cloning repo git@bitbucket.org:zynkmobile/resourcemanager.git to /opt/apps/resourcemanager/shared/cached-copy
    Cloning into ‘/opt/apps/resourcemanager/shared/cached-copy’…

    • clone from git@bitbucket.org:zynkmobile/resourcemanager.git into /opt/apps/resourcemanager/shared/cached-copy[2013-07-23T14:08:55+00:00] WARN: Error on deploying /opt/apps/resourcemanager/releases/af5c7ff84dd852cf1c0b5ca60ea92546a20dda01: Expected process to exit with [0], but received ‘128’
      ---- Begin output of git checkout -b deploy af5c7ff84dd852cf1c0b5ca60ea92546a20dda01 ----
      STDOUT:
      STDERR: fatal: reference is not a tree: af5c7ff84dd852cf1c0b5ca60ea92546a20dda01
      ---- End output of git checkout -b deploy af5c7ff84dd852cf1c0b5ca60ea92546a20dda01 ----
      Ran git checkout -b deploy af5c7ff84dd852cf1c0b5ca60ea92546a20dda01 returned 128
      [2013-07-23T14:08:55+00:00] INFO: Removing failed deploy /opt/apps/resourcemanager/releases/af5c7ff84dd852cf1c0b5ca60ea92546a20dda01

    • remove failed deploy /opt/apps/resourcemanager/releases/af5c7ff84dd852cf1c0b5ca60ea92546a20dda01[2013-07-23T14:08:55+00:00] DEBUG: Creating directory /var/chef/cache/revision-deploys

As can be seen, it finds the Git revision to deploy, clones the directory, but then cannot find the revision. I poked on the cached-copy, and it doesn’t have the branch from which I’m deploying, neither local nor origin/branch, which would certainly explain the error. What I don’t understand is why the branch is not there, since it’s definitely in the repo (and was found before, via git ls I presume).

Is there something else that should be set on the application resource so that this error would not occur?

Cheers,

Cassiano Leal
http://cassianoleal.com
http://twitter.com/cassianoleal

Is anyone deleting these branches, or force-pushing?
I’m not sure I understand your question. The deploy is managed 100% by the application_ruby cookbook. We’re still stabilising some of the features, so we’re deploying from the develop branch.

If you force push, commits may no longer be reachable from any branch/tag. This is an area of git that I haven’t explored before, but it’s described a little bit here:

http://stefan.saasen.me/articles/git-clone-in-haskell-from-the-bottom-up/

I managed to work around the issue manually by doing (after letting it fail with the error above):

cd /opt/apps/resourcemanager/shared/cached-copy

export GIT_SSH=/opt/apps/resourcemanager/deploy-ssh-wrapper

git fetch origin develop

And then re-running chef-client. Why hasn’t this branch been pulled from the repo in the first place?

When you do that, does it deploy the same reference it failed to deploy the first time? E.g, af5c7ff84dd852cf1c0b5ca60ea92546a20dda01 from your example?

Yes:

$ ls -l /opt/apps/resourcemanager/releases/
total 4
drwxr-xr-x 15 resourcemanager apps 4096 Jul 23 15:24 af5c7ff84dd852cf1c0b5ca60ea92546a20dda01


Daniel DeLeo

On Tuesday, July 23, 2013 at 2:05 PM, Cassiano Leal wrote:

Seems relevant to add — I’m referencing the branch name:

application_ruby “resourcemanager” do

revision ‘develop’

end

Could this be the source of my problems? If so, how do I fix it, or what should I be passing to ‘revision’ instead of the branch name?
It shouldn't matter how you specify the revision. If it's not a commit hash, the provider uses git ls-remote to figure out what the commit is, then uses that from then on.

Looks like the repo you get from git clone doesn't contain the SHA it's supposed to have, which is pretty odd since a subsequent git fetch picks it up. From the command line, with the same git version, does it fail if you

COMMIT_ID = git ls-remote GIT_URL BRANCH
git clone REPO
git checkout -b deploy COMMIT_ID

--
Daniel DeLeo

It works:

resourcemanager@ip-10-0-1-105:/tmp$ COMMIT_ID=git ls-remote git@bitbucket.org:zynkmobile/resourcemanager.git develop
resourcemanager@ip-10-0-1-105:/tmp$ git clone git@bitbucket.org:zynkmobile/resourcemanager.git
Cloning into ‘resourcemanager’…
remote: Counting objects: 1703, done.
remote: Compressing objects: 100% (729/729), done.
remote: Total 1703 (delta 1193), reused 1349 (delta 948)
Receiving objects: 100% (1703/1703), 20.07 MiB | 183 KiB/s, done.
Resolving deltas: 100% (1193/1193), done.
resourcemanager@ip-10-0-1-105:/tmp$ cd resourcemanager/
resourcemanager@ip-10-0-1-105:/tmp/resourcemanager$ git checkout -b deploy COMMIT_ID fatal: Cannot update paths and switch to branch 'deploy' at the same time. Did you intend to checkout 'refs/heads/develop' which can not be resolved as commit? resourcemanager@ip-10-0-1-105:/tmp/resourcemanager echo COMMIT_ID af5c7ff84dd852cf1c0b5ca60ea92546a20dda01 refs/heads/develop resourcemanager@ip-10-0-1-105:/tmp/resourcemanager git checkout -b deploy echo $COMMIT_ID | cut -d' ' -f1
Switched to a new branch 'deploy’
resourcemanager@ip-10-0-1-105:/tmp/resourcemanager$

Another interesting thing is that if I list branched on this manually cloned repo, here’s what I get:

resourcemanager@ip-10-0-1-105:/tmp/resourcemanager$ git branch -a

  • deploy
    master
    remotes/origin/HEAD -> origin/master
    remotes/origin/develop
    remotes/origin/master

On the one that was cloned by Chef, on the other hand:

resourcemanager@ip-10-0-1-105:/opt/apps/resourcemanager/shared/cached-copy$ git branch -a

  • master
    remotes/origin/HEAD -> origin/master
    remotes/origin/master

The develop branch is not present. My only guess is that Chef’s way of cloning does not clone the whole thing, and for some reason leaves the branch I need out.


Cassiano Leal
http://cassianoleal.com
http://twitter.com/cassianoleal

On July 23, 2013 at 22:07:47, Daniel DeLeo (dan@kallistec.com) wrote:

On Tuesday, July 23, 2013 at 2:05 PM, Cassiano Leal wrote:
Seems relevant to add — I’m referencing the branch name:

application_ruby “resourcemanager” do

revision ‘develop’

end

Could this be the source of my problems? If so, how do I fix it, or what should I be passing to ‘revision’ instead of the branch name?
It shouldn’t matter how you specify the revision. If it’s not a commit hash, the provider uses git ls-remote to figure out what the commit is, then uses that from then on.

Looks like the repo you get from git clone doesn’t contain the SHA it’s supposed to have, which is pretty odd since a subsequent git fetch picks it up. From the command line, with the same git version, does it fail if you

COMMIT_ID = git ls-remote GIT_URL BRANCH
git clone REPO
git checkout -b deploy COMMIT_ID


Daniel DeLeo

On Wednesday, July 24, 2013 at 6:54 AM, Cassiano Leal wrote:

It works:

resourcemanager@ip-10-0-1-105:/tmp$ COMMIT_ID=git ls-remote git@bitbucket.org (mailto:git@bitbucket.org):zynkmobile/resourcemanager.git develop
resourcemanager@ip-10-0-1-105:/tmp$ git clone git@bitbucket.org (mailto:git@bitbucket.org):zynkmobile/resourcemanager.git
Cloning into 'resourcemanager'...
remote: Counting objects: 1703, done.
remote: Compressing objects: 100% (729/729), done.
remote: Total 1703 (delta 1193), reused 1349 (delta 948)
Receiving objects: 100% (1703/1703), 20.07 MiB | 183 KiB/s, done.
Resolving deltas: 100% (1193/1193), done.
resourcemanager@ip-10-0-1-105:/tmp$ cd resourcemanager/
resourcemanager@ip-10-0-1-105:/tmp/resourcemanager$ git checkout -b deploy $COMMIT_ID
fatal: Cannot update paths and switch to branch 'deploy' at the same time.
Did you intend to checkout 'refs/heads/develop' which can not be resolved as commit?
resourcemanager@ip-10-0-1-105:/tmp/resourcemanager$ echo $COMMIT_ID
af5c7ff84dd852cf1c0b5ca60ea92546a20dda01 refs/heads/develop
resourcemanager@ip-10-0-1-105:/tmp/resourcemanager$ git checkout -b deploy echo $COMMIT_ID | cut -d' ' -f1
Switched to a new branch 'deploy'
resourcemanager@ip-10-0-1-105:/tmp/resourcemanager$

Another interesting thing is that if I list branched on this manually cloned repo, here’s what I get:

resourcemanager@ip-10-0-1-105:/tmp/resourcemanager$ git branch -a

  • deploy
    master
    remotes/origin/HEAD -> origin/master
    remotes/origin/develop
    remotes/origin/master

On the one that was cloned by Chef, on the other hand:

resourcemanager@ip-10-0-1-105:/opt/apps/resourcemanager/shared/cached-copy$ git branch -a

  • master
    remotes/origin/HEAD -> origin/master
    remotes/origin/master

The develop branch is not present. My only guess is that Chef’s way of cloning does not clone the whole thing, and for some reason leaves the branch I need out.
The only options chef passes to git clone are -o REMOTE if remote is not set to "origin" (default) and --depth DEPTH if you specified a clone depth. Are you setting shallow_clone true? Does it work if you explicitly pass shallow_clone false ?

--
Daniel DeLeo

After a whole day trying to make beam.smp stop consuming a whole core of my chef server, I was able to try that.

Indeed setting shallow_clone to false solves the problem. It this the best approach, though? And why is that, since the revision that I’m trying to deploy is much newer than whatever is on master?

Thanks for your help!

Cassiano Leal
http://cassianoleal.com
http://twitter.com/cassianoleal

On July 24, 2013 at 12:24:32, Daniel DeLeo (dan@kallistec.com) wrote:

On Wednesday, July 24, 2013 at 6:54 AM, Cassiano Leal wrote:

It works:

resourcemanager@ip-10-0-1-105:/tmp$ COMMIT_ID=git ls-remote git@bitbucket.org:zynkmobile/resourcemanager.git develop
resourcemanager@ip-10-0-1-105:/tmp$ git clone git@bitbucket.org:zynkmobile/resourcemanager.git
Cloning into ‘resourcemanager’…
remote: Counting objects: 1703, done.
remote: Compressing objects: 100% (729/729), done.
remote: Total 1703 (delta 1193), reused 1349 (delta 948)
Receiving objects: 100% (1703/1703), 20.07 MiB | 183 KiB/s, done.
Resolving deltas: 100% (1193/1193), done.
resourcemanager@ip-10-0-1-105:/tmp$ cd resourcemanager/
resourcemanager@ip-10-0-1-105:/tmp/resourcemanager$ git checkout -b deploy COMMIT_ID fatal: Cannot update paths and switch to branch 'deploy' at the same time. Did you intend to checkout 'refs/heads/develop' which can not be resolved as commit? resourcemanager@ip-10-0-1-105:/tmp/resourcemanager echo COMMIT_ID af5c7ff84dd852cf1c0b5ca60ea92546a20dda01 refs/heads/develop resourcemanager@ip-10-0-1-105:/tmp/resourcemanager git checkout -b deploy echo $COMMIT_ID | cut -d' ' -f1
Switched to a new branch 'deploy’
resourcemanager@ip-10-0-1-105:/tmp/resourcemanager$

Another interesting thing is that if I list branched on this manually cloned repo, here’s what I get:

resourcemanager@ip-10-0-1-105:/tmp/resourcemanager$ git branch -a

  • deploy
    master
    remotes/origin/HEAD -> origin/master
    remotes/origin/develop
    remotes/origin/master

On the one that was cloned by Chef, on the other hand:

resourcemanager@ip-10-0-1-105:/opt/apps/resourcemanager/shared/cached-copy$ git branch -a

  • master
    remotes/origin/HEAD -> origin/master
    remotes/origin/master

The develop branch is not present. My only guess is that Chef’s way of cloning does not clone the whole thing, and for some reason leaves the branch I need out.
The only options chef passes to git clone are -o REMOTE if remote is not set to “origin” (default) and --depth DEPTH if you specified a clone depth. Are you setting shallow_clone true? Does it work if you explicitly pass shallow_clone false ?


Daniel DeLeo

On Thursday, July 25, 2013 at 6:50 AM, Cassiano Leal wrote:

After a whole day trying to make beam.smp stop consuming a whole core of my chef server, I was able to try that.

Indeed setting shallow_clone to false solves the problem. It this the best approach, though? And why is that, since the revision that I’m trying to deploy is much newer than whatever is on master?
It's probably the only thing you can do that doesn't involve patching chef. I'm not sure exactly how clone depth works in git, in particular how it handles branches other than master. Here's what the manual says:

   --depth <depth>
       Create a shallow clone with a history truncated to the specified number of revisions. A shallow repository has a
       number of limitations (you cannot clone or fetch from it, nor push from nor into it), but is adequate if you are
       only interested in the recent history of a large project with a long history, and would want to send in fixes as
       patches.

There is a --branch option to clone:

   --branch <name>, -b <name>
       Instead of pointing the newly created HEAD to the branch pointed to by the cloned repository's HEAD, point to
       <name> branch instead.  --branch can also take tags and treat them like detached HEAD. In a non-bare repository,
       this is the branch that will be checked out.

If you'd like to investigate further, you can see whether using --depth and --branch together results in a usable repo. I did a quick check and it seems to work for branches and tags, so it might be possible to fix this problem by patching chef to use -b.

Thanks for your help!

Cassiano Leal
http://cassianoleal.com
http://twitter.com/cassianoleal

--
Daniel DeLeo

On July 25, 2013 at 12:45:54, Daniel DeLeo (dan@kallistec.com) wrote:

On Thursday, July 25, 2013 at 6:50 AM, Cassiano Leal wrote:

Indeed setting shallow_clone to false solves the problem. It this the best approach, though? And why is that, since the revision that I’m trying to deploy is much newer than whatever is on master?
It’s probably the only thing you can do that doesn’t involve patching chef. I’m not sure exactly how clone depth works in git, in particular how it handles branches other than master. Here’s what the manual says:

   --depth <depth>
       Create a shallow clone with a history truncated to the specified number of revisions. A shallow repository has a
       number of limitations (you cannot clone or fetch from it, nor push from nor into it), but is adequate if you are
       only interested in the recent history of a large project with a long history, and would want to send in fixes as
       patches.

There is a --branch option to clone:

   --branch <name>, -b <name>
       Instead of pointing the newly created HEAD to the branch pointed to by the cloned repository's HEAD, point to
       <name> branch instead.  --branch can also take tags and treat them like detached HEAD. In a non-bare repository,
       this is the branch that will be checked out.

If you’d like to investigate further, you can see whether using --depth and --branch together results in a usable repo. I did a quick check and it seems to work for branches and tags, so it might be possible to fix this problem by patching chef to use -b.
I see. I’ll keep shallow_clone false for now, then.

I’ll spare some time later on to investigate whether combining the two flags solves the problem. If so, I’ll open a ticket and have a go at providing a patches for the deploy resource and the application cookbook to implement a branch attribute.

Thanks!


Daniel DeLeo

Cassiano Leal
http://cassianoleal.com
http://twitter.com/cassianoleal

Hello

I would like to mention I just ran into the same issue: git resources
shallow clones cannot work with a branch different than master. I'm working
on a project with 8 years of heavy history; so this is very annoying for me.
Also, from what I know Chef git resource is heavily inspired from
Capistrano, right? From what I could see, Capistrano had the very same
issue but then they patched Capistrano to add the -b option:

I think such patch would be welcome in Chef.

Regards.