All deployments suddenly start failing due to native gem extensions failing


#1

We had run a bunch of deployments on Friday and Saturday and everything was
going smooth.

We come back today and run some new deployments, and they start failing.
Always, the issue is bundle install fails because compiling gem extension
fails for json gem (I don’t think this is related to that gem, it just
happens to be the first on the list requiring a native extension).

This is the error trace:

Installing json (1.7.7)
Gem::Installer::ExtensionBuildError: ERROR: Failed to build gem native
extension.

    /usr/local/rubies/1.9.3-p392/bin/ruby extconf.rb

creating Makefile

make
sh: make: command not found

I do not understand the make: command not found part. For one, gem
install and even bundle install works fine when I run it manually(Our temp
workaround is actually to comment out our bundle install commands in the
recipe and run it manually after deployment). How can make suddenly
disappear from Chef’s path? We are on Chef 10.24.0. This issue suddenly
appeared across ALL our deployments: remote AWS instances, local CENTOS
development servers and even newly spinned off AWS instances. We’re
honestly buggered. Hope someone knows what’s going on!


#2

It look like ‘make’ is installed or isn’t in the path for the user your automation runs as. When it runs manually later are you using the same user? If so maybe make is getting installed by a recipe after that.

At least that’s where I’d start my troubleshooting if I saw that error

Michael Glenney
Sent from my iPhone

On Mar 18, 2013, at 5:15 AM, Prajwal Manjunath prajwal@elitmus.com wrote:

We had run a bunch of deployments on Friday and Saturday and everything was going smooth.

We come back today and run some new deployments, and they start failing. Always, the issue is bundle install fails because compiling gem extension fails for json gem (I don’t think this is related to that gem, it just happens to be the first on the list requiring a native extension).

This is the error trace:

Installing json (1.7.7)
Gem::Installer::ExtensionBuildError: ERROR: Failed to build gem native extension.

    /usr/local/rubies/1.9.3-p392/bin/ruby extconf.rb 

creating Makefile

make
sh: make: command not found

I do not understand the make: command not found part. For one, gem install and even bundle install works fine when I run it manually(Our temp workaround is actually to comment out our bundle install commands in the recipe and run it manually after deployment). How can make suddenly disappear from Chef’s path? We are on Chef 10.24.0. This issue suddenly appeared across ALL our deployments: remote AWS instances, local CENTOS development servers and even newly spinned off AWS instances. We’re honestly buggered. Hope someone knows what’s going on!