Compiling Chef Client source on Windows


#1

I am not able to find any help on compilation of Chef Client source on Windows. Available help topics on github are written considering non-windows platform.
Can anyone help me out compiling Chef Client on windows?
I am particularly seeking re-compilation because i would like to remove one .kitchen file from the source. This is not allowed to be present as per legal department of our company.

Thanks,
Kailas


#2

Hi Kailas,

Not too sure what you mean by compilation (do you mean packaging?) but I can help, please reach out to me directly at stuart@chef.io with some more details of what you are trying to achieve, and on which version of Windows.

Thanks,
Stuart


#3

Stuart,

Thanks for a quick response.
Yes, I am trying to create an installable Chef Client MSI for Windows 2012 R2.
Below are the steps:

  1. Installed ruby 2.4.4p296 (2018-03-28 revision 63013) [x64-mingw32]
  2. Installed MSYS2 x64
  3. gem install bundler
  4. gem intsall omnibus
  5. bundle install
  6. cd to downloaded chef source code C:\chef\omnibus
  7. bundle exec omnibus build chef
    Shows up this error
    Error is - C:/chef-12.21.3/chef/omnibus/vendor/cache/ruby/2.4.0/bundler/gems/omnibus-software-266a8ab07b03/config/software/ruby.rb:241:in block (2 levels) in evaluate': undefined methoddowncase' for nil:NilClass (NoMethodError)

I checked this file and found that ENV["MSYSTEM"] is undefined.
What this variable to be set to?

And what ENV["OMNIBUS_TOOLCHAIN_INSTALL_DIR"] set to be to?
I tried installing OMNIBUS_TOOLCHAIN but not found, where will I get this?

I had already installed MSYS2 and it got installed at C:\msys2.
I found that 'libgcc_s_seh' named dlls are present at C:\msys2\mingw64\bin
so I set MSYSTEM to 'mingw64' but the build command for chef again failed with error - C:/chef-12.21.3/chef/omnibus/vendor/cache/ruby/2.4.0/bundler/gems/omnibus-software-266a8ab07b03/config/software/ruby.rb:247:in `block (2 levels) in evaluate': Cannot find required DLL needed for dynamic linking: C:/msys2/mingw64/bin/libgcc_s_dw2-1.dll (RuntimeError)

Please guide on this.

Thanks,
Kailas


#4

Is there a reason you can't use Test Kitchen to set up your machine for building? As per the instructions in https://github.com/chef/chef/tree/master/omnibus#kitchen-based-build-environment

The three bits that look missing:

  1. The omnibus cookbook downloads and installs all the correct dependencies for build and sets up the environment correctly. https://supermarket.chef.io/cookbooks/omnibus
  2. The kitchen.yml for omnibus has the steps, and attributes that need to be set.
  3. Omnibus cookbook plants a load_omnibus_toolchain.bat that you should use to set all these variables before you try and build.

I would also note that signing the MSI will not work, but the MSI will be produced. Luckily signing the MSI is the very last step.

I'm curious as to why you're trying to build Chef 12.21.3 though, there have been multiple security patches since then and we do not support/condone the use of it, even in source form. The last stable build of Chef 12 was 12.22.5 and the installer is still available (though, unsupported) - https://downloads.chef.io/chef/stable/12.22.5

I'm also unclear as to your request to remove ".kitchen" from the distribution - could you help me understand the licensing concern? (I did provide my email address in my earlier message, if it's easier to discuss privately).

Thanks,
Stuart


#5

Thy build your own? What are you trying to generate or modify in your custom-built chef-client?

Even building on various Linux systems, I've personally encountered very real difficulty getting the omnibus bootstraps to run to completion. Build systems often have many more version sensitive software requirements than modular, operational systems.


#6

Hi,

I am also facing same issue.
Thanks Stuartpreston for https://supermarket.chef.io/cookbooks/omnibus link

I have downloaded all the cookbooks which were required for omnibus.
Following cookbooks are required.
omnibus-toolchain
omnibus
build-essential
chef-sugar
chef-ingredient
git
homebrew
remote_install
seven_zip
wix
mingw
windows

I created my chef folder wherein created cookbooks named folder and pasted all above downloaded cookbooks. And included the omnibus::default recipe in node's run list.
And ran the cookbooks using chef-solo command.

All the cookbooks were successfully executed and Omnibus cookbook planted a load_omnibus_toolchain.bat that I ran to set all required environment variables.

Finally tried the build with command - bundle exec omnibus build chef
Build took some time and ended with an error -
Error:

.libs/iconv.o:iconv.c:(.text+0x17198): undefined reference to `aliases2_lookup'

.libs/iconv.o:iconv.c:(.text+0x176e6): undefined reference to `aliases2_lookup'
collect2.exe: error: ld returned 1 exit status
make[1]: *** [Makefile:71: libiconv.la] Error 1
make: *** [Makefile:33: all] Error 2

C:/chef-12.21.3/chef/omnibus/vendor/cache/ruby/2.4.0/bundler/gems/omnibus-0212438d74eb/lib/omnibus/util.rb:139:in `rescue in shellout!'

Am I missing anything to get successful build.

Thanks,
Dipali


#7

You're also stuck on the exact same problem as @KAILAS, using the exact same versions of Chef? Are you on the same team?

I expect part of the error is that at a point in time (when we originally built that version) the versions of Chef, Omnibus, Omnibus-Toolchain, Ruby, RubyDevkit and all the dependencies worked together on the latest stable version of all of them. So to recreate a full build of Chef 12.21.3 you would need to go back in time and figure out what all those dependencies were at that point.

I'm still a little confused/perplexed at why there's a need to build/package Chef 12.21.3 for Windows though. This is a pretty unsupportable configuration. I'd like to understand your needs more though. Please can one of you email me with the requested info?


#8

Yes, we are using exact version of chef and are in same team.

We need to work on this chef version only because of release dependency.

We are building the source code because the chef third party needs to go from approval process and kitchen.yml file was asked to be removed from source code so this compilation is required to verify that after updating source executable works smoothly.

We have already successfully built the source of the same chef version on Linux RHEL 7.2.
Now the turn is of Windows and stuck up in such compilation issues.

It would be great if you could provide the exact dependencies to get a successful build on Windows also for the version we are interested in.

We understand it is bit old version but because of release dependency we can not upgrade it to latest.

Yes, you are right that compilation issues could be because of gcc or many other software which are of latest version and may not be compatible with chef-client - 12.21.3 version.
But we need to identify that what all exact versions are required and for which software so that compilation would work smoothly.

Thanks,
Dipali


#9

I'm afraid I do not have a simple way of telling you the version numbers for packages and cookbooks that build our build system at the time that we ran the build, nor should you trust that building the source from the latest version of compilers and dependencies will not introduce defects.

Can you help me understand more about the requirement to "remove kitchen.yml from the source code" and where it comes from and specifically where you think the kitchen.yml becomes part of the Chef Client distribution.

As it appears you are building software for redistribution, I also want to remind you of your obligations around the Apache2 license too, if you could get in touch at stuart@chef.io I would appreciate it.


#10

Hello,

We have removed .kitchen.yml from chef source code from omnibus folder.

We see after installation of chef-client-12.21.3-1-x64.msi at following installed location.

  1. C:\opscode\chef\embedded\lib\ruby\gems\2.3.0\gems\chef-12.21.3-universal-mingw32\acceptance\basics
  2. C:\opscode\chef\embedded\lib\ruby\gems\2.3.0\gems\chef-12.21.3-universal-mingw32\acceptance\data-collector
  3. C:\opscode\chef\embedded\lib\ruby\gems\2.3.0\gems\chef-12.21.3-universal-mingw32\acceptance\data-collector
  4. C:\opscode\chef\embedded\lib\ruby\gems\2.3.0\gems\chef-12.21.3-universal-mingw32\acceptance\trivial
  5. C:\opscode\chef\embedded\lib\ruby\gems\2.3.0\gems\chef-12.21.3-universal-mingw32\acceptance\windows-service

We are not sure whether it impacts the distribution. We checked that above all files just contains the OS version and name of suite to be installed in it. Would you please let us know that are all above files comes just because of presence of .kitchen.yml in chef client source code omnibus folder.

Does all these files come in the chef-client installed folder just because of presence of .kitchen.yml in the source code?

If we do not remove this file from source code still the compilation errors persists and those are same too.
We just wanted to know the impact of .kitchen.yml in source code as we are failing to compile the source code on Windows.

Thanks,
Dipali


#11

The presence of .kitchen.yml (which contains configuration instructions for the Test Kitchen tool) has zero bearing on compilation/packaging or running of Chef Client. .kitchen.yml is only used during development and build activities.

We would recommend you use the official installers, however Chef 12 is not supported and I can offer no further guidance on how to build it. I recommend you spend the time updating to a supported version of Chef.

Stuart


#12

Stuart,

Thanks for your inputs.
Only reason we want to exclude 'omnibus/.kitchen.yml' is that this file does not contain any license information.

Thanks,


#13

Thanks Stuart,

Good to see that omnibus/.kitchen.yml does not have any bearing on running of Chef Client.
I understand that Chef 12 is not supported, however, upgrading to the supported version is time consuming for us and we would like to stick to this version. Any quick help/insights on compilation would be of a great help.

Thanks,
Kailas


#14

We do not distribute the omnibus/.kitchen.yml file so this is a bit of a moot point. The route you are taking is pretty much cutting off any opportunity to use newer Chef features and should be avoided. I appreciate you're on the ground and just want things to work but this is not the right route and we cannot help you any further to compile/package Chef 12.

If you want to discuss please have someone from your project leadership/management team reach out to me directly so we can discuss how best to help you, I have given my email address several times in this thread.

Stuart


#15

Wait: the presence of the LICENSE file in the top of the git repo is not considered enough? What kind of legel environment are you dealing with where the licensing of the source code repository is not sufficient? Why not check the git log for the submitters of the relevant file or files and encourage a license comment if needed?