I know that ChefDK meant for dev machines, I can find quite a few statements here in this mail lists that confirms that, but all of it pretty old (back from 2014).
ChefDK has evolved over the years, so as we, and probably it is time to have a fresh look on the issue.
ChefDK provides a sustainable baseline. In my humble opinion - people should be using ChefDK on CI machines too. It gives an upgrade path, it focusing community on using stable versions of gems and eliminates the struggle with resolving all that ‘dependency hell’ we all had to deal with before ChefDK. Having to cook your CI server for Chef on your own leaves so many room for mistakes, especially that there are no (as far as I know) ‘official’ HowTo doc for this. Something tells me these days pretty much everyone (except probably some legacy setups) are using ChefDK on CI servers, at least it sounds like a common sense to me.
However, as of now ChefDK is pretty much locked down in terms of it’s ingredients. I can see why though, having people (especially newcomers) able to mess up with their /opt/chefdk locally was causing many false-negative feedback and was leading to spending time investigating not real issues. However, for managed CI-environment it is not the case. Either it is a chef-controlled machine or a container, there is no way to mess up with chefdk folder to a degree to not be able to recover, as these would not be any manual actions. In the same time, having the ability to change versions of some gems in the chefdk is crucial at a times to unblock the progress. To get a fix into ChefDK as of now can easily take months, because it involves having the fix done, having a new gem released and only then having that gem shipped into ChefDK. Agile teams these days cannot afford waiting for months, they want to move fast (and break things hehe). There is more background on my specific case here https://github.com/chef/chef-dk/issues/1559 - I started it as an issue and then being told mailing list is the right place to have a conversation like that.
Instead of cooking my own CI env with bundler, it sounds much more safe to have ChefDK installed as a baseline and then overriding some versions of some gems. It is safe in a managed environments and also provides a way to manage technical debt - I can leave a comment in my recipe or a Dockerfile with the link to the issue we’re waiting a fix for and later on be safely able to remove that ‘patch’ override and turn back to clean ChefDK installation. In opposite, having your own bundler-based setup leaves you on your own to find out a way to keep your CI environment as close to ChefDK and community effort as possible (IF you are smart enough to understand that you’d better be close).
To sum up - we have a problem statement: no ability to override gems in a managed CI env.
I don’t know what the actual solution would be, but just to start some discussion here are a few options from the top of my head:
- Having ChefDK version pins somewhat relaxed. Maybe getting rid of appbundler leaving with just a Gemfile.lock, or some other option that does not lock you down with /opt/chefdk/bin overrides.
- Having a new tool or improvements to appbundle-updater so that I can feed a single source of truth to it (like a Gemfile) and having it to rebuild /opt/chefdk for me. Having a docs for that if this is something already possible with a reasonable effort…
- Having a separate ChefDK bundle for CI servers, somewhat aligned with or generated from ChefDK, maybe in a form of Dockerfile (but not just a Dockerfile that installs the package since it will not solve a problem statement above).