Hi,
We just started using chef for our application deployments which are rpms so at
the moment we are using the package resource to handle this installation. I
was directed to the application cookbook
(https://github.com/opscode-cookbooks/application) but it seems this cookbook
doesn’t support rpm’s and also requires the application to be pulled from a
repository. Are there plans to update the application cookbook to support rpms
or any other suggestions?
In addition I wanted to get some suggestions on how best to manage the version
number of the application rpm being deployed into qa/staging/production. Since
we have a separate continuous integration pipeline for our infrastructure code
(cookbooks) and the rev’ing of the cookbook in the production.json file is
automated through our CI system I would rather not store the application rpm
version in the environment files. I was thinking of storing this in the
application role and namespace it like so [‘app’][‘version’][‘production’] =
“1.0”. Any thoughts around this or suggested approaches?
Thanks,
Jay
On Thu, Jul 12, 2012 at 9:13 AM, JayP bosoxjay@gmail.com wrote:
In addition I wanted to get some suggestions on how best to manage the version
number of the application rpm being deployed into qa/staging/production. Since
we have a separate continuous integration pipeline for our infrastructure code
(cookbooks) and the rev'ing of the cookbook in the production.json file is
automated through our CI system I would rather not store the application rpm
Mapping application versions to environments is a good use for a data
bag. At my former company we did this initially and eventually
replaced the data bag with a separate web service that provided
similar data but could run custom business logic to manage version
promotion. This was for deploying maven artifacts (java shop) but a
similar strategy should work for rpm versions. I should bug those guys
about presenting on that stuff someday.
KC
On Thu, Jul 12, 2012 at 6:13 PM, JayP bosoxjay@gmail.com wrote:
Hi,
We just started using chef for our application deployments which are rpms so at
the moment we are using the package resource to handle this installation. I
was directed to the application cookbook
(https://github.com/opscode-cookbooks/application) but it seems this cookbook
doesn't support rpm's and also requires the application to be pulled from a
repository. Are there plans to update the application cookbook to support rpms
or any other suggestions?
I've thought about that while working on that cookbook, and while I
get the reasoning about this, I'm still kind of conflicted.
On the one hand, if you simply need to install rpms, manage some files
with templates and start/restart a service, you won't get much out of
the application cookbook. You can do all that with your own recipes,
without having to learn a whole new set of conventions.
For example, the application cookbook keeps the last few checkouts,
and creates a symlink to the "current" version, and tries to do that
in a reliable way.
How would you map that to installation via RPMs?
On the other hand, it would be nice to use the same DSL regardless of
where the app comes from.
In my case, I can see myself using git in staging and pre-build
archives (coming from Jenkins) in production.
In other words, once you've gone past the learning curve with the
application cookbook, you'll want to manage everything in the same
way.
In addition I wanted to get some suggestions on how best to manage the version
number of the application rpm being deployed into qa/staging/production. Since
we have a separate continuous integration pipeline for our infrastructure code
(cookbooks) and the rev'ing of the cookbook in the production.json file is
automated through our CI system I would rather not store the application rpm
version in the environment files. I was thinking of storing this in the
application role and namespace it like so ['app']['version']['production'] =
"1.0". Any thoughts around this or suggested approaches?
I think this is very appropriate, that's pretty much what I do.
Andrea
Thanks KC, any reason you chose a data bag over using a role to store the
version info? Did you check in the data bag you kept bumping the version
on or it was just pushed to the chef server each time and it was a databag
that wasn't stored in source control? I just don't like that our CI server
will need to check in the data bag or role so I think we'll just keep a
copy of the databag in our CI server.
I'm also interested if you could get those those guys to present.
Thanks,
Jay
On Thu, Jul 12, 2012 at 2:25 PM, KC Braunschweig
kcbraunschweig@gmail.comwrote:
On Thu, Jul 12, 2012 at 9:13 AM, JayP bosoxjay@gmail.com wrote:
In addition I wanted to get some suggestions on how best to manage the
version
number of the application rpm being deployed into qa/staging/production.
Since
we have a separate continuous integration pipeline for our
infrastructure code
(cookbooks) and the rev'ing of the cookbook in the production.json file
is
automated through our CI system I would rather not store the application
rpm
Mapping application versions to environments is a good use for a data
bag. At my former company we did this initially and eventually
replaced the data bag with a separate web service that provided
similar data but could run custom business logic to manage version
promotion. This was for deploying maven artifacts (java shop) but a
similar strategy should work for rpm versions. I should bug those guys
about presenting on that stuff someday.
KC
Hi Andrea,
I think I will stick with using my own cookbooks for now but would like to
have something like the application cookbook that we can use for all our
application deploys which will be rpms. Basically I want to avoid of
repeating the same code for each application.
Thanks for the reassurance that putting the version info in a role is an
acceptable solution and not a bad practice.
Thanks,
Jay
On Thu, Jul 12, 2012 at 3:12 PM, Andrea Campi
andrea.campi@zephirworks.comwrote:
On Thu, Jul 12, 2012 at 6:13 PM, JayP bosoxjay@gmail.com wrote:
Hi,
We just started using chef for our application deployments which are
rpms so at
the moment we are using the package resource to handle this
installation. I
was directed to the application cookbook
(GitHub - poise/application: A Chef cookbook to deploy applications.) but it seems this
cookbook
doesn't support rpm's and also requires the application to be pulled
from a
repository. Are there plans to update the application cookbook to
support rpms
or any other suggestions?
I've thought about that while working on that cookbook, and while I
get the reasoning about this, I'm still kind of conflicted.
On the one hand, if you simply need to install rpms, manage some files
with templates and start/restart a service, you won't get much out of
the application cookbook. You can do all that with your own recipes,
without having to learn a whole new set of conventions.
For example, the application cookbook keeps the last few checkouts,
and creates a symlink to the "current" version, and tries to do that
in a reliable way.
How would you map that to installation via RPMs?
On the other hand, it would be nice to use the same DSL regardless of
where the app comes from.
In my case, I can see myself using git in staging and pre-build
archives (coming from Jenkins) in production.
In other words, once you've gone past the learning curve with the
application cookbook, you'll want to manage everything in the same
way.
In addition I wanted to get some suggestions on how best to manage the
version
number of the application rpm being deployed into qa/staging/production.
Since
we have a separate continuous integration pipeline for our
infrastructure code
(cookbooks) and the rev'ing of the cookbook in the production.json file
is
automated through our CI system I would rather not store the application
rpm
version in the environment files. I was thinking of storing this in the
application role and namespace it like so
['app']['version']['production'] =
"1.0". Any thoughts around this or suggested approaches?
I think this is very appropriate, that's pretty much what I do.
Andrea
On Thu, Jul 12, 2012 at 6:43 PM, Jay Perry bosoxjay@gmail.com wrote:
Thanks KC, any reason you chose a data bag over using a role to store the
version info? Did you check in the data bag you kept bumping the version on
or it was just pushed to the chef server each time and it was a databag that
wasn't stored in source control? I just don't like that our CI server will
need to check in the data bag or role so I think we'll just keep a copy of
the databag in our CI server.
Mapping versions onto environments is totally arbitrary data from the
node's perspective so a data bag made more sense. If your version is
in the role, your role goes to all environments when you upload a
change so you lose your staged rollouts. You could have different
attributes per environment in the role but that seems hacky,
repetitive and prone to error. You could have different roles for
different environments, but then you're not deploying exactly the same
thing in prod as dev and test which is a CD antipattern.
We made our source repo the source of truth. You can have the chef
server be the source of truth for your data bag if you want. Either
way works as long as you pick one and only one. However, I recommend
the source tree because it is truth for everything else, why split?
There are tools out there to have jenkins or whatever you use push
stuff to the chef server in relatively sane ways.
That being said, we didn't stick with databags for long. Pretty
quickly became apparent that we wanted custom logic to manage multiple
CI/CD pipelines for different products so data bags got replaced by a
service fairly quickly. I'll hit them up again about talking about it.
KC
One more question 
How is using a databag give you more control over staged deployments. Could you provide and example of a databag for an application?
Thanks,
-- Jay
On Jul 13, 2012, at 2:18 AM, KC Braunschweig kcbraunschweig@gmail.com wrote:
On Thu, Jul 12, 2012 at 6:43 PM, Jay Perry bosoxjay@gmail.com wrote:
Thanks KC, any reason you chose a data bag over using a role to store the
version info? Did you check in the data bag you kept bumping the version on
or it was just pushed to the chef server each time and it was a databag that
wasn't stored in source control? I just don't like that our CI server will
need to check in the data bag or role so I think we'll just keep a copy of
the databag in our CI server.
Mapping versions onto environments is totally arbitrary data from the
node's perspective so a data bag made more sense. If your version is
in the role, your role goes to all environments when you upload a
change so you lose your staged rollouts. You could have different
attributes per environment in the role but that seems hacky,
repetitive and prone to error. You could have different roles for
different environments, but then you're not deploying exactly the same
thing in prod as dev and test which is a CD antipattern.
We made our source repo the source of truth. You can have the chef
server be the source of truth for your data bag if you want. Either
way works as long as you pick one and only one. However, I recommend
the source tree because it is truth for everything else, why split?
There are tools out there to have jenkins or whatever you use push
stuff to the chef server in relatively sane ways.
That being said, we didn't stick with databags for long. Pretty
quickly became apparent that we wanted custom logic to manage multiple
CI/CD pipelines for different products so data bags got replaced by a
service fairly quickly. I'll hit them up again about talking about it.
KC
And also a follow up to the CI Server, when you build a newer version of
your application did you have your CI server check in the data bag before
pushing to the chef server? Did you ever run into any issues with this
approach?
Thanks,
Jay
On Fri, Jul 13, 2012 at 7:43 AM, Jason Perry bosoxjay@gmail.com wrote:
One more question
How is using a databag give you more control over staged deployments.
Could you provide and example of a databag for an application?
Thanks,
-- Jay
On Jul 13, 2012, at 2:18 AM, KC Braunschweig kcbraunschweig@gmail.com
wrote:
On Thu, Jul 12, 2012 at 6:43 PM, Jay Perry bosoxjay@gmail.com wrote:
Thanks KC, any reason you chose a data bag over using a role to store
the
version info? Did you check in the data bag you kept bumping the
version on
or it was just pushed to the chef server each time and it was a databag
that
wasn't stored in source control? I just don't like that our CI server
will
need to check in the data bag or role so I think we'll just keep a copy
of
the databag in our CI server.
Mapping versions onto environments is totally arbitrary data from the
node's perspective so a data bag made more sense. If your version is
in the role, your role goes to all environments when you upload a
change so you lose your staged rollouts. You could have different
attributes per environment in the role but that seems hacky,
repetitive and prone to error. You could have different roles for
different environments, but then you're not deploying exactly the same
thing in prod as dev and test which is a CD antipattern.
We made our source repo the source of truth. You can have the chef
server be the source of truth for your data bag if you want. Either
way works as long as you pick one and only one. However, I recommend
the source tree because it is truth for everything else, why split?
There are tools out there to have jenkins or whatever you use push
stuff to the chef server in relatively sane ways.
That being said, we didn't stick with databags for long. Pretty
quickly became apparent that we wanted custom logic to manage multiple
CI/CD pipelines for different products so data bags got replaced by a
service fairly quickly. I'll hit them up again about talking about it.
KC
On Fri, Jul 13, 2012 at 4:43 AM, Jason Perry bosoxjay@gmail.com wrote:
How is using a databag give you more control over staged deployments. Could you provide and example of a databag for an application?
A databag doesn't in and of itself but a role definitely puts you in a
corner. You could put the versions in environment files rather than
the data bag. That'd work. In our case we wanted more than that
though, we wanted a place to store (1) arbitrary data about an
artifact regardless of node or environment, e.g. deployment method and
(2) artifact configuration specific to an environment. Maybe that
extra complexity shows why the databag based solution didn't last?
KC
On Fri, Jul 13, 2012 at 5:56 AM, Jay Perry bosoxjay@gmail.com wrote:
And also a follow up to the CI Server, when you build a newer version of
your application did you have your CI server check in the data bag before
pushing to the chef server? Did you ever run into any issues with this
approach?
The databag based solution was gone before it mattered. However this
should be manageable. The CI server creates your versioned artifact
and puts it in a repo (maven artifacts in nexus in our case). This
should always happen first. You make the version available but nothing
will use it. Then you update your databag or whatever to tell an
environment to use the new version you made available. No node takes
action based on the repo so the only thing to work around is what if
your upload of the data bag fails? You probably can start with manual
intervention at that point because you have a bug in your CI code
generating a bad databag item or your chef server is hosed and you
have bigger problems.
KC
Thanks for all the great info. It really helps in my decision of what direction to go.
Thanks,
-- Jay
On Jul 13, 2012, at 4:19 PM, KC Braunschweig kcbraunschweig@gmail.com wrote:
On Fri, Jul 13, 2012 at 5:56 AM, Jay Perry bosoxjay@gmail.com wrote:
And also a follow up to the CI Server, when you build a newer version of
your application did you have your CI server check in the data bag before
pushing to the chef server? Did you ever run into any issues with this
approach?
The databag based solution was gone before it mattered. However this
should be manageable. The CI server creates your versioned artifact
and puts it in a repo (maven artifacts in nexus in our case). This
should always happen first. You make the version available but nothing
will use it. Then you update your databag or whatever to tell an
environment to use the new version you made available. No node takes
action based on the repo so the only thing to work around is what if
your upload of the data bag fails? You probably can start with manual
intervention at that point because you have a bug in your CI code
generating a bad databag item or your chef server is hosed and you
have bigger problems.
KC