Chef-solo server deploy with many services


#1

Hello,

I’m totally newbie to chef and this is my first post on the list. I need to
deploy several servers with the following requirements:

  • Servers are mostly identical only some variables are changed. Further
    details on [1].
  • Servers are geographically dispersed.
  • I won’t do any local administration on those servers on a daily basis,
    just eventually if local admins have problems.
  • There might be some connections issues so decentralization is a must
  • The solution “should” run in all major linux distros available.
  • I need to ensure the installation and configuration of the following
    services: DHCP server, DNS server (Bind), NTP server, NFS server, Samba
    server and LDAP server, as other stuff like users, groups and folders
    creation.

For all of the above reasons I thought using chef-solo for deploy instead
of regular chef.

This is the workflow that I thought.

  • I prepare and maintain a git repository (or/and deb / rpm package)
  • The local admin clones (or install the package) and executes the main
    program which invokes chef-solo

I’ve been reading and researching for a module to configure a ISC DHCP
server. I tried the dhcp cookbook
http://community.opscode.com/cookbooks/dhcp which seems to be broken. (I
won’t post the log because is not the focus of this email).

I thought that perhaps using my files as templates and just changing the
variable values perhaps would be easier. I have a repository with the
config files here: https://github.com/ccinfra/conf/tree/master/srv/etc

Is this a reasonable approach for the project description? What would you
experts recommend? Using my conf files or using cookbooks, if cookbooks
which ones are the most featured ones?

Any good reading / recommendation on how to do this?

[1] The most important vars is $id which is used for instance as part of
the: DHCP domain, DNS zone, NFS domain, Samba workgroup, LDAP dn, etc etc.
So a user from one network can access the same services just changing the
$id

Many thanks!

Albert.

http://www.albertdelafuente.com


#2

On Wednesday, June 19, 2013 at 2:03 PM, Albert Vonpupp wrote:

Hello,

I’m totally newbie to chef and this is my first post on the list. I need to deploy several servers with the following requirements:

  • Servers are mostly identical only some variables are changed. Further details on [1].
  • Servers are geographically dispersed.
  • I won’t do any local administration on those servers on a daily basis, just eventually if local admins have problems.
  • There might be some connections issues so decentralization is a must
  • The solution “should” run in all major linux distros available.
  • I need to ensure the installation and configuration of the following services: DHCP server, DNS server (Bind), NTP server, NFS server, Samba server and LDAP server, as other stuff like users, groups and folders creation.

For all of the above reasons I thought using chef-solo for deploy instead of regular chef.

This is the workflow that I thought.

  • I prepare and maintain a git repository (or/and deb / rpm package)
  • The local admin clones (or install the package) and executes the main program which invokes chef-solo
    Quick question: how are these git repos set up or distributed? For example, if your primary git server is in data center A, and the link between data centers A and B goes down, what do you expect to accomplish on nodes in data center B? No matter what distribution mechanism you choose, you won’t be able to ship updated configuration code to the nodes in B, right? Contrarily, you would be able to create new nodes in data center B with the old config by copying the configuration code from a local copy. If that scenario is an absolute must for you, then your chef-solo based scheme makes sense, but otherwise, you’re losing cool stuff like search that could potentially make stuff like your DNS setup a breeze.

I’ve been reading and researching for a module to configure a ISC DHCP server. I tried the dhcp cookbook http://community.opscode.com/cookbooks/dhcp which seems to be broken. (I won’t post the log because is not the focus of this email).

I thought that perhaps using my files as templates and just changing the variable values perhaps would be easier. I have a repository with the config files here: https://github.com/ccinfra/conf/tree/master/srv/etc

Is this a reasonable approach for the project description? What would you experts recommend? Using my conf files or using cookbooks, if cookbooks which ones are the most featured ones?
You can think about this as a similar decision as using any open source software vs. building your own: If you write your own, it will do exactly what you want and work the way you think, but you’re responsible for every bug or new feature you want. Contrarily, using community cookbooks, a lot of work is done for you already, particularly in the area of multiple platform support, but you may have to put up with file hierarchies you don’t like, etc. Also note that there’s a pretty big range of popularity/activity/quality in different community cookbooks, so you may encounter a few “bad apples”.

Any good reading / recommendation on how to do this?

https://learnchef.opscode.com to get started, http://docs.opscode.com/chef/ for reference. There are a good number of 3rd party “getting started with chef” blog posts that may be helpful.

[1] The most important vars is $id which is used for instance as part of the: DHCP domain, DNS zone, NFS domain, Samba workgroup, LDAP dn, etc etc. So a user from one network can access the same services just changing the $id

Many thanks!

Albert.

http://www.albertdelafuente.com

Good luck!

Daniel DeLeo


#3

Thanks for your answer Daniel.

I’m having just one git repo that should be available all the time (i.e.
github or similar). My goal would be continuous delivery of an rpm / deb
package with the cookbooks within.

My main concern is if using templates of configuration files instead of
cookbooks would be better or not. Ideally I would prefer cookbooks, but I
don’t seem to find all of them, i.e: dhcp-server (working) with rndc-keys.

Any suggestion?

Thanks a lot.

On Thu, Jun 20, 2013 at 12:39 PM, Daniel DeLeo dan@kallistec.com wrote:

On Wednesday, June 19, 2013 at 2:03 PM, Albert Vonpupp wrote:

Hello,

I’m totally newbie to chef and this is my first post on the list. I need
to deploy several servers with the following requirements:

  • Servers are mostly identical only some variables are changed. Further
    details on [1].
  • Servers are geographically dispersed.
  • I won’t do any local administration on those servers on a daily basis,
    just eventually if local admins have problems.
  • There might be some connections issues so decentralization is a must
  • The solution “should” run in all major linux distros available.
  • I need to ensure the installation and configuration of the following
    services: DHCP server, DNS server (Bind), NTP server, NFS server, Samba
    server and LDAP server, as other stuff like users, groups and folders
    creation.

For all of the above reasons I thought using chef-solo for deploy instead
of regular chef.

This is the workflow that I thought.

  • I prepare and maintain a git repository (or/and deb / rpm package)
  • The local admin clones (or install the package) and executes the main
    program which invokes chef-solo

Quick question: how are these git repos set up or distributed? For
example, if your primary git server is in data center A, and the link
between data centers A and B goes down, what do you expect to accomplish on
nodes in data center B? No matter what distribution mechanism you choose,
you won’t be able to ship updated configuration code to the nodes in B,
right? Contrarily, you would be able to create new nodes in data center B
with the old config by copying the configuration code from a local copy. If
that scenario is an absolute must for you, then your chef-solo based
scheme makes sense, but otherwise, you’re losing cool stuff like search
that could potentially make stuff like your DNS setup a breeze.

I’ve been reading and researching for a module to configure a ISC DHCP
server. I tried the dhcp cookbook
http://community.opscode.com/cookbooks/dhcp which seems to be broken. (I
won’t post the log because is not the focus of this email).

I thought that perhaps using my files as templates and just changing the
variable values perhaps would be easier. I have a repository with the
config files here: https://github.com/ccinfra/conf/tree/master/srv/etc

Is this a reasonable approach for the project description? What would you
experts recommend? Using my conf files or using cookbooks, if cookbooks
which ones are the most featured ones?

You can think about this as a similar decision as using any open source
software vs. building your own: If you write your own, it will do exactly
what you want and work the way you think, but you’re responsible for every
bug or new feature you want. Contrarily, using community cookbooks, a lot
of work is done for you already, particularly in the area of multiple
platform support, but you may have to put up with file hierarchies you
don’t like, etc. Also note that there’s a pretty big range of
popularity/activity/quality in different community cookbooks, so you may
encounter a few “bad apples”.

Any good reading / recommendation on how to do this?

https://learnchef.opscode.com to get started,
http://docs.opscode.com/chef/ for reference. There are a good number of
3rd party “getting started with chef” blog posts that may be helpful.

[1] The most important vars is $id which is used for instance as part of
the: DHCP domain, DNS zone, NFS domain, Samba workgroup, LDAP dn, etc etc.
So a user from one network can access the same services just changing the
$id

Many thanks!

Albert.

http://www.albertdelafuente.com

Good luck!

Daniel DeLeo

Albert.

http://www.albertdelafuente.com


#4

On Wednesday, June 26, 2013 at 5:40 PM, Albert Vonpupp wrote:

Thanks for your answer Daniel.

I’m having just one git repo that should be available all the time (i.e. github or similar). My goal would be continuous delivery of an rpm / deb package with the cookbooks within.

My main concern is if using templates of configuration files instead of cookbooks would be better or not. Ideally I would prefer cookbooks, but I don’t seem to find all of them, i.e: dhcp-server (working) with rndc-keys.
Templates are part of cookbooks. I’m assuming you mean using a community cookbook vs. writing your own with templates based on your current config files?

Any suggestion?

Thanks a lot.
If you have the time, the best thing to do is fork the cookbooks on github, patch them to provide the feature you need, and submit a pull request. Use your forked version in the meantime while working with the maintainer to get your patch in.

If you don’t have time for that, or feel like the existing cookbook is way off from your use case, then write your own that does just what you need.


Daniel DeLeo