Problems with the standalone installation - using an external database and git repository

Chef Infra (archive)
1

Hi,

I’m new to Chef and I’m having some trouble setting up our Chef server, or
at least it seems that we’re trying to do something thats uncommon…

I’ll start by explaining our setup a little, or at least how we’d like it
to become.

  • 2 management nodes (frontend):
    • We have two frontend servers which function as our management
      nodes. In theory we can run two standalone chef’s on both and
      replicate the
      DB. Not what we want, but I believe that should work without too much
      trouble.
  • *2 database servers *(backend, but not specifically for Chef):
    • We have a dedicated database platform which hosts PostgreSQL
      (active backup setup).
    • We also have a Git repository on this platform, redundant on both.
    • This platform runs all our production databases (chef being one of
      them).

Since Chef is new in our infrastructure we want it to use the existing
database platform. We’d also like Chef to use the existing Git repository.
The way we see this is that data will be on de DB servers (PostgreSQL and
Git) and the management node will only contain configuration or as little
data as possible.

We’ve tried to edit /etc/opscode/chef-server.rb to point to a different DB
server but when we do this it simply won’t work.

The difference I’m noticing is that the standalone installation (local
pgsql) will run a bunch of DB queries, create databases, etc. but when I
set a different DB server it won’t do any of those.

Below the contents of our current chef-server.rb:

postgresql[‘enable’] = false
postgresql[‘vip’] = 'postgresql.server.local’
postgresql[‘port’] = '5432’
postgresql[‘version’] = '9.1’
postgresql[‘sql_user’] = 'opscode_pgsql’
postgresql[‘sql_password’] = '’
postgresql[‘sql_ro_user’] = 'opscode_pgsql_ro’
postgresql[‘sql_ro_password’] = '’
oc_id[‘sql_user’] = 'oc_id’
oc_id[‘sql_password’] = '’
oc_bifrost[‘sql_user’] = 'bifrost’
oc_bifrost[‘sql_password’] = '’
oc_bifrost[‘sql_ro_user’] = 'bifrost_ro’
oc_bifrost[‘sql_ro_password’] = ‘****’

When looking at the local pgsql I can see there are 4 databases and a few
users. I’ve created all of those on our DB server. But even then no luck,
it seems the databases aren’t populated and thus Chef won’t work.

One thing I don’t want to do is put all chef software on the DB server and
create a Chef backend (as documented), that would still create a second DB
instead of using the one that’s already there. Another reason for us not
trying this is that we don’t want the Chef software to populate the DB
server, it has a dedicated Job as a DB server and we’d like to keep that as
clean as possible.

This is basically where I’m stuck, I’ve tried allot of options but I can’t
seem to get this to work.

The installation is on a CentOS 6.6 server, up to this point the
installation has been from scratch so there’s data at this point.

Does anyone have experience with moving the DB away from a standalone
installation? What am I missing?

Also, since I havent even gotten to the Git part yet, will I get similar
issues when I try to use an external Git for Chef? Is it even possible?

Thanks in advance!

–
Bart G.

2

Le 2015-02-06 10:38, Bart a écrit :

When looking at the local pgsql I can see there are 4 databases and a few users. I've created all of those on our DB server. But even then no luck, it seems the databases aren't populated and thus Chef won't work.

[...] we don't want the Chef software to populate the DB server, it has a dedicated Job as a DB server and we'd like to keep that as clean as possible.

Don't get you there, you don't want chef to populate the DB but you want
it to populate the DB ????

You can try dumping the local DBs to remote and then point chef-server
to the remote...

Also, since I havent even gotten to the Git part yet, will I get similar issues when I try to use an external Git for Chef? Is it even possible?

There's no link between chef-server and git, the git repo are used to
store/version the cookbooks before being uploaded to the chef server.
(Or I have missed something somewhere)

The chef-server is made of more moving parts than just api frontend and
DB backend, there's bookshelf too which will store the cookbooks and is
an s3 compatible service storing files by checksum (global idea).

If you're starting with chef, I would advise starting to learn it with a
simple standalone install before trying to make a distributed one...

Using a remote DB sounds overkill to me and may get you in unknown and
untested problems.
This won't give you a zero downtime system without managing to replicate
bookshelf, rabbitmq and solr indexes too (at least).

In MY opinion if you want to achieve HA on the chef server, you'd
better:

  • look at the HA commercial addon (replication)
    or
  • take time to understand deeply each part an how they works together
    before trying to hack over them.
3

Hi Tensibai,

Thanks for the reply, this clarified some of the questions I've had.

About the populating sentence, it's indeed a little confusing, sorry about
that! What I meant was that I don't want to install the Chef backend
components on our DB servers (I want to keep those servers as clean as
possible).

For now I think we'll start with a standalone installation and replicate
the contents to the second management host. I think your right about the HA
setting, we'll work on that in the future when our environment grows and
when we understand the Chef components better.

Thanks again for the answers!

-- Bart

2015-02-06 15:58 GMT+01:00 Tensibai tensibai@iabis.net:

Le 2015-02-06 10:38, Bart a écrit :

When looking at the local pgsql I can see there are 4 databases and a few
users. I've created all of those on our DB server. But even then no luck,
it seems the databases aren't populated and thus Chef won't work.

[...] we don't want the Chef software to populate the DB server, it has a
dedicated Job as a DB server and we'd like to keep that as clean as
possible.

Don't get you there, you don't want chef to populate the DB but you want
it to populate the DB ????
You can try dumping the local DBs to remote and then point chef-server
to the remote...

Also, since I havent even gotten to the Git part yet, will I get similar
issues when I try to use an external Git for Chef? Is it even possible?

There's no link between chef-server and git, the git repo are used to
store/version the cookbooks before being uploaded to the chef server. (Or I
have missed something somewhere)

The chef-server is made of more moving parts than just api frontend and DB
backend, there's bookshelf too which will store the cookbooks and is an s3
compatible service storing files by checksum (global idea).

If you're starting with chef, I would advise starting to learn it with a
simple standalone install before trying to make a distributed one...

Using a remote DB sounds overkill to me and may get you in unknown and
untested problems.
This won't give you a zero downtime system without managing to replicate
bookshelf, rabbitmq and solr indexes too (at least).

In my opinion if you want to achieve HA on the chef server, you'd
better:

  • look at the HA commercial addon (replication)
    or
  • take time to understand deeply each part an how they works together
    before trying to hack over them.

--
Bart G.