We’re currently running Chef Server with the monolithic installer, with everything running on one node; the only exception is that the bookshelf is backed by S3.
I’ve been asked to investigate resiliency, and in that effort, would like to split Chef Server up by its individual components, and offload what I can to managed services (like Redis And Postgres on Elasticache and RDS, respectively).
Suppose I were to split each component into its own Docker container (which isn’t what I’m set on, necessarily, but for the sake of discussion); how would I do so? Is it possible via the RPM/DEB packages to just install individual components? Also, is it safe to assume that all state is present solely in Redis, Postgres, and Solr (and the bookshelf, of course)?
Could you do it via docker, sure. There are a number of different
approaches you could take. I would suggest looking into the backup and
restoration of your existing chef server: Backup and Restore a Standalone or Frontend install After this I would suggest
green-fielding an HA solution, and transferring your data to that
implementation, if that's the route that you decide to take.
Best,
Bryant
On Thu, Sep 24, 2015 at 4:20 AM, Ameir A. ameirh@gmail.com wrote:
Hello,
We're currently running Chef Server with the monolithic installer, with
everything running on one node; the only exception is that the bookshelf is
backed by S3.
I've been asked to investigate resiliency, and in that effort, would like
to split Chef Server up by its individual components, and offload what I
can to managed services (like Redis And Postgres on Elasticache and RDS,
respectively).
Suppose I were to split each component into its own Docker container
(which isn't what I'm set on, necessarily, but for the sake of discussion);
how would I do so? Is it possible via the RPM/DEB packages to just install
individual components? Also, is it safe to assume that all state is
present solely in Redis, Postgres, and Solr (and the bookshelf, of course)?
We (at Chef Software) are working on a project to allow customers to
externalize many of the data stores utilized by Chef Server for
exactly this purpose -- for example, being able to use an external
PostgreSQL (in RDS, if you want), CloudSearch instead of Solr, and so
on. Is that what you're asking?
Julian
On Thu, Sep 24, 2015 at 5:20 AM, Ameir A. ameirh@gmail.com wrote:
Hello,
We're currently running Chef Server with the monolithic installer, with everything running on one node; the only exception is that the bookshelf is backed by S3.
I've been asked to investigate resiliency, and in that effort, would like to split Chef Server up by its individual components, and offload what I can to managed services (like Redis And Postgres on Elasticache and RDS, respectively).
Suppose I were to split each component into its own Docker container (which isn't what I'm set on, necessarily, but for the sake of discussion); how would I do so? Is it possible via the RPM/DEB packages to just install individual components? Also, is it safe to assume that all state is present solely in Redis, Postgres, and Solr (and the bookshelf, of course)?
Any insight would be well-appreciated.
Thanks,
Ameir
--
[ Julian C. Dunn jdunn@aquezada.com * Sorry, I'm ]
[ WWW: http://www.aquezada.com/staff/julian * only Web 1.0 ]
[ gopher://sdf.org/1/users/keymaker/ * compliant! ]
[ PGP: 91B3 7A9D 683C 7C16 715F 442C 6065 D533 FDC2 05B9 ]
We're currently running Chef Server with the monolithic installer, with everything running on one node; the only exception is that the bookshelf is backed by S3.
I've been asked to investigate resiliency, and in that effort, would like to split Chef Server up by its individual components, and offload what I can to managed services (like Redis And Postgres on Elasticache and RDS, respectively).
Suppose I were to split each component into its own Docker container (which isn't what I'm set on, necessarily, but for the sake of discussion); how would I do so? Is it possible via the RPM/DEB packages to just install individual components? Also, is it safe to assume that all state is present solely in Redis, Postgres, and Solr (and the bookshelf, of course)?
Any insight would be well-appreciated.
The general way the Chef HA packages do this IIRC is to install the whole omnibus package on each node but disable services that aren't needed there so the DB machines turn off erchef/nginx/bifrost/etc and vice versa. This can be done via the chef-server.rb config file.
Thanks, Julian; that's exactly what I'm looking for. Are you able to share
further information on that project? It sounds like it'd solve our itch
quite well.
Thanks,
Ameir
On Sep 24, 2015 6:10 PM, "Julian C. Dunn" jdunn@aquezada.com wrote:
We (at Chef Software) are working on a project to allow customers to
externalize many of the data stores utilized by Chef Server for
exactly this purpose -- for example, being able to use an external
PostgreSQL (in RDS, if you want), CloudSearch instead of Solr, and so
on. Is that what you're asking?
Julian
On Thu, Sep 24, 2015 at 5:20 AM, Ameir A. ameirh@gmail.com wrote:
Hello,
We're currently running Chef Server with the monolithic installer, with
everything running on one node; the only exception is that the bookshelf is
backed by S3.
I've been asked to investigate resiliency, and in that effort, would
like to split Chef Server up by its individual components, and offload what
I can to managed services (like Redis And Postgres on Elasticache and RDS,
respectively).
Suppose I were to split each component into its own Docker container
(which isn't what I'm set on, necessarily, but for the sake of discussion);
how would I do so? Is it possible via the RPM/DEB packages to just install
individual components? Also, is it safe to assume that all state is
present solely in Redis, Postgres, and Solr (and the bookshelf, of course)?
Any insight would be well-appreciated.
Thanks,
Ameir
--
[ Julian C. Dunn jdunn@aquezada.com * Sorry, I'm ]
[ WWW: http://www.aquezada.com/staff/julian * only Web 1.0 ]
[ gopher://sdf.org/1/users/keymaker/ * compliant! ]
[ PGP: 91B3 7A9D 683C 7C16 715F 442C 6065 D533 FDC2 05B9 ]
Sure. The work basically looks something like this.
Our customers have told us loud & clear that they want a real HA
solution, i.e. in the distributed systems lingo, an AP system. We're
building this first on AWS because it forces you to address network
partitions and unavailable instances as part of life, so it's a great
petri dish.
Architecturally, what this means is:
Where possible, Chef Server will be configured to utilize native AWS
primitives to externalize all the data stores -- ELB, RDS, S3 for
cookbooks (no change from today), and CloudSearch as a stretch goal
instead of Solr.
Every server node at that point that sits behind the ELB will just
be stateless, hosting the services like nginx, redis, oc_id, erchef,
authz i.e. bifrost, bookshelf, etc. but won't hold any data.
Countermeasures for addressing the loss of "C" (consistency) in the
system design are improving our backup & restore procedure and
simplifying the # of services in Chef server (e.g. removal of
opscode-expander & RabbitMQ).
We hope to have a beta available by the beginning of Q4 for select
customers that we're working with individually, with a final build
sometime in Q4.
Does that help?
Julian
On Thu, Sep 24, 2015 at 6:42 PM, Ameir Abdeldayem ameirh@gmail.com wrote:
Thanks, Julian; that's exactly what I'm looking for. Are you able to share
further information on that project? It sounds like it'd solve our itch
quite well.
We (at Chef Software) are working on a project to allow customers to
externalize many of the data stores utilized by Chef Server for
exactly this purpose -- for example, being able to use an external
PostgreSQL (in RDS, if you want), CloudSearch instead of Solr, and so
on. Is that what you're asking?
Julian
On Thu, Sep 24, 2015 at 5:20 AM, Ameir A. ameirh@gmail.com wrote:
Hello,
We're currently running Chef Server with the monolithic installer, with
everything running on one node; the only exception is that the bookshelf is
backed by S3.
I've been asked to investigate resiliency, and in that effort, would
like to split Chef Server up by its individual components, and offload what
I can to managed services (like Redis And Postgres on Elasticache and RDS,
respectively).
Suppose I were to split each component into its own Docker container
(which isn't what I'm set on, necessarily, but for the sake of discussion);
how would I do so? Is it possible via the RPM/DEB packages to just install
individual components? Also, is it safe to assume that all state is present
solely in Redis, Postgres, and Solr (and the bookshelf, of course)?
Any insight would be well-appreciated.
Thanks,
Ameir
--
[ Julian C. Dunn jdunn@aquezada.com * Sorry, I'm ]
[ WWW: http://www.aquezada.com/staff/julian * only Web 1.0 ]
[ gopher://sdf.org/1/users/keymaker/ * compliant! ]
[ PGP: 91B3 7A9D 683C 7C16 715F 442C 6065 D533 FDC2 05B9 ]
--
[ Julian C. Dunn jdunn@aquezada.com * Sorry, I'm ]
[ WWW: http://www.aquezada.com/staff/julian * only Web 1.0 ]
[ gopher://sdf.org/1/users/keymaker/ * compliant! ]
[ PGP: 91B3 7A9D 683C 7C16 715F 442C 6065 D533 FDC2 05B9 ]
