Chef Server for HA

Chef Infra (archive)
1

Hi All,

I would like to implement this use case of Chef server for High
Availability
(http://docs.opscode.com/enterprise/server_deploy_febe.html)

Followed the basic approach :

a) Installed chef and git repository on administrative
workstation, also configured knife on this system

b) Used Hosted enterprise chef as chef server

c) Installed chef-client on various clients machines using
bootstrap from workstation.

I am using AWS infrastructure and for assuring High Availability, I
would like to have chef, managed from multiple AZ’s

My questions:

a) I had planned to use ELB as the load balancer for front end
servers. ELB will handle requests from chef-clients, workstations and
chef server web interface. Is this the right approach??

b) I will have two chef instances (workstation) on each AZ’s
(us-east-1a,us-east-1b), from where I can make changes on cookbooks and
configure clients. From each instances, I can upload the changed
cookbooks to hosted enterprise chef server.

c) What is the best approach to implement the shared VIP ??

c.1 Can I have one EC2 machine, which will monitor the two chef
instances through some shell script, and attach/detach Network
interface card from primary to secondary chef server.

c.2 Domain Name change using Route53

d) How my both the git repositories will be synced in different
AZ’s ??

Any Ideas how to implement this use case???

Thanks & Regards,

Sachin Kumar

2

I am unsure what you are trying to accomplish. That documentation is
for setting up an Enterprise Chef server in HA mode, but you state:

b) Used Hosted enterprise chef as chef server

so why do you need to do all this?

  • Julian

On Tue, Apr 8, 2014 at 7:59 AM, Sachin Kumar Sachin.Kumar@infogain.com wrote:

Hi All,

I would like to implement this use case of Chef server for High
Availability (http://docs.opscode.com/enterprise/server_deploy_febe.html)

Followed the basic approach :

a) Installed chef and git repository on administrative workstation,
also configured knife on this system

b) Used Hosted enterprise chef as chef server

c) Installed chef-client on various clients machines using bootstrap
from workstation.

I am using AWS infrastructure and for assuring High Availability, I would
like to have chef, managed from multiple AZ's

My questions:

a) I had planned to use ELB as the load balancer for front end
servers. ELB will handle requests from chef-clients, workstations and chef
server web interface. Is this the right approach??

b) I will have two chef instances (workstation) on each AZ's
(us-east-1a,us-east-1b), from where I can make changes on cookbooks and
configure clients. From each instances, I can upload the changed cookbooks
to hosted enterprise chef server.

c) What is the best approach to implement the shared VIP ??

c.1 Can I have one EC2 machine, which will monitor the two chef
instances through some shell script, and attach/detach Network interface
card from primary to secondary chef server.

c.2 Domain Name change using Route53

d) How my both the git repositories will be synced in different AZ's ??

Any Ideas how to implement this use case???

Thanks & Regards,

Sachin Kumar

--
[ Julian C. Dunn jdunn@aquezada.com * Sorry, I'm ]
[ WWW: http://www.aquezada.com/staff/julian * only Web 1.0 ]
[ gopher://sdf.org/1/users/keymaker/ * compliant! ]
[ PGP: 91B3 7A9D 683C 7C16 715F 442C 6065 D533 FDC2 05B9 ]

3

Hi Julian,

My aim is to install chef server for high availability in different AZ's
in AWS. So that if one AZ goes down, my chef server will be running in
different availability zone.

I am ok, if I have to use Enterprise chef server.

Thanks & Regards,
Sachin Kumar

-----Original Message-----
From: Julian C. Dunn [mailto:jdunn@aquezada.com]
Sent: 08 April 2014 17:56
To: chef@lists.opscode.com
Subject: [chef] Re: Chef Server for HA

I am unsure what you are trying to accomplish. That documentation is for
setting up an Enterprise Chef server in HA mode, but you state:

b) Used Hosted enterprise chef as chef server

so why do you need to do all this?

  • Julian

On Tue, Apr 8, 2014 at 7:59 AM, Sachin Kumar Sachin.Kumar@infogain.com
wrote:

Hi All,

I would like to implement this use case of Chef server for High
Availability
(http://docs.opscode.com/enterprise/server_deploy_febe.html)

Followed the basic approach :

a) Installed chef and git repository on administrative
workstation,
also configured knife on this system

b) Used Hosted enterprise chef as chef server

c) Installed chef-client on various clients machines using
bootstrap
from workstation.

I am using AWS infrastructure and for assuring High Availability, I
would like to have chef, managed from multiple AZ's

My questions:

a) I had planned to use ELB as the load balancer for front end
servers. ELB will handle requests from chef-clients, workstations and
chef server web interface. Is this the right approach??

b) I will have two chef instances (workstation) on each AZ's
(us-east-1a,us-east-1b), from where I can make changes on cookbooks
and configure clients. From each instances, I can upload the changed
cookbooks to hosted enterprise chef server.

c) What is the best approach to implement the shared VIP ??

c.1 Can I have one EC2 machine, which will monitor the two
chef
instances through some shell script, and attach/detach Network
interface card from primary to secondary chef server.

c.2 Domain Name change using Route53

d) How my both the git repositories will be synced in different
AZ's ??

Any Ideas how to implement this use case???

Thanks & Regards,

Sachin Kumar

--
[ Julian C. Dunn jdunn@aquezada.com * Sorry, I'm ]
[ WWW: http://www.aquezada.com/staff/julian * only Web 1.0 ]
[ gopher://sdf.org/1/users/keymaker/ * compliant! ]
[ PGP: 91B3 7A9D 683C 7C16 715F 442C 6065 D533 FDC2 05B9 ]

4

On Tue, Apr 8, 2014 at 9:03 AM, Sachin Kumar Sachin.Kumar@infogain.com wrote:

My aim is to install chef server for high availability in different AZ's
in AWS. So that if one AZ goes down, my chef server will be running in
different availability zone.

I am ok, if I have to use Enterprise chef server.

So just to clarify some terminology then:

  • Hosted Enterprise Chef == the Enterprise Chef installation we run at
    manage.opscode.com. As it's "hosted", you don't need to set up a
    server at all.
  • Private Enterprise Chef == an on-premise Enterprise Chef server that
    you would run in your own infrastructure (be that AWS or a data
    center)

I am assuming that you are talking about the latter.

The current deployment scenario for Private Enterprise Chef does not
support AWS. In particular, the HA scenario is incompatible with AWS
because the shared DRBD volume requires a low-latency link between the
two back-end machines for fast failover.

That said, if you want to experiment outside of the bounds of what's
supported, here is the scenario with which I would deploy Private
Enterprise Chef into AWS. Disclaimer: no warranty express or implied,
my employer doesn't officially support this, etc. :slight_smile:

  • Use a VPC. Put the FE boxes in a public subnet, the BE box in a private one.
  • Set up N frontends, as many as you need, behind an ELB.
  • Set up an autoscaling group for 1 backend. If it dies, Autoscaling
    will bring up another one. Set up DHCP options so that this new
    instance always gets the same IP.
  • Store Chef's data in a separate EBS volume that you can reattach
    automatically to a reborn backend instance. Snapshot the EBS
    regularly.
  • Store the generated keys from the bootstrap procedure in S3 or
    somewhere that you can retrieve them if the backend dies.
  • Optionally, store the cookbooks in S3.
  • Use CloudFormation to orchestrate this entire setup.

If you want to hit me up off-list to talk about this, I'm happy to,
since this probably isn't of much interest to folks who aren't
purchasing Private Enterprise Chef.

  • Julian

--
[ Julian C. Dunn jdunn@aquezada.com * Sorry, I'm ]
[ WWW: http://www.aquezada.com/staff/julian * only Web 1.0 ]
[ gopher://sdf.org/1/users/keymaker/ * compliant! ]
[ PGP: 91B3 7A9D 683C 7C16 715F 442C 6065 D533 FDC2 05B9 ]

5

Thanks John the inputs, will work on the private Enterprise chef and
evaluate the scenarios in AWS as suggested.

Thanks & Regards,
Sachin Kumar
-----Original Message-----
From: Julian C. Dunn [mailto:jdunn@aquezada.com]
Sent: 09 April 2014 08:16
To: chef@lists.opscode.com
Subject: [chef] Re: RE: Re: Chef Server for HA

On Tue, Apr 8, 2014 at 9:03 AM, Sachin Kumar Sachin.Kumar@infogain.com
wrote:

My aim is to install chef server for high availability in different
AZ's in AWS. So that if one AZ goes down, my chef server will be
running in different availability zone.

I am ok, if I have to use Enterprise chef server.

So just to clarify some terminology then:

  • Hosted Enterprise Chef == the Enterprise Chef installation we run at
    manage.opscode.com. As it's "hosted", you don't need to set up a server
    at all.
  • Private Enterprise Chef == an on-premise Enterprise Chef server that
    you would run in your own infrastructure (be that AWS or a data
    center)

I am assuming that you are talking about the latter.

The current deployment scenario for Private Enterprise Chef does not
support AWS. In particular, the HA scenario is incompatible with AWS
because the shared DRBD volume requires a low-latency link between the
two back-end machines for fast failover.

That said, if you want to experiment outside of the bounds of what's
supported, here is the scenario with which I would deploy Private
Enterprise Chef into AWS. Disclaimer: no warranty express or implied, my
employer doesn't officially support this, etc. :slight_smile:

  • Use a VPC. Put the FE boxes in a public subnet, the BE box in a
    private one.
  • Set up N frontends, as many as you need, behind an ELB.
  • Set up an autoscaling group for 1 backend. If it dies, Autoscaling
    will bring up another one. Set up DHCP options so that this new instance
    always gets the same IP.
  • Store Chef's data in a separate EBS volume that you can reattach
    automatically to a reborn backend instance. Snapshot the EBS regularly.
  • Store the generated keys from the bootstrap procedure in S3 or
    somewhere that you can retrieve them if the backend dies.
  • Optionally, store the cookbooks in S3.
  • Use CloudFormation to orchestrate this entire setup.

If you want to hit me up off-list to talk about this, I'm happy to,
since this probably isn't of much interest to folks who aren't
purchasing Private Enterprise Chef.

  • Julian

--
[ Julian C. Dunn jdunn@aquezada.com * Sorry, I'm ]
[ WWW: http://www.aquezada.com/staff/julian * only Web 1.0 ]
[ gopher://sdf.org/1/users/keymaker/ * compliant! ]
[ PGP: 91B3 7A9D 683C 7C16 715F 442C 6065 D533 FDC2 05B9 ]