# Setting up hosts between multiple boxes on EC2

**URL:** <https://discourse.chef.io/t/setting-up-hosts-between-multiple-boxes-on-ec2/2191>\
**Category:** Chef Infra (archive)\
**Created:** [November 20, 2011, 1:32pm UTC](https://discourse.chef.io/t/setting-up-hosts-between-multiple-boxes-on-ec2/2191 "2011-11-20T13:32:49Z")\
**Posts on this page:** 7\
**Page:** 1

<div class="post-metadata">

**Author:** ![Luis\_Correa\_d\_Almeid](https://sea2.discourse-cdn.com/flex016/user_avatar/discourse.chef.io/luis_correa_d_almeid/32/777_2.png) [@Luis\_Correa\_d\_Almeid](https://discourse.chef.io/u/Luis_Correa_d_Almeid)\
**Post date:** [November 20, 2011, 1:32pm UTC](https://discourse.chef.io/t/setting-up-hosts-between-multiple-boxes-on-ec2/2191/1 "2011-11-20T13:32:49Z")

</div>

Let’s say I provision two boxes on EC2: db1 and app1

What’s the best way to make each box aware of the other? I need to  
tell app1 the IP/host/FQDN of the db1 machine so it can connect to it.

One option is to use hosts in the config files - for example, in the  
app1 box the config references a host (db1) that I can set up either  
in the hosts file or in the local DNS. But I still wouldn’t know the  
IP of db1 to populate the hosts file with when running chef. At this  
point, I need to change my data-bags once I find out the address of  
db1 and then provision app1, running a recipe that adds the right  
entry to the hosts file with the newly found db1 IP.

Just wondering if there is a better way.

Thanks,

Luis

---

<div class="post-metadata">

**Author:** ![Matt\_Ray\_OPSCODE](https://sea2.discourse-cdn.com/flex016/user_avatar/discourse.chef.io/matt_ray_opscode/32/459_2.png) [@Matt\_Ray\_OPSCODE](https://discourse.chef.io/u/Matt_Ray_OPSCODE)\
**Post date:** [November 20, 2011, 2:31pm UTC](https://discourse.chef.io/t/setting-up-hosts-between-multiple-boxes-on-ec2/2191/2 "2011-11-20T14:31:06Z")

</div>

You'll want to take a good look at how Chef utilizes search within  
recipes. Rather than try to manage dns, you use search in the recipes  
for your "app1" to find the machines that have the recipe or roles  
that identify your "db1" machine (and vice versa). Many recipes do  
this sort of work, load balancers looking for web apps, clients  
looking for servers and workers looking for masters. IMHO, it's one of  
the cooler features of working with Chef.

[http://wiki.opscode.com/display/chef/Search](http://wiki.opscode.com/display/chef/Search)  
and more specifically  
[http://wiki.opscode.com/display/chef/Search#Search-UsingSearchinRecipes](http://wiki.opscode.com/display/chef/Search#Search-UsingSearchinRecipes)

Thanks,  
Matt Ray  
Senior Technical Evangelist | Opscode Inc.  
[matt@opscode.com](mailto:matt@opscode.com) | (512) 731-2218  
Twitter, IRC, GitHub: mattray

On Sun, Nov 20, 2011 at 1:32 PM, Luis Correa d'Almeida  
[luis.ca@gmail.com](mailto:luis.ca@gmail.com) wrote:

> Let's say I provision two boxes on EC2: db1 and app1
> 
> What's the best way to make each box aware of the other? I need to  
> tell app1 the IP/host/FQDN of the db1 machine so it can connect to it.
> 
> One option is to use hosts in the config files - for example, in the  
> app1 box the config references a host (db1) that I can set up either  
> in the hosts file or in the local DNS. But I still wouldn't know the  
> IP of db1 to populate the hosts file with when running chef. At this  
> point, I need to change my data-bags once I find out the address of  
> db1 and then provision app1, running a recipe that adds the right  
> entry to the hosts file with the newly found db1 IP.
> 
> Just wondering if there is a better way.
> 
> Thanks,
> 
> Luis

---

<div class="post-metadata">

**Author:** ![Brad\_Knowles](https://sea2.discourse-cdn.com/flex016/user_avatar/discourse.chef.io/brad_knowles/32/716_2.png) [@Brad\_Knowles](https://discourse.chef.io/u/Brad_Knowles)\
**Post date:** [November 20, 2011, 2:38pm UTC](https://discourse.chef.io/t/setting-up-hosts-between-multiple-boxes-on-ec2/2191/3 "2011-11-20T14:38:28Z")

</div>

On Nov 20, 2011, at 8:31 AM, Matt Ray wrote:

> You'll want to take a good look at how Chef utilizes search within  
> recipes. Rather than try to manage dns, you use search in the recipes  
> for your "app1" to find the machines that have the recipe or roles  
> that identify your "db1" machine (and vice versa). Many recipes do  
> this sort of work, load balancers looking for web apps, clients  
> looking for servers and workers looking for masters. IMHO, it's one of  
> the cooler features of working with Chef.

And I've recently gotten this sort of thing working with the "shorewall" firewall toolkit. I can tell you that it feels like a pretty good achievement when you finally get something like this working on your own, for the first time.

As I understand it, one thing to be concerned about with regards to using data bags is that you don't want to have multiple nodes potentially writing to the same data bag at the same time -- that could cause a race condition in writing the data bag contents back to the chef server, which might cause data to be lost.

So, if you were in a situation where you wanted to use search to populate the data bags and then be more circumspect about when information is purged from the data bags, then you would want to have that process done by a back-end monitoring server, and have read-only access to the data bags on all the rest of the nodes.

--  
Brad Knowles [bknowles@ihiji.com](mailto:bknowles@ihiji.com)  
SAGE Level IV, Chef Level 0.0.1

---

<div class="post-metadata">

**Author:** ![Haim\_Ashkenazi](https://avatars.discourse-cdn.com/v4/letter/h/ed655f/32.png) [@Haim\_Ashkenazi](https://discourse.chef.io/u/Haim_Ashkenazi)\
**Post date:** [November 20, 2011, 3:40pm UTC](https://discourse.chef.io/t/setting-up-hosts-between-multiple-boxes-on-ec2/2191/4 "2011-11-20T15:40:25Z")

</div>

Hi

I've actually implemented something like that with route53 (amazon't DNS  
service). For each important host (e.g, db) I've created a name (e.g.  
[db.domain.com](http://db.domain.com)) and added it as CNAME to the host's internal address (this  
does not work between regions though). The reason I chose route53 was that  
I think (It's not yet checked) that It helps reduces TTL since I already  
query the amazon DNS.

The update script looks like this:

#!/usr/bin/env ruby

NODENAME = \<%= %{'#{node.name}'} %\>  
HOSTNAME = \<%= %{'#{node[:ec2][:hostname]}'} %\>  
DOMAINNAME = \<%= %{'#{node[:company][:internal\_domain]}'} %\>  
EC2\_KEY\_ID = \<%= %{'#{@api['EC2\_KEY\_ID']}'} %\>  
EC2\_SECRET\_KEY = \<%= %{'#{@api['EC2\_SECRET\_KEY']}'} %\>

require 'rubygems'  
require 'right\_aws'

module Route53Update  
extend self

def r53  
@r53 ||= RightAws::Route53Interface.new(EC2\_KEY\_ID, EC2\_SECRET\_KEY)  
end

def zone\_id  
return @zone\_id if @zone\_id  
zone = r53.list\_hosted\_zones.select { |zone|  
zone[:name] =~ /^#{DOMAINNAME}.?$/  
}.first  
@zone\_id = zone ? zone[:aws\_id] :  
raise("Zone #{DOMAINNAME} doesn't seem to be registered!")  
end

def create\_cname  
delete\_cname  
resource\_record\_sets = [{  
:name =\> NODENAME,  
:type =\> 'CNAME',  
:ttl =\> 300,  
:resource\_records =\> [HOSTNAME]  
}]  
r53.create\_resource\_record\_sets(zone\_id, resource\_record\_sets, "")  
end

def delete\_cname  
r53.list\_resource\_record\_sets(zone\_id).each do |record|  
if record[:name] == "#{NODENAME}."  
r53.delete\_resource\_record\_sets(zone\_id, [record], "")  
break  
end  
end  
end

def validate\_cname  
res = r53.list\_resource\_record\_sets(zone\_id).select { |record|  
record[:name] =~ /#{NODENAME}.?$/  
}.first  
res && res[:resource\_records].include?(HOSTNAME) ? true : false  
end

def update\_cname  
return if validate\_cname  
create\_cname  
# TODO: validate sync!  
end  
end

if $0 == **FILE**  
Route53Update.update\_cname  
end

You have to install right\_aws gem and add an init script (so it'll run on  
boot). You can probably figure out the variables.

Using DNS helps with not having to restart application every time a host  
changes it IP, but you have to consider DNS caching (e.g, Java caches dns  
results depending on running environment).

HTH

Haim

On Sun, Nov 20, 2011 at 3:32 PM, Luis Correa d'Almeida [luis.ca@gmail.com](mailto:luis.ca@gmail.com)wrote:

> Let's say I provision two boxes on EC2: db1 and app1
> 
> What's the best way to make each box aware of the other? I need to  
> tell app1 the IP/host/FQDN of the db1 machine so it can connect to it.
> 
> One option is to use hosts in the config files - for example, in the  
> app1 box the config references a host (db1) that I can set up either  
> in the hosts file or in the local DNS. But I still wouldn't know the  
> IP of db1 to populate the hosts file with when running chef. At this  
> point, I need to change my data-bags once I find out the address of  
> db1 and then provision app1, running a recipe that adds the right  
> entry to the hosts file with the newly found db1 IP.
> 
> Just wondering if there is a better way.
> 
> Thanks,
> 
> Luis

--  
Haim

---

<div class="post-metadata">

**Author:** ![Haim\_Ashkenazi](https://avatars.discourse-cdn.com/v4/letter/h/ed655f/32.png) [@Haim\_Ashkenazi](https://discourse.chef.io/u/Haim_Ashkenazi)\
**Post date:** [November 20, 2011, 3:42pm UTC](https://discourse.chef.io/t/setting-up-hosts-between-multiple-boxes-on-ec2/2191/5 "2011-11-20T15:42:22Z")

</div>

One more thing, In this setup the node is named with the general name (  
[db.domain.com](http://db.domain.com)). Otherwise change the NODENAME variable.

On Sun, Nov 20, 2011 at 5:40 PM, Haim Ashkenazi [haim.ashkenazi@gmail.com](mailto:haim.ashkenazi@gmail.com)wrote:

> Hi
> 
> I've actually implemented something like that with route53 (amazon't DNS  
> service). For each important host (e.g, db) I've created a name (e.g.  
> [db.domain.com](http://db.domain.com)) and added it as CNAME to the host's internal address (this  
> does not work between regions though). The reason I chose route53 was that  
> I think (It's not yet checked) that It helps reduces TTL since I already  
> query the amazon DNS.
> 
> The update script looks like this:
> 
> #!/usr/bin/env ruby
> 
> NODENAME = \<%= %{'#{node.name}'} %\>  
> HOSTNAME = \<%= %{'#{node[:ec2][:hostname]}'} %\>  
> DOMAINNAME = \<%= %{'#{node[:company][:internal\_domain]}'} %\>  
> EC2\_KEY\_ID = \<%= %{'#{@api['EC2\_KEY\_ID']}'} %\>  
> EC2\_SECRET\_KEY = \<%= %{'#{@api['EC2\_SECRET\_KEY']}'} %\>
> 
> require 'rubygems'  
> require 'right\_aws'
> 
> module Route53Update  
> extend self
> 
> def r53  
> @r53 ||= RightAws::Route53Interface.new(EC2\_KEY\_ID, EC2\_SECRET\_KEY)  
> end
> 
> def zone\_id  
> return @zone\_id if @zone\_id  
> zone = r53.list\_hosted\_zones.select { |zone|  
> zone[:name] =~ /^#{DOMAINNAME}.?$/  
> }.first  
> @zone\_id = zone ? zone[:aws\_id] :  
> raise("Zone #{DOMAINNAME} doesn't seem to be registered!")  
> end
> 
> def create\_cname  
> delete\_cname  
> resource\_record\_sets = [{  
> :name =\> NODENAME,  
> :type =\> 'CNAME',  
> :ttl =\> 300,  
> :resource\_records =\> [HOSTNAME]  
> }]  
> r53.create\_resource\_record\_sets(zone\_id, resource\_record\_sets, "")  
> end
> 
> def delete\_cname  
> r53.list\_resource\_record\_sets(zone\_id).each do |record|  
> if record[:name] == "#{NODENAME}."  
> r53.delete\_resource\_record\_sets(zone\_id, [record], "")  
> break  
> end  
> end  
> end
> 
> def validate\_cname  
> res = r53.list\_resource\_record\_sets(zone\_id).select { |record|  
> record[:name] =~ /#{NODENAME}.?$/  
> }.first  
> res && res[:resource\_records].include?(HOSTNAME) ? true : false  
> end
> 
> def update\_cname  
> return if validate\_cname  
> create\_cname  
> # TODO: validate sync!  
> end  
> end
> 
> if $0 == **FILE**  
> Route53Update.update\_cname  
> end
> 
> You have to install right\_aws gem and add an init script (so it'll run on  
> boot). You can probably figure out the variables.
> 
> Using DNS helps with not having to restart application every time a host  
> changes it IP, but you have to consider DNS caching (e.g, Java caches dns  
> results depending on running environment).
> 
> HTH
> 
> Haim
> 
> On Sun, Nov 20, 2011 at 3:32 PM, Luis Correa d'Almeida [luis.ca@gmail.com](mailto:luis.ca@gmail.com)wrote:
> 
> > Let's say I provision two boxes on EC2: db1 and app1
> > 
> > What's the best way to make each box aware of the other? I need to  
> > tell app1 the IP/host/FQDN of the db1 machine so it can connect to it.
> > 
> > One option is to use hosts in the config files - for example, in the  
> > app1 box the config references a host (db1) that I can set up either  
> > in the hosts file or in the local DNS. But I still wouldn't know the  
> > IP of db1 to populate the hosts file with when running chef. At this  
> > point, I need to change my data-bags once I find out the address of  
> > db1 and then provision app1, running a recipe that adds the right  
> > entry to the hosts file with the newly found db1 IP.
> > 
> > Just wondering if there is a better way.
> > 
> > Thanks,
> > 
> > Luis
> 
> --  
> Haim

--  
Haim

---

<div class="post-metadata">

**Author:** ![Luis\_Correa\_d\_Almeid](https://sea2.discourse-cdn.com/flex016/user_avatar/discourse.chef.io/luis_correa_d_almeid/32/777_2.png) [@Luis\_Correa\_d\_Almeid](https://discourse.chef.io/u/Luis_Correa_d_Almeid)\
**Post date:** [November 21, 2011, 11:58am UTC](https://discourse.chef.io/t/setting-up-hosts-between-multiple-boxes-on-ec2/2191/6 "2011-11-21T11:58:15Z")

</div>

Great, thanks. Using search works well for my setup.

On Sun, Nov 20, 2011 at 3:42 PM, Haim Ashkenazi  
[haim.ashkenazi@gmail.com](mailto:haim.ashkenazi@gmail.com) wrote:

> One more thing, In this setup the node is named with the general name  
> ([db.domain.com](http://db.domain.com)). Otherwise change the NODENAME variable.
> 
> On Sun, Nov 20, 2011 at 5:40 PM, Haim Ashkenazi [haim.ashkenazi@gmail.com](mailto:haim.ashkenazi@gmail.com)  
> wrote:
> 
> > Hi  
> > I've actually implemented something like that with route53 (amazon't DNS  
> > service). For each important host (e.g, db) I've created a name (e.g.  
> > [db.domain.com](http://db.domain.com)) and added it as CNAME to the host's internal address (this  
> > does not work between regions though). The reason I chose route53 was that I  
> > think (It's not yet checked) that It helps reduces TTL since I already query  
> > the amazon DNS.  
> > The update script looks like this:  
> > #!/usr/bin/env ruby  
> > NODENAME = \<%= %{'#{node.name}'} %\>  
> > HOSTNAME = \<%= %{'#{node[:ec2][:hostname]}'} %\>  
> > DOMAINNAME = \<%= %{'#{node[:company][:internal\_domain]}'} %\>  
> > EC2\_KEY\_ID = \<%= %{'#{@api['EC2\_KEY\_ID']}'} %\>  
> > EC2\_SECRET\_KEY = \<%= %{'#{@api['EC2\_SECRET\_KEY']}'} %\>  
> > require 'rubygems'  
> > require 'right\_aws'  
> > module Route53Update  
> > extend self  
> > def r53  
> > @r53 ||= RightAws::Route53Interface.new(EC2\_KEY\_ID, EC2\_SECRET\_KEY)  
> > end  
> > def zone\_id  
> > return @zone\_id if @zone\_id  
> > zone = r53.list\_hosted\_zones.select { |zone|  
> > zone[:name] =~ /^#{DOMAINNAME}.?$/  
> > }.first  
> > @zone\_id = zone ? zone[:aws\_id] :  
> > raise("Zone #{DOMAINNAME} doesn't seem to be registered!")  
> > end  
> > def create\_cname  
> > delete\_cname  
> > resource\_record\_sets = [{  
> > :name =\> NODENAME,  
> > :type =\> 'CNAME',  
> > :ttl =\> 300,  
> > :resource\_records =\> [HOSTNAME]  
> > }]  
> > r53.create\_resource\_record\_sets(zone\_id, resource\_record\_sets, "")  
> > end  
> > def delete\_cname  
> > r53.list\_resource\_record\_sets(zone\_id).each do |record|  
> > if record[:name] == "#{NODENAME}."  
> > r53.delete\_resource\_record\_sets(zone\_id, [record], "")  
> > break  
> > end  
> > end  
> > end  
> > def validate\_cname  
> > res = r53.list\_resource\_record\_sets(zone\_id).select { |record|  
> > record[:name] =~ /#{NODENAME}.?$/  
> > }.first  
> > res && res[:resource\_records].include?(HOSTNAME) ? true : false  
> > end  
> > def update\_cname  
> > return if validate\_cname  
> > create\_cname  
> > # TODO: validate sync!  
> > end  
> > end  
> > if $0 == **FILE**  
> > Route53Update.update\_cname  
> > end  
> > You have to install right\_aws gem and add an init script (so it'll run on  
> > boot). You can probably figure out the variables.  
> > Using DNS helps with not having to restart application every time a host  
> > changes it IP, but you have to consider DNS caching (e.g, Java caches dns  
> > results depending on running environment).  
> > HTH  
> > Haim  
> > On Sun, Nov 20, 2011 at 3:32 PM, Luis Correa d'Almeida [luis.ca@gmail.com](mailto:luis.ca@gmail.com)  
> > wrote:
> > 
> > > Let's say I provision two boxes on EC2: db1 and app1
> > > 
> > > What's the best way to make each box aware of the other? I need to  
> > > tell app1 the IP/host/FQDN of the db1 machine so it can connect to it.
> > > 
> > > One option is to use hosts in the config files - for example, in the  
> > > app1 box the config references a host (db1) that I can set up either  
> > > in the hosts file or in the local DNS. But I still wouldn't know the  
> > > IP of db1 to populate the hosts file with when running chef. At this  
> > > point, I need to change my data-bags once I find out the address of  
> > > db1 and then provision app1, running a recipe that adds the right  
> > > entry to the hosts file with the newly found db1 IP.
> > > 
> > > Just wondering if there is a better way.
> > > 
> > > Thanks,
> > > 
> > > Luis
> > 
> > --  
> > Haim
> 
> --  
> Haim

--  
Luis Corrêa d'Almeida  
tribe.fm/luis

---

<div class="post-metadata">

**Author:** ![Brad\_Knowles](https://sea2.discourse-cdn.com/flex016/user_avatar/discourse.chef.io/brad_knowles/32/716_2.png) [@Brad\_Knowles](https://discourse.chef.io/u/Brad_Knowles)\
**Post date:** [November 21, 2011, 3:12pm UTC](https://discourse.chef.io/t/setting-up-hosts-between-multiple-boxes-on-ec2/2191/7 "2011-11-21T15:12:34Z")

</div>

On Nov 20, 2011, at 9:40 AM, Haim Ashkenazi wrote:

> I've actually implemented something like that with route53 (amazon't DNS service). For each important host (e.g, db) I've created a name (e.g. [db.domain.com](http://db.domain.com)) and added it as CNAME to the host's internal address (this does not work between regions though). The reason I chose route53 was that I think (It's not yet checked) that It helps reduces TTL since I already query the amazon DNS.

BTW, there's already a cookbook for doing this kind of stuff with DynECT, and we have implemented similar functionality using a curl-based mechanism to Zerigo for our nodes on Rackspace Cloud.

I know that Fog is the "right" way to do this sort of thing (and I'm sure the DynECT cookbook uses this method), but the interface between Zerigo and Fog wasn't obvious to us, and the existing cookbook from Zerigo for using their tools ... well, let's just say that we found it "sub-optimal". So, we kit-bashed together the critical pieces we needed based on their documentation for their REST API and their curl-based mechanisms.

All of which gets us back to the mantra TIMTOWTDI.

--  
Brad Knowles [bknowles@ihiji.com](mailto:bknowles@ihiji.com)  
SAGE Level IV, Chef Level 0.0.1