Configure server to integer mapping


#1

Hi everyone,

For a few services like zookeeper or kafka, they require a server to integer
mapping like this,

server a - 1
server b - 2

This is particularly difficult to do in Chef (outside of manually configuring
this) using things like search. Some methods I have tried or seen,

  1. Search for nodes and sort nodes by name. Use the index of each server as
    their id. This can cause servers to use the same id for a moment if
    provisioning servers concurrently. This can also cause problems if a new node
    doesn’t get appended to the end causing a number of servers to change their
    ids.

  2. Convert ipaddress or some other attribute into an integer. Unfortunately
    this produces an integer too large as services like zookeeper only allow ids
    between 0-255.

  3. Set this id on the node during provisioning and track the current id
    somewhere. Then you can use search to just get the mapping.

Does anyone have other suggestions? It seems like 3 is the best option though
not terrible easy to do. It could be easier than I think if I use something
like chef-metal and define each machine with an id.

Bryan


#2

Hi Bryan,

You could also store the entire cluster configuration in Chef, and pull
out the node’s ID based on the IP address/hostname/etc.

For example:
default[‘zk’][‘cluster’][‘192.168.1.20’][‘id’] = 1
default[‘zk’][‘cluster’][‘192.168.1.21’][‘id’] = 2

default[‘zk’][‘cluster’][‘192.168.1.22’][‘id’] = 3

ip = node[‘ipaddress’]
myid = node[‘zk’][‘cluster’][ip][‘id’]
if myid.nil?
raise 'Cannot find ME in cluster’
end

template ‘/etc/zookeeper/conf/myid’ do
variables({ :myid => myid })
end

Template contents

<%= @myid %>


Cheers,
Andrew

On 2014-10-29, 10:50 PM, "bjbq4d@gmail.com" bjbq4d@gmail.com wrote:

Hi everyone,

For a few services like zookeeper or kafka, they require a server to
integer
mapping like this,

server a - 1
server b - 2

This is particularly difficult to do in Chef (outside of manually
configuring
this) using things like search. Some methods I have tried or seen,

  1. Search for nodes and sort nodes by name. Use the index of each server
    as
    their id. This can cause servers to use the same id for a moment if
    provisioning servers concurrently. This can also cause problems if a new
    node
    doesn’t get appended to the end causing a number of servers to change
    their
    ids.

  2. Convert ipaddress or some other attribute into an integer.
    Unfortunately
    this produces an integer too large as services like zookeeper only allow
    ids
    between 0-255.

  3. Set this id on the node during provisioning and track the current id
    somewhere. Then you can use search to just get the mapping.

Does anyone have other suggestions? It seems like 3 is the best option
though
not terrible easy to do. It could be easier than I think if I use
something
like chef-metal and define each machine with an id.

Bryan


#3

I do something near to get a minute of execution for different servers,
for your case you may use:

oldrand=srand “server_name”.sum # set the random number generator to a
known value (sum of the hostname)

id=rand(254) # Get a random (but still the same for this server) number

srand oldrand # reset the random number generator to its previous seed

this would give the same origin to random and ends up with the same
number at end.

With that you’ll have a fixed id for a server name.

!WARNING! you may have a collision if two server use the same letters in
name:

irb(main):023:0> “abc”.sum
=> 294
irb(main):024:0> “bac”.sum
=> 294
irb(main):025:0> “cba”.sum
=> 294

Here we’re naming servers like srv0001 and increasing and I’m using the
numeric part for srand instead of sum,

this gives something like oldrand = srand
"server_name"[/([0-9]+),1].to_i

For now I didn’t hit the wall with collision in the same cluster.

Le 2014-10-30 03:50, bjbq4d@gmail.com a écrit :

Hi everyone,

For a few services like zookeeper or kafka, they require a server to integer
mapping like this,

server a - 1
server b - 2

This is particularly difficult to do in Chef (outside of manually configuring
this) using things like search. Some methods I have tried or seen,

  1. Search for nodes and sort nodes by name. Use the index of each server as
    their id. This can cause servers to use the same id for a moment if
    provisioning servers concurrently. This can also cause problems if a new node
    doesn’t get appended to the end causing a number of servers to change their
    ids.

  2. Convert ipaddress or some other attribute into an integer. Unfortunately
    this produces an integer too large as services like zookeeper only allow ids
    between 0-255.

  3. Set this id on the node during provisioning and track the current id
    somewhere. Then you can use search to just get the mapping.

Does anyone have other suggestions? It seems like 3 is the best option though
not terrible easy to do. It could be easier than I think if I use something
like chef-metal and define each machine with an id.

Bryan