Attributes in templates

Chef Infra (archive)
1

Hello all,

I have a config file which looks like this:

[cache:a]
MAX_CACHE_SIZE = 2000
MAX_UPDATES_PER_SECOND = 500
MAX_CREATES_PER_MINUTE = 100
LOG_UPDATES = False

[cache:b]
MAX_CACHE_SIZE = 2000
MAX_UPDATES_PER_SECOND = 500
MAX_CREATES_PER_MINUTE = 100
LOG_UPDATES = True

[cache:c]
MAX_CACHE_SIZE = 5000
MAX_UPDATES_PER_SECOND = 300
MAX_CREATES_PER_MINUTE = 200
LOG_UPDATES = False

I’ve created the following attributes:

default['app']['cache_instance_name']     = ['a','b','c']
default['app']['max_cache_size']          = [2000, 2000, 5000]
default['app']['max_updates_per_second']  = [500, 500, 300]
default['app']['max_creates_per_minutes'] = [100, 100, 200]
default['app']['log_updates']             = ['False', 'True', 'False']

What I need and it seems to be complicated is to have a template file which with help from mighty ERuby, will generate a config file that looks like the original above

What I was thinking of is something like:

[cache:<%= node['app']['cache_instance_name'].each do |max_cache_size, max_updates_per_second, max_creates_per_minutes, log_updates| %>]
MAX_CACHE_SIZE =<%="{max_cache_size}%>
MAX_UPDATES_PER_SECOND =<%="{max_updates_per_second}%>
MAX_CREATES_PER_MINUTE =<%="{max_creates_per_minutes}%>
LOG_UPDATES =<%="{log_updates}%>
<% end %>

Is this a good approach? Is the best? Is there any better one?

Thank you,
Gabriel

2

My personal thoughts here is that you should NEVER have node attributes in templates. The ONLY place you ever want to see node attributes are attribute files and recipies. Why? You are less likely left scratching your head wondering where values are appearing from.

In your case, I’d just have simple variables in the template and feed them with attributes in the recipe.

3

Could you please give me an example?
Like this?

template 'app.conf' do
	path "/local/conf/app.conf"
	source "app.conf.erb"
	owner 'user'
	group 'user'
	mode "0644",
	variables(:max_cache_size => node['app']['max_cache_size'])
end

And in the template file like:
MAX_CACHE_SIZE =<%=max_cache_size%>
?
Thank you.

4

Thats exactly it.

5

I am still having a dilemma on how to get the config file to look like how it has to.
In the recipe now I have this:

template 'app.conf' do
path "/local/conf/app.conf"
source "app.conf.erb"
owner 'user'
group 'user'
mode "0644"
variables(
:cache_instance_name => node['app']['cache_instance_name'],
:max_cache_size => node['app']['max_cache_size'],
:max_updates_per_second => node['app']['max_updates_per_second'],
:max_creates_per_minutes => node['app']['max_creates_per_minutes'],
:log_updates => node['app']['log_updates']
)
end

The app.conf.erb looks like this:

<% @cache_instance_name.each do |cache_instance_name, max_cache_size, max_updates_per_second, max_creates_per_minutes, log_updates| %>
[cache:<%= cache_instance_name %>]
MAX_CACHE_SIZE = <%= "#{max_cache_size}" %>
MAX_UPDATES_PER_SECOND = <%= "#{max_updates_per_second}" %>
MAX_CREATES_PER_MINUTE = <%= "#{max_creates_per_minutes}" %>
LOG_UPDATES = <%= "#{log_updates}" %>
<% end %>

And the resulted config although shows the instances names, does not show any other variable:

[cache:a]
MAX_CACHE_SIZE =
MAX_UPDATES_PER_SECOND =
MAX_CREATES_PER_MINUTE =
LOG_UPDATES =
[cache:b]
MAX_CACHE_SIZE =
MAX_UPDATES_PER_SECOND =
MAX_CREATES_PER_MINUTE =
LOG_UPDATES =
[cache:c]
MAX_CACHE_SIZE =
MAX_UPDATES_PER_SECOND =
MAX_CREATES_PER_MINUTE =
LOG_UPDATES =

What do I do wrong?

Thank you,
Gabriel

6

Do the values need to be in quotes? Are the template values repeated or did you want different values per cache?

I think the variables are “top level”, this has more to do with erubis and binding contexts.

Try prefixing your variables with the at sign, in the template

<% @cache_instance_name.each do |cache_instance_name| %>
[cache:<%= cache_instance_name %>]
MAX_CACHE_SIZE = <%= "#{@max_cache_size}" %>
MAX_UPDATES_PER_SECOND = <%= "#{@max_updates_per_second}" %>
MAX_CREATES_PER_MINUTE = <%= "#{@max_creates_per_minutes}" %>
LOG_UPDATES = <%= "#{@log_updates}" %>
<% end %>
7

Hello and thank you for the fast reply.

My attributes/default.rb looks like this:

default['app']['cache_instance_name']     = ['a','b','c']
default['app']['max_cache_size']          = [2000, 2000, 5000]
default['app']['max_updates_per_second']  = [500, 500, 300]
default['app']['max_creates_per_minutes'] = [100, 100, 200]
default['app']['log_updates']             = ['False', 'True', 'False']

Because the conf file should look like this in the end:

[cache:a]
MAX_CACHE_SIZE = 2000
MAX_UPDATES_PER_SECOND = 500
MAX_CREATES_PER_MINUTE = 100
LOG_UPDATES = False

[cache:b]
MAX_CACHE_SIZE = 2000
MAX_UPDATES_PER_SECOND = 500
MAX_CREATES_PER_MINUTE = 100
LOG_UPDATES = True

[cache:c]
MAX_CACHE_SIZE = 5000
MAX_UPDATES_PER_SECOND = 300
MAX_CREATES_PER_MINUTE = 200
LOG_UPDATES = False

But I need to be able to scale to more instances.
If I add 5 more instances, like in:

default['app']['cache_instance_name'] = ['a','b','c','d','inst1','inst2','in32','testinstance']

In the end in the config file I would need to have eight

[cache:<instance_name>]
MAX_CACHE_SIZE = <max cache size value for every max_cache_size of every instance>
MAX_UPDATES_PER_SECOND = <and so on>
MAX_CREATES_PER_MINUTE = <and so on>
LOG_UPDATES = <and so on>

If we take the first instance, ‘a’, then the MAX_CACHE_SIZE would need to take the value 2000, the MAX_UPDATES_PER_SECOND would need to the the value 500 and so on, but if we get to the 3rd instance, ‘c’, then the MAX_CACHE_SIZE would get the value 5000, the MAX_UPDATES_PER_SECOND would get the value 300 and so on. So depending on the instance, the other variables should take other values.
I might have define the attributes wrong…

Does that make any sense?

Thanks again.

8

Ok.Thanks

You need to iterate through several arrays, the code you had just iterated through a single array.

Could change to iterate ALL of the arrays

<% (0..@cache_instance_name.length-1).each do |idx|  %>
[cache:<%= cache_instance_name[idx] %>]
MAX_CACHE_SIZE = <%= "#{@max_cache_size[idx]}" %>
MAX_UPDATES_PER_SECOND = <%= "#{@max_updates_per_second[idx]}" %>
MAX_CREATES_PER_MINUTE = <%= "#{@max_creates_per_minutes[idx]}" %>
LOG_UPDATES = <%= "#{@log_updates[idx]}" %>
<% end %>

But if it were me I’d consider storing the data as Hash or represent the Hash via attributes, for example
default[‘app’][‘a’][‘max_cache_size’] = 2000
default[‘app’][‘a’][‘max_updates_per_second’] = 500
default[‘app’][‘a’][‘max_creates_per_minutes’] = 100
default[‘app’][‘a’][‘log_updates’] = ‘False’

default[‘app’][‘b’[‘max_cache_size’] = 2000
default[‘app’][‘b’][‘max_updates_per_second’] = 500
default[‘app’][‘b’][‘max_creates_per_minutes’] = 100
default[‘app’][‘b’][‘log_updates’] = ‘True’

default[‘app’]’‘c’][‘max_cache_size’] = 5000
default[‘app’]['c"][‘max_updates_per_second’] = 300
default[‘app’][‘c’][‘max_creates_per_minutes’] = 200
default[‘app’][‘c’][‘log_updates’] = ‘False’

Then …

template 'app.conf' do
...
variables(:instances => node['app'])
end

And in the template

<% @instances.each_pair do |name, settings| %>
[cache:<%= name %>]
MAX_CACHE_SIZE = <%= "#{settings['max_cache_size']}" %>
MAX_UPDATES_PER_SECOND = <%= "#{settings['max_updates_per_second']}" %>
MAX_CREATES_PER_MINUTE = <%= "#{settings['max_creates_per_minutes']}" %>
LOG_UPDATES = <%= "#{settings['log_updates']}" %>
<% end %>

Code isn’t tested, written inline so I apologize up front if it doesn’t work as is.

9

Thank you for your help.
I’ve tried now with:

default['app']['a']['max_cache_size'] = 2000
default['app']['a']['max_updates_per_second'] = 500
default['app']['a']['max_creates_per_minutes'] = 100
default['app']['a']['log_updates'] = 'False'

default['app']['b']['max_cache_size'] = 2000
default['app']['b']['max_updates_per_second'] = 500
default['app']['b']['max_creates_per_minutes'] = 100
default['app']['b']['log_updates'] = 'True'

default['app']['c']['max_cache_size'] = 5000
default['app']['c']['max_updates_per_second'] = 300
default['app']['c']['max_creates_per_minutes'] = 200
default['app']['c']['log_updates'] = 'False'

and the template looks like this:

<% @instances.each_pair do |name, settings| %>
[cache:<%= name %>]
MAX_CACHE_SIZE = <%= "#{settings['max_cache_size']}" %>
MAX_UPDATES_PER_SECOND = <%= "#{settings['max_updates_per_second']}" %>
MAX_CREATES_PER_MINUTE = <%= "#{settings['max_creates_per_minutes']}" %>
LOG_UPDATES = <%= "#{settings['log_updates']}" %>
<% end %>

But when I try to run it, I get the following error:

Compiled Resource:
------------------
# Declared in /var/chef/cache/cookbooks/app/recipes/default.rb:63:in `from_file'

template("app.conf") do
  action [:create]
  retries 0
  retry_delay 2
  default_guard_interpreter :default
  source "app.conf.erb"
  variables {:instances=>{"a"=>{"max_cache_size"=>2000, "max_updates_per_second"=>500, "max_creates_per_minutes"=>100, "log_updates"=>"False"}, "b"=>{"max_cache_size"=>2000, "max_updates_per_second"=>500, "max_creates_per_minutes"=>100, "log_updates"=>"True"}, "c"=>{"max_cache_size"=>5000, "max_updates_per_second"=>300, "max_creates_per_minutes"=>200, "log_updates"=>"False"}, "relay_line_receiver_port"=>2003, "relay_pickle_receiver_port"=>2004, "relay_max_queue_size"=>10000, "max_datapoints_per_message"=>500, "relay_method"=>"consistent-hashing", "relay_destinations"=>"127.0.0.1"}}
  declared_type :template
  cookbook_name "app"
  recipe_name "default"
  path "/local/app/conf/app.conf"
  owner "user"
  group "user"
  mode "0644"
  atomic_update true
end

Template Context:
-----------------
on line #3
  1: <% @instances.each_pair do |name, settings| %>
  2: [cache:<%= name %>]
  3: MAX_CACHE_SIZE = <%= "#{settings['max_cache_size']}" %>
  4: MAX_UPDATES_PER_SECOND = <%= "#{settings['max_updates_per_second']}" %>
  5: MAX_CREATES_PER_MINUTE = <%= "#{settings['max_creates_per_minutes']}" %>


Running handlers:
[2016-05-19T09:48:10+00:00] ERROR: Running exception handlers
Running handlers complete
[2016-05-19T09:48:10+00:00] ERROR: Exception handlers complete
Chef Client failed. 1 resources updated in 17 seconds
[2016-05-19T09:48:10+00:00] FATAL: Stacktrace dumped to /var/chef/cache/chef-stacktrace.out
[2016-05-19T09:48:10+00:00] FATAL: Please provide the contents of the stacktrace.out file if you file a bug report
[2016-05-19T09:48:10+00:00] ERROR:

Chef::Mixin::Template::TemplateError (no implicit conversion of String into Integer) on line #3:

  1: <% @instances.each_pair do |name, settings| %>
  2: [cache:<%= name %>]
  3: MAX_CACHE_SIZE = <%= "#{settings['max_cache_size']}" %>
  4: MAX_UPDATES_PER_SECOND = <%= "#{settings['max_updates_per_second']}" %>
  5: MAX_CREATES_PER_MINUTE = <%= "#{settings['max_creates_per_minutes']}" %>

  (erubis):3:in `[]'
  (erubis):3:in `block (2 levels) in evaluate'
  (erubis):1:in `each_pair'
  (erubis):1:in `block in evaluate'

What am I missing?

Thanks again.

10

Hi,

Not sure, I’ve just tried it out and works for me.

Attributes file

default['app']['a']['max_cache_size'] = 2000
default['app']['a']['max_updates_per_second'] = 500
default['app']['a']['max_creates_per_minutes'] = 100
default['app']['a']['log_updates'] = 'False'

default['app']['b']['max_cache_size'] = 2000
default['app']['b']['max_updates_per_second'] = 500
default['app']['b']['max_creates_per_minutes'] = 100
default['app']['b']['log_updates'] = 'True'

default['app']['c']['max_cache_size'] = 5000
default['app']['c']['max_updates_per_second'] = 300
default['app']['c']['max_creates_per_minutes'] = 200
default['app']['c']['log_updates'] = 'False'

Recipe File

template 'c:\temp\test.cfg' do
	source "test.erb"
	variables(
		:instances => node['app']
	)
end

Template File

<% @instances.each_pair do |name,settings| %>
[cache:<%= name %>]
MAX_CACHE_SIZE = <%= "#{settings['max_cache_size']}" %>
MAX_UPDATES_PER_SECOND = <%= "#{settings['max_updates_per_second']}" %>
MAX_CREATES_PER_MINUTE = <%= "#{settings['max_creates_per_minutes']}" %>
LOG_UPDATES = <%= "#{settings['log_updates']}" %>
<% end %>

The output was a file in c:\temp named test.cfg with the content of

[cache:a]
MAX_CACHE_SIZE = 2000
MAX_UPDATES_PER_SECOND = 500
MAX_CREATES_PER_MINUTE = 100
LOG_UPDATES = False
[cache:b]
MAX_CACHE_SIZE = 2000
MAX_UPDATES_PER_SECOND = 500
MAX_CREATES_PER_MINUTE = 100
LOG_UPDATES = True
[cache:c]
MAX_CACHE_SIZE = 5000
MAX_UPDATES_PER_SECOND = 300
MAX_CREATES_PER_MINUTE = 200
LOG_UPDATES = False

What I would add is that the template uses string interpolation when it doesn’t need to, therefore

MAX_CACHE_SIZE = <%= "#{settings['max_cache_size']}" %>

can become

MAX_CACHE_SIZE = <%= settings['max_cache_size'] %>

If you need to quote the values in your configuration file it will be

MAX_CACHE_SIZE = "<%= settings['max_cache_size'] %>"
1 Like
11

And it worked. Thank you.

Now for the last part…
The last piece of the config file should look like this:

DESTINATIONS = 127.0.0.1:2000:a, 127.0.0.1:2000:b, 127.0.0.1:5000:c

So for each instance, in the DESTINATION field, will appear something like

iplocalhost(127.0.0.1):max_cache_size-for each instance(2000):instance_name(a)

The attribute I should have is this:

default['app']['ip_localhost'] = '127.0.0.1'

But if I add it to my default.rb attribute file, I get an error.
The first thing I should do is to create another attribute file? How would I address it?

The second thing. How would I use the attributes from default.rb with the one from the second attribute file?

What other choice do I have?

Thank you again.

12

It’s pretty much more of the same, you want

DESTINATIONS = 127.0.0.1:2000:a, 127.0.0.1:2000:b, 127.0.0.1:5000:c

Is that data to be built from the existing attributes or a new value in the attributes?

Creating a new attributes file does not partition the attributes in any way, you just gain clarity by being able to group like attributes together.

I suspect the error you are getting is because the code iterates through node[‘app’] and expects another hash, your attribute [‘ip_localhosrt’] is a single value.

You would have to make a different attribute name I’m afraid, such as

default['destinations'] = '127.0.0.1'
13

Hello.

Thank you for your response.
The solution I used is this:

<%
destinations = ""
@instances.each_pair do |name,settings|
	destinations = destinations + "#{settings['relay_destinations']}:#{settings['pickle_receiver_port']}:#{name}, "
end
destinations = destinations.chomp(", ")
%>

[relay]
DESTINATIONS = <%= destinations %>

Best regards,
Gabriel

14

Hi,
I have to do similar thing & generate an XML application properties file as below from the template file

 <servers>
     <server hostname="all">
        <property name="a.ext.connections" value="5" />
        <property name="a.max.connections" value="15" />
        <property name="userid" value="WebUser" />
        <property name="cryptolabel" value=“fiware-authn-c***” />

        <!--		This Property will be used to set to options for		enabling/disabling risk engine in analyze calls Option's		available currently are ALL,DEVICE_ONLY, RISK_ONLY and NONE	-->
        <property name="runrisktype" value="DEVICE_ONLY" />
    </server>
    <server hostname=“DEV”>
        <property name=“link”			url="https://dev.services.ext:2017/Auth/services/Authentication" />
    </server>
    <server hostname=“UAT”>
        <property name=“link”			url="https://stage.services.ext: 2017/Auth/services/Authentication" />
    </server>
    <server hostname=“PROD”>
        <property name=“link”			url="https://stage.services.ext 2017/Auth/services/AAuthentication" />
    </server>
</servers>

Can anyone help how this can be achieved?