Each on array inside bash resource

Chef Infra (archive)
1

Hi, I'm struggling to get my head round a concept, which I need to use to DRY out one of my cookbooks.

Currently I have a bash resource which creates a bunch of Kafka topics...

/opt/kafka/bin/kafka-topic-sh --create --topic TOPIC1 --zookeeper localhost:2182 --partitions 10 --relication-factor 10
/opt/kafka/bin/kafka-topic-sh --create --topic TOPIC2 --zookeeper localhost:2182 --partitions 10 --relication-factor 10
etc.

But I'm trying to do something along the lines of :-

bash 'create kafka topics' do
code <<-EOH
systemctl start kafka
#{ %w(TOPIC1 TOPIC2).each do |topic| }
/opt/kafka/bin/kafka-topics.sh ..... --topic #{topic} --zookeeper....
#{end}
end

However, when I try to fire this up in test kitchen it fails miserably with error messages about "syntax error, unexpected tSTRING_DEND" and "unknown regex options - kafka"

Perhaps it's getting late in the day, but I just can't think of the correct way to do this.

Hopefully it's clear what I'm trying to achieve here.

Any pointers much appreciated. I've tried various googlings, but getting more confused.

2

The immediate source of the syntax error is that you are missing a closing EOH on your heredoc. https://www.rubyguides.com/2018/11/ruby-heredoc/

After that though, I think there will be a problem with your interpolated values. It looks like you are using some control flow on different lines like you might expect in a template erb file, but in strings and heredocs each interpolation (each set of #{}) has its own context and needs to produce a string. Depending on how many topics you have you may want to just write them out, it will probably be easier to read:

bash 'create kafka topics' do
  code <<-EOH
    systemctl start kafka
    /opt/kafka/bin/kafka-topics.sh ..... --topic #{TOPIC1} --zookeeper....
    /opt/kafka/bin/kafka-topics.sh ..... --topic #{TOPIC2} --zookeeper....
  EOH
end

If you do need loops or similar blocks though you will have to keep it all inside one set of #{} or perhaps build the string outside of the heredoc and just save it as a variable which you pass to code.

3

why not just loop the resource

Also remember to add in some guards so it won't run before

service kafka do
  action :start
end

#{ %w(TOPIC1 TOPIC2).each do |topic| }
bash "create kafka topic #{topic}" do
  code "/opt/kafka/bin/kafka-topics.sh ..... --topic #{topic} --zookeeper...."
  only_if 'ruby code to check topic does not already exist'
end
4

Will that actually work though? As the post above says, it's not like a template and the loop through the array would simply be an empty loop, and the code would run once with #{topic} being empty as it's not defined in the scope.

5

Currently it's 15 topics, which is why I thought it was time to DRY it. Perhaps I should use a template?

6

Fixed it. I was forgetting that you can simply put ruby straight into the cookbooks. The topics are all in a data bag now, but this could have just been a hash.

topics.each do |topic|
script 'create topic' do
code "kafka-topics.sh --create --topic {topic[name]} --partitions #{topic['partitions']} ....
not_if "kafka-topics.sh --list .... | grep #{topic['name']}
end
end

It also copes with having different number of partitions now depending on topic.

1 Like