"Arbitrary" configs?

Habitat Help
1

This is VERY closely related to the other topic I just posted about composite packages, but I felt it was enough of a different topic to deserve its own post.

Basically, I just built a consul-client package which allows running the Consul binary in “client” or “agent” mode so that it just communicates with a Consul cluster and doesn’t participate in the raft/consensus protocol.

Basically, consul allows you to register arbitrary services and drop the service config in a “config directory” (traditionally /etc/consul.d) where the services will be loaded and reported back to the cluster. Then you’re able to query the consul cluster via dns for your service. This provides significant benefit in cases where you can’t run the hab service or only have the ability to do DNS lookups. (selfishly, I’d like to start working our way away from Consul for service discovery and just use pure Habitat, but I’m not sold that this is the end-all-be-all solution… AND being able to do both provides some ability to “lift and shift” gradually rather than all at once…)

Is there a mechanism for applying “arbitrary” config files to habitat packages? With Chef we have a generic template that just parses node attributes and generates the appropriate JSON in /etc/consul.d. We could conceivably create /hab/svc/consul-client/data/consul.d and just drop files in there with Chef, which may be a fine interim solution, but it feels like this is the wrong approach and goes against the ethos of Habitat as a whole. I really don’t want to create a disparate consul-client package for each potential service config (besides I’m not sure what the logistics of running multiple composite packages natively that all use the same base qubitrenegade/consul-client package you can’t run multiple instances of a package…)

My Chef looks like (this is off the top of my head and my not be 100% accurate but I think gets the gist across):

node['consul-client']['service'].each do |service_name, config|
  template "/etc/consul.d/#{service_name}.json" do
    source 'consul-service.erb'
    variables config: config.to_json
    action :create
  end
end

I mean, maybe the answer is that Chef should still be generating these configs. Although I think it would be cool to be able to just provide that information in the user.toml… I’m not really sold on one way or the other so open to any suggestions!

Thanks!

2

We don’t yet have a way to generate multiple separate files from a single template, though there have been a few requests for something like this. Right now, there is a 1-to-1 relationship between templates and rendered files.

That being said, I could think of some hacky ways you could play around with something like this, given the current capabilities. Since there is a 1-to-1 relationship between templates and rendered files, which affects when the Supervisor is signaled to restart, you could have a “dummy” template that basically exists to just render out all the data that you would ultimately render into separate files, perhaps delimited in a particular way. Then at the beginning of your run hook, you could use a little Bash to split them out into individual files, before running your main service.

Something maybe like this:

{{ #each cfg.configs as |name, data| }}
# {{name}}.cfg
{{data.one_thing}}
{{data.another_thing}}
{{data.yet_one_more_thing}}
<<THIS_IS_MY_DELIMITER>>
{{ /each }}

Then in your run hook, rip through that rendered file, splitting it at the delimiter string, and using the value from {{name}}.cfg to get the name of the individual file it should be written to.

I haven’t tested any of that; it’s just me spit-balling off the top of my head. It’s a filthy hack, but I think it should work, and could be a basis from which to experiment and perhaps develop a “real” feature from.

1 Like
3

Interesting… This is totally worth a try.