"System", "Auto-generated", or "default" defaults

Habitat Help
1

I understand that you can set and override defaults with default.toml and user.toml which works great for well known values.

What about cases where the “default” value can/should be “autodiscovered” or generally relies on “dynamic” data. (My initial thought was to call it “Automatic” data because that’s sort of what it would be in Chef.)

What I really want to do is:

# default.toml
[server]
bind_address = "{{sys.ip}}"

# user.toml
[server]
bind_address = "0.0.0.0"

Say I have a service that has an API and a Cluster address. In most cases I want the “api” and “cluster” addresses to be the same, and those both to be the IP on the first device (which I’m assuming is what sys.ip is… but that’s probably worth a separate thread). But sometimes I’ll run my cluster behind a load balancer, then in that case I’d want to broadcast the IP of the load balancer instead of each of my individual IPs as my “API Address” and have my nodes communicate with each other on their “cluster address” in the private segment behind the LB.

Another use case is when running in a container, I very well may not know my IP, so populating my config file with {{sys.ip}} may be mostly sufficient until I want to put my containers behind a LB. I really don’t want to compile a new package just to update one config setting… then would I have two packages with just that difference?

Or what about an application connecting to a database. Eventually I want to get my database cluster habitized, but for now I’m using an external database cluster. So maybe to help me transition I do:

# default.toml
[database]
address = {{bind.database.leader.sys.ip}}

# user.toml
[database]
address = 192.168.100.152

Then when I’m ready I just remove my user.toml

I can also see a use-case for populating {{sys.org}} and {{sys.group}} in my default.toml. “When running in production we just use “production” everywhere, but when testing we configure our apps differently with specific naming.”

2

I also keep running into cases where I’d really like to be able to template or calculate some defaults and my config templates get pretty gnarly and un-DRY working around it.

Some sort of optional hook scripts for pre- and post-processing the config tree could be really great but I worry that would add too big a crack in habitat’s straightforwardness

3

@qubitrenegade I often do something like this:

{{#if bind.database }}
{{#eachAlive bind.database.members as |member| }}
db_host = {{ member.sys.ip}}
{{/eachAlive}}
{{else}}
db_host = {{cfg.database.address}}
{{/if}}
1 Like
4

Awesome! This gives me the behavior I’m after. Thanks!

{
  "datacenter": "{{cfg.client.datacenter}}",
  {{ #if cfg.data_dir }}
  "data_dir": "{{cfg.client.data_dir}}",
  {{ else }}
  "data_dir": "{{pkg.svc_data_path}}",
  {{ /if }}
  "log_level": "{{cfg.client.loglevel}}",
  {{ #if cfg.bind_addr }}
  "bind_addr": "{{cfg.client.bind_addr}}",
  {{ else }}
  "bind_addr": "{{sys.ip}}",
  {{ /if }}
  {{ #if cfg.advertise_addr }}
  "advertise_addr": "{{cfg.client.advertise_addr}}",
  {{ #else }}
  "advertise_addr": "{{sys.ip}}",
  {{ /if }}
  "client_addr": "{{cfg.client.client_addr}}",
  "retry_join": [
  {{~#eachAlive bind.consul-server.members as |member|}}
    "{{member.sys.ip}}"{{~#unless @last}},{{/unless}}
  {{~/eachAlive}}
  ],
  "ports": {
    "dns": {{cfg.ports.dns}},
    "http": {{cfg.ports.http}},
    "https": {{cfg.ports.https}},
    "serf_lan": {{cfg.ports.serf_lan}},
    "serf_wan": {{cfg.ports.serf_wan}},
    "server": {{cfg.ports.server}}
  }
}

(There’s actually a bug somewhere in that… :confused: )

Compare that to:

{
  "datacenter": "{{cfg.client.datacenter}}",
  "data_dir": "{{cfg.client.data_dir}}",
  "log_level": "{{cfg.client.loglevel}}",
  "bind_addr": "{{cfg.client.bind_addr}}",
  "client_addr": "{{cfg.client.client_addr}}",
  "retry_join": [
  {{~#eachAlive bind.consul-server.members as |member|}}
    "{{member.sys.ip}}"{{~#unless @last}},{{/unless}}
  {{~/eachAlive}}
  ],
  "ports": {
    "dns": {{cfg.ports.dns}},
    "http": {{cfg.ports.http}},
    "https": {{cfg.ports.https}},
    "serf_lan": {{cfg.ports.serf_lan}},
    "serf_wan": {{cfg.ports.serf_wan}},
    "server": {{cfg.ports.server}}
  }
}

I could actually almost replace my entire template with:

{{ toJson cfg }}

If I could specify things like:

[client]
bind_addr = "{{sys.ip}}"

in my default.toml

In my opinion, this would be a much better experience:

[client]
bind_addr = "{{sys.ip}}"

"bind_addr": "{{cfg.client.bind_addr}}",

Than this:

[client]
# uncomment this to override default {{sys.ip}}
# bind_addr = "0.0.0.0"

  {{ #if cfg.client.bind_addr }}
  "bind_addr": "{{cfg.client.bind_addr}}",
  {{ else }}
  "bind_addr": "{{sys.ip}}",
  {{ /if }}

Or at least:

"bind_addr": "{{cfg.client.bind_addr || sys.ip }}"

Having “hidden” or testing for undefined variables seems like anti-pattern to me though. Especially because the default.toml is one of the first things a consumer of a package would generally see.

I think my above example using sys.ip in default.toml is more explicit in saying “The default for this value is {{sys.ip}}” whereas having a series of if statements to decide defaults makes the default behavior opaque to the user.

Anyway, just my thoughts :slight_smile: Thanks again for the suggestion!

2 Likes
5

Can someone explain how difficult would be to render default.toml, as suggested, before its used? And when it happen"?

The case with external LB, was not answered - I may provide env specific user.toml to assign LB address used, or I may have “side care” that will read it, but then I have a need to set it to plan cfg. “With: will read it - I mean even more complex scenario, If that is not LB but k8s policy or scheduler option I really may want to call it to get current valid value.)

Actually I find even interesting to set some values from composite plan, to its component plans. What was the rational to have publicly visible only exposed variables vs all by default?

6

Sorry @epcim I don’t quite follow what you need?

The LB example above doesn’t quite apply towards service rails in k8s. The idea for these configuration files is to write them in a way that they can query their state from the ring and template out based on that global state data. So in the case that you’re deploying a load balancer alongside some services, you would follow the same pattern that @elliott-davis provided above for database. The binding in that case is database but if your service is bound as some thing else, say haproxy-appfoo you can query that same information and use it however makes sense.
The idea is to have all of the global state data available in the ring so that the user can craft a package to be self-updating and autonomous.

I’m not sure if this is helpful or not but happy to discuss further if i’ve missed the point!