Overriding attributes

Jeffrey_Jones · June 28, 2013, 8:55am

Hello all

I have a list of default users I would like for my DB and am attempting
to combine them with ones set in an attribute.json file used by chef
solo. The relevant code is below

gist.github.com

https://gist.github.com/rurounijones/5883385

attributes.json

// This is a json file used by chef-solo with the -j option

{
    "postgresql" : {
        "pg_hba": [
            { "type":"host", "db":"all", "user":"master", "addr":"10.80.79.0/24", "method":"md5"}
        ]
    }
}

database.rb

# This is a recipe file

default_db_users = [
  { :type => 'host', :db => 'all', :user => 'postgres', :addr => '127.0.0.1/32', :method => 'md5'},
  { :type => 'host', :db => 'all', :user => 'administrator', :addr => '192.168.80.10/32', :method => 'md5' },
  { :type => 'host', :db => 'all', :user => 'connector', :addr => '127.0.0.1/32', :method => 'md5' }
]

node.force_override['postgresql']['pg_hba'] = default_db_users + node['postgresql']['pg_hba']

This file has been truncated. show original

If there is nothing set in the json file then it write the 3 default
values to pg_hba.conf

However if there are values set in the json then ONLY the values in the
.json file are used. They are not merged in with the default_db_users
despite my setting of them (I tried everything up to force_override
which, as far as I see it, should take precedence),

Am I missing something here?

Cheers

Jeff

DV1 · March 18, 2014, 6:55pm

Ran into this issue today so thought I'd revive this thread.

I'm using postgresql cookbook with chef-solo and cookbook attributes
aren't merged together with attributes set in .json file passed to
chef-solo. Instead .json file attributes override those set in
cookbook.

Here's what's in postgresql/attributes/default.rb:
...
default['postgresql']['pg_hba'] = [
{:type => 'local', :db => 'all', :user => 'postgres', :addr => nil,
:method => 'ident'},
{:type => 'local', :db => 'all', :user => 'all', :addr => nil,
:method => 'ident'},
{:type => 'host', :db => 'all', :user => 'all', :addr =>
'127.0.0.1/32', :method => 'md5'},
{:type => 'host', :db => 'all', :user => 'all', :addr => '::1/128',
:method => 'md5'}
]
...

Here's what in my .json file:
...
{
...
"postgresql": {
...
"pg_hba": [
{ "type": "host", "db": "replication", "user": "postgres",
"addr": "10.0.0.0/8", "method": "trust" }
],
...
"run_list": [ "recipe[postgresql::server]" ]
}

Resulting pg_hba.conf has only one line in it:

host replication postgres 10.0.0.0/8 trust

Since node[:postgresq][:pg_hba] is an array attribute, it should be
merged rather than overwritten.

The behavior is same with chef-solo 10.28.0 and 11.6.0.

I'll go ahead and submit that as a defect unless someone thinks it's
intended behavior.

On Fri, Jun 28, 2013 at 1:55 AM, Jeffrey Jones jjones@toppan-f.co.jp wrote:

Hello all

I have a list of default users I would like for my DB and am attempting to
combine them with ones set in an attribute.json file used by chef solo. The
relevant code is below

attributes.json · GitHub

If there is nothing set in the json file then it write the 3 default values
to pg_hba.conf

However if there are values set in the json then ONLY the values in the
.json file are used. They are not merged in with the default_db_users
despite my setting of them (I tried everything up to force_override which,
as far as I see it, should take precedence),

Am I missing something here?

Cheers

Jeff

--
Best regards, Dmitriy V.

kallistec · March 19, 2014, 3:17pm

On Tuesday, March 18, 2014 at 11:55 AM, DV wrote:

Ran into this issue today so thought I'd revive this thread.

I'm using postgresql cookbook with chef-solo and cookbook attributes
aren't merged together with attributes set in .json file passed to
chef-solo. Instead .json file attributes override those set in
cookbook.

Here's what's in postgresql/attributes/default.rb:
...
default['postgresql']['pg_hba'] = [
{:type => 'local', :db => 'all', :user => 'postgres', :addr => nil,
:method => 'ident'},
{:type => 'local', :db => 'all', :user => 'all', :addr => nil,
:method => 'ident'},
{:type => 'host', :db => 'all', :user => 'all', :addr =>
'127.0.0.1/32', :method => 'md5'},
{:type => 'host', :db => 'all', :user => 'all', :addr => '::1/128',
:method => 'md5'}
]
...

Here's what in my .json file:
...
{
...
"postgresql": {
...
"pg_hba": [
{ "type": "host", "db": "replication", "user": "postgres",
"addr": "10.0.0.0/8", "method": "trust" }
],
...
"run_list": [ "recipe[postgresql::server]" ]
}

Resulting pg_hba.conf has only one line in it:

host replication postgres 10.0.0.0/8 trust

Since node[:postgresq][:pg_hba] is an array attribute, it should be
merged rather than overwritten.

The behavior is same with chef-solo 10.28.0 and 11.6.0.

I'll go ahead and submit that as a defect unless someone thinks it's
intended behavior.

On Fri, Jun 28, 2013 at 1:55 AM, Jeffrey Jones <jjones@toppan-f.co.jp (mailto:jjones@toppan-f.co.jp)> wrote:

Hello all

I have a list of default users I would like for my DB and am attempting to
combine them with ones set in an attribute.json file used by chef solo. The
relevant code is below

attributes.json · GitHub

If there is nothing set in the json file then it write the 3 default values
to pg_hba.conf

However if there are values set in the json then ONLY the values in the
.json file are used. They are not merged in with the default_db_users
despite my setting of them (I tried everything up to force_override which,
as far as I see it, should take precedence),

Am I missing something here?

Cheers

Jeff

--
Best regards, Dmitriy V.

Arrays are difficult to get right, because the behavior you want totally depends on what your use case is. For example, you might use an array to specify a list of ports that a server should bind to. So your cookbook might specify, say, port 80 and 443 for an HTTP server. If arrays merged in a purely additive way, then you would not be able to specify that the server should listen only on, say, 8000 and 8443.

Chef attempts to reach a compromise by merging arrays within precedence levels, that is, default attributes regardless of source (environment, role, cookbook) are merged, as are override attributes from those sources. Arrays are replaced between different precedence levels, however, so when you use -j SOME_JSON, these become “normal” level attributes, where arrays will replace anything set at the “default” level (which is what cookbooks typically do).

Your best bet is to work with the cookbook maintainer to come up with a non-array-based attributes API, which will behave the way you want.

--
Daniel DeLeo

DV1 · March 21, 2014, 11:11pm

Thanks Daniel, now it makes a little more sense. I guess it would be
helpful if precedence of attributes could be defined in the JSON files
that are passed to chef-solo so that there's a choice of what happens
to them.

On Wed, Mar 19, 2014 at 8:17 AM, Daniel DeLeo dan@kallistec.com wrote:

On Tuesday, March 18, 2014 at 11:55 AM, DV wrote:

Ran into this issue today so thought I'd revive this thread.

I'm using postgresql cookbook with chef-solo and cookbook attributes
aren't merged together with attributes set in .json file passed to
chef-solo. Instead .json file attributes override those set in
cookbook.

Here's what's in postgresql/attributes/default.rb:
...
default['postgresql']['pg_hba'] = [
{:type => 'local', :db => 'all', :user => 'postgres', :addr => nil,
:method => 'ident'},
{:type => 'local', :db => 'all', :user => 'all', :addr => nil,
:method => 'ident'},
{:type => 'host', :db => 'all', :user => 'all', :addr =>
'127.0.0.1/32', :method => 'md5'},
{:type => 'host', :db => 'all', :user => 'all', :addr => '::1/128',
:method => 'md5'}
]
...

Here's what in my .json file:
...
{
...
"postgresql": {
...
"pg_hba": [
{ "type": "host", "db": "replication", "user": "postgres",
"addr": "10.0.0.0/8", "method": "trust" }
],
...
"run_list": [ "recipe[postgresql::server]" ]
}

Resulting pg_hba.conf has only one line in it:

host replication postgres 10.0.0.0/8 trust

Since node[:postgresq][:pg_hba] is an array attribute, it should be
merged rather than overwritten.

The behavior is same with chef-solo 10.28.0 and 11.6.0.

I'll go ahead and submit that as a defect unless someone thinks it's
intended behavior.

On Fri, Jun 28, 2013 at 1:55 AM, Jeffrey Jones <jjones@toppan-f.co.jp (mailto:jjones@toppan-f.co.jp)> wrote:

Hello all

I have a list of default users I would like for my DB and am attempting to
combine them with ones set in an attribute.json file used by chef solo. The
relevant code is below

attributes.json · GitHub

If there is nothing set in the json file then it write the 3 default values
to pg_hba.conf

However if there are values set in the json then ONLY the values in the
.json file are used. They are not merged in with the default_db_users
despite my setting of them (I tried everything up to force_override which,
as far as I see it, should take precedence),

Am I missing something here?

Cheers

Jeff

--
Best regards, Dmitriy V.

Arrays are difficult to get right, because the behavior you want totally depends on what your use case is. For example, you might use an array to specify a list of ports that a server should bind to. So your cookbook might specify, say, port 80 and 443 for an HTTP server. If arrays merged in a purely additive way, then you would not be able to specify that the server should listen only on, say, 8000 and 8443.

Chef attempts to reach a compromise by merging arrays within precedence levels, that is, default attributes regardless of source (environment, role, cookbook) are merged, as are override attributes from those sources. Arrays are replaced between different precedence levels, however, so when you use -j SOME_JSON, these become “normal” level attributes, where arrays will replace anything set at the “default” level (which is what cookbooks typically do).

Your best bet is to work with the cookbook maintainer to come up with a non-array-based attributes API, which will behave the way you want.

--
Daniel DeLeo

--
Best regards, Dmitriy V.

Topic		Replies	Views
Override attributes using JSON Chef Infra (archive)	1	582	August 3, 2010
Overriding default attributes Chef Infra (archive)	5	1812	February 6, 2014
How to override attribute values at chef-client runtime Chef Infra (archive)	11	15885	November 7, 2017
Attribute overrides in wrapper cookbook Chef Infra (archive)	7	385	January 17, 2014
Overriding Chef Attributes in an Array Chef Infra (archive)	1	2153	November 30, 2015

Overriding attributes

Related topics