Ohai, number ldap accounts

Chef Infra (archive)
1

Fellow chef’s,

I have a question about ldap accounts that ohai pulls down. We are a
large shop with thousands of servers and we have a large number of ldap
accounts (only a few are currently managed by chef). As per the issue
in OHAI-165 http://tickets.opscode.com/browse/OHAI-165 , I believe it
may be affecting performance on our chef server.

One solution proffered is to place Ohai::Config[:disabled_plugins] = [
“passwd” ] in the client config to disable this. However, if we do
this, won’t the ldap accounts be unavailable in resources? Meaning, If
I assign the ownership of a file to an ldap account in a file resource,
won’t that cause a failure. This since chef will not know anything
about that account…

file “/tmp/myfile” do

owner “ldapacct1”

group “ldapgroup1”

mode “0600”

end

http://tickets.opscode.com/browse/OHAI-165

Anyone have any guidance on this issue? I would like to keep all the
ldap info out of ohai, but still be able to use ldap accounts in a
resource.

Thanks

Randy

2

Hi Randy,

We are experiencing this very same issue and were recently looking for a solution. We use centrify to pull in AD accounts for authorization. The nodes are pulling in tons of data for all of these user groups and accounts.

If we’re not doing anything with specific user accounts in recipes/templates, should we be safe to just disable the Ohai plugin that pulls in the user accounts from AD?

Ian D. Rossi
CD - Server Infrastructure
Phone 23834

From: Van Fossan,Randy [vanfossr@oclc.org]
Sent: Wednesday, March 28, 2012 11:10 AM
To: chef@lists.opscode.com
Subject: [chef] ohai, number ldap accounts

Fellow chef’s,

I have a question about ldap accounts that ohai pulls down. We are a large shop with thousands of servers and we have a large number of ldap accounts (only a few are currently managed by chef). As per the issue in OHAI-165http://tickets.opscode.com/browse/OHAI-165, I believe it may be affecting performance on our chef server.

One solution proffered is to place Ohai::Config[:disabled_plugins] = [ “passwd” ] in the client config to disable this. However, if we do this, won’t the ldap accounts be unavailable in resources? Meaning, If I assign the ownership of a file to an ldap account in a file resource, won’t that cause a failure. This since chef will not know anything about that account…

file “/tmp/myfile” do
owner “ldapacct1”
group “ldapgroup1”
mode "0600"
end

http://tickets.opscode.com/browse/OHAI-165

Anyone have any guidance on this issue? I would like to keep all the ldap info out of ohai, but still be able to use ldap accounts in a resource.

Thanks
Randy

3

On Wednesday, March 28, 2012 at 8:25 AM, IDROSSI@jw.org wrote:

Hi Randy,

We are experiencing this very same issue and were recently looking for a solution. We use centrify to pull in AD accounts for authorization. The nodes are pulling in tons of data for all of these user groups and accounts.

If we're not doing anything with specific user accounts in recipes/templates, should we be safe to just disable the Ohai plugin that pulls in the user accounts from AD?
Chef doesn't use ohai data to lookup user/group data for file/template/directory or execute/script or user/group resources. It uses Ruby's Etc module, which in turn uses the getpwent family of functions.

So it's totally safe to remove the plugin if you don't need the data for other reasons.

--
Dan DeLeo

4

That is good news. I need to incorporate that in the chef-client cookbook so that when the node is bootstrapped, it doesn't pull the data in after the first run. Anyone, Know how I can incorporate the plugin disable when doing a bootstrap? I do not want to delete the plugin as there may be reason to use it in the future or on a particular node.

I am not sure where to incorporate this...
Ohai::Config[:disabled_plugins] = [ "passwd" ]

Thanks all
Randy

-----Original Message-----
From: Daniel DeLeo [mailto:ddeleo@kallistec.com] On Behalf Of Daniel DeLeo
Sent: Wednesday, March 28, 2012 11:58 AM
To: chef@lists.opscode.com
Cc: GJZULAUF@jw.org; KAKiner@jw.org; EMENDOZA@jw.org
Subject: [chef] Re: RE: ohai, number ldap accounts

On Wednesday, March 28, 2012 at 8:25 AM, IDROSSI@jw.org wrote:

Hi Randy,

We are experiencing this very same issue and were recently looking for a solution. We use centrify to pull in AD accounts for authorization. The nodes are pulling in tons of data for all of these user groups and accounts.

If we're not doing anything with specific user accounts in recipes/templates, should we be safe to just disable the Ohai plugin that pulls in the user accounts from AD?
Chef doesn't use ohai data to lookup user/group data for file/template/directory or execute/script or user/group resources. It uses Ruby's Etc module, which in turn uses the getpwent family of functions.

So it's totally safe to remove the plugin if you don't need the data for other reasons.

--
Dan DeLeo

5

While this probably isn't the 'proper' way, I added:

<% if node.attribute?("ohai") && node["ohai"].attribute?("disabled_plugins") -%>

Ohai::Config[:disabled_plugins] = [<%=
node["ohai"]["disabled_plugins"].join(",") %>]
<% end -%>

to the client.rb template in the chef-client cookbook. Yes, it does go
away if you upgrade your cookbook, but for me thats been pretty safe
since it takes an act of nature to upgrade anything.

On Wed, Mar 28, 2012 at 9:52 AM, Van Fossan,Randy vanfossr@oclc.org wrote:

That is good news. I need to incorporate that in the chef-client cookbook so that when the node is bootstrapped, it doesn't pull the data in after the first run. Anyone, Know how I can incorporate the plugin disable when doing a bootstrap? I do not want to delete the plugin as there may be reason to use it in the future or on a particular node.

I am not sure where to incorporate this...
Ohai::Config[:disabled_plugins] = [ "passwd" ]

Thanks all
Randy

-----Original Message-----
From: Daniel DeLeo [mailto:ddeleo@kallistec.com] On Behalf Of Daniel DeLeo
Sent: Wednesday, March 28, 2012 11:58 AM
To: chef@lists.opscode.com
Cc: GJZULAUF@jw.org; KAKiner@jw.org; EMENDOZA@jw.org
Subject: [chef] Re: RE: ohai, number ldap accounts

On Wednesday, March 28, 2012 at 8:25 AM, IDROSSI@jw.org wrote:

Hi Randy,

We are experiencing this very same issue and were recently looking for a solution. We use centrify to pull in AD accounts for authorization. The nodes are pulling in tons of data for all of these user groups and accounts.

If we're not doing anything with specific user accounts in recipes/templates, should we be safe to just disable the Ohai plugin that pulls in the user accounts from AD?
Chef doesn't use ohai data to lookup user/group data for file/template/directory or execute/script or user/group resources. It uses Ruby's Etc module, which in turn uses the getpwent family of functions.

So it's totally safe to remove the plugin if you don't need the data for other reasons.

--
Dan DeLeo

--
Debian GNU/Linux comes with ABSOLUTELY NO WARRANTY, to the extent
permitted by applicable law.

6

I'm glad to hear that as well. I would also like to know how to do the same thing...with bootstrap.

Ian D. Rossi
CD - Server Infrastructure
Phone 23834

From: Van Fossan,Randy [vanfossr@oclc.org]
Sent: Wednesday, March 28, 2012 12:52 PM
To: chef@lists.opscode.com
Cc: Zulauf, Graham; Kiner, Kari; EMENDOZA@jw.org
Subject: [chef] RE: Re: RE: ohai, number ldap accounts

That is good news. I need to incorporate that in the chef-client cookbook so that when the node is bootstrapped, it doesn't pull the data in after the first run. Anyone, Know how I can incorporate the plugin disable when doing a bootstrap? I do not want to delete the plugin as there may be reason to use it in the future or on a particular node.

I am not sure where to incorporate this...
Ohai::Config[:disabled_plugins] = [ "passwd" ]

Thanks all
Randy

-----Original Message-----
From: Daniel DeLeo [mailto:ddeleo@kallistec.com] On Behalf Of Daniel DeLeo
Sent: Wednesday, March 28, 2012 11:58 AM
To: chef@lists.opscode.com
Cc: GJZULAUF@jw.org; KAKiner@jw.org; EMENDOZA@jw.org
Subject: [chef] Re: RE: ohai, number ldap accounts

On Wednesday, March 28, 2012 at 8:25 AM, IDROSSI@jw.org wrote:

Hi Randy,

We are experiencing this very same issue and were recently looking for a solution. We use centrify to pull in AD accounts for authorization. The nodes are pulling in tons of data for all of these user groups and accounts.

If we're not doing anything with specific user accounts in recipes/templates, should we be safe to just disable the Ohai plugin that pulls in the user accounts from AD?
Chef doesn't use ohai data to lookup user/group data for file/template/directory or execute/script or user/group resources. It uses Ruby's Etc module, which in turn uses the getpwent family of functions.

So it's totally safe to remove the plugin if you don't need the data for other reasons.

--
Dan DeLeo

7

An alternative approach we used is to edit /etc/logins.defs (on debian
based systems anyway) . And lower UID_MAX below where your ldap range
starts. That way you keep the local users in ohai.

On Wed, Mar 28, 2012 at 8:25 AM, IDROSSI@jw.org wrote:

Hi Randy,

We are experiencing this very same issue and were recently looking for a
solution. We use centrify to pull in AD accounts for authorization. The
nodes are pulling in tons of data for all of these user groups and accounts.

If we're not doing anything with specific user accounts in
recipes/templates, should we be safe to just disable the Ohai plugin that
pulls in the user accounts from AD?

Ian D. Rossi
CD - Server Infrastructure
Phone 23834

From: Van Fossan,Randy [vanfossr@oclc.org]
Sent: Wednesday, March 28, 2012 11:10 AM
To: chef@lists.opscode.com
Subject: [chef] ohai, number ldap accounts

Fellow chef’s,

I have a question about ldap accounts that ohai pulls down. We are a
large shop with thousands of servers and we have a large number of ldap
accounts (only a few are currently managed by chef). As per the issue in
OHAI-165 http://tickets.opscode.com/browse/OHAI-165, I believe it may
be affecting performance on our chef server.

One solution proffered is to place Ohai::Config[:disabled_plugins] = [
"passwd" ] in the client config to disable this. However, if we do
this, won’t the ldap accounts be unavailable in resources? Meaning, If I
assign the ownership of a file to an ldap account in a file resource, won’t
that cause a failure. This since chef will not know anything about that
account..

file “/tmp/myfile” do

owner “ldapacct1”

group “ldapgroup1”

mode "0600"

end

http://tickets.opscode.com/browse/OHAI-165

Anyone have any guidance on this issue? I would like to keep all the
ldap info out of ohai, but still be able to use ldap accounts in a resource.

Thanks

Randy

8

On Wed, Mar 28, 2012 at 6:52 PM, Van Fossan,Randy vanfossr@oclc.org wrote:

That is good news. I need to incorporate that in the chef-client cookbook so that when the node is bootstrapped, it doesn't pull the data in after the first run. Anyone, Know how I can incorporate the plugin disable when doing a bootstrap? I do not want to delete the plugin as there may be reason to use it in the future or on a particular node.

I see you're making progress incorporating this into the chef-client
cookbook, that's good.
However both you and Ian were also asking about bootstrap, and I
didn't see a reply about that.

knife bootstrap uses a distro-specific template file that you can
specify with -d DISTRO.
Those files live inside the chef gem, in the lib/chef/knife/bootstrap
directory. You can also look at them on GH [1].

To start customizing them, just copy them to your chef-repo:
${CHEF_REPO}/.chef/bootstrap/

You will find something like this:

(
cat <<'EOP'
<%= config_content %>
EOP
) > /etc/chef/client.rb

Changing that to:

(
cat <<'EOP'
<%= config_content %>

Ohai::Config[:disabled_plugins] = [ "passwd" ]
EOP
) > /etc/chef/client.rb

should be what you need.

Then of course if you use the chef-client, you will still need to make
sure it does the right thing. But this will take care of the first
run.

Hope this helps.
Andrea

1: https://github.com/opscode/chef/tree/master/chef/lib/chef/knife/bootstrap

9

Hi Andrea,

I did get the change into the chef-client cookbook and since my first run (bootstrapped) configures ldap, it currently only picks up the local accounts on the first pass before the disable takes affect. However, I will add that to my custom bootstrapper.

My custom bootstrapper detects the OS and version and installs the ruby and chef-client rpms from a local source for that OS / version.

Thanks again.

-----Original Message-----
From: Andrea Campi [mailto:andrea.campi@zephirworks.com]
Sent: Thursday, March 29, 2012 3:20 AM
To: chef@lists.opscode.com
Cc: GJZULAUF@jw.org; KAKiner@jw.org; EMENDOZA@jw.org
Subject: [chef] Re: RE: Re: RE: ohai, number ldap accounts

On Wed, Mar 28, 2012 at 6:52 PM, Van Fossan,Randy vanfossr@oclc.org wrote:

That is good news. I need to incorporate that in the chef-client cookbook so that when the node is bootstrapped, it doesn't pull the data in after the first run. Anyone, Know how I can incorporate the plugin disable when doing a bootstrap? I do not want to delete the plugin as there may be reason to use it in the future or on a particular node.

I see you're making progress incorporating this into the chef-client cookbook, that's good.
However both you and Ian were also asking about bootstrap, and I didn't see a reply about that.

knife bootstrap uses a distro-specific template file that you can specify with -d DISTRO.
Those files live inside the chef gem, in the lib/chef/knife/bootstrap directory. You can also look at them on GH [1].

To start customizing them, just copy them to your chef-repo:
${CHEF_REPO}/.chef/bootstrap/

You will find something like this:

(
cat <<'EOP'
<%= config_content %>
EOP
) > /etc/chef/client.rb

Changing that to:

(
cat <<'EOP'
<%= config_content %>

Ohai::Config[:disabled_plugins] = [ "passwd" ] EOP
) > /etc/chef/client.rb

should be what you need.

Then of course if you use the chef-client, you will still need to make sure it does the right thing. But this will take care of the first run.

Hope this helps.
Andrea

1: https://github.com/opscode/chef/tree/master/chef/lib/chef/knife/bootstrap

10

Hi Andrea,

I tried including that in the ubuntu10.04-apt.erb bootstrap file and it didn't seem to make the change.

Also,

(
cat <<'EOP'
<%= config_content %>
EOP
) > /etc/chef/client.rb

wasn't there. That is only in ubuntu10.04-gems.erb.

Would this work?

echo 'Ohai::Config[:disabled_plugins] = ["passwd", "dmi", "dmi_common"]' >> /etc/chef/client.rb

Ian D. Rossi
CD - Server Infrastructure
Phone 23834

From: Andrea Campi [andrea.campi@zephirworks.com]
Sent: Thursday, March 29, 2012 3:19 AM
To: chef@lists.opscode.com
Cc: Zulauf, Graham; Kiner, Kari; EMENDOZA@jw.org
Subject: [chef] Re: RE: Re: RE: ohai, number ldap accounts

On Wed, Mar 28, 2012 at 6:52 PM, Van Fossan,Randy vanfossr@oclc.org wrote:

That is good news. I need to incorporate that in the chef-client cookbook so that when the node is bootstrapped, it doesn't pull the data in after the first run. Anyone, Know how I can incorporate the plugin disable when doing a bootstrap? I do not want to delete the plugin as there may be reason to use it in the future or on a particular node.

I see you're making progress incorporating this into the chef-client
cookbook, that's good.
However both you and Ian were also asking about bootstrap, and I
didn't see a reply about that.

knife bootstrap uses a distro-specific template file that you can
specify with -d DISTRO.
Those files live inside the chef gem, in the lib/chef/knife/bootstrap
directory. You can also look at them on GH [1].

To start customizing them, just copy them to your chef-repo:
${CHEF_REPO}/.chef/bootstrap/

You will find something like this:

(
cat <<'EOP'
<%= config_content %>
EOP
) > /etc/chef/client.rb

Changing that to:

(
cat <<'EOP'
<%= config_content %>

Ohai::Config[:disabled_plugins] = [ "passwd" ]
EOP
) > /etc/chef/client.rb

should be what you need.

Then of course if you use the chef-client, you will still need to make
sure it does the right thing. But this will take care of the first
run.

Hope this helps.
Andrea

1: https://github.com/opscode/chef/tree/master/chef/lib/chef/knife/bootstrap

11

Since I have disabled the password plugin in Ohai, my nagios recipe now fails as the owner and group on a remote_directory resource is from ldap.

Back to the drawing board! :frowning:

Randy

-----Original Message-----
From: Andrea Campi [mailto:andrea.campi@zephirworks.com]
Sent: Thursday, March 29, 2012 3:20 AM
To: chef@lists.opscode.com
Cc: GJZULAUF@jw.org; KAKiner@jw.org; EMENDOZA@jw.org
Subject: [chef] Re: RE: Re: RE: ohai, number ldap accounts

On Wed, Mar 28, 2012 at 6:52 PM, Van Fossan,Randy vanfossr@oclc.org wrote:

That is good news. I need to incorporate that in the chef-client cookbook so that when the node is bootstrapped, it doesn't pull the data in after the first run. Anyone, Know how I can incorporate the plugin disable when doing a bootstrap? I do not want to delete the plugin as there may be reason to use it in the future or on a particular node.

I see you're making progress incorporating this into the chef-client cookbook, that's good.
However both you and Ian were also asking about bootstrap, and I didn't see a reply about that.

knife bootstrap uses a distro-specific template file that you can specify with -d DISTRO.
Those files live inside the chef gem, in the lib/chef/knife/bootstrap directory. You can also look at them on GH [1].

To start customizing them, just copy them to your chef-repo:
${CHEF_REPO}/.chef/bootstrap/

You will find something like this:

(
cat <<'EOP'
<%= config_content %>
EOP
) > /etc/chef/client.rb

Changing that to:

(
cat <<'EOP'
<%= config_content %>

Ohai::Config[:disabled_plugins] = [ "passwd" ] EOP
) > /etc/chef/client.rb

should be what you need.

Then of course if you use the chef-client, you will still need to make sure it does the right thing. But this will take care of the first run.

Hope this helps.
Andrea

1: https://github.com/opscode/chef/tree/master/chef/lib/chef/knife/bootstrap

12

Yo,

I would suggest managing your systems into the required state instead
of relying on the state of a centralized user directory, that is to
say: do not store the users required for convergence in LDAP -- unless
you are inserting them into LDAP with Chef, too. Even then I would
suggest redoubling your familiarity with LDAP, libnss-ldap and the
LDAP related PAM modifications.

I'm mostly confident others have managed libnss-ldap at high scale
without problems. I choose a more pragmatic approach these days
(system users & operators via databags), with state clearly defined
within the management bounds of Chef.

With more than 1,000 users I can see databags getting out of hand, and
perhaps another tool (or knife plugin) to be created for managing
those data bag files. It would be trivial to emit data-bag
configurations for users based on a template similar to the 'adduser'
functionality, even from a web service. Even from a web service
talking to LDAP, or a combination of back ends across multiple legacy
platforms.

On 30 March 2012 05:52, Van Fossan,Randy vanfossr@oclc.org wrote:

Since I have disabled the password plugin in Ohai, my nagios recipe now fails as the owner and group on a remote_directory resource is from ldap.

I thought I had read 2+ posts back that the ohai 'passwd' plugin has
nothing to do with remote_directory (or directory, or file),
providers -- they use the Etc ruby module to getent, getgrent,
getpwent, etc. That is my recollection of the code as well. To be
sure, check it yourself.

I hope this helps your drawing board a little.

--AJ

Back to the drawing board! :frowning:

Randy

-----Original Message-----
From: Andrea Campi [mailto:andrea.campi@zephirworks.com]
Sent: Thursday, March 29, 2012 3:20 AM
To: chef@lists.opscode.com
Cc: GJZULAUF@jw.org; KAKiner@jw.org; EMENDOZA@jw.org
Subject: [chef] Re: RE: Re: RE: ohai, number ldap accounts

On Wed, Mar 28, 2012 at 6:52 PM, Van Fossan,Randy vanfossr@oclc.org wrote:

That is good news. I need to incorporate that in the chef-client cookbook so that when the node is bootstrapped, it doesn't pull the data in after the first run. Anyone, Know how I can incorporate the plugin disable when doing a bootstrap? I do not want to delete the plugin as there may be reason to use it in the future or on a particular node.

I see you're making progress incorporating this into the chef-client cookbook, that's good.
However both you and Ian were also asking about bootstrap, and I didn't see a reply about that.

knife bootstrap uses a distro-specific template file that you can specify with -d DISTRO.
Those files live inside the chef gem, in the lib/chef/knife/bootstrap directory. You can also look at them on GH [1].

To start customizing them, just copy them to your chef-repo:
${CHEF_REPO}/.chef/bootstrap/

You will find something like this:

(
cat <<'EOP'
<%= config_content %>
EOP
) > /etc/chef/client.rb

Changing that to:

(
cat <<'EOP'
<%= config_content %>

Ohai::Config[:disabled_plugins] = [ "passwd" ] EOP
) > /etc/chef/client.rb

should be what you need.

Then of course if you use the chef-client, you will still need to make sure it does the right thing. But this will take care of the first run.

Hope this helps.
Andrea

1: https://github.com/opscode/chef/tree/master/chef/lib/chef/knife/bootstrap