I noticed the client looks for cookbooks in /var/chef, which is what I
have the cookbooks path set to on the server. Is it getting this value from
the server, or is that a default value?
It seems that the client must have all the cookbooks locally, it can’t add
new cookbooks from the server that it doesn’t have. Is that right? Seems
like that would be a desirable feature, although I can see how calculating
all the dependencies could be a pain. Or maybe an alternative could be the
client can set a list of cookbooks in the attributes somewhere, and then
download whatever cookbooks are not present at each run? So then say you
want to push out a new recipe to a bunch of clients you just update their
attributes on the server and you are done.
Chef client will cache all cookbooks available from the Chef server
each time.
Templates and Files are lazy loaded.
Regards,
AJ
On 17/04/2009, at 7:18 PM, snacktime wrote:
I noticed the client looks for cookbooks in /var/chef, which is
what I have the cookbooks path set to on the server. Is it getting
this value from the server, or is that a default value?
It seems that the client must have all the cookbooks locally, it
can't add new cookbooks from the server that it doesn't have. Is
that right? Seems like that would be a desirable feature, although
I can see how calculating all the dependencies could be a pain. Or
maybe an alternative could be the client can set a list of cookbooks
in the attributes somewhere, and then download whatever cookbooks
are not present at each run? So then say you want to push out a new
recipe to a bunch of clients you just update their attributes on the
server and you are done.
Ah misconfigured server on my end. No cookbooks_path.
Chris
On Fri, Apr 17, 2009 at 1:02 AM, AJ Christensen aj@junglist.gen.nz wrote:
Chef client will cache all cookbooks available from the Chef server each
time.
Templates and Files are lazy loaded.
Regards,
AJ
On 17/04/2009, at 7:18 PM, snacktime wrote:
I noticed the client looks for cookbooks in /var/chef, which is what I
have the cookbooks path set to on the server. Is it getting this value from
the server, or is that a default value?
It seems that the client must have all the cookbooks locally, it can't
add new cookbooks from the server that it doesn't have. Is that right?
Seems like that would be a desirable feature, although I can see how
calculating all the dependencies could be a pain. Or maybe an alternative
could be the client can set a list of cookbooks in the attributes somewhere,
and then download whatever cookbooks are not present at each run? So then
say you want to push out a new recipe to a bunch of clients you just update
their attributes on the server and you are done.
Chef client will cache all cookbooks available from the Chef server
each time.
Does anyone else see a problem with this?This means any node managed
by a chef server will have access to recipes, files and templates
meant for any other node. If one node is compromised, the atacker will
have access to sensitive data for that node and for any other node
just by looking at the Chef client cache.
I know that the chef server is dumb and all the smarts are on the chef
client by design, but perhaps a smart move would be to give some
recipe processing brains to Chef server. At least to let it make
decisions about what coockbooks to provide to each client based on
"include_recipe" directives.
For now, I'll have to leave my multi-user machines outside of chef
management because of that.
I know that this may not be a big problem for the typical use of Chef
that I'm aware of (Web app server farms), for for my usage scenario
(Computer center of a major University in Portugal, with servers,
multi-user clusters and personal workstations) it is.
This is a major problem, and something we've discussed although no
concrete decisions going forward have been made.
I'm interested in potential solutions for this, but walking recipes
(server) included on a node for include_recipe and only caching those
ones seems like a pretty good solution.
On 17/04/2009, at 11:48 PM, Miguel Cabeça wrote:
Hi,
Chef client will cache all cookbooks available from the Chef server
each time.
Does anyone else see a problem with this?This means any node managed
by a chef server will have access to recipes, files and templates
meant for any other node. If one node is compromised, the atacker
will have access to sensitive data for that node and for any other
node just by looking at the Chef client cache.
I know that the chef server is dumb and all the smarts are on the
chef client by design, but perhaps a smart move would be to give
some recipe processing brains to Chef server. At least to let it
make decisions about what coockbooks to provide to each client based
on "include_recipe" directives.
For now, I'll have to leave my multi-user machines outside of chef
management because of that.
I know that this may not be a big problem for the typical use of
Chef that I'm aware of (Web app server farms), for for my usage
scenario (Computer center of a major University in Portugal, with
servers, multi-user clusters and personal workstations) it is.
I actually have a similar need and was looking at how difficult it would be
to add domains to chef. So a domain could contain it's own cookbooks,
nodes, etc..
Chris
On Fri, Apr 17, 2009 at 5:39 AM, AJ Christensen aj@junglist.gen.nz wrote:
This is a major problem, and something we've discussed although no concrete
decisions going forward have been made.
I'm interested in potential solutions for this, but walking recipes
(server) included on a node for include_recipe and only caching those ones
seems like a pretty good solution.
On 17/04/2009, at 11:48 PM, Miguel Cabeça wrote:
Hi,
Chef client will cache all cookbooks available from the Chef server each
time.
Does anyone else see a problem with this?This means any node managed by a
chef server will have access to recipes, files and templates meant for any
other node. If one node is compromised, the atacker will have access to
sensitive data for that node and for any other node just by looking at the
Chef client cache.
I know that the chef server is dumb and all the smarts are on the chef
client by design, but perhaps a smart move would be to give some recipe
processing brains to Chef server. At least to let it make decisions about
what coockbooks to provide to each client based on "include_recipe"
directives.
For now, I'll have to leave my multi-user machines outside of chef
management because of that.
I know that this may not be a big problem for the typical use of Chef
that I'm aware of (Web app server farms), for for my usage scenario
(Computer center of a major University in Portugal, with servers, multi-user
clusters and personal workstations) it is.
Support for tags already exists, but we don't use it. This is
essentially allowing you to easily tag/untag a given node, from the
ui, an attribute file, or a recipe. Tags might be something like
'dl380', 'ec2', etc.
Roles
Right now, a node is configured based on the list of recipes it should
apply. That will remain the same, but we're going to add another
layer on top, which is a 'role'. In addition to the recipes that
would be applied, it will also contain a list of the cookbooks needed
to fulfill that role, and potentially any attributes that would need
to be set.
A node wind up in a given role in one of two ways: manual placement or
tag matching. Each role would have a set of tags that, if present on
a node, cause the node to automatically receive a given role.
Nodes can have multiple roles.
Infrastructures
Infrastructures are where you can define what roles are available for
a given environment. Think 'production', 'testing', etc. It would
also allow you to define a 'policy', which would be evaluated against
a given node and place it dynamically into roles.
The ways this would change chef:
A) We would be able to compute the exact set of recipes for a given
node, making sure we only ship the minimal set.
B) It would allow for automatic assignment of recipes based on policies.
C) It would eliminate the need for lots of small cookbooks that do
nothing but contain a default recipe full of 'include_recipe' - for
example, no more 'web_server' cookbooks.
I actually have a similar need and was looking at how difficult it would be
to add domains to chef. So a domain could contain it's own cookbooks,
nodes, etc..
Chris
On Fri, Apr 17, 2009 at 5:39 AM, AJ Christensen aj@junglist.gen.nz wrote:
This is a major problem, and something we've discussed although no
concrete decisions going forward have been made.
I'm interested in potential solutions for this, but walking recipes
(server) included on a node for include_recipe and only caching those ones
seems like a pretty good solution.
On 17/04/2009, at 11:48 PM, Miguel Cabeça wrote:
Hi,
Chef client will cache all cookbooks available from the Chef server each
time.
Does anyone else see a problem with this?This means any node managed by a
chef server will have access to recipes, files and templates meant for any
other node. If one node is compromised, the atacker will have access to
sensitive data for that node and for any other node just by looking at the
Chef client cache.
I know that the chef server is dumb and all the smarts are on the chef
client by design, but perhaps a smart move would be to give some recipe
processing brains to Chef server. At least to let it make decisions about
what coockbooks to provide to each client based on "include_recipe"
directives.
For now, I'll have to leave my multi-user machines outside of chef
management because of that.
I know that this may not be a big problem for the typical use of Chef
that I'm aware of (Web app server farms), for for my usage scenario
(Computer center of a major University in Portugal, with servers, multi-user
clusters and personal workstations) it is.
What are your thoughts on that?
Best Regards
Miguel Cabeça
--
Opscode, Inc.
Adam Jacob, CTO
T: (206) 508-4759 E: adam@opscode.com
That sounds amazing to me, especially role configuration. I have yet to need
to use Chef (as my project is not ready to be deployed), but as someone
watching it very closely, I would definitely say that roles are crucial.
david
On Fri, Apr 17, 2009 at 1:01 PM, Adam Jacob adam@opscode.com wrote:
We've got some thoughts on how this might be accomplished, which I
just posted to the wiki at:
Support for tags already exists, but we don't use it. This is
essentially allowing you to easily tag/untag a given node, from the
ui, an attribute file, or a recipe. Tags might be something like
'dl380', 'ec2', etc.
Roles
Right now, a node is configured based on the list of recipes it should
apply. That will remain the same, but we're going to add another
layer on top, which is a 'role'. In addition to the recipes that
would be applied, it will also contain a list of the cookbooks needed
to fulfill that role, and potentially any attributes that would need
to be set.
A node wind up in a given role in one of two ways: manual placement or
tag matching. Each role would have a set of tags that, if present on
a node, cause the node to automatically receive a given role.
Nodes can have multiple roles.
Infrastructures
Infrastructures are where you can define what roles are available for
a given environment. Think 'production', 'testing', etc. It would
also allow you to define a 'policy', which would be evaluated against
a given node and place it dynamically into roles.
The ways this would change chef:
A) We would be able to compute the exact set of recipes for a given
node, making sure we only ship the minimal set.
B) It would allow for automatic assignment of recipes based on policies.
C) It would eliminate the need for lots of small cookbooks that do
nothing but contain a default recipe full of 'include_recipe' - for
example, no more 'web_server' cookbooks.
I actually have a similar need and was looking at how difficult it would
be
to add domains to chef. So a domain could contain it's own cookbooks,
nodes, etc..
Chris
On Fri, Apr 17, 2009 at 5:39 AM, AJ Christensen aj@junglist.gen.nz
wrote:
This is a major problem, and something we've discussed although no
concrete decisions going forward have been made.
I'm interested in potential solutions for this, but walking recipes
(server) included on a node for include_recipe and only caching those
ones
seems like a pretty good solution.
On 17/04/2009, at 11:48 PM, Miguel Cabeça wrote:
Hi,
Chef client will cache all cookbooks available from the Chef server
each
time.
Does anyone else see a problem with this?This means any node managed by
a
chef server will have access to recipes, files and templates meant for
any
other node. If one node is compromised, the atacker will have access to
sensitive data for that node and for any other node just by looking at
the
Chef client cache.
I know that the chef server is dumb and all the smarts are on the chef
client by design, but perhaps a smart move would be to give some recipe
processing brains to Chef server. At least to let it make decisions
about
what coockbooks to provide to each client based on "include_recipe"
directives.
For now, I'll have to leave my multi-user machines outside of chef
management because of that.
I know that this may not be a big problem for the typical use of Chef
that I'm aware of (Web app server farms), for for my usage scenario
(Computer center of a major University in Portugal, with servers,
multi-user
clusters and personal workstations) it is.
What are your thoughts on that?
Best Regards
Miguel Cabeça
--
Opscode, Inc.
Adam Jacob, CTO
T: (206) 508-4759 E: adam@opscode.com
Support for tags already exists, but we don't use it. This is
essentially allowing you to easily tag/untag a given node, from the
ui, an attribute file, or a recipe. Tags might be something like
'dl380', 'ec2', etc.
Roles
Right now, a node is configured based on the list of recipes it should
apply. That will remain the same, but we're going to add another
layer on top, which is a 'role'. In addition to the recipes that
would be applied, it will also contain a list of the cookbooks needed
to fulfill that role, and potentially any attributes that would need
to be set.
A node wind up in a given role in one of two ways: manual placement or
tag matching. Each role would have a set of tags that, if present on
a node, cause the node to automatically receive a given role.
Nodes can have multiple roles.
Infrastructures
Infrastructures are where you can define what roles are available for
a given environment. Think 'production', 'testing', etc. It would
also allow you to define a 'policy', which would be evaluated against
a given node and place it dynamically into roles.
The ways this would change chef:
A) We would be able to compute the exact set of recipes for a given
node, making sure we only ship the minimal set.
B) It would allow for automatic assignment of recipes based on policies.
C) It would eliminate the need for lots of small cookbooks that do
nothing but contain a default recipe full of 'include_recipe' - for
example, no more 'web_server' cookbooks.
I actually have a similar need and was looking at how difficult it would be
to add domains to chef. So a domain could contain it's own cookbooks,
nodes, etc..
Chris
On Fri, Apr 17, 2009 at 5:39 AM, AJ Christensen aj@junglist.gen.nz wrote:
This is a major problem, and something we've discussed although no
concrete decisions going forward have been made.
I'm interested in potential solutions for this, but walking recipes
(server) included on a node for include_recipe and only caching those ones
seems like a pretty good solution.
On 17/04/2009, at 11:48 PM, Miguel Cabeça wrote:
Hi,
Chef client will cache all cookbooks available from the Chef server each
time.
Does anyone else see a problem with this?This means any node managed by a
chef server will have access to recipes, files and templates meant for any
other node. If one node is compromised, the atacker will have access to
sensitive data for that node and for any other node just by looking at the
Chef client cache.
I know that the chef server is dumb and all the smarts are on the chef
client by design, but perhaps a smart move would be to give some recipe
processing brains to Chef server. At least to let it make decisions about
what coockbooks to provide to each client based on "include_recipe"
directives.
For now, I'll have to leave my multi-user machines outside of chef
management because of that.
I know that this may not be a big problem for the typical use of Chef
that I'm aware of (Web app server farms), for for my usage scenario
(Computer center of a major University in Portugal, with servers, multi-user
clusters and personal workstations) it is.
Support for tags already exists, but we don't use it. This is
essentially allowing you to easily tag/untag a given node, from the
ui, an attribute file, or a recipe. Tags might be something like
'dl380', 'ec2', etc.
Roles
Right now, a node is configured based on the list of recipes it should
apply. That will remain the same, but we're going to add another
layer on top, which is a 'role'. In addition to the recipes that
would be applied, it will also contain a list of the cookbooks needed
to fulfill that role, and potentially any attributes that would need
to be set.
A node wind up in a given role in one of two ways: manual placement or
tag matching. Each role would have a set of tags that, if present on
a node, cause the node to automatically receive a given role.
Nodes can have multiple roles.
Infrastructures
Infrastructures are where you can define what roles are available for
a given environment. Think 'production', 'testing', etc. It would
also allow you to define a 'policy', which would be evaluated against
a given node and place it dynamically into roles.
The ways this would change chef:
A) We would be able to compute the exact set of recipes for a given
node, making sure we only ship the minimal set.
B) It would allow for automatic assignment of recipes based on policies.
C) It would eliminate the need for lots of small cookbooks that do
nothing but contain a default recipe full of 'include_recipe' - for
example, no more 'web_server' cookbooks.
I actually have a similar need and was looking at how difficult it would
be
to add domains to chef. So a domain could contain it's own cookbooks,
nodes, etc..
Chris
On Fri, Apr 17, 2009 at 5:39 AM, AJ Christensen aj@junglist.gen.nz
wrote:
This is a major problem, and something we've discussed although no
concrete decisions going forward have been made.
I'm interested in potential solutions for this, but walking recipes
(server) included on a node for include_recipe and only caching those
ones
seems like a pretty good solution.
On 17/04/2009, at 11:48 PM, Miguel Cabeça wrote:
Hi,
Chef client will cache all cookbooks available from the Chef server
each
time.
Does anyone else see a problem with this?This means any node managed by
a
chef server will have access to recipes, files and templates meant for
any
other node. If one node is compromised, the atacker will have access to
sensitive data for that node and for any other node just by looking at
the
Chef client cache.
I know that the chef server is dumb and all the smarts are on the chef
client by design, but perhaps a smart move would be to give some recipe
processing brains to Chef server. At least to let it make decisions
about
what coockbooks to provide to each client based on "include_recipe"
directives.
For now, I'll have to leave my multi-user machines outside of chef
management because of that.
I know that this may not be a big problem for the typical use of Chef
that I'm aware of (Web app server farms), for for my usage scenario
(Computer center of a major University in Portugal, with servers,
multi-user
clusters and personal workstations) it is.
What are your thoughts on that?
Best Regards
Miguel Cabeça
--
Opscode, Inc.
Adam Jacob, CTO
T: (206) 508-4759 E: adam@opscode.com
Support for tags already exists, but we don't use it. This is
essentially allowing you to easily tag/untag a given node, from the
ui, an attribute file, or a recipe. Tags might be something like
'dl380', 'ec2', etc.
Roles
Right now, a node is configured based on the list of recipes it should
apply. That will remain the same, but we're going to add another
layer on top, which is a 'role'. In addition to the recipes that
would be applied, it will also contain a list of the cookbooks needed
to fulfill that role, and potentially any attributes that would need
to be set.
A node wind up in a given role in one of two ways: manual placement or
tag matching. Each role would have a set of tags that, if present on
a node, cause the node to automatically receive a given role.
Nodes can have multiple roles.
Infrastructures
Infrastructures are where you can define what roles are available for
a given environment. Think 'production', 'testing', etc. It would
also allow you to define a 'policy', which would be evaluated against
a given node and place it dynamically into roles.
The ways this would change chef:
A) We would be able to compute the exact set of recipes for a given
node, making sure we only ship the minimal set.
B) It would allow for automatic assignment of recipes based on policies.
C) It would eliminate the need for lots of small cookbooks that do
nothing but contain a default recipe full of 'include_recipe' - for
example, no more 'web_server' cookbooks.
I actually have a similar need and was looking at how difficult it would
be
to add domains to chef. So a domain could contain it's own cookbooks,
nodes, etc..
Chris
On Fri, Apr 17, 2009 at 5:39 AM, AJ Christensen aj@junglist.gen.nz
wrote:
This is a major problem, and something we've discussed although no
concrete decisions going forward have been made.
I'm interested in potential solutions for this, but walking recipes
(server) included on a node for include_recipe and only caching those
ones
seems like a pretty good solution.
On 17/04/2009, at 11:48 PM, Miguel Cabeça wrote:
Hi,
Chef client will cache all cookbooks available from the Chef server
each
time.
Does anyone else see a problem with this?This means any node managed by
a
chef server will have access to recipes, files and templates meant for
any
other node. If one node is compromised, the atacker will have access to
sensitive data for that node and for any other node just by looking at
the
Chef client cache.
I know that the chef server is dumb and all the smarts are on the chef
client by design, but perhaps a smart move would be to give some recipe
processing brains to Chef server. At least to let it make decisions
about
what coockbooks to provide to each client based on "include_recipe"
directives.
For now, I'll have to leave my multi-user machines outside of chef
management because of that.
I know that this may not be a big problem for the typical use of Chef
that I'm aware of (Web app server farms), for for my usage scenario
(Computer center of a major University in Portugal, with servers,
multi-user
clusters and personal workstations) it is.
B) It would allow for automatic assignment of recipes based on
policies.
I think there are four new concepts in this description, the fourth
being the policy that was mentioned on B). As I told Adam on Chef Infra (archive),
there are four new concepts to add to the existing Nodes, Cookbooks,
Recipes, Definitions, Attributes, Libraries, Files, Templates,
Resources, Providers and Search Indexes. It will steep even more the
chef learning curve.
I think this is a bit of over-engeneering, at least to solve the
original all-cookbooks-to-all-clients security problem I've mentioned.
And it may not solve the problem completely. If the chef server unit
of access control is the cookbook, it will be impossible to have
sensitive data in one cookbook that has recipes for multiple types of
servers. For example I have a postfix cookbook with
postfix::sattelite, postfix::relay and postfix::store recipes, the
postfix::relay serves remote files with sensitive data that the
postfix::sattelite should not have acess to.
I have some (maybe uninformed) opinions on the new concepts introduced:
Tags : I don't see the diference of this and an attribute of the
node named 'tags' that could be set automaticaly or manually in a
recipe.
Roles: I've implemented this already as a toplevel cookbook with
several recipes in it. Each node has one or more of this recipes like
for example role::mailstore, or role::ldapmaster or role::kerberoskdc.
It's not dynamic, but is there a need for this to be dynamic?
I didn't understand the use for this concept. Doesn't this overlapp
with roles? A 'testing' infrastructure should have nodes with
'testingX' roles
I didn't understand this policy concept either.
I'm afraid that the complexities involved in implementing all this
features will postpone the resolution of the all-cookbooks-to-all-
clients problem.
I was thinking of an approach based on scanning all recipes on chef
server for include_recipes, remote_file and remote_directory
resources. That would build an ACL that would allow only authorized
nodes to acess the resources.
As a side note, there's no ORM for chef is there? IE, rather than being
able to use something like ActiveRecord, the complexity Miguel is
flagging must be dealt with directly via ferret searches(?)
I'd agree with Miguel that his approach makes sense to solve the
security problem he raised; tags / roles / etc seems like a separate set
of enhancements best considered afterwards, in the interests of
improving security as quickly as possible.
just my 2c
David
Miguel Cabeça wrote:
Hi,
On 2009/04/17, at 21:01, Adam Jacob wrote:
We've got some thoughts on how this might be accomplished, which I
just posted to the wiki at:
B) It would allow for automatic assignment of recipes based on policies.
I think there are four new concepts in this description, the fourth
being the policy that was mentioned on B). As I told Adam on Chef Infra (archive),
there are four new concepts to add to the existing Nodes, Cookbooks,
Recipes, Definitions, Attributes, Libraries, Files, Templates,
Resources, Providers and Search Indexes. It will steep even more the
chef learning curve.
I think this is a bit of over-engeneering, at least to solve the
original all-cookbooks-to-all-clients security problem I've mentioned.
And it may not solve the problem completely. If the chef server unit of
access control is the cookbook, it will be impossible to have sensitive
data in one cookbook that has recipes for multiple types of servers. For
example I have a postfix cookbook with postfix::sattelite,
postfix::relay and postfix::store recipes, the postfix::relay serves
remote files with sensitive data that the postfix::sattelite should not
have acess to.
I have some (maybe uninformed) opinions on the new concepts introduced:
Tags : I don't see the diference of this and an attribute of the node
named 'tags' that could be set automaticaly or manually in a recipe.
Roles: I've implemented this already as a toplevel cookbook with
several recipes in it. Each node has one or more of this recipes like
for example role::mailstore, or role::ldapmaster or role::kerberoskdc.
It's not dynamic, but is there a need for this to be dynamic?
I didn't understand the use for this concept. Doesn't this overlapp
with roles? A 'testing' infrastructure should have nodes with 'testingX'
roles
I didn't understand this policy concept either.
I'm afraid that the complexities involved in implementing all this
features will postpone the resolution of the
all-cookbooks-to-all-clients problem.
I was thinking of an approach based on scanning all recipes on chef
server for include_recipes, remote_file and remote_directory resources.
That would build an ACL that would allow only authorized nodes to acess
the resources.
Best Regards
Miguel Cabeça
--
David Lee
Application Development Coordinator
Kanji Group Pty Ltd
02 8272 9483
david.lee@kanji.com.au
This message and any attachment are confidential and may be privileged
or otherwise protected from disclosure. If you are not the intended
recipient, please telephone or email the sender and delete this message
and any attachment from your system. If you are not the intended
recipient you must not copy this message or attachment or disclose the
contents to any other person.
This email message does not constitute legal, financial or any other
kind of advice and reliance must not be placed on its contents. Any
advice will be prefixed with a notice to that effect - and unless such a
notice is affixed all liability for the contents of this email is
disclaimed. The integrity of this email, its contents or any attachments
is not certified in any way by the sender.
Liability limited by a scheme approved under Professional Standards
Legislation
I think an rbac system would probably solve most of the problems and be more
flexible.
Something like this.
Roles can be assigned to cookbooks and recipes.
Clients are added to roles at the cookbook or recipe level.
Permissions are added to roles
Roles can be inherited, so if you have a role on a cookbook, you
automatically have that role on all it's recipes.
Permissions might not be necessary, but they are a simple add on and allow a
lot more customization at little expense.
I wrote a rails plugin that does exactly the above a while back. Would
probably take a couple of nights hacking to get it working with couchdb
given that I've never written a couchdb view and the main sql query is
moderately complex.
As a side note, there's no ORM for chef is there? IE, rather than being
able to use something like ActiveRecord, the complexity Miguel is flagging
must be dealt with directly via ferret searches(?)
I'd agree with Miguel that his approach makes sense to solve the security
problem he raised; tags / roles / etc seems like a separate set of
enhancements best considered afterwards, in the interests of improving
security as quickly as possible.
just my 2c
David
Miguel Cabeça wrote:
Hi,
On 2009/04/17, at 21:01, Adam Jacob wrote:
We've got some thoughts on how this might be accomplished, which I
B) It would allow for automatic assignment of recipes based on policies.
I think there are four new concepts in this description, the fourth being
the policy that was mentioned on B). As I told Adam on Chef Infra (archive), there are four
new concepts to add to the existing Nodes, Cookbooks, Recipes, Definitions,
Attributes, Libraries, Files, Templates, Resources, Providers and Search
Indexes. It will steep even more the chef learning curve.
I think this is a bit of over-engeneering, at least to solve the original
all-cookbooks-to-all-clients security problem I've mentioned. And it may not
solve the problem completely. If the chef server unit of access control is
the cookbook, it will be impossible to have sensitive data in one cookbook
that has recipes for multiple types of servers. For example I have a postfix
cookbook with postfix::sattelite, postfix::relay and postfix::store recipes,
the postfix::relay serves remote files with sensitive data that the
postfix::sattelite should not have acess to.
I have some (maybe uninformed) opinions on the new concepts introduced:
Tags : I don't see the diference of this and an attribute of the node
named 'tags' that could be set automaticaly or manually in a recipe.
Roles: I've implemented this already as a toplevel cookbook with
several recipes in it. Each node has one or more of this recipes like for
example role::mailstore, or role::ldapmaster or role::kerberoskdc. It's not
dynamic, but is there a need for this to be dynamic?
I didn't understand the use for this concept. Doesn't this overlapp
with roles? A 'testing' infrastructure should have nodes with 'testingX'
roles
I didn't understand this policy concept either.
I'm afraid that the complexities involved in implementing all this
features will postpone the resolution of the all-cookbooks-to-all-clients
problem.
I was thinking of an approach based on scanning all recipes on chef server
for include_recipes, remote_file and remote_directory resources. That would
build an ACL that would allow only authorized nodes to acess the resources.
Best Regards
Miguel Cabeça
--
David Lee
Application Development Coordinator
Kanji Group Pty Ltd
02 8272 9483
david.lee@kanji.com.au
This message and any attachment are confidential and may be privileged or
otherwise protected from disclosure. If you are not the intended recipient,
please telephone or email the sender and delete this message and any
attachment from your system. If you are not the intended recipient you must
not copy this message or attachment or disclose the contents to any other
person.
This email message does not constitute legal, financial or any other kind
of advice and reliance must not be placed on its contents. Any advice will
be prefixed with a notice to that effect - and unless such a notice is
affixed all liability for the contents of this email is disclaimed. The
integrity of this email, its contents or any attachments is not certified in
any way by the sender.
Liability limited by a scheme approved under Professional Standards
Legislation
First off, this email is great, Miguel. Thanks for putting so much
thought in to how to make Chef better.
It's 12:24am PST right now, but I wanted to get something out before I
go to bed. I'll follow up with more detail in the morning.
On Wed, Apr 22, 2009 at 5:57 AM, Miguel Cabeça cabeca@ist.utl.pt wrote:
I think there are four new concepts in this description, the fourth being
the policy that was mentioned on B). As I told Adam on Chef Infra (archive), there are four
new concepts to add to the existing Nodes, Cookbooks, Recipes, Definitions,
Attributes, Libraries, Files, Templates, Resources, Providers and Search
Indexes. It will steep even more the chef learning curve.
That may be true. I think there are two parts to Chef having a high
learning curve right now: one is that it is early enough in
development to be in flux, the other is that we still need to write
documentation that guides users through the process clearly. Roles
and Infrastructures are both things that people already essentially do
with Chef today - we just don't call them that.
I think this is a bit of over-engeneering, at least to solve the original
all-cookbooks-to-all-clients security problem I've mentioned. And it may not
solve the problem completely. If the chef server unit of access control is
the cookbook, it will be impossible to have sensitive data in one cookbook
that has recipes for multiple types of servers. For example I have a postfix
cookbook with postfix::sattelite, postfix::relay and postfix::store recipes,
the postfix::relay serves remote files with sensitive data that the
postfix::sattelite should not have acess to.
In the case outlined for Roles and Infrastructures, you should only be
served the components of the cookbook that are required for your Role.
Essentially, a postfix::relay machine would get the postfix::relay
recipe, but none of the others in that cookbook. Files and Templates
both get transferred when they are used, not up-front, so any
sensitive data you store there would only live on the Chef Server and
the nodes themselves.
You bring up a good point here (and again later) regarding the fact
that any chef client can request any file or template, which isn't
resolved by this mechanism, and I think may lead to a change in the
cookbook file structure. I'll talk more about this tomorrow.
I have some (maybe uninformed) opinions on the new concepts introduced:
Tags : I don't see the diference of this and an attribute of the node
named 'tags' that could be set automaticaly or manually in a recipe.
That is exactly what the implementation would be. (In fact, it's
already their - we just aren't using it.)
Roles: I've implemented this already as a toplevel cookbook with several
recipes in it. Each node has one or more of this recipes like for example
role::mailstore, or role::ldapmaster or role::kerberoskdc. It's not dynamic,
but is there a need for this to be dynamic?
There are a couple reasons why roles are handy: they could be defined
easily without any need to write code (processing the ruby recipes on
the server is less than ideal, given the fact that it's just ruby -
who knows what the code might do, especially if it came from a third
party repository,) they let us group cookbooks together for
distribution, and they provide for dynamic assignment.
The dynamic assignment use-case is useful, but I agree it's not
super-high on the list of things that need to get done. As one
example, you might have a specific set of configuration based on
whether the system is an HP DL380, running Ubuntu 9.10 and a member of
a Hadoop cluster. All of that could be accomplished with a cookbook
and some fancy footwork, but now we're really starting to overload the
things you can/should use recipes for - and we would require that to
happen at the edge, which means we need to ship the ruby there before
we execute it.
I didn't understand the use for this concept. Doesn't this overlapp with
roles? A 'testing' infrastructure should have nodes with 'testingX' roles
You might have different sets of roles, or even different sets of
cookbooks, based on the infrastructure.
I didn't understand this policy concept either.
I think we didn't do a very good job of explaining it! I'll
expound on this tomorrow.
I'm afraid that the complexities involved in implementing all this features
will postpone the resolution of the all-cookbooks-to-all-clients problem.
I'm definitely interested in getting it fixed sooner rather than
later. I think we all agree on that.
I was thinking of an approach based on scanning all recipes on chef server
for include_recipes, remote_file and remote_directory resources. That would
build an ACL that would allow only authorized nodes to acess the resources.
Scanning the recipes on the server is something I'm trying really hard
to avoid. The prime reason is a strong desire to not be executing
arbitrary code on the system responsible for distributing the
configuration information to the edges.
More to come, but feel free to follow up before (my) morning.
Regards,
Adam
--
Opscode, Inc.
Adam Jacob, CTO
T: (206) 508-4759 E: adam@opscode.com
As a side note, there's no ORM for chef is there? IE, rather than being able
to use something like ActiveRecord, the complexity Miguel is flagging must
be dealt with directly via ferret searches(?)
I'm not sure I understand how the search indexes help solve Miguel's problem?
I'd agree with Miguel that his approach makes sense to solve the security
problem he raised; tags / roles / etc seems like a separate set of
enhancements best considered afterwards, in the interests of improving
security as quickly as possible.
I think he's totally on the right track for what the sort of controls
need to be - I'm nervous about how it gets implemented, particularly
if it's some sort of server-side processing of cookbook contents.
Adam
--
Opscode, Inc.
Adam Jacob, CTO
T: (206) 508-4759 E: adam@opscode.com
Scanning the recipes on the server is something I'm trying really hard
to avoid. The prime reason is a strong desire to not be executing
arbitrary code on the system responsible for distributing the
configuration information to the edges.
With the approach Miguel outlined, I would imagine you would be reading
the files as text (not interpreting them as templates ) and simply
matching any occurences of include_recipe.
The parser would need to be intelligent enough to deal with recursion
sanely, but it should still be something you'd accomplish in an evening
or so.
Aside: apart from the security advantages, delivering a subset of
recipes would also prevent some quite unexpected behaviour which is
possible due to the fact attributes are executed for ALL recipes, not
just ones that the node is supposed to execute: you would no longer need
to add guards to attribute files to prevent this.
Additional intelligence could be added to the parser later to take into
account roles / etc.
cheers,
David
--
David Lee
Application Development Coordinator
Kanji Group Pty Ltd
02 8272 9483
david.lee@kanji.com.au
This message and any attachment are confidential and may be privileged
or otherwise protected from disclosure. If you are not the intended
recipient, please telephone or email the sender and delete this message
and any attachment from your system. If you are not the intended
recipient you must not copy this message or attachment or disclose the
contents to any other person.
This email message does not constitute legal, financial or any other
kind of advice and reliance must not be placed on its contents. Any
advice will be prefixed with a notice to that effect - and unless such a
notice is affixed all liability for the contents of this email is
disclaimed. The integrity of this email, its contents or any attachments
is not certified in any way by the sender.
Liability limited by a scheme approved under Professional Standards
Legislation
As a side note, there's no ORM for chef is there? IE, rather than being able
to use something like ActiveRecord, the complexity Miguel is flagging must
be dealt with directly via ferret searches(?)
I'm not sure I understand how the search indexes help solve Miguel's problem?
I was just curious about how difficult the implementation is likely to
be. An ActiveRecord, with polymorphic joins etc, would likely require
less work than one using native database interaction.
But I gather it would have to be built directly on top of couch; I have
little sense of the difficulty involved at this point, and I'm curious.
cheers,
David Lee
David Lee
Application Development Coordinator
Kanji Group Pty Ltd
02 8272 9483
david.lee@kanji.com.au
This message and any attachment are confidential and may be privileged
or otherwise protected from disclosure. If you are not the intended
recipient, please telephone or email the sender and delete this message
and any attachment from your system. If you are not the intended
recipient you must not copy this message or attachment or disclose the
contents to any other person.
This email message does not constitute legal, financial or any other
kind of advice and reliance must not be placed on its contents. Any
advice will be prefixed with a notice to that effect - and unless such a
notice is affixed all liability for the contents of this email is
disclaimed. The integrity of this email, its contents or any attachments
is not certified in any way by the sender.
Liability limited by a scheme approved under Professional Standards
Legislation
As a side note, there's no ORM for chef is there? IE, rather than
being able
to use something like ActiveRecord, the complexity Miguel is
flagging must
be dealt with directly via ferret searches(?)
I'm not sure I understand how the search indexes help solve
Miguel's problem?
I was just curious about how difficult the implementation is likely
to be. An ActiveRecord, with polymorphic joins etc, would likely
require less work than one using native database interaction.
Less work than simple RESTful requests?
But I gather it would have to be built directly on top of couch; I
have little sense of the difficulty involved at this point, and I'm
curious.
We've already built a layer on top of couch, chef::couchdb - I too am
not sure what you're getting at here?
I have done a little work replacing chef::rest and chef::couchdb with
Heroku's Rest Client[1] and jchris' CouchRest[2], and the CouchRest
extended document class is quite ORMy, creates a few default views
(Obj.all, ..) and has the validation system from DataMapper ported
across to it.
I believe Adam was working on an Opscode wrapper[3] for Rest Client
that would give it some capabilities we require for communications wit
the chef-server.
I'm not sure I understand how the search indexes help solve Miguel's
problem?
I was just curious about how difficult the implementation is likely to be.
An ActiveRecord, with polymorphic joins etc, would likely require less work
than one using native database interaction.
But I gather it would have to be built directly on top of couch; I have
little sense of the difficulty involved at this point, and I'm curious.
CouchDB is particularly ill-suited to ad-hoc queries of that sort -
while you would be easily able to pull out single objects, you really
don't have the ability to string together arbitrary queries in the way
you are thinking. This is a side-effect of being schema-free and
document oriented, and it's why something like a full text index for
the CouchDB documents is necessary.
We're chatting about ways to make this better in the long term - we
would love to hear your thoughts on the matter.
Adam
--
Opscode, Inc.
Adam Jacob, CTO
T: (206) 508-4759 E: adam@opscode.com
