Bring up the discussion for knife-ec2 --node-name-prefix option

[AJ_Christensen]

Tensibai, others;

I'd like to apologize for the late night post-rant final reply here.

It was not my intention to come across as elitist, and I hope you and the
community can forgive me. I stepped over the line. Trying to speak from
experience without relaying the full context often causes this problem for
me. I got annoyed and banged out a dumb reply.

As Chris and Dan have pointed out, there are a few issues at hand here and
I am sure that Chef will continue to be flexible in this matter. I
personally prescribe to CB's methodology re: not spending time building
naming schemes.

If there is one final commentary I can make: I'd highly advise against
reflecting on node names as meta data for any kind of recipe flow. I would
however recommend ensuring you have appropriate systems that allow for
consistent machine identification and address resolution, and generally
speaking; I think the node name is an appropriate source of data for that
case although any similar data would do.

Deepest apologies,

AJ
On Oct 23, 2012 1:39 AM, "Tensibai" tensibai@iabis.net wrote:

**

Well, I'm not using ec2 but reading this thread at this point makes 2 or 3
points in my mind:

1. I'm promoting chef, at this stage I really can't tell others to stop
   using manually defined machine names without shooting a bullet in chef
   future.

2. We do use others tools (really ? yes) wich are not so easy to connect
   to chef, basing a decision on system name about OS/type could be easiest.

So well, it could be a "best practice" to have not meaningfull node names,
but that could be helpfull too in some cases.

I see not point avoiding this kind of use for mid-way teams. But I acn see
the point having this feature to help mid-way teams migrate from one step
to the next...

So, sorry but this time some 'great' people from this list sounds elitist
on how you may/should use chef ...

Cheers

Le 2012-10-22 08:45, AJ Christensen a écrit :

I'm going to go ahead and say yeah, I am Iron Man, and we are so done in
this thread.

Ship the code. Resolve the ticket. Let the community use it if they want.
You can give machines friendly human snowflake pet names if you like. We
will not.

--AJ
On Oct 22, 2012 3:56 PM, "Sascha Bates" sascha.bates@gmail.com wrote:

With Jenkins, there are tools to copy workspaces from one job to another, to copy workspace data from one system to another, to archive artifacts, etc.... You should be making use of those kinds of tools so that you never need to go out to a specific slave to find out what went wrong with a given job.

And if we lived in a perfect world, I'm sure we'd never need to ssh anywhere and could just sit in the middle of an interactive matrix like Tony Stark. Regardless, this is an active development environment and we are in and out of Jenkins slaves and development team VMs, and just generally being the diagnostic fixit guys. The information doesn't always come to you. Sometimes you have to go to the information. And while I get a lot of information from the Jenkins GUI, sometimes there's no substitute for going out and looking at something.

Also, the decision was made (before I came along) to have all the dev teams use their own chef servers, so there is no unified environment in dev. Don't get me started on the discrepancies between this arena and other environments. I'm hoping to move toward homogenization. But we have hundreds of developers, and something like 50 Openstack tenants (all dev teams). The infrastructure started small with a couple of teams and then grew to include a large portion of the enterprise rather quickly.

Or possibly I'm the only person who doesn't work in a perfect world. Everyone else, back to your regularly scheduled Ironman Matrix.

Sascha Bates | sascha.bates@gmail.com | 612 850 0444 | sascha.bates@skype| sascha_bates@yahoo|
On 10/21/12 7:57 PM, Brad Knowles wrote:

On Oct 21, 2012, at 5:35 PM, Sascha Bates sascha.bates@gmail.com sascha.bates@gmail.com wrote:

How are people working with these servers that have no useful labels? I really want to know. We ssh all over the place in our infrastructure and if I had to go out and look up a special snowflake ec2-blah-blah or IP every time I wanted to get to a server, I'd probably take a baseball bat to my infrastructure.

IMO, if you're manually ssh'ing around in a large infrastructure, you're doing it wrong.

If you've got a small infrastructure, and you can keep all the hostnames in your head at the same time, and you can manage your IP address space with simple /etc/hosts files, then manually ssh'ing around may be required on occasion and is probably okay.

However, if you've got a larger infrastructure, then you should have better tools to get the information you need and put it in a place where you can get to it, without you having to manually ssh around. Those tools might end up having to use ssh to get the information, but that part of the process should be hidden from you.

For example, we run a Jenkins setup with 20 slaves. The slaves are rebuilt often using openstack and chef. I'm currently writing a knife plugin to set up ssh aliases for nodes in a role that make it easy for us to do things like 'ssh slave20' as we are constantly helping people figure out stuff about their jobs on the slaves or diagnosing VM issues (I'd actually really like to take a baseball bat to our Openstack infra).

With Jenkins, there are tools to copy workspaces from one job to another, to copy workspace data from one system to another, to archive artifacts, etc.... You should be making use of those kinds of tools so that you never need to go out to a specific slave to find out what went wrong with a given job.

--
Brad Knowles brad@shub-internet.org brad@shub-internet.org
LinkedIn Profile: http://tinyurl.com/y8kpxu http://tinyurl.com/y8kpxu

[Tensibai]

There's no need to apologize in my case

I just wanted to point
out some use cases where it's not trivial to bind chef and another tool
which have a built-in capability to make decision based on node names
...

For a quick exemple here, our antivirus guess the OS
(linux/windows) based on a node name filter (just to know how it should
connect to it).

Indeed I may try to replace that with an API call, but
it will probably takes me quite long just to guess if I really can and
this is time I won't spend on recipes to do things we have to do
manually for now. That's why I do think we're not ready to forget our
anming scheme and still need to enforce it on new servers to keep
existing running smoothly.

Tensibai

Le 2012-10-22 18:54, AJ
Christensen a écrit :

Tensibai, others;

I'd like to apologize
for the late night post-rant final reply here.

It was not my
intention to come across as elitist, and I hope you and the community
can forgive me. I stepped over the line. Trying to speak from experience
without relaying the full context often causes this problem for me. I
got annoyed and banged out a dumb reply.

As Chris and Dan have
pointed out, there are a few issues at hand here and I am sure that Chef
will continue to be flexible in this matter. I personally prescribe to
CB's methodology re: not spending time building naming schemes.

If
there is one final commentary I can make: I'd highly advise against
reflecting on node names as meta data for any kind of recipe flow. I
would however recommend ensuring you have appropriate systems that allow
for consistent machine identification and address resolution, and
generally speaking; I think the node name is an appropriate source of
data for that case although any similar data would do.

Deepest
apologies,

AJ
On Oct 23, 2012 1:39 AM, "Tensibai"
tensibai@iabis.net wrote:

Well, I'm not using ec2 but reading
this thread at this point makes 2 or 3 points in my mind:

1. I'm
   promoting chef, at this stage I really can't tell others to stop using
   manually defined machine names without shooting a bullet in chef future.

2. We do use others tools (really ? yes) wich are not so easy to
   connect to chef, basing a decision on system name about OS/type could be
   easiest.

So well, it could be a "best practice" to have not
meaningfull node names, but that could be helpfull too in some cases.

I see not point avoiding this kind of use for mid-way teams. But
I acn see the point having this feature to help mid-way teams migrate
from one step to the next...

So, sorry but this time some
'great' people from this list sounds elitist on how you may/should use
chef ...

Cheers

Le 2012-10-22 08:45, AJ Christensen a
écrit :

I'm going to go ahead and say yeah, I am Iron Man, and
we are so done in this thread.

Ship the code. Resolve the
ticket. Let the community use it if they want. You can give machines
friendly human snowflake pet names if you like. We will not.

--AJ

On Oct 22, 2012 3:56 PM, "Sascha Bates"
sascha.bates@gmail.com wrote:

With Jenkins, there are tools
to copy workspaces from one job to another, to copy workspace data from
one system to another, to archive artifacts, etc.... You should be
making use of those kinds of tools so that you never need to go out to a
specific slave to find out what went wrong with a given job.

And if we lived in a perfect world, I'm sure we'd never need to ssh
anywhere and could just sit in the middle of an interactive matrix like
Tony Stark. Regardless, this is an active development environment and we
are in and out of Jenkins slaves and development team VMs, and just
generally being the diagnostic fixit guys. The information doesn't
always come to you. Sometimes you have to go to the information. And
while I get a lot of information from the Jenkins GUI, sometimes there's
no substitute for going out and looking at something.

Also,
the decision was made (before I came along) to have all the dev teams
use their own chef servers, so there is no unified environment in dev.
Don't get me started on the discrepancies between this arena and other
environments. I'm hoping to move toward homogenization. But we have
hundreds of developers, and something like 50 Openstack tenants (all dev
teams). The infrastructure started small with a couple of teams and then
grew to include a large portion of the enterprise rather quickly.

Or possibly I'm the only person who doesn't work in a perfect
world. Everyone else, back to your regularly scheduled Ironman
Matrix.

Sascha Bates | sascha.bates@gmail.com | 612 850 0444
| sascha.bates@skype | sascha_bates@yahoo | On 10/21/12 7:57 PM, Brad
Knowles wrote:

On Oct 21, 2012, at 5:35 PM, Sascha Bates
sascha.bates@gmail.com wrote:

How are people working
with these servers that have no useful labels? I really want to know. We
ssh all over the place in our infrastructure and if I had to go out and
look up a special snowflake ec2-blah-blah or IP every time I wanted to
get to a server, I'd probably take a baseball bat to my
infrastructure.

IMO, if you're manually ssh'ing around in a
large infrastructure, you're doing it wrong.

If you've got
a small infrastructure, and you can keep all the hostnames in your head
at the same time, and you can manage your IP address space with simple
/etc/hosts files, then manually ssh'ing around may be required on
occasion and is probably okay.

However, if you've got a
larger infrastructure, then you should have better tools to get the
information you need and put it in a place where you can get to it,
without you having to manually ssh around. Those tools might end up
having to use ssh to get the information, but that part of the process
should be hidden from you.

For example, we run a Jenkins
setup with 20 slaves. The slaves are rebuilt often using openstack and
chef. I'm currently writing a knife plugin to set up ssh aliases for
nodes in a role that make it easy for us to do things like 'ssh slave20'
as we are constantly helping people figure out stuff about their jobs on
the slaves or diagnosing VM issues (I'd actually really like to take a
baseball bat to our Openstack infra).

With Jenkins, there
are tools to copy workspaces from one job to another, to copy workspace
data from one system to another, to archive artifacts, etc.... You
should be making use of those kinds of tools so that you never need to
go out to a specific slave to find out what went wrong with a given
job.

--
Brad Knowles brad@shub-internet.org

LinkedIn Profile: http://tinyurl.com/y8kpxu [1]

Links:

[1]
http://tinyurl.com/y8kpxu

[millisami_r]

Liked the opinion of @Leo

The argument by @Sascha is strong enough.

All those opnions for no need to use names is a little more succint.
Yes, its best to stick with role names, query the chef, or any knife ssh commands.

But all the new chefs doesn't work or have the need to have a cluster/farm of 100s of nodes at a time or they all doesn't work at companies like 37-signals, engine-yard, and all the big players.

If you consider that chef should be used by the big companies that have more than 50s/100s of servers, then I've nothing to say and don't even bother to read below.

But if the new devs/startups wants to give a push to chef to handle their deployment, then they certainly start out with just a couple of servers. (maybe 1 for prod, 1 for stag and 1 for dev)

To scale out to multiple instances, I believe one doesn't start out with several instances, but just with one first.

So, in the course of development that has its production server running at the same time, and you do:

$ knife node list
i-fj3j3j3j
i-2j2k3k34
i-3j3jjk4k

Then say I want to terminate the dev instance and launch a new one, then I'ld do knife node list and now you've to fire up the aws ec2 web console and map with instance is hosting what env of the app.

In contrary one might argue that just use the -N (node name prefix: say production-server or dev-server) then later if I want to do the same, ie. delete the dev instance, then firing up the same knife cmd knife node list

$ knife node list
production-server
dev-server
stage-server

Now if I fire up knife ec2 server delete i-fj3j3j3j -y, it just deletes the ec2 instance on the aws, but the node and client is still in the chef-server sitting around abandoned. To overcome this, I have to do:

knife node delete prod-wtf
knife client delete prod-wtf
knife node delete dev-wtf
knife client delete dev-wtf

Still from the opposite direction, deleting the node and client first, then trying to delete the ec2 instance as well, (to save aws billing or relaunch a new instance to test out the new changes of the cookbooks) one have to still map to ec2 web console or just some information from the node to extract out that i-dkdkdkk id thing, its too boring, tedious and repeating the same process, which I believe becomes cumber-some.

---

@millisami
~ Sachin Sagar Rai
Ruby on Rails Developer
http://tfm.com.np
http://nepalonrails.tumblr.com
Sent with Sparrow (http://www.sparrowmailapp.com/?sig)

On Tuesday, October 23, 2012 at 1:51 PM, Tensibai wrote:

There's no need to apologize in my case
I just wanted to point out some use cases where it's not trivial to bind chef and another tool which have a built-in capability to make decision based on node names ...
For a quick exemple here, our antivirus guess the OS (linux/windows) based on a node name filter (just to know how it should connect to it).
Indeed I may try to replace that with an API call, but it will probably takes me quite long just to guess if I really can and this is time I won't spend on recipes to do things we have to do manually for now. That's why I do think we're not ready to forget our anming scheme and still need to enforce it on new servers to keep existing running smoothly.
Tensibai
Le 2012-10-22 18:54, AJ Christensen a écrit :

Tensibai, others;
I'd like to apologize for the late night post-rant final reply here.
It was not my intention to come across as elitist, and I hope you and the community can forgive me. I stepped over the line. Trying to speak from experience without relaying the full context often causes this problem for me. I got annoyed and banged out a dumb reply.
As Chris and Dan have pointed out, there are a few issues at hand here and I am sure that Chef will continue to be flexible in this matter. I personally prescribe to CB's methodology re: not spending time building naming schemes.
If there is one final commentary I can make: I'd highly advise against reflecting on node names as meta data for any kind of recipe flow. I would however recommend ensuring you have appropriate systems that allow for consistent machine identification and address resolution, and generally speaking; I think the node name is an appropriate source of data for that case although any similar data would do.
Deepest apologies,
AJ
On Oct 23, 2012 1:39 AM, "Tensibai" <tensibai@iabis.net (mailto:tensibai@iabis.net)> wrote:

Well, I'm not using ec2 but reading this thread at this point makes 2 or 3 points in my mind:

1. I'm promoting chef, at this stage I really can't tell others to stop using manually defined machine names without shooting a bullet in chef future.
2. We do use others tools (really ? yes) wich are not so easy to connect to chef, basing a decision on system name about OS/type could be easiest.

So well, it could be a "best practice" to have not meaningfull node names, but that could be helpfull too in some cases.
I see not point avoiding this kind of use for mid-way teams. But I acn see the point having this feature to help mid-way teams migrate from one step to the next...

So, sorry but this time some 'great' people from this list sounds elitist on how you may/should use chef ...

Cheers

Le 2012-10-22 08:45, AJ Christensen a écrit :

I'm going to go ahead and say yeah, I am Iron Man, and we are so done in this thread.
Ship the code. Resolve the ticket. Let the community use it if they want. You can give machines friendly human snowflake pet names if you like. We will not.
--AJ
On Oct 22, 2012 3:56 PM, "Sascha Bates" <sascha.bates@gmail.com (mailto:sascha.bates@gmail.com)> wrote:

With Jenkins, there are tools to copy workspaces from one job to another, to copy workspace data from one system to another, to archive artifacts, etc.... You should be making use of those kinds of tools so that you never need to go out to a specific slave to find out what went wrong with a given job. And if we lived in a perfect world, I'm sure we'd never need to ssh anywhere and could just sit in the middle of an interactive matrix like Tony Stark. Regardless, this is an active development environment and we are in and out of Jenkins slaves and development team VMs, and just generally being the diagnostic fixit guys. The information doesn't always come to you. Sometimes you have to go to the information. And while I get a lot of information from the Jenkins GUI, sometimes there's no substitute for going out and looking at something. Also, the decision was made (before I came along) to have all the dev teams use their own chef servers, so there is no unified environment in dev. Don't get me started on the discrepancies between this arena and other environments. I'm hoping to move toward homogenization. But we have hundreds of developers, and something like 50 Openstack tenants (all dev teams). The infrastructure started small with a couple of teams and then grew to include a large portion of the enterprise rather quickly. Or possibly I'm the only person who doesn't work in a perfect world. Everyone else, back to your regularly scheduled Ironman Matrix.
Sascha Bates | sascha.bates@gmail.com (mailto:sascha.bates@gmail.com) | 612 850 0444 | sascha.bates@skype | sascha_bates@yahoo |
On 10/21/12 7:57 PM, Brad Knowles wrote:

On Oct 21, 2012, at 5:35 PM, Sascha Bates sascha.bates@gmail.com (mailto:sascha.bates@gmail.com) wrote:

How are people working with these servers that have no useful labels? I really want to know. We ssh all over the place in our infrastructure and if I had to go out and look up a special snowflake ec2-blah-blah or IP every time I wanted to get to a server, I'd probably take a baseball bat to my infrastructure.

IMO, if you're manually ssh'ing around in a large infrastructure, you're doing it wrong. If you've got a small infrastructure, and you can keep all the hostnames in your head at the same time, and you can manage your IP address space with simple /etc/hosts files, then manually ssh'ing around may be required on occasion and is probably okay. However, if you've got a larger infrastructure, then you should have better tools to get the information you need and put it in a place where you can get to it, without you having to manually ssh around. Those tools might end up having to use ssh to get the information, but that part of the process should be hidden from you.

For example, we run a Jenkins setup with 20 slaves. The slaves are rebuilt often using openstack and chef. I'm currently writing a knife plugin to set up ssh aliases for nodes in a role that make it easy for us to do things like 'ssh slave20' as we are constantly helping people figure out stuff about their jobs on the slaves or diagnosing VM issues (I'd actually really like to take a baseball bat to our Openstack infra).

With Jenkins, there are tools to copy workspaces from one job to another, to copy workspace data from one system to another, to archive artifacts, etc.... You should be making use of those kinds of tools so that you never need to go out to a specific slave to find out what went wrong with a given job. -- Brad Knowles brad@shub-internet.org (mailto:brad@shub-internet.org) LinkedIn Profile: http://tinyurl.com/y8kpxu (Brad Knowles - Amazon Web Services (AWS) | LinkedIn)

[Mike]

Now if I fire up knife ec2 server delete i-fj3j3j3j -y, it just deletes
the ec2 instance on the aws, but the node and client is still in the
chef-server sitting around abandoned. To overcome this, I have to do:

See the --purge flag, as mentioned here:

GitHub - chef/knife-ec2: Chef knife plug-in for AWS EC2

On Tue, Oct 23, 2012 at 5:58 AM, Sachin Sagar Rai millisami@gmail.com wrote:

Liked the opinion of @Leo

The argument by @Sascha is strong enough.

All those opnions for no need to use names is a little more succint.
Yes, its best to stick with role names, query the chef, or any knife ssh
commands.

But all the new chefs doesn't work or have the need to have a cluster/farm
of 100s of nodes at a time or they all doesn't work at companies like
37-signals, engine-yard, and all the big players.

If you consider that chef should be used by the big companies that have more
than 50s/100s of servers, then I've nothing to say and don't even bother to
read below.

But if the new devs/startups wants to give a push to chef to handle their
deployment, then they certainly start out with just a couple of servers.
(maybe 1 for prod, 1 for stag and 1 for dev)

To scale out to multiple instances, I believe one doesn't start out with
several instances, but just with one first.

So, in the course of development that has its production server running at
the same time, and you do:

$ knife node list
i-fj3j3j3j
i-2j2k3k34
i-3j3jjk4k

Then say I want to terminate the dev instance and launch a new one, then
I'ld do knife node list and now you've to fire up the aws ec2 web console
and map with instance is hosting what env of the app.

In contrary one might argue that just use the -N (node name prefix: say
production-server or dev-server) then later if I want to do the same, ie.
delete the dev instance, then firing up the same knife cmd knife node list

$ knife node list
production-server
dev-server
stage-server

Now if I fire up knife ec2 server delete i-fj3j3j3j -y, it just deletes
the ec2 instance on the aws, but the node and client is still in the
chef-server sitting around abandoned. To overcome this, I have to do:

knife node delete prod-wtf
knife client delete prod-wtf
knife node delete dev-wtf
knife client delete dev-wtf

Still from the opposite direction, deleting the node and client first, then
trying to delete the ec2 instance as well, (to save aws billing or relaunch
a new instance to test out the new changes of the cookbooks) one have to
still map to ec2 web console or just some information from the node to
extract out that i-dkdkdkk id thing, its too boring, tedious and repeating
the same process, which I believe becomes cumber-some.

---

@millisami
~ Sachin Sagar Rai
Ruby on Rails Developer
http://tfm.com.np
http://nepalonrails.tumblr.com
Sent with Sparrow

On Tuesday, October 23, 2012 at 1:51 PM, Tensibai wrote:

There's no need to apologize in my case

I just wanted to point out some use cases where it's not trivial to bind
chef and another tool which have a built-in capability to make decision
based on node names ...

For a quick exemple here, our antivirus guess the OS (linux/windows) based
on a node name filter (just to know how it should connect to it).

Indeed I may try to replace that with an API call, but it will probably
takes me quite long just to guess if I really can and this is time I won't
spend on recipes to do things we have to do manually for now. That's why I
do think we're not ready to forget our anming scheme and still need to
enforce it on new servers to keep existing running smoothly.

Tensibai

Le 2012-10-22 18:54, AJ Christensen a écrit :

Tensibai, others;

I'd like to apologize for the late night post-rant final reply here.

It was not my intention to come across as elitist, and I hope you and the
community can forgive me. I stepped over the line. Trying to speak from
experience without relaying the full context often causes this problem for
me. I got annoyed and banged out a dumb reply.

As Chris and Dan have pointed out, there are a few issues at hand here and I
am sure that Chef will continue to be flexible in this matter. I personally
prescribe to CB's methodology re: not spending time building naming schemes.

If there is one final commentary I can make: I'd highly advise against
reflecting on node names as meta data for any kind of recipe flow. I would
however recommend ensuring you have appropriate systems that allow for
consistent machine identification and address resolution, and generally
speaking; I think the node name is an appropriate source of data for that
case although any similar data would do.

Deepest apologies,

AJ

On Oct 23, 2012 1:39 AM, "Tensibai" tensibai@iabis.net wrote:

Well, I'm not using ec2 but reading this thread at this point makes 2 or 3
points in my mind:

1. I'm promoting chef, at this stage I really can't tell others to stop
   using manually defined machine names without shooting a bullet in chef
   future.

2. We do use others tools (really ? yes) wich are not so easy to connect to
   chef, basing a decision on system name about OS/type could be easiest.

So well, it could be a "best practice" to have not meaningfull node names,
but that could be helpfull too in some cases.

I see not point avoiding this kind of use for mid-way teams. But I acn see
the point having this feature to help mid-way teams migrate from one step to
the next...

So, sorry but this time some 'great' people from this list sounds elitist on
how you may/should use chef ...

Cheers

Le 2012-10-22 08:45, AJ Christensen a écrit :

I'm going to go ahead and say yeah, I am Iron Man, and we are so done in
this thread.

Ship the code. Resolve the ticket. Let the community use it if they want.
You can give machines friendly human snowflake pet names if you like. We
will not.

--AJ

On Oct 22, 2012 3:56 PM, "Sascha Bates" sascha.bates@gmail.com wrote:

With Jenkins, there are tools to copy workspaces from one job to another, to
copy workspace data from one system to another, to archive artifacts,
etc.... You should be making use of those kinds of tools so that you never
need to go out to a specific slave to find out what went wrong with a given
job.

And if we lived in a perfect world, I'm sure we'd never need to ssh anywhere
and could just sit in the middle of an interactive matrix like Tony Stark.
Regardless, this is an active development environment and we are in and out
of Jenkins slaves and development team VMs, and just generally being the
diagnostic fixit guys. The information doesn't always come to you.
Sometimes you have to go to the information. And while I get a lot of
information from the Jenkins GUI, sometimes there's no substitute for going
out and looking at something.

Also, the decision was made (before I came along) to have all the dev teams
use their own chef servers, so there is no unified environment in dev.
Don't get me started on the discrepancies between this arena and other
environments. I'm hoping to move toward homogenization. But we have
hundreds of developers, and something like 50 Openstack tenants (all dev
teams). The infrastructure started small with a couple of teams and then
grew to include a large portion of the enterprise rather quickly.

Or possibly I'm the only person who doesn't work in a perfect world.
Everyone else, back to your regularly scheduled Ironman Matrix.

Sascha Bates | sascha.bates@gmail.com | 612 850 0444 | sascha.bates@skype |
sascha_bates@yahoo |
On 10/21/12 7:57 PM, Brad Knowles wrote:

On Oct 21, 2012, at 5:35 PM, Sascha Bates sascha.bates@gmail.com wrote:

How are people working with these servers that have no useful labels? I
really want to know. We ssh all over the place in our infrastructure and if
I had to go out and look up a special snowflake ec2-blah-blah or IP every
time I wanted to get to a server, I'd probably take a baseball bat to my
infrastructure.

IMO, if you're manually ssh'ing around in a large infrastructure, you're
doing it wrong.

If you've got a small infrastructure, and you can keep all the hostnames in
your head at the same time, and you can manage your IP address space with
simple /etc/hosts files, then manually ssh'ing around may be required on
occasion and is probably okay.

However, if you've got a larger infrastructure, then you should have better
tools to get the information you need and put it in a place where you can
get to it, without you having to manually ssh around. Those tools might end
up having to use ssh to get the information, but that part of the process
should be hidden from you.

For example, we run a Jenkins setup with 20 slaves. The slaves are rebuilt
often using openstack and chef. I'm currently writing a knife plugin to set
up ssh aliases for nodes in a role that make it easy for us to do things
like 'ssh slave20' as we are constantly helping people figure out stuff
about their jobs on the slaves or diagnosing VM issues (I'd actually really
like to take a baseball bat to our Openstack infra).

With Jenkins, there are tools to copy workspaces from one job to another, to
copy workspace data from one system to another, to archive artifacts,
etc.... You should be making use of those kinds of tools so that you never
need to go out to a specific slave to find out what went wrong with a given
job.

--
Brad Knowles brad@shub-internet.org
LinkedIn Profile: http://tinyurl.com/y8kpxu

[Lamont_Granquist_OLD]

I've used fairly lightweight human readable hostnames for a long time
now, at very large scales and while there's issues with that approach,
i've seen silliness at the level of a hundred or so nodes where people
who need to talk about a specific server start abbreviating the random
number digits based on fractions of the hostname and i've become tired
of using search and copy and pasting random numbers around. If serial
numbers is really the wave of the future then tooling needs to improve
to the point where I don't need to see the serial numbers, and right now
that simply isn't the state that we're in, and I'd vastly prefer to be
oncall at 3am in the morning and be able to say "i'm going to reboot
foo07" rather than "i'm going to reboot 8df34c1185d". While its nice to
think that humans being involved in the whole process is dirty and
unclean and a symptom of a problem and we should just automate them all
away, I think its drinking the kool aid way too hard. Human names have
many benefits, in addition to their downsides, and there is simply no
clear winner in this argument.

IMO chef should be perfectly neutral about this issue.

On 10/22/12 7:16 AM, Christopher Brown wrote:

Tensibai (and others who started the thread),
These are good points. As it is with many things in this industry,
one size does not fit all.

I've be responsible for very large infrastructure deployments and have
argued for most of my career against spending time and energy on a
naming scheme. In a large infrastructure, relying on decipherable
hostnames generally means other non-scalable practices follow, exactly
because you are thinking at the single host level.

Nonetheless, we (Chef users and web-scale administrators) are quick to
jump on the "SCALE IS KING" bandwagon and expect those techniques to
be absolute truth. They are not. If your infrastructure is
manageable now, is not expected to grow dramatically, and you are
working with other tools and existing processes, don't let other
people tell you they know your business best. Most of us are on a
path of change and, over time, you may find you adopt these techniques
when it's necessary or reasonable.

Until then, keep asking how Chef can help with what you do now, and
what you want to do later.

-C

Christopher Brown
Chief Technology Officer, Opscode
Twitter: @skeptomai

From: Tensibai <tensibai@iabis.net mailto:tensibai@iabis.net>
Reply-To: "chef@lists.opscode.com mailto:chef@lists.opscode.com"
<chef@lists.opscode.com mailto:chef@lists.opscode.com>
Date: Monday, October 22, 2012 5:38 AM
To: "chef@lists.opscode.com mailto:chef@lists.opscode.com"
<chef@lists.opscode.com mailto:chef@lists.opscode.com>
Subject: [chef] Re: *** PROBABLY SPAM *** Re: Re: Re: Bring up the
discussion for knife-ec2 --node-name-prefix option

Well, I'm not using ec2 but reading this thread at this point makes 2
or 3 points in my mind:

1. I'm promoting chef, at this stage I really can't tell others to
   stop using manually defined machine names without shooting a bullet in
   chef future.

2. We do use others tools (really ? yes) wich are not so easy to
   connect to chef, basing a decision on system name about OS/type could
   be easiest.

So well, it could be a "best practice" to have not meaningfull node
names, but that could be helpfull too in some cases.

I see not point avoiding this kind of use for mid-way teams. But I acn
see the point having this feature to help mid-way teams migrate from
one step to the next...

So, sorry but this time some 'great' people from this list sounds
elitist on how you may/should use chef ...

Cheers

Le 2012-10-22 08:45, AJ Christensen a écrit :

I'm going to go ahead and say yeah, I am Iron Man, and we are so done
in this thread.

Ship the code. Resolve the ticket. Let the community use it if they
want. You can give machines friendly human snowflake pet names if you
like. We will not.

--AJ

On Oct 22, 2012 3:56 PM, "Sascha Bates" <sascha.bates@gmail.com
mailto:sascha.bates@gmail.com> wrote:

/With Jenkins, there are tools to copy workspaces from one job to another, to copy workspace data from one system to another, to archive artifacts, etc....  You should be making use of those kinds of tools so that you never need to go out to a specific slave to find out what went wrong with a given job./

And if we lived in a perfect world, I'm sure we'd never need to ssh anywhere and could just sit in the middle of an interactive matrix like Tony Stark.  Regardless, this is an active development environment and we are in and out of Jenkins slaves and development team VMs, and just generally being the diagnostic fixit guys.  The information doesn't always come to you. Sometimes you have to go to the information. And while I get a lot of information from the Jenkins GUI, sometimes there's no substitute for going out and looking at something.

Also, the decision was made (before I came along) to have all the dev teams use their own chef servers, so there is no unified environment in dev.  Don't get me started on the discrepancies between this arena and other environments. I'm hoping to move toward homogenization.  But we have hundreds of developers, and something like 50 Openstack tenants (all dev teams). The infrastructure started small with a couple of teams and then grew to include a large portion of the enterprise rather quickly.

Or possibly I'm the only person who doesn't work in a perfect world. Everyone else, back to your regularly scheduled Ironman Matrix.
  

Sascha Bates | sascha.bates@gmail.com
<mailto:sascha.bates@gmail.com> | 612 850 0444 |
sascha.bates@skype | sascha_bates@yahoo |
On 10/21/12 7:57 PM, Brad Knowles wrote:

On Oct 21, 2012, at 5:35 PM, Sascha Bates<sascha.bates@gmail.com>  <mailto:sascha.bates@gmail.com>  wrote:

How are people working with these servers that have no useful labels?  I really want to know.  We ssh all over the place in our infrastructure and if I had to go out and look up a special snowflake ec2-blah-blah or IP every time I wanted to get to a server, I'd probably take a baseball bat to my infrastructure.

IMO, if you're manually ssh'ing around in a large infrastructure, you're doing it wrong.

If you've got a small infrastructure, and you can keep all the hostnames in your head at the same time, and you can manage your IP address space with simple /etc/hosts files, then manually ssh'ing around may be required on occasion and is probably okay.

However, if you've got a larger infrastructure, then you should have better tools to get the information you need and put it in a place where you can get to it, without you having to manually ssh around.  Those tools might end up having to use ssh to get the information, but that part of the process should be hidden from you.

For example, we run a Jenkins setup with 20 slaves.  The slaves are rebuilt often using openstack and chef.  I'm currently writing a knife plugin to set up ssh aliases for nodes in a role that make it easy for us to do things like 'ssh slave20' as we are constantly helping people figure out stuff about their jobs on the slaves or diagnosing VM issues (I'd actually really like to take a baseball bat to our Openstack infra).

With Jenkins, there are tools to copy workspaces from one job to another, to copy workspace data from one system to another, to archive artifacts, etc....  You should be making use of those kinds of tools so that you never need to go out to a specific slave to find out what went wrong with a given job.

--
Brad Knowles<brad@shub-internet.org>  <mailto:brad@shub-internet.org>
LinkedIn Profile:<http://tinyurl.com/y8kpxu>  <http://tinyurl.com/y8kpxu>

[Christopher_Brown]

This misses the point in exactly the same way it missed the point when we had this conversation at Amazon 8 years ago.
The point is not to have humans trying to track and use serial numbers directly, as if we were good at that. We are not.

The point is to use an appropriate mapping when and where possible. (As you said “then tooling needs to improve to the point where I don’t need to see the serial numbers”). We keep deflecting. Hostname arguments are a symptom and are neither the actual problem nor the cure. It’s also not about being “neutral”; it’s about being amenable to either concern. If manipulating hostnames, at some scale, is part of a current process, we should not make that difficult. We should also make rich mapping easy and first-class at scale.

-C

From: Lamont Granquist <lamont@opscode.commailto:lamont@opscode.com>
Reply-To: "chef@lists.opscode.commailto:chef@lists.opscode.com" <chef@lists.opscode.commailto:chef@lists.opscode.com>
Date: Wednesday, October 24, 2012 9:11 AM
To: "chef@lists.opscode.commailto:chef@lists.opscode.com" <chef@lists.opscode.commailto:chef@lists.opscode.com>
Subject: [chef] Re: Bring up the discussion for knife-ec2 --node-name-prefix option

I’ve used fairly lightweight human readable hostnames for a long time now, at very large scales and while there’s issues with that approach, i’ve seen silliness at the level of a hundred or so nodes where people who need to talk about a specific server start abbreviating the random number digits based on fractions of the hostname and i’ve become tired of using search and copy and pasting random numbers around. If serial numbers is really the wave of the future then tooling needs to improve to the point where I don’t need to see the serial numbers, and right now that simply isn’t the state that we’re in, and I’d vastly prefer to be oncall at 3am in the morning and be able to say “i’m going to reboot foo07” rather than “i’m going to reboot 8df34c1185d”. While its nice to think that humans being involved in the whole process is dirty and unclean and a symptom of a problem and we should just automate them all away, I think its drinking the kool aid way too hard. Human names have many benefits, in addition to their downsides, and there is simply no clear winner in this argument.

IMO chef should be perfectly neutral about this issue.

On 10/22/12 7:16 AM, Christopher Brown wrote:
Tensibai (and others who started the thread),
These are good points. As it is with many things in this industry, one size does not fit all.

I’ve be responsible for very large infrastructure deployments and have argued for most of my career against spending time and energy on a naming scheme. In a large infrastructure, relying on decipherable hostnames generally means other non-scalable practices follow, exactly because you are thinking at the single host level.

Nonetheless, we (Chef users and web-scale administrators) are quick to jump on the “SCALE IS KING” bandwagon and expect those techniques to be absolute truth. They are not. If your infrastructure is manageable now, is not expected to grow dramatically, and you are working with other tools and existing processes, don’t let other people tell you they know your business best. Most of us are on a path of change and, over time, you may find you adopt these techniques when it’s necessary or reasonable.

Until then, keep asking how Chef can help with what you do now, and what you want to do later.

-C

Christopher Brown
Chief Technology Officer, Opscode
Twitter: @skeptomai

From: Tensibai <tensibai@iabis.netmailto:tensibai@iabis.net>
Reply-To: "chef@lists.opscode.commailto:chef@lists.opscode.com" <chef@lists.opscode.commailto:chef@lists.opscode.com>
Date: Monday, October 22, 2012 5:38 AM
To: "chef@lists.opscode.commailto:chef@lists.opscode.com" <chef@lists.opscode.commailto:chef@lists.opscode.com>
Subject: [chef] Re: *** PROBABLY SPAM *** Re: Re: Re: Bring up the discussion for knife-ec2 --node-name-prefix option

Well, I’m not using ec2 but reading this thread at this point makes 2 or 3 points in my mind:

1. I’m promoting chef, at this stage I really can’t tell others to stop using manually defined machine names without shooting a bullet in chef future.

2. We do use others tools (really ? yes) wich are not so easy to connect to chef, basing a decision on system name about OS/type could be easiest.

So well, it could be a “best practice” to have not meaningfull node names, but that could be helpfull too in some cases.

I see not point avoiding this kind of use for mid-way teams. But I acn see the point having this feature to help mid-way teams migrate from one step to the next…

So, sorry but this time some ‘great’ people from this list sounds elitist on how you may/should use chef …

Cheers

Le 2012-10-22 08:45, AJ Christensen a écrit :

I’m going to go ahead and say yeah, I am Iron Man, and we are so done in this thread.

Ship the code. Resolve the ticket. Let the community use it if they want. You can give machines friendly human snowflake pet names if you like. We will not.

–AJ

On Oct 22, 2012 3:56 PM, “Sascha Bates” <sascha.bates@gmail.commailto:sascha.bates@gmail.com> wrote:

With Jenkins, there are tools to copy workspaces from one job to another, to copy workspace data from one system to another, to archive artifacts, etc… You should be making use of those kinds of tools so that you never need to go out to a specific slave to find out what went wrong with a given job.

And if we lived in a perfect world, I’m sure we’d never need to ssh anywhere and could just sit in the middle of an interactive matrix like Tony Stark. Regardless, this is an active development environment and we are in and out of Jenkins slaves and development team VMs, and just generally being the diagnostic fixit guys. The information doesn’t always come to you. Sometimes you have to go to the information. And while I get a lot of information from the Jenkins GUI, sometimes there’s no substitute for going out and looking at something.

Also, the decision was made (before I came along) to have all the dev teams use their own chef servers, so there is no unified environment in dev. Don’t get me started on the discrepancies between this arena and other environments. I’m hoping to move toward homogenization. But we have hundreds of developers, and something like 50 Openstack tenants (all dev teams). The infrastructure started small with a couple of teams and then grew to include a large portion of the enterprise rather quickly.

Or possibly I’m the only person who doesn’t work in a perfect world. Everyone else, back to your regularly scheduled Ironman Matrix.

Sascha Bates | sascha.bates@gmail.commailto:sascha.bates@gmail.com | 612 850 0444 | sascha.bates@skype | sascha_bates@yahoo |
On 10/21/12 7:57 PM, Brad Knowles wrote:

On Oct 21, 2012, at 5:35 PM, Sascha Bates sascha.bates@gmail.commailto:sascha.bates@gmail.com wrote:

How are people working with these servers that have no useful labels? I really want to know. We ssh all over the place in our infrastructure and if I had to go out and look up a special snowflake ec2-blah-blah or IP every time I wanted to get to a server, I’d probably take a baseball bat to my infrastructure.

IMO, if you’re manually ssh’ing around in a large infrastructure, you’re doing it wrong.

If you’ve got a small infrastructure, and you can keep all the hostnames in your head at the same time, and you can manage your IP address space with simple /etc/hosts files, then manually ssh’ing around may be required on occasion and is probably okay.

However, if you’ve got a larger infrastructure, then you should have better tools to get the information you need and put it in a place where you can get to it, without you having to manually ssh around. Those tools might end up having to use ssh to get the information, but that part of the process should be hidden from you.

For example, we run a Jenkins setup with 20 slaves. The slaves are rebuilt often using openstack and chef. I’m currently writing a knife plugin to set up ssh aliases for nodes in a role that make it easy for us to do things like ‘ssh slave20’ as we are constantly helping people figure out stuff about their jobs on the slaves or diagnosing VM issues (I’d actually really like to take a baseball bat to our Openstack infra).

With Jenkins, there are tools to copy workspaces from one job to another, to copy workspace data from one system to another, to archive artifacts, etc… You should be making use of those kinds of tools so that you never need to go out to a specific slave to find out what went wrong with a given job.

–
Brad Knowles brad@shub-internet.orgmailto:brad@shub-internet.org
LinkedIn Profile: http://tinyurl.com/y8kpxuhttp://tinyurl.com/y8kpxu

[Lamont_Granquist_OLD]

Well, I think we're always going to have humans interacting with
individual servers. I think trying to eliminate that entirely is
probably reducible to the halting problem, and you'll just keep kicking
the can down the road, or up another level. At the same time I think
that by pushing the use of serial numbers and removing humans as much as
possible that we'll likely make a lot of useful process in the tooling
that isn't going to happen if the industry/community/whatever doesn't
try. So, its probably better off to ignore me and push for the use of
serial numbers in order to make progress, however, I just don't buy that
its that clear that its the ideal solution or that anything else doesn't
scale.

And you're probably right that we need more than being neutral, but to
embrace both ways of doing it. I'll certainly buy that. That has been
one of the strengths so far of chef which is that however you fall along
religious lines, chef tends to just support it.

On 10/24/12 9:50 AM, Christopher Brown wrote:

This misses the point in exactly the same way it missed the point when
we had this conversation at Amazon 8 years ago.
The point is not to have humans trying to track and use serial numbers
directly, as if we were good at that. We are not.

The point is to use an appropriate mapping when and where possible.
(As you said "then tooling needs to improve to the point where I don't
need to see the serial numbers"). We keep deflecting. Hostname
arguments are a symptom and are neither the actual problem nor the
cure. It's also not about being "neutral"; it's about being amenable
to either concern. If manipulating hostnames, at some scale, is part
of a current process, we should not make that difficult. We should
also make rich mapping easy and first-class at scale.

-C

From: Lamont Granquist <lamont@opscode.com mailto:lamont@opscode.com>
Reply-To: "chef@lists.opscode.com mailto:chef@lists.opscode.com"
<chef@lists.opscode.com mailto:chef@lists.opscode.com>
Date: Wednesday, October 24, 2012 9:11 AM
To: "chef@lists.opscode.com mailto:chef@lists.opscode.com"
<chef@lists.opscode.com mailto:chef@lists.opscode.com>
Subject: [chef] Re: Bring up the discussion for knife-ec2
--node-name-prefix option

I've used fairly lightweight human readable hostnames for a long time
now, at very large scales and while there's issues with that approach,
i've seen silliness at the level of a hundred or so nodes where people
who need to talk about a specific server start abbreviating the random
number digits based on fractions of the hostname and i've become tired
of using search and copy and pasting random numbers around. If serial
numbers is really the wave of the future then tooling needs to improve
to the point where I don't need to see the serial numbers, and right
now that simply isn't the state that we're in, and I'd vastly prefer
to be oncall at 3am in the morning and be able to say "i'm going to
reboot foo07" rather than "i'm going to reboot 8df34c1185d". While
its nice to think that humans being involved in the whole process is
dirty and unclean and a symptom of a problem and we should just
automate them all away, I think its drinking the kool aid way too
hard. Human names have many benefits, in addition to their downsides,
and there is simply no clear winner in this argument.

IMO chef should be perfectly neutral about this issue.

On 10/22/12 7:16 AM, Christopher Brown wrote:

Tensibai (and others who started the thread),
These are good points. As it is with many things in this industry,
one size does not fit all.

I've be responsible for very large infrastructure deployments and
have argued for most of my career against spending time and energy on
a naming scheme. In a large infrastructure, relying on decipherable
hostnames generally means other non-scalable practices follow,
exactly because you are thinking at the single host level.

Nonetheless, we (Chef users and web-scale administrators) are quick
to jump on the "SCALE IS KING" bandwagon and expect those techniques
to be absolute truth. They are not. If your infrastructure is
manageable now, is not expected to grow dramatically, and you are
working with other tools and existing processes, don't let other
people tell you they know your business best. Most of us are on a
path of change and, over time, you may find you adopt these
techniques when it's necessary or reasonable.

Until then, keep asking how Chef can help with what you do now, and
what you want to do later.

-C

Christopher Brown
Chief Technology Officer, Opscode
Twitter: @skeptomai

From: Tensibai <tensibai@iabis.net mailto:tensibai@iabis.net>
Reply-To: "chef@lists.opscode.com mailto:chef@lists.opscode.com"
<chef@lists.opscode.com mailto:chef@lists.opscode.com>
Date: Monday, October 22, 2012 5:38 AM
To: "chef@lists.opscode.com mailto:chef@lists.opscode.com"
<chef@lists.opscode.com mailto:chef@lists.opscode.com>
Subject: [chef] Re: *** PROBABLY SPAM *** Re: Re: Re: Bring up the
discussion for knife-ec2 --node-name-prefix option

Well, I'm not using ec2 but reading this thread at this point makes 2
or 3 points in my mind:

1. I'm promoting chef, at this stage I really can't tell others to
   stop using manually defined machine names without shooting a bullet
   in chef future.

2. We do use others tools (really ? yes) wich are not so easy to
   connect to chef, basing a decision on system name about OS/type could
   be easiest.

So well, it could be a "best practice" to have not meaningfull node
names, but that could be helpfull too in some cases.

I see not point avoiding this kind of use for mid-way teams. But I
acn see the point having this feature to help mid-way teams migrate
from one step to the next...

So, sorry but this time some 'great' people from this list sounds
elitist on how you may/should use chef ...

Cheers

Le 2012-10-22 08:45, AJ Christensen a écrit :

I'm going to go ahead and say yeah, I am Iron Man, and we are so
done in this thread.

Ship the code. Resolve the ticket. Let the community use it if they
want. You can give machines friendly human snowflake pet names if
you like. We will not.

--AJ

On Oct 22, 2012 3:56 PM, "Sascha Bates" <sascha.bates@gmail.com
mailto:sascha.bates@gmail.com> wrote:

/With Jenkins, there are tools to copy workspaces from one job to another, to copy workspace data from one system to another, to archive artifacts, etc....  You should be making use of those kinds of tools so that you never need to go out to a specific slave to find out what went wrong with a given job./

And if we lived in a perfect world, I'm sure we'd never need to ssh anywhere and could just sit in the middle of an interactive matrix like Tony Stark.  Regardless, this is an active development environment and we are in and out of Jenkins slaves and development team VMs, and just generally being the diagnostic fixit guys.  The information doesn't always come to you. Sometimes you have to go to the information. And while I get a lot of information from the Jenkins GUI, sometimes there's no substitute for going out and looking at something.

Also, the decision was made (before I came along) to have all the dev teams use their own chef servers, so there is no unified environment in dev.  Don't get me started on the discrepancies between this arena and other environments. I'm hoping to move toward homogenization.  But we have hundreds of developers, and something like 50 Openstack tenants (all dev teams). The infrastructure started small with a couple of teams and then grew to include a large portion of the enterprise rather quickly.

Or possibly I'm the only person who doesn't work in a perfect world. Everyone else, back to your regularly scheduled Ironman Matrix.
  

Sascha Bates | sascha.bates@gmail.com
<mailto:sascha.bates@gmail.com> | 612 850 0444 |
sascha.bates@skype | sascha_bates@yahoo |
On 10/21/12 7:57 PM, Brad Knowles wrote:

On Oct 21, 2012, at 5:35 PM, Sascha Bates<sascha.bates@gmail.com>  <mailto:sascha.bates@gmail.com>  wrote:

How are people working with these servers that have no useful labels?  I really want to know.  We ssh all over the place in our infrastructure and if I had to go out and look up a special snowflake ec2-blah-blah or IP every time I wanted to get to a server, I'd probably take a baseball bat to my infrastructure.

IMO, if you're manually ssh'ing around in a large infrastructure, you're doing it wrong.

If you've got a small infrastructure, and you can keep all the hostnames in your head at the same time, and you can manage your IP address space with simple /etc/hosts files, then manually ssh'ing around may be required on occasion and is probably okay.

However, if you've got a larger infrastructure, then you should have better tools to get the information you need and put it in a place where you can get to it, without you having to manually ssh around.  Those tools might end up having to use ssh to get the information, but that part of the process should be hidden from you.

For example, we run a Jenkins setup with 20 slaves.  The slaves are rebuilt often using openstack and chef.  I'm currently writing a knife plugin to set up ssh aliases for nodes in a role that make it easy for us to do things like 'ssh slave20' as we are constantly helping people figure out stuff about their jobs on the slaves or diagnosing VM issues (I'd actually really like to take a baseball bat to our Openstack infra).

With Jenkins, there are tools to copy workspaces from one job to another, to copy workspace data from one system to another, to archive artifacts, etc....  You should be making use of those kinds of tools so that you never need to go out to a specific slave to find out what went wrong with a given job.

--
Brad Knowles<brad@shub-internet.org>  <mailto:brad@shub-internet.org>
LinkedIn Profile:<http://tinyurl.com/y8kpxu>  <http://tinyurl.com/y8kpxu>