I need chef to wait until it has been able to successfully SSH into a remote box/node.
I know that I can use the following logic to determine whether I am able to SSH from Node A/user A to Node B/user B:
ssh -q userB@NodeB exit
If ? / Return code is 0 then ssh is successful
Else IF ? / Return code is 255 then ssh is unsuccessful.
So I need to know how can I implement this in a resource/Ruby block within chef:
Step1: Execute test SSH connection
Step2: UNTIL (return code is 0)
wait 60 seconds
print a info log message to chef::log
GO TO Step1
END
Step3: Proceed to next chef resource in the recipe only when you come out of the above wait block, no matter how much time it takes
It looks like the question was already answered, so if I may ask, what were you trying to accomplish with this? Does an application deploy depend on an event that occurred after you login into another box?
I think I can tweak the retries option to a higher number and achieve a indefinite wait.
If you are wondering why I canāt afford to just error out, fix the problem and retry chef-client execution, its because I have quite a lot of resources āpreceedingā this test SSH connection resource, which have to do with generating certificates, adding the certificate to the trust-store etc. (>100 resources) i.e. I cannot simply abort in the middle, fix the problem and retry because the keystores etc. would already have been created and cert import commands fail because a certificate already exists. Therefore, the only way I can do this is to wipe the VM and start clean - something I donāt want to waste lot of time on during my chef code development. (I know I can take VM snapshots, but wont go into that here for some other reasons)
Rilindo >> Yes, effectively I am installing 2 parts of an application on different VMs, and these 2 parts talk to each other over SSL/keystore/truststore on a bi-directional basis. Both these VMs are built by 2 different cookbooks. I need one cookbook, running on VM #2, to wait until it can successfully copy the root cert/public key/trust certs from VM #1 - for which I need to setup SSH keys in the background so that I can do a SFTP via Chef.
You may ask why I cant setup the SSH Keys via Chef itself i.e. import the public key from VM#2 into authorized_keys file of VM#1, but Chef itself is creating the application user account (OS) and therefore since passoword auth is disabled, I need to do this āenable SFTP via SSHā activity manually via a different OS user ID.
Then your recipe is not idempotent (not testing if thereās something to do) and thus should be improved.
Keep in mind chef is designed to be run periodically and success at each run, enforcing your described state at each run.
If you wish to have a bunch of sequential commands, chef is not the right tool IMHO. If you wan to do a system configuration management, you MUST take into account it may have to be done in two or three pass sometimes, and write your recipes accordingly using guards/writing custom resources when really necessary, reviewing how you plan it at all.
Considering ssh keys, thereās no reason you canāt generate them before even creating the VMs, so you should be able to rethink it, generating each machine key, provisioning each machine and setting it up with the keys generated beforehand.
I think I am realizing that when I need to check something before I DO something within Chef, I run into having to know a bit of Ruby syntax. Trying to learn that now.
For example, BEFORE I tell Chef to run a execute block, I know that I can use the guard not_if to check for something.
e.g.
execute "unzip #{folder}/installer.zip -d #{node[:app][:installer_dir]}" do
user node[:app][:user]
group node[:app][:group]
not_if { File.exists?("#{node[:app][:installer_dir]}/app") }
end
What I need to figure out is some examples and āpossible options that I can useā for the not_if section.
For example, if I can run a bash command and grep for a particular exit code or output AS PART OF the not_if block, I can then determine if the cert already exists and if so, not execute the āimport certā execute block and that way make the āimport certā execute block idempotent.
In your example, it may be better to set that execute to action :nothing and then notify it from the place where you download installer.zip, so that it will re-unzip files in the case where the zip
had changed. Your not_if approach will do nothing in that circumstance.
As an aside, a system package rather than a zip is going to be more
idempotent-while-supporting-interesting-changes approach.
So a couple of points on guards which might help you out:
Thereās only_if as well as not_if, which is the reverse
You can execute a local shell command instead of ruby by omitting the
curly braces in a guard, i.e. only_if āgrep āmyhost.example.com http://myhost.example.comā /etc/hostsā**
Looking for a specific exit code from a guard isnāt supported. Write to
your guard to pass or fail
** Note that you are shelling out here, which is slower than using pure
ruby, so avoid it when you know how to do it in ruby. That being said if
youāre getting frustrated and just want something to work, or your guard is
getting too big then itās an option.
Having looked at a few of your posts in the last few days I think a bit of
a mentality shift on your part would be useful. Think of the code youāre
writing in Chef as describing a state for your infrastructure, rather than
a series of chained ad-hoc tasks. As the author of the code you should be
aware of the dependencies of your resources and factor that into the code
so that all the cogs turn nicely, dependencies are created before theyāre
are needed and your script could fail at any step and be re-run without
incident.
As mentioned earlier idempotency is also important to your code, and for
the sake of idempotency (and the speed related to idempotency checks) you
might want to spent a bit of time thinking about how you implement.
As youāre probably noticing Chef has a bit of a learning curve, itās
definitely a technology that shines when you take a deep dive and learn as
much as you can about it (and ruby), but when you do the reward is that it
becomes quite easy to do pretty much anything.
To complement @yoshiwaanās answer above, here is the Guard documentation, with some examples, as well as details on how they behave (return 0 to trigger the guard,etc).
According to your example, the ark cookbook could be of use too, taking care of getting a remote file, unzip it, do simlinks, etc.
I think I am at that point of the Chef learning lifecycle where after getting to grip with key resources (bash, execute, template, remote_file etc.) used in a āvanillaā sort-of-way, now I am slowing moving into idempotency/Guards. I need to next further improve the cookbooks using notifications.
May I ask if any of you know of existing Sublime Text 3 snippets for all chef resources (with ALL possible options included) - I am finding writing my own snippets does cut down on the programming time quite a lot, in the absence of a dedicated IDE for Chef scripting.
I think as @yoshiwaan pointed out, it does require a bit of analyzing things in a different way - it feels almost as though Chef is intended to work in non-sequential way (which is NOT what I am used to with Bash scripting etc. where one thing leads to another methodically, usually).
If indeed Chef is meant to work non-sequentially and in sort of neural network way, I am also surprised to not find multi-threading capabilities where for example i should have to ability to spawn multiple resources (especially ones that take long time like downloading or unzipping something) and then make use of notifications to then trigger dependent resources. Sure, I suppose I could do this via a execute block and nohup & a command but then I lose the ability to track that PIDās exit status within Chef.
Interesting times, though, and definitely something to look forward to in the New Year.
Absolutely not, chef will run (converge) resources in the order specified in the recipe (minus the compile vs converge times, but thatās another story and thereās a lot of pages on it), and the recipes in the order specified in the run list. You may get a more complex āexpanded runlistā with roles runlist and all include_recipe calls (search āwrapper cookbookā for clues here)
What does really change between a bash script and chef is that Chef is meant to be run in a regular schedule and enforce the state described, the main difference is here, you describe a state more than a path to a desired state.
Because of this, you should never assume the state anything will be in when chef run, the goal is to end the run in the desired state, whatever was the starting state. This involve doing things in an idempotent manner, do something only if necessary, and if thereās nothing to do, donāt do anything (or for worst cases do a no-op thing, like executing something which will have no effect if the configuration is already ok, but will correct it if not)
Iāve the feeling you should take a day or two to go through https://learn.chef.io to get the hands on with this principal change and how to tackle things with Chef. As a rule of thumb, using script (bash or else) or execute resources is usually a sign youāre doing something wrong.
thereās side cases really precise where itās unavoidable to use bash/script or execute, but 90% of time you can take advantage of existing cookbooks (for their LWRP/custom resources) or even base chef resources.
So as a bit of context as to WHY you should use resources other than
execute, instead of just āyouāre doing it wrongā
Pretty much all of the core resources in Chef other than execute/script
have inbuilt logic to deal with current and desired state (and thus
idempotency) as well as clear interfaces (in the form of attributes to the
resource) for managing the resource.
When reading a recipe itās a lot easier to see a file resource and know
that it manages a file, and see the parameters of that resource there in an
understandable format than to have grok an execute statement.
When the code converges the logic of those resources obtains the current
state of the resource and can work out any changes to make to bring it to
the desired state (which also allows notifications from one resource to
another on state change). It can also implement those changes. This is done
without shelling out (which slows down the Chef run) and without guards, as
the resource logic handles whether to take actions or not.
An execute/script resource is triggered by Chef but the action that takes
isnāt understood by the Chef run, thus the need for guards and the fact
execute resources are often specified with action :nothing and triggered
via notification.
So itās always worth asking the question ācan I do what this execute/script
resource using other Chef resources (including custom resources from
cookbooks)?ā because if the answer is yes youāll have a faster, cleaner
result with more clarity.
I apologize, if it gets quoted, it's that it sounds rude.
My goal was to highlight the fact @yoshiwaan explained very well that you should think twice before using those kind of resources, but leaving the why as an exercise.
Sorry again if it was rude, it was not the intention.
Well, I didnāt write that because it sounded rude. More for someone new
coming along and reading through this thread and wondering why itās
wrong, which is something I know I struggled with when I was learning.
Also, for the benefit of new forum members/new chefers (is that the right term) who may not have done this already, I also suggest requesting training material from https://www.chef.io/contact/open-training/ - they send you a Box shared folder link which has very useful presentations on Basics and Intermediary topics of Chef which I found very useful. I felt its also a bit simpler to understand and structured in an easy to understand way, especially for newbies like me.
