Problem with Chef::Mixin::ShellOut

Chef Infra (archive)
1

I have a following disk in my system. What is important is that it has GPT and one XFS formatted partition.

[root@myhost ~]# parted -s /dev/sdb print 2>/dev/null | grep -c 'Partition Table: gpt'
1

[root@myhost ~]# lsblk -no FSTYPE /dev/sdb1 | grep xfs
xfs

I’m tryin to check if the drive is in the state as shown above and if not make necessary changes in chef.

Now I have my helper

module MyHelper
require 'mixlib/shellout'

# IS IT XFS?
def has_xfs_partition?(block_device)
 
  cmd = Mixlib::ShellOut.new("lsblk -no FSTYPE #{block_device} | grep xfs")
  cmd.run_command    

  puts "block_device: " + block_device
  puts "cmd.stderr: " + cmd.stderr
  puts "cmd.stdout: " + cmd.stdout
  puts "cmd.status: " + cmd.status.to_s
  puts "cmd.exitstatus: " + cmd.exitstatus.to_s

  if !cmd.stderr.empty?
    raise cmd.stderr
  end

  if cmd.stdout.to_s.empty?
    Chef::Log.info("#{block_device}: NO XFS ")
    return false
  else
    Chef::Log.info("#{block_device}: XFS")
    return true
  end
end

# DOES IT HAVE GPT?
def has_partition_table?(block_device)
  cmd = Mixlib::ShellOut.new("parted -s #{block_device} print 2>/dev/null|grep -c 'Partition Table: gpt'")
  cmd.run_command    

  puts "block_device: " + block_device
  puts "cmd.stderr: " + cmd.stderr
  puts "cmd.stdout: " + cmd.stdout
  puts "cmd.status: " + cmd.status.to_s
  puts "cmd.exitstatus: " + cmd.exitstatus.to_s

  unless cmd.stderr.empty?
    raise cmd.stderr
  end

  if cmd.stdout.to_i != 0
    Chef::Log.info("YES")
    return true
  else
    Chef::Log.info("NO")
    return false
  end
end

end

And I use these functions in the following simple recipe

# v['dev_path'] contains my device, here /dev/sdb
# DO WE HAVE GPT ON THE DEVICE?
Chef::Log.info("LAUNCHING has_partition_table?#{v['dev_path']}")
unless has_partition_table?(v['dev_path'])
  puts "IT DIDNT HAVE PARTITION TABLE. HERE DOING SOMETHING"
end

Chef::Log.info("LAUNCHING has_xfs_partition?(#{v['dev_path']}1)")
unless has_xfs_partition?("#{v['dev_path']}1")
  puts "#{v['dev_path']}1 WAS NOT XFS FORMATTED. HERE DOING SOMETHING"
else
  puts "#{v['dev_path']}1 IS XFS FORMATTED"
end

Now when I run chef-client I get dissapointing result

Compiling Cookbooks...
[2016-12-01T08:24:45+01:00] INFO: LAUNCHING has_partition_table?/dev/sdb
block_device: /dev/sdb
cmd.stderr: 
cmd.stdout: 1
cmd.status: pid 31431 exit 0
cmd.exitstatus: 0
[2016-12-01T08:24:45+01:00] INFO: YES
[2016-12-01T08:24:45+01:00] INFO: LAUNCHING has_xfs_partition?(/dev/sdb1)
block_device: /dev/sdb1
cmd.stderr: 
cmd.stdout: 
cmd.status: pid 31439 exit 1
cmd.exitstatus: 1
[2016-12-01T08:24:45+01:00] INFO: /dev/sdb1: NO XFS 
/dev/sdb1 WAS NOT XFS FORMATTED. HERE DOING SOMETHING

When I comment out the first function and leave only checking if the thing is XFS partition then everything is OK

Compiling Cookbooks...
[2016-12-01T08:31:40+01:00] INFO: LAUNCHING has_xfs_partition?(/dev/sdb1)
block_device: /dev/sdb1
cmd.stderr: 
cmd.stdout: xfs
cmd.status: pid 31651 exit 0
cmd.exitstatus: 0
[2016-12-01T08:31:40+01:00] INFO: /dev/sdb1: XFS
/dev/sdb1 IS XFS FORMATTED

I probably made some stupid mistake but unfortunately I can’t figure out where. As you see when I run these two commands directly in console everything’s fine but when I try to run them both in a recipe they fail. Why it’s OK to run them separately?

2

Any reason to not use ohai data ?
Something along the line node['filesystem2']['by_device'].include?("/dev/sdb1") to check the partition exists and node['filesystem2']['by_device']['/dev/sdb1']['fs_type'] == 'xfs' to check the formatting sounds enough to me.

3

Yeah, I know but I’m trying to understand Chef::Mixing::ShellOut behaviour.

4

I would try this way:

cmd = Mixlib::ShellOut.new("lsblk -no FSTYPE #{block_device}")

And then

if cmd.stdout != "xfs" instead of if cmd.stdout_to.s.empty?

This way the stdout of lsblk will be shown and not filtered by grep, you may have more insight on what’s going wrong.

5

I tried it.

Anyway as you see in the log I provided I’m printing cmd.stdout and it’s empty. So is stderr and in status you see the proces exited with code 1. No idea why, though.

6

You’re printing stdout of a pipeline where you filter the output to contain only xfs, so obvisouly it’s empty if there’s not xfs in the line but we have no idea what it does contain at first.

The exit status is 1 because either lsblk did fail or because grep didn’t find xfs in the output of lsblk, with a pipeline it’s impossible to know which one did return 1.

7

So now it's

def has_xfs_partition?(block_device)
 
  cmd = Mixlib::ShellOut.new("lsblk -no FSTYPE #{block_device}")
  cmd.run_command    

  puts "block_device: " + block_device
  puts "cmd.stderr: " + cmd.stderr
  puts "cmd.stdout: " + cmd.stdout
  puts "cmd.status: " + cmd.status.to_s
  puts "cmd.exitstatus: " + cmd.exitstatus.to_s

  if !cmd.stderr.empty?
    raise cmd.stderr
  end

 if cmd.stdout.delete!("\n") != 'xfs' # stdout had newline at the end
    Chef::Log.info("#{block_device}: NO XFS ")
    return false
  else
    Chef::Log.info("#{block_device}: XFS")
    return true
  end
end

and I get

Compiling Cookbooks...
[2016-12-01T13:20:29+01:00] INFO: LAUNCHING has_partition_table?/dev/sdb
block_device: /dev/sdb
cmd.stderr:
cmd.stdout: 1
cmd.status: pid 3081 exit 0
cmd.exitstatus: 0
[2016-12-01T13:20:29+01:00] INFO: YES
[2016-12-01T13:20:29+01:00] INFO: LAUNCHING has_xfs_partition?(/dev/sdb1)
block_device:/dev/sdb1
block_device: /dev/sdb1
cmd.stderr:
cmd.stdout:
cmd.status: pid 3088 exit 0
cmd.exitstatus: 0
[2016-12-01T13:20:29+01:00] INFO: /dev/sdb1: NO XFS
/dev/sdb1 WAS NOT XFS FORMATTED. HERE DOING SOMETHING

When I do it myself
[root@myhost ~]# lsblk -no FSTYPE /dev/sdb1
xfs

lsblk clearly gives me some output

I don't know why but stdout is empty and empty string | grep "something" returns 1. That's why I had code=1 before. Still no idea why stdout is empty. The same code launched without first function (one which is checking for GPT) works just fine :/.

With recipe which has only has_xfs_partition?("#{v['dev_path']}1")
instead of

has_partition_table?(v['dev_path']) and
has_xfs_partition?("#{v['dev_path']}1")
I get the following output which is correct.

Compiling Cookbooks...
[2016-12-01T13:32:10+01:00] INFO: LAUNCHING has_xfs_partition?(/dev/sdb1)
block_device:/dev/sdb1
block_device: /dev/sdb1
cmd.stderr:
cmd.stdout: xfs
cmd.status: pid 4147 exit 0
cmd.exitstatus: 0
[2016-12-01T13:32:10+01:00] INFO: /dev/sdb1: XFS
/dev/sdb1 IS XFS FORMATTED
8

I always forgot it, instead of .delete("\n") you can use .chomp. No need for replacing in place (the ! version).[quote="veluu, post:7, topic:9958"]
[2016-12-01T13:20:29+01:00] INFO: LAUNCHING has_xfs_partition?(/dev/sdb1)
block_device:/dev/sdb1
block_device: /dev/sdb1
[/quote]

Why do you have two times the block_device line? The output you're showing doesn't match your code, there's something unclean here.

I can't reproduce your case... so I've really no idea of why this is happening. sorry

9

I added one more print before doing cmd.stdout. For a second I had something on my mind and wanted to printout parameter before I actually do something.

It’s a pitty you don’t have any chef instance available to reproduce. Long story short.

When I launch:
has_partition_table?
has_xfs_partition?

stdout in the second function is empty.

If I comment out has_partition_table? so in my recipe there’s only:
has_xfs_partition?

stdout is not empty any more and has desired value.

10

You didn’t understand me, when I say I can’t repro, I did copy/paste your code in a cookbook and used it, and I can’t get this behavior in chef 11.16, chef 12.4 or chef 12.14. I don’t have isntances with gpt partitions, but that shouldn’t change much. So I really don’t get what’s going on with your case.

11

Ok I dropped looking at chef code and returned to console.

There’s a problem with parted.

while [ 1 ]; do block_device="/dev/sdb"; parted -s "$block_device" print 2>/dev/null|grep -c 'Partition Table: gpt' && lsblk -no FSTYPE "$block_device"1 | grep xfs; done

Gives me: lsblk: /dev/sdb1: not a block device
Either I have to add sleep 1s between these two command or use blkid instead of parted which works without using sleep. My problem had nothing to do with chef. Thank you for your time.