# aws\_ebs\_raid fails on ec2 node, but only when it's rebooted

**URL:** <https://discourse.chef.io/t/aws-ebs-raid-fails-on-ec2-node-but-only-when-its-rebooted/4141>\
**Category:** Chef Infra (archive)\
**Created:** [June 26, 2013, 11:06am UTC](https://discourse.chef.io/t/aws-ebs-raid-fails-on-ec2-node-but-only-when-its-rebooted/4141 "2013-06-26T11:06:10Z")\
**Posts on this page:** 3\
**Page:** 1

<div class="post-metadata">

**Author:** ![Jorge\_Bianquetti](https://sea2.discourse-cdn.com/flex016/user_avatar/discourse.chef.io/jorge_bianquetti/32/432_2.png) [@Jorge\_Bianquetti](https://discourse.chef.io/u/Jorge_Bianquetti)\
**Post date:** [June 26, 2013, 11:06am UTC](https://discourse.chef.io/t/aws-ebs-raid-fails-on-ec2-node-but-only-when-its-rebooted/4141/1 "2013-06-26T11:06:10Z")

</div>

Hi!

Something strange it’s happening with one mongoDB node that needs and  
EBS raid with provisioned IOPS, using Chef 10.26.0

First, node creates EBS raid type 10 with two disks without problems,  
and then use it to hosts mongoDB databases.

Then, I reboot or stops the ec2 node.

When the node finished the boot process, EBS raid devices are  
attached to the machine, but when I run chef-client, it fails with  
message:

FATAL: ArgumentError: ruby\_block[wait\_mongodb\_data\_raid]  
(/var/chef/cache/cookbooks/aws/providers/ebs\_raid.rb line 182) had an  
error: ArgumentError: You tried to set a nested key, where the parent  
is not a hash-like object:  
aws/raid//var/lib/mongodb/device\_map/sdi1/sdi2/sdi1

Node’s info seems right to me:

```
"aws": {
  "raid": {
    "/var/lib/mongodb": {
      "raid_dev": "md0",
      "device_map": {
        "sdi1": "vol-6bba1647",
        "sdi2": "vol-bbbb1797"
      }
    }
  },
  "ebs_volume": {
    "sdi1": {
      "volume_id": "vol-6bba1647"
    },
    "sdi2": {
      "volume_id": "vol-bbbb1797"
    }
  }
}

```

So, I tried with different aws cookbook versions: 0.101.3 0.101.2  
0.100. All fails in the same steps.

Any suggestions?

---

<div class="post-metadata">

**Author:** ![Nathaniel\_Eliot](https://sea2.discourse-cdn.com/flex016/user_avatar/discourse.chef.io/nathaniel_eliot/32/506_2.png) [@Nathaniel\_Eliot](https://discourse.chef.io/u/Nathaniel_Eliot)\
**Post date:** [June 26, 2013, 4:30pm UTC](https://discourse.chef.io/t/aws-ebs-raid-fails-on-ec2-node-but-only-when-its-rebooted/4141/2 "2013-06-26T16:30:42Z")

</div>

I suspect that something in ohai overwriting the existing array, but  
only once the ebs volume has mounted.

Are you looking at the failed run log, or the node saved on the Chef  
server? Node info only saves on a successful run.

Using 'chef-shell -z' (or 'shef -z', if you're 10.X) will drop you  
into an irb shell right after everything loads for a chef run. From  
there, you can switch to recipe mode, and start digging into  
node['attr'] values as they would be presented to your recipe code.

The other option is dropping Chef::Log.info statements into the aws  
cookbook. That's got a longer debug loop time, but gives you a bit  
more power; you don't have to approximate what the cookbook is doing,  
you can just take it straight off the line.

Cheers,

--  
Nathaniel Eliot  
T9 Productions

On Wed, Jun 26, 2013 at 6:06 AM, Jorge Bianquetti [jbianquetti@gmail.com](mailto:jbianquetti@gmail.com) wrote:

> Hi!
> 
> Something strange it's happening with one mongoDB node that needs and  
> EBS raid with provisioned IOPS, using Chef 10.26.0
> 
> First, node creates EBS raid type 10 with two disks without problems,  
> and then use it to hosts mongoDB databases.
> 
> Then, I reboot or stops the ec2 node.
> 
> When the node finished the boot process, EBS raid devices are  
> attached to the machine, but when I run chef-client, it fails with  
> message:
> 
> FATAL: ArgumentError: ruby\_block[wait\_mongodb\_data\_raid]  
> (/var/chef/cache/cookbooks/aws/providers/ebs\_raid.rb line 182) had an  
> error: ArgumentError: You tried to set a nested key, where the parent  
> is not a hash-like object:  
> aws/raid//var/lib/mongodb/device\_map/sdi1/sdi2/sdi1
> 
> Node's info seems right to me:
> 
> ```
> "aws": {
> "raid": {
> "/var/lib/mongodb": {
> "raid_dev": "md0",
> "device_map": {
> "sdi1": "vol-6bba1647",
> "sdi2": "vol-bbbb1797"
> }
> }
> },
> "ebs_volume": {
> "sdi1": {
> "volume_id": "vol-6bba1647"
> },
> "sdi2": {
> "volume_id": "vol-bbbb1797"
> }
> }
> }
> 
> ```
> 
> So, I tried with different aws cookbook versions: 0.101.3 0.101.2  
> 0.100. All fails in the same steps.
> 
> Any suggestions?

---

<div class="post-metadata">

**Author:** ![kallistec](https://sea2.discourse-cdn.com/flex016/user_avatar/discourse.chef.io/kallistec/32/23_2.png) [@kallistec](https://discourse.chef.io/u/kallistec)\
**Post date:** [June 26, 2013, 4:31pm UTC](https://discourse.chef.io/t/aws-ebs-raid-fails-on-ec2-node-but-only-when-its-rebooted/4141/3 "2013-06-26T16:31:31Z")

</div>

On Wednesday, June 26, 2013 at 4:06 AM, Jorge Bianquetti wrote:

> Hi!
> 
> Something strange it's happening with one mongoDB node that needs and  
> EBS raid with provisioned IOPS, using Chef 10.26.0
> 
> First, node creates EBS raid type 10 with two disks without problems,  
> and then use it to hosts mongoDB databases.
> 
> Then, I reboot or stops the ec2 node.
> 
> When the node finished the boot process, EBS raid devices are  
> attached to the machine, but when I run chef-client, it fails with  
> message:
> 
> FATAL: ArgumentError: ruby\_block[wait\_mongodb\_data\_raid]  
> (/var/chef/cache/cookbooks/aws/providers/ebs\_raid.rb line 182) had an  
> error: ArgumentError: You tried to set a nested key, where the parent  
> is not a hash-like object:  
> aws/raid//var/lib/mongodb/device\_map/sdi1/sdi2/sdi1
> 
> Node's info seems right to me:
> 
> "aws": {  
> "raid": {  
> "/var/lib/mongodb": {  
> "raid\_dev": "md0",  
> "device\_map": {  
> "sdi1": "vol-6bba1647",  
> "sdi2": "vol-bbbb1797"  
> }  
> }  
> },  
> "ebs\_volume": {  
> "sdi1": {  
> "volume\_id": "vol-6bba1647"  
> },  
> "sdi2": {  
> "volume\_id": "vol-bbbb1797"  
> }  
> }  
> }
> 
> So, I tried with different aws cookbook versions: 0.101.3 0.101.2  
> 0.100. All fails in the same steps.
> 
> Any suggestions?  
> What does the ruby\_block look like? Also, if you can try it with Chef 11, the attribute system is reworked so you should not see such inscrutable errors.

--  
Daniel DeLeo