Daniel, doing some tcpdump in the chef server…
RST sent from server
07:30:35.013370 IP chef.server.https > mynode.49776: Flags [R], seq
2457752489, win 0, length 0
RST from client
07:30:35.472340 IP mynode.49779 >chef.server.https: Flags [R], seq
3250774265, win 0, length 0
So im a little bit blind here…
This is the chef-server-ctl tail output when chef-client it is executed…
==> /var/log/chef-server/nginx/access.log <==
mynode - - [16/May/2014:07:35:34 +0000] "GET /nodes/mynode HTTP/1.1"
200 “0.007” 15767 “-” “Chef Client/11.12.4 (ruby-1.9.3-p484;
ohai-7.0.4; i386-mingw32; +http://opscode.com)” “127.0.0.1:8000” "200"
“0.007” “11.12.4” “algorithm=sha1;version=1.0;” “mynode”
“2014-05-16T07:35:38Z” “2jmj7l5rSw0yVb/vlWAYkK/YBwk=” 999
mynode - - [16/May/2014:07:35:34 +0000] “POST
/reports/nodes/mynode/runs HTTP/1.1” 404 “0.044” 173 “-” “Chef
Client/11.12.4 (ruby-1.9.3-p484; ohai-7.0.4; i386-mingw32;
+http://opscode.com)” “127.0.0.1:8000” “404” “0.001” "11.12.4"
“algorithm=sha1;version=1.0;” “mynode” “2014-05-16T07:35:38Z”
“XemsHTvp8BHcocDMY1kiYRTf0dg=” 1214
mynode - - [16/May/2014:07:35:34 +0000] “POST
/environments/_default/cookbook_versions HTTP/1.1” 200 “0.056” 864 “-”
“Chef Client/11.12.4 (ruby-1.9.3-p484; ohai-7.0.4; i386-mingw32;
+http://opscode.com)” “127.0.0.1:8000” “200” “0.009” "11.12.4"
“algorithm=sha1;version=1.0;” “mynode” “2014-05-16T07:35:39Z”
“oxopNq5HHhVWRiVIxCseXf3iHmU=” 1107
mynode - - [16/May/2014:07:35:35 +0000] "PUT /nodes/mynode HTTP/1.1"
200 “0.067” 15758 “-” “Chef Client/11.12.4 (ruby-1.9.3-p484;
ohai-7.0.4; i386-mingw32; +http://opscode.com)” “127.0.0.1:8000” "200"
“0.041” “11.12.4” “algorithm=sha1;version=1.0;” “mynode”
“2014-05-16T07:35:39Z” “AI4nEjYvOw1tAkONEqnxBehhvu8=” 124188
==> /var/log/chef-server/chef-solr/current <==
2014-05-16_07:35:35.25015 May 16, 2014 7:35:35 AM
org.apache.solr.core.SolrDeletionPolicy onInit
2014-05-16_07:35:35.25018 INFO: SolrDeletionPolicy.onInit: commits:num=1
2014-05-16_07:35:35.25018
commit{dir=/var/opt/chef-server/chef-solr/data/index,segFN=segments_13,version=1399982839679,generation=39,filenames=[_9.tis,
_7_1.del, _8.tis, _v.fdx, _q.prx, _d.tii, _9.tii, _3.fdt, _c.tii,
_k.fnm, _7.fnm, _7.tis, _4.frq, _c.tis, _h.fdt, _10.tii, _y.nrm,
_h.fdx, _k_1.del, _d.tis, _w.fnm, _u.tii, _10.tis, _a.prx, _g.fdt,
_2.prx, _g.fdx, _u.tis, _5_2.del, _7.tii, _8.frq, _9.fdt, _2.frq,
_h.frq, _9.fdx, _1.fdx, _10.prx, _8.tii, _g.nrm, _1.fdt, _0.frq,
_6.fdt, _t.fdt, _a.fnm, _2.nrm, _0_4.del, _r.nrm, _y.tis, _6.fdx,
_u.fdx, _t.frq, _y.tii, _r_1.del, _6.tis, _x_1.del, _r.fdt, _m.tis,
_w_1.del, _r.fdx, _p.tii, _v.prx, _i.fdx, _m.tii, _y.fnm, _i.fdt,
_2_2.del, _g.prx, _x.fnm, _p.tis, _c_3.del, _z.prx, _i.frq, _10.frq,
_6.tii, _4.nrm, _l_1.del, _5.frq, _g.frq, _2.fnm, _3.fdx, _b.frq,
_x.prx, _a.nrm, _f.tis, _0.tis, _g.tis, _p.fdx, _5.prx, _p.fdt,
_m.prx, _4.prx, _c.fdx, _4.fdx, _c.fdt, _i.tis, _o.prx, _q_1.del,
_p.nrm, _5.nrm, _t_1.del, _g.tii, _7.frq, _i.tii, _s.nrm, _f.tii,
_m.fnm, _e.fnm, _o.fnm, _h.fnm, _j_1.del, _h.tis, _j.tis, _u.nrm,
_1_2.del, _0.tii, _h.tii, _j.tii, _u.fdt, _q.fnm, _6.nrm, _h_1.del,
_z.fdt, _x.nrm, _3.tii, _t.fnm, _z.fdx, _6.frq, _u.frq, _s.fnm,
_a.fdt, _b.nrm, _e.tis, _w.prx, _t.prx, _a.fdx, _s.tii, _e.tii,
_l.tis, _l.tii, _n.nrm, _10.fnm, _5.fnm, _n.prx, _b.prx, _x.frq,
_4.fdt, _b.tis, _b.tii, _8.prx, _s.tis, _f_1.del, _6.prx, _b.fnm,
_b.fdt, _8.fdx, _b.fdx, _8.fdt, _3.tis, _s.fdt, _z.frq, _1.frq,
_a.tii, _m.frq, _e.frq, _2.tii, _y.fdt, _v_2.del, _7.nrm, _y.fdx,
_n.frq, _e_1.del, _q.nrm, _c.nrm, _v.fnm, _g_1.del, _x.tis, _o.frq,
_a.tis, _l.nrm, _d.frq, _w.fdt, _9.prx, _l.prx, _w.fdx, _q.frq,
_c.prx, _2.fdt, _7.fdx, _x.tii, _6_1.del, _2.fdx, _z.nrm, _7.fdt,
_f.fdt, _2.tis, _f.fdx, _l.fdx, segments_13, _l.fdt, _c.frq, _j.fdx,
_x.fdt, _d_1.del, _j.fdt, _x.fdx, _k.prx, _l.frq, _3.fnm, _e.fdx,
_y_1.del, _p.frq, _v.tis, _n.tis, _r.fnm, _o_1.del, _u.fnm, _k.nrm,
_e.fdt, _j.nrm, _q.tis, _n.tii, _1.tii, _j.prx, _e.nrm, _o.fdt,
_q.tii, _1.tis, _o.fdx, _s_1.del, _n.fdt, _9.fnm, _3_2.del, _n.fdx,
_m.nrm, _y.prx, _v.nrm, _d.prx, _r.prx, _d.fdx, _8.nrm, _d.fdt,
_o.nrm, _m.fdt, _v.tii, _m.fdx, _d.nrm, _6.fnm, _c.fnm, _j.frq,
_f.frq, _p.prx, _h.prx, _5.fdt, _r.tii, _a_1.del, _q.fdx, _4.fnm,
_z.fnm, _q.fdt, _o.tis, _1.fnm, _h.nrm, _8_1.del, _k.tii, _v.frq,
_5.tis, _w.tii, _9_1.del, _t.tis, _4_1.del, _0.prx, _n_1.del, _n.fnm,
_k.tis, _w.tis, _3.nrm, _f.nrm, _w.frq, _0.fnm, _t.tii, _a.frq,
_s.prx, _d.fnm, _3.prx, _s.frq, _5.tii, _1.prx, _8.fnm, _f.prx,
_u.prx, _o.tii, _t.nrm, _e.prx, _r.tis, _1.nrm, _i.prx, _i.nrm,
_3.frq, _10.fdt, _z.tii, _u_1.del, _l.fnm, _p.fnm, _9.nrm, _k.fdt,
_y.frq, _z.tis, _k.fdx, _m_1.del, _p_1.del, _10.fdx, _g.fnm, _z_1.del,
_w.nrm, _r.frq, _4.tis, _0.nrm, _4.tii, _v.fdt, _9.frq, _7.prx,
_s.fdx, _f.fnm, _j.fnm, _t.fdx, _5.fdx, _0.fdx, _10.nrm, _k.frq,
_i.fnm, _0.fdt]
2014-05-16_07:35:35.25077 May 16, 2014 7:35:35 AM
org.apache.solr.core.SolrDeletionPolicy updateCommits
2014-05-16_07:35:35.25078 INFO: newest commit = 1399982839679
2014-05-16_07:35:35.26603 May 16, 2014 7:35:35 AM
org.apache.solr.update.processor.LogUpdateProcessor finish
2014-05-16_07:35:35.26606 INFO: {add=[0000000000007c1093a019810992da7a]} 0 31
2014-05-16_07:35:35.26648 May 16, 2014 7:35:35 AM
org.apache.solr.core.SolrCore execute
2014-05-16_07:35:35.26650 INFO: [] webapp=/solr path=/update params={}
status=0 QTime=31
==> /var/log/chef-server/chef-expander/current <==
2014-05-16_07:35:35.26821 [Fri, 16 May 2014 07:35:35 +0000] INFO:
indexed node[0000000000007c1093a019810992da7a]
database[chef_00000000000000000000000000000000] transit,xml,solr-post
| 0,0.10456681251525879,0.14017462730407715 |
==> /var/log/chef-server/erchef/erchef.log.1 <==
2014-05-16T07:35:34Z erchef@127.0.0.1 INFO
req_id=I1u0R4xTNbOQrNCbQ19eCg==; status=200; method=GET;
path=/nodes/mynode; user=mynode; msg=[]; req_time=5; rdbms_time=0;
rdbms_count=2
2014-05-16T07:35:34Z erchef@127.0.0.1 INFO
req_id=BUGbBDUTtkSFO5WwvomZGg==; status=200; method=POST;
path=/environments/_default/cookbook_versions; user=mynode; msg=[];
req_time=7; rdbms_time=2; rdbms_count=4
2014-05-16T07:35:35Z erchef@127.0.0.1 INFO
req_id=DiEmZOzphC/ivnnOmhpvwg==; status=200; method=PUT;
path=/nodes/mynode; user=mynode; msg=[]; req_time=39; rdbms_time=13;
rdbms_count=3
On Fri, May 16, 2014 at 8:58 AM, Carlos Camacho
carlos.camacho@frontiersin.org wrote:
Daniel, i have the default iptables rules from rackspace.
It is a very long list and i jut run this to clean it a little bit.
iptables --policy INPUT ACCEPT
iptables --policy OUTPUT ACCEPT
iptables --policy FORWARD ACCEPT
iptables -F INPUT
iptables -F OUTPUT
iptables -F FORWARD
iptables -F RS-RackConnect-INBOUND
iptables -X RS-RackConnect-INBOUND
Now the output is this.
Chain INPUT (policy ACCEPT)
target prot opt source destination
Chain FORWARD (policy ACCEPT)
target prot opt source destination
Chain OUTPUT (policy ACCEPT)
target prot opt source destination
Also in windows i disabled the windows firewall (It is important to
tell that this problem only happens with Windows nodes).
Now the chef-client output changed a little bit…
This is now…
[2014-05-16T06:53:30+00:00] INFO: *** Chef 11.12.4 ***
[2014-05-16T06:53:30+00:00] INFO: Chef-client pid: 3924
[2014-05-16T06:53:37+00:00] INFO: Run List is [recipe[frontiers_test]]
[2014-05-16T06:53:37+00:00] INFO: Run List expands to [frontiers_test]
[2014-05-16T06:53:37+00:00] INFO: Starting Chef Run for slwnone09
[2014-05-16T06:53:37+00:00] INFO: Running start handlers
[2014-05-16T06:53:37+00:00] INFO: Start handlers complete.
[2014-05-16T06:53:37+00:00] INFO: HTTP Request Returned 404 Object Not Found:
resolving cookbooks for run list: [“frontiers_test”]
[2014-05-16T06:53:38+00:00] INFO: Loading cookbooks [cookbook_test@0.1.0]
Synchronizing Cookbooks:
- cookbook_test
Compiling Cookbooks…
Converging 1 resources
Recipe:cookbook_test::default
- directory[C:\hello_test] action create[2014-05-16T06:53:38+00:00]
INFO: Processing directory[C:\hello_test
] action create (cookbook_test::default line 10)
(up to date)
Running handlers:
[2014-05-16T06:53:38+00:00] ERROR: Running exception handlers
Running handlers complete
[2014-05-16T06:53:38+00:00] ERROR: Exception handlers complete
[2014-05-16T06:53:38+00:00] FATAL: Stacktrace dumped to
c:/chef/cache/chef-stacktrace.out
Chef Client failed. 0 resources updated in 10.324711 seconds
[2014-05-16T06:53:38+00:00] FATAL: EOFError: end of file reached
PS C:\Users\Administrator>
And before the output was…
Starting Chef Client, version 11.12.4
[2014-05-14T14:02:26+00:00] INFO: *** Chef 11.12.4 ***
[2014-05-14T14:02:26+00:00] INFO: Chef-client pid: 2936
================================================================================
Chef encountered an error attempting to load the node data for “mynode”
Unexpected Error:
EOFError: end of file reached
[2014-05-14T14:02:34+00:00] FATAL: Stacktrace dumped to
c:/chef/cache/chef-stacktrace.out
Chef Client failed. 0 resources updated in 10.154057 seconds
[2014-05-14T14:02:34+00:00] FATAL: EOFError: end of file reached
So, it is really annoying not to have any help from rackspace, they
only tells me, it is a server problem, not network issue…
Daniel, if you have any idea im open minded!
Cheers!
On Thu, May 15, 2014 at 7:08 PM, Daniel DeLeo dan@kallistec.com wrote:
On Thursday, May 15, 2014 at 10:06 AM, Carlos Camacho wrote:
Daniel! Thanks for your answer,
The RST package it is being send from the chef server, but i don’t
know why, or the service which is failing.
Can you recommend me things to test?!?!
Thanks!
Is there a firewall on the server itself?
–
Daniel DeLeo
