Re: Re: Berks upload seems very slow

Chef Infra (archive)
1

Good call but my .kitchen folder is 1 meg, so I don't think that is it. All
of terida-base is 1.5 megs.

On Thu Dec 11 2014 at 2:45:32 PM Peter Burkholder pburkholder@getchef.com
wrote:

Make sure you don't have any 'huge' files that are being uploaded. You may
need a 'chefignore' file to ignore .kitchen/ if you're doing KitchenCI.
E.g. set 'terrida-base/chefignore' to include a line with '.kitchen'.

--Peter

On Thu, Dec 11, 2014 at 1:07 PM, Jeff MacDonald <
mr.jeffmacdonald@gmail.com> wrote:

Hi,

I'm uploading from a MacBook with 16 gigs of ram, to a VM running chef
server with 4 gigs of ram. 30mbit/s upload speed.

To do a berks upload of about 6 cookbooks takes over 3 minutes. Seems
kind of slow to me, but I don't understand all of its inner workings.

Just looking for some feedback to understand this or make it better..

time berks upload --force
Uploaded apt (2.6.0) to: 'https://chef:443/organizations/terida'
Uploaded build-essential (2.1.3) to: 'https://chef:443/organizations/terida'
Uploaded mongodb (0.16.2) to: 'https://chef:443/organizations/terida'
Uploaded python (1.4.6) to: 'https://chef:443/organizations/terida'
Uploaded runit (1.5.10) to: 'https://chef:443/organizations/terida'
Uploaded terida-base (0.1.0) to: 'https://chef:443/organizations/terida'
Uploaded yum (3.5.1) to: 'https://chef:443/organizations/terida'
Uploaded yum-epel (0.5.3) to: 'https://chef:443/organizations/terida'

real 2m58.414s
user 0m7.241s
sys 0m1.433s

2

Hi Jeff,

Here are some debugging ideas:

  1. use berks debug logging (-d). It should print timestamps, use
    those to figure out what parts of the upload are slow.
  2. Point berks at a local chef-zero instance. This will be uploading
    the cookbooks to memory, it should be fast, if it is slow, you can
    rule out your chef server.
  3. Take a look at the server logs (start with
    /var/log/opscode/nginx/access.log or /var/log/chef-server/access.log)
    and see how long your chef-server thinks each request is taking.
    Cookbook uploads involve a number of different HTTP requests, however,
    they all flow through nginx in your chef-server. The nginx logs
    lines look something like

1.2.3.4 - - [11/Aug/2014:11:06:18 +0100] "PUT
/bookshelf/organization-baed8c... HTTP/1.1" 204 "0.066" 0 "-" "Chef
Knife/11.14.2 (ruby-1.9.3-p484; ohai-7.2.0; x86_64-linux;
+http://opscode.com)" "127.0.0.1:4321" "204" "0.064" "11.14.2"
"algorithm=sha1;version=1.0;" "myuser" "2014-08-11T10:06:17Z"
"6NQz7YnDiWTasa1jtj0CCLAECMU=" 4315

The bit after HTTP/1.1 is the response times in miliseconds.

I would start with (1). If you post the full debug output to a gist
or something similar, I'm sure many on this list would be happy to
take a look.

Cheers,

Steven

On Fri, Dec 12, 2014 at 12:46 PM, Jeff MacDonald
mr.jeffmacdonald@gmail.com wrote:

Good call but my .kitchen folder is 1 meg, so I don't think that is it. All
of terida-base is 1.5 megs.

On Thu Dec 11 2014 at 2:45:32 PM Peter Burkholder pburkholder@getchef.com
wrote:

Make sure you don't have any 'huge' files that are being uploaded. You may
need a 'chefignore' file to ignore .kitchen/ if you're doing KitchenCI.
E.g. set 'terrida-base/chefignore' to include a line with '.kitchen'.

--Peter

On Thu, Dec 11, 2014 at 1:07 PM, Jeff MacDonald
mr.jeffmacdonald@gmail.com wrote:

Hi,

I'm uploading from a MacBook with 16 gigs of ram, to a VM running chef
server with 4 gigs of ram. 30mbit/s upload speed.

To do a berks upload of about 6 cookbooks takes over 3 minutes. Seems
kind of slow to me, but I don't understand all of its inner workings.

Just looking for some feedback to understand this or make it better..

time berks upload --force
Uploaded apt (2.6.0) to: 'https://chef:443/organizations/terida'
Uploaded build-essential (2.1.3) to:
'https://chef:443/organizations/terida'
Uploaded mongodb (0.16.2) to: 'https://chef:443/organizations/terida'
Uploaded python (1.4.6) to: 'https://chef:443/organizations/terida'
Uploaded runit (1.5.10) to: 'https://chef:443/organizations/terida'
Uploaded terida-base (0.1.0) to: 'https://chef:443/organizations/terida'
Uploaded yum (3.5.1) to: 'https://chef:443/organizations/terida'
Uploaded yum-epel (0.5.3) to: 'https://chef:443/organizations/terida'

real 2m58.414s
user 0m7.241s
sys 0m1.433s