Knife -- safe to run multiple times in parallel on a single host?


#1

If I wanted to upload 100 chef git repos to 100 chef orgs could I kick off 100 knife commands, all running in parallel, without problems?


#2

Unconstrained hypotheticals like this is hard to answer meaningfully on any given day but I’d say “Yes”, so long as the machine had threads and resources available. I feel like there is probably much more to this question so I won’t speculate much more.


#3

I have a situation where I need to upload about 50 git repositories to roughly 150 chef orgs across 3 separate chef servers.

I wrote a small process to oversee this through automation, but it’s currently doing so serially. It takes a long time to say the least. When I try to parallelize it, I start receiving a bunch of errors. Below are some of the most common. The more knife processes I fork in parallel, the more I see the errors. I am limiting the number of knife processes to one per core.

  • ERROR: cookbooks failed to write: HTTP error writing: 403 “Forbidden”
  • ERROR: internal server error