If I wanted to upload 100 chef git repos to 100 chef orgs could I kick off 100 knife commands, all running in parallel, without problems?
Unconstrained hypotheticals like this is hard to answer meaningfully on any given day but I’d say “Yes”, so long as the machine had threads and resources available. I feel like there is probably much more to this question so I won’t speculate much more.
I have a situation where I need to upload about 50 git repositories to roughly 150 chef orgs across 3 separate chef servers.
I wrote a small process to oversee this through automation, but it’s currently doing so serially. It takes a long time to say the least. When I try to parallelize it, I start receiving a bunch of errors. Below are some of the most common. The more knife processes I fork in parallel, the more I see the errors. I am limiting the number of knife processes to one per core.
- ERROR: cookbooks failed to write: HTTP error writing: 403 “Forbidden”
- ERROR: internal server error