How can I find a way to easily speed up bootstrapping nodes?

Chef Infra (archive)
1

I have been struggling with speeding up bootstrapping nodes and it seems quite difficult to find an easy way.
First, I tried using Squid proxy but chef bootstrap uses https communication to download packages and my squid proxy doesn’t cache the contents with https communication. I have tried Squid to cache for https communication but it seems pretty complicating and not recommended by its design purpose.
So second, I have tried making local apt repo(since I usually work with Ubuntu) to speed up downloading the packages but it seems there is no option for ‘chef bootstrap’ to download chef packages from a specific URL.
Can anybody help me to solve this problem? Am I doing wrong or misunderstanding?
Please help.

Thank you,
Dongwon

2

Bootstrapping basically just involves copying a script over to the node and executing it. The script is not very complicated, actually; it just sets up /etc/chef and adds a basic configuration file and a validation key, installs chef and then runs it.

There are several options:

  • You can create your own bootstrapping script, based on the default one.
  • You can install the chef client before bootstrapping. The package manager (apt in your case) should recognize that the package is already installed, and not re-download it.

In my own system, I am taking the second approach. My systems are CentOS and specific to my site, so my script would be fairly useless for you, but the general idea might help.

My OS gets installed with kickstarter in an unattended install. I simply added installing chef to the kickstarter script. The kickstarter script also sets up the validation key and configuration file in /etc/chef, and then launches the initial chef run.

That’s pretty much all the bootstrapping process does, so my systems don’t really need to be bootstrapped at all; it happens automatically at OS install time.

Kevin Keane
Whom the IT Pros Call
The NetTech
http://www.4nettech.com
Our values: Privacy, Liberty, Justice
See https://www.4nettech.com/corp/the-nettech-values.html

3

I will also choose the second one.
Thank you so much!

4

Use packer.io to automate making new golden images on a schedule.

Packer includes a ‘chef-client’ and ‘chef-solo’ provisioner that will install chef for you, and optionally run any cookbooks.

Having chef preinstalled on the image, will save several minutes bootstrapping.

Here is a snippet on a windows packer image

    {
      "type": "chef-solo",
      "guest_os_type": "windows",
      "install_command": "powershell.exe -Command \". { iwr -useb https://omnitruck.chef.io/install.ps1 } | iex; install\"",
      "cookbook_paths": ["cookbooks"],
      "run_list": []
    },

On ubuntu images, we just use a simple bash script

    "type": "shell",
    "environment_vars": [
      "HOME_DIR=/home/vagrant",
      "http_proxy={{user `http_proxy`}}",
      "https_proxy={{user `https_proxy`}}",
      "no_proxy={{user `no_proxy`}}"
    ],
    "execute_command": "echo 'vagrant' | {{.Vars}} sudo -S -E sh -eux '{{.Path}}'",
    "scripts": [
      "scripts/ubuntu/common/install-chef.sh",
      "scripts/ubuntu/common/vagrant.sh",
      "scripts/ubuntu/common/cleanup.sh",
      "scripts/ubuntu/common/minimize.sh"
    ]

where install-chef.sh looks like:

#!/bin/bash
curl -L https://omnitruck.chef.io/install.sh | sudo bash
5

Thank you for the recommendation!
I will try that soon as well.