Customizing bootstrap

Hi Chefs!

Objective: Avoid going out to internet for downloading chef-client rpm everytime I bootstrap a machine.

Approach followed:

  1. Copied install.sh to locally hosted http server
  2. Edited chef-full.erb to point it to install.sh from my http server
  3. Edited install.sh to point chef-client rpm that is on my http server

Result:

techops@exaple:~/chefser12work/chef-repo$ sudo knife bootstrap 192.168.0.15 -sudo -x root -P wc5ack3 --distro test
Connecting to 1.1.1.1
1.1.1.1 --2014-11-26 15:41:43-- http://buildrepo.examplel.com/Adapters/chef-utils/install.sh
1.1.1.1 Resolving buildrepo.examplel.com… 2.2.2.2
1.1.1.1 Connecting to buildrepo.examplel.com|2.2.2.2|:80… connected.
1.1.1.1 HTTP request sent, awaiting response… 200 OK
1.1.1.1 Length: 16592 (16K) [application/x-sh]
1.1.1.1 Saving to: `STDOUT’
1.1.1.1
100%[======================================>] 16,592 --.-K/s in 0.02s
1.1.1.1
1.1.1.1 2014-11-26 15:41:43 (852 KB/s) - written to stdout [16592/16592]
1.1.1.1
1.1.1.1 Downloading Chef 11.16.2 for el…
1.1.1.1 downloading https://testing.exmaple.com/chef/metadata?v=11.16.2&prerelease=false&nightlies=false&p=el&pv=6&m=x86_64
1.1.1.1 to file /tmp/install.sh.1628/metadata.txt
1.1.1.1 trying wget…
1.1.1.1 ERROR 404
1.1.1.1 Unable to retrieve a valid package!
1.1.1.1 Please file a bug report at https://github.com/opscode/opscode-omnitruck/issues/new
1.1.1.1 Version: 11.16.2
1.1.1.1
1.1.1.1 Please detail your operating system type, version and any other relevant details
1.1.1.1 Metadata URL: https://testing.exmaple.com/chef/metadata?v=11.16.2&prerelease=false&nightlies=false&p=el&pv=6&m=x86_64
1.1.1.1 bash: line 89: chef-client: command not found

Regards,
PullaReddy
TechOps

Hi,

Can you show us your modified install.sh? The current install.sh
script makes use of a service called "omnitruck" [0] and it appears
that your script is still expecting that API to exist [1]. Namely, we
are expecting

https://testing.exmaple.com/chef/metadata?v=11.16.2&prerelease=false&nightlies=false&p=el&pv=6&m=x86_64

to return something like:

url     https://opscode-omnibus-packages.s3.amazonaws.com/el/6/x86_64/chef-11.16.2-1.el6.x86_64.rpm
md5     8731b6558009fc322f6469b415a759f8
sha256  dc7bc9d6084d29ffec67664fdae455406c3e7657c4aebcbef19f5a9ba459db37

where the url is the actual location of the package to download. To
host the packages yourself, you can either:

  1. Host the packages on your local server and then modify install.sh
    to simply point at the package and not need the omnitruck API
  2. Host the packages in a local package repository and modify
    install.sh to configure your system to use that repository

You could probably also run your own omnitruck:

GitHub - chef/omnitruck: Web service to automate the release of Omnibus artifacts

but I think the other two options are much easier.

I hope this info helps debug what might be going on.

Cheers,

Steven

[0] An omnitruck delivers the omnibuses.
[1] Omnitruck API
On Wed, Nov 26, 2014 at 10:16 AM, Malli Pulla Reddy
mpreddy@jamcracker.com wrote:

Hi Chefs!

Objective: Avoid going out to internet for downloading chef-client rpm
everytime I bootstrap a machine.

Approach followed:

  1. Copied install.sh to locally hosted http server

  2. Edited chef-full.erb to point it to install.sh from my http server

  3. Edited install.sh to point chef-client rpm that is on my http server

Result:

techops@exaple:~/chefser12work/chef-repo$ sudo knife bootstrap 192.168.0.15
-sudo -x root -P wc5ack3 --distro test

Connecting to 1.1.1.1

1.1.1.1 --2014-11-26 15:41:43--
http://buildrepo.examplel.com/Adapters/chef-utils/install.sh

1.1.1.1 Resolving buildrepo.examplel.com... 2.2.2.2

1.1.1.1 Connecting to buildrepo.examplel.com|2.2.2.2|:80... connected.

1.1.1.1 HTTP request sent, awaiting response... 200 OK

1.1.1.1 Length: 16592 (16K) [application/x-sh]

1.1.1.1 Saving to: `STDOUT'

1.1.1.1

100%[======================================>] 16,592 --.-K/s in 0.02s

1.1.1.1

1.1.1.1 2014-11-26 15:41:43 (852 KB/s) - written to stdout [16592/16592]

1.1.1.1

1.1.1.1 Downloading Chef 11.16.2 for el...

1.1.1.1 downloading
https://testing.exmaple.com/chef/metadata?v=11.16.2&prerelease=false&nightlies=false&p=el&pv=6&m=x86_64

1.1.1.1 to file /tmp/install.sh.1628/metadata.txt

1.1.1.1 trying wget...

1.1.1.1 ERROR 404

1.1.1.1 Unable to retrieve a valid package!

1.1.1.1 Please file a bug report at
Sign in to GitHub · GitHub

1.1.1.1 Version: 11.16.2

1.1.1.1

1.1.1.1 Please detail your operating system type, version and any other
relevant details

1.1.1.1 Metadata URL:
https://testing.exmaple.com/chef/metadata?v=11.16.2&prerelease=false&nightlies=false&p=el&pv=6&m=x86_64

1.1.1.1 bash: line 89: chef-client: command not found

Regards,

PullaReddy

TechOps

On Nov 26, 2014, at 5:16 AM, Malli Pulla Reddy mpreddy@jamcracker.com wrote:

Hi Chefs!

Objective: Avoid going out to internet for downloading chef-client rpm everytime I bootstrap a machine.

Approach followed:

  1. Copied install.sh to locally hosted http server
  2. Edited chef-full.erb to point it to install.sh from my http server
  3. Edited install.sh to point chef-client rpm that is on my http server

Why not just put the desired RPM on your locally-hosted HTTP server directly? Example:

https://github.com/juliandunn/chef-bootstrap-templates/blob/master/chef-full-nointernet.erb#L22-L38 https://github.com/juliandunn/chef-bootstrap-templates/blob/master/chef-full-nointernet.erb#L22-L38

  • Julian

[ Julian C. Dunn jdunn@aquezada.com * Sorry, I'm ]
[ WWW: Julian Dunn's Blog - Commentary on media, technology, and everything in between. * only Web 1.0 ]
[ gopher://sdf.org/1/users/keymaker/ * compliant! ]
[ PGP: 91B3 7A9D 683C 7C16 715F 442C 6065 D533 FDC2 05B9 ]

Hi Steven,

I made below changes in install.sh , in my local http server I'm uploaded chef-client rpm.

metadata_filename="$tmp_dir/metadata.txt"

case "$project" in
"chef")
metadata_url="https://testing.exmaple.com/chef/metadata?v=${version}&prerelease=${prerelease}&nightlies=${nightlies}&p=${platform}&pv=${platform_version}&m=${machine}"
;;
"angrychef")
metadata_url="https://www.opscode.com/chef/metadata-angrychef?v=${version}&prerelease=${prerelease}&nightlies=${nightlies}&p=${platform}&pv=${platform_version}&m=${machine}"
;;
"server")
metadata_url="https://www.opscode.com/chef/metadata-server?v=${version}&prerelease=${prerelease}&nightlies=${nightlies}&p=${platform}&pv=${platform_version}&m=${machine}"
;;
"chefdk")
metadata_url="https://www.opscode.com/chef/metadata-chefdk?v=${version}&prerelease=${prerelease}&nightlies=${nightlies}&p=${platform}&pv=${platform_version}&m=${machine}"
;;
"container")
metadata_url="https://www.opscode.com/chef/metadata-container?v=${version}&prerelease=${prerelease}&nightlies=${nightlies}&p=${platform}&pv=${platform_version}&m=${machine}"
;;
*)

-----Original Message-----
From: Steven Danna [mailto:steve@opscode.com]
Sent: Wednesday, November 26, 2014 7:28 PM
To: chef@lists.opscode.com
Cc: Rajesh Pappu
Subject: [chef] Re: customizing bootstrap

Hi,

Can you show us your modified install.sh? The current install.sh script makes use of a service called "omnitruck" [0] and it appears that your script is still expecting that API to exist [1]. Namely, we are expecting

https://testing.exmaple.com/chef/metadata?v=11.16.2&prerelease=false&nightlies=false&p=el&pv=6&m=x86_64

to return something like:

url     https://opscode-omnibus-packages.s3.amazonaws.com/el/6/x86_64/chef-11.16.2-1.el6.x86_64.rpm
md5     8731b6558009fc322f6469b415a759f8
sha256  dc7bc9d6084d29ffec67664fdae455406c3e7657c4aebcbef19f5a9ba459db37

where the url is the actual location of the package to download. To host the packages yourself, you can either:

  1. Host the packages on your local server and then modify install.sh to simply point at the package and not need the omnitruck API
  2. Host the packages in a local package repository and modify install.sh to configure your system to use that repository

You could probably also run your own omnitruck:

GitHub - chef/omnitruck: Web service to automate the release of Omnibus artifacts

but I think the other two options are much easier.

I hope this info helps debug what might be going on.

Cheers,

Steven

[0] An omnitruck delivers the omnibuses.
[1] Omnitruck API
On Wed, Nov 26, 2014 at 10:16 AM, Malli Pulla Reddy mpreddy@jamcracker.com wrote:

Hi Chefs!

Objective: Avoid going out to internet for downloading chef-client rpm
everytime I bootstrap a machine.

Approach followed:

  1. Copied install.sh to locally hosted http server

  2. Edited chef-full.erb to point it to install.sh from my http server

  3. Edited install.sh to point chef-client rpm that is on my http
    server

Result:

techops@exaple:~/chefser12work/chef-repo$ sudo knife bootstrap
192.168.0.15 -sudo -x root -P wc5ack3 --distro test

Connecting to 1.1.1.1

1.1.1.1 --2014-11-26 15:41:43--
http://buildrepo.examplel.com/Adapters/chef-utils/install.sh

1.1.1.1 Resolving buildrepo.examplel.com... 2.2.2.2

1.1.1.1 Connecting to buildrepo.examplel.com|2.2.2.2|:80... connected.

1.1.1.1 HTTP request sent, awaiting response... 200 OK

1.1.1.1 Length: 16592 (16K) [application/x-sh]

1.1.1.1 Saving to: `STDOUT'

1.1.1.1

100%[======================================>] 16,592 --.-K/s in 0.02s

1.1.1.1

1.1.1.1 2014-11-26 15:41:43 (852 KB/s) - written to stdout
[16592/16592]

1.1.1.1

1.1.1.1 Downloading Chef 11.16.2 for el...

1.1.1.1 downloading
https://testing.exmaple.com/chef/metadata?v=11.16.2&prerelease=false&n
ightlies=false&p=el&pv=6&m=x86_64

1.1.1.1 to file /tmp/install.sh.1628/metadata.txt

1.1.1.1 trying wget...

1.1.1.1 ERROR 404

1.1.1.1 Unable to retrieve a valid package!

1.1.1.1 Please file a bug report at
Sign in to GitHub · GitHub

1.1.1.1 Version: 11.16.2

1.1.1.1

1.1.1.1 Please detail your operating system type, version and any
other relevant details

1.1.1.1 Metadata URL:
https://testing.exmaple.com/chef/metadata?v=11.16.2&prerelease=false&n
ightlies=false&p=el&pv=6&m=x86_64

1.1.1.1 bash: line 89: chef-client: command not found

Regards,

PullaReddy

TechOps

On 11/26/14, 5:57 AM, Steven Danna wrote:

Hi,

Can you show us your modified install.sh? The current install.sh
script makes use of a service called "omnitruck" [0] and it appears
that your script is still expecting that API to exist [1]. Namely, we
are expecting

 https://testing.exmaple.com/chef/metadata?v=11.16.2&prerelease=false&nightlies=false&p=el&pv=6&m=x86_64

to return something like:

 url     https://opscode-omnibus-packages.s3.amazonaws.com/el/6/x86_64/chef-11.16.2-1.el6.x86_64.rpm
 md5     8731b6558009fc322f6469b415a759f8
 sha256  dc7bc9d6084d29ffec67664fdae455406c3e7657c4aebcbef19f5a9ba459db37

where the url is the actual location of the package to download. To
host the packages yourself, you can either:

  1. Host the packages on your local server and then modify install.sh
    to simply point at the package and not need the omnitruck API
  2. Host the packages in a local package repository and modify
    install.sh to configure your system to use that repository

I wouldn't use install.sh or omnitruck at all. Just download the RPM or
deb packages, set up your own internal repo, and modify bootstrap to
configure the repo and install the package. If you want to host
internal packages you'll want that internal repo anyway.

Hi,

Yes, I agree with this. In both of these options you can really just
s/install.sh/bootstrap script/.

Sincerely,

Steven

On Wed, Nov 26, 2014 at 6:02 PM, Lamont Granquist lamont@opscode.com wrote:

On 11/26/14, 5:57 AM, Steven Danna wrote:

Hi,

Can you show us your modified install.sh? The current install.sh
script makes use of a service called "omnitruck" [0] and it appears
that your script is still expecting that API to exist [1]. Namely, we
are expecting

https://testing.exmaple.com/chef/metadata?v=11.16.2&prerelease=false&nightlies=false&p=el&pv=6&m=x86_64

to return something like:

 url

https://opscode-omnibus-packages.s3.amazonaws.com/el/6/x86_64/chef-11.16.2-1.el6.x86_64.rpm
md5 8731b6558009fc322f6469b415a759f8
sha256
dc7bc9d6084d29ffec67664fdae455406c3e7657c4aebcbef19f5a9ba459db37

where the url is the actual location of the package to download. To
host the packages yourself, you can either:

  1. Host the packages on your local server and then modify install.sh
    to simply point at the package and not need the omnitruck API
  2. Host the packages in a local package repository and modify
    install.sh to configure your system to use that repository

I wouldn't use install.sh or omnitruck at all. Just download the RPM or deb
packages, set up your own internal repo, and modify bootstrap to configure
the repo and install the package. If you want to host internal packages
you'll want that internal repo anyway.