Hello,
I’m wanting to build Hashicorp Nomad into a Habitat plan, but I’m running into an issue when building my plan. For reference the plan can be found here.
Stepping into the plan, the unzipped file is the file that is expected, but ldd and bash report “file not found”:
[1][default:/src:0]# build
: Loading /src/plan.sh
nomad: Plan loaded
nomad: Validating plan metadata
nomad: Using HAB_BIN=/hab/pkgs/core/hab/0.55.0/20180321220925/bin/hab for installs, signing, and hashing
nomad: hab-plan-build setup
nomad: Writing pre_build file
nomad: Resolving build dependencies
» Installing core/unzip from channel 'stable'
→ Using core/unzip/6.0/20180310001951
★ Install of core/unzip/6.0/20180310001951 complete with 0 new packages installed.
nomad: Resolved build dependency 'core/unzip' to /hab/pkgs/core/unzip/6.0/20180310001951
» Installing core/strace from channel 'stable'
→ Using core/strace/4.19/20171011085951
★ Install of core/strace/4.19/20171011085951 complete with 0 new packages installed.
nomad: Resolved build dependency 'core/strace' to /hab/pkgs/core/strace/4.19/20171011085951
nomad: Resolving run dependencies
» Installing core/linux from channel 'stable'
→ Using core/linux/4.11.1/20170807154945
★ Install of core/linux/4.11.1/20170807154945 complete with 0 new packages installed.
nomad: Resolved dependency 'core/linux' to /hab/pkgs/core/linux/4.11.1/20170807154945
» Installing core/glibc from channel 'stable'
→ Using core/glibc/2.22/20170513201042
★ Install of core/glibc/2.22/20170513201042 complete with 0 new packages installed.
nomad: Resolved dependency 'core/glibc' to /hab/pkgs/core/glibc/2.22/20170513201042
nomad: Setting up environment
nomad: Populating runtime environment from dependencies
nomad: WARN No RUNTIME_ENVIRONMENT metadata file found for core/linux/4.11.1/20170807154945; falling back to PATH metadata
nomad: WARN Final generated PATH for /hab/pkgs/core/linux/4.11.1/20170807154945: /hab/pkgs/core/glibc/2.22/20170513201042/bin
nomad: WARN No RUNTIME_ENVIRONMENT metadata file found for core/glibc/2.22/20170513201042; falling back to PATH metadata
nomad: WARN Final generated PATH for /hab/pkgs/core/glibc/2.22/20170513201042: /hab/pkgs/core/glibc/2.22/20170513201042/bin
nomad: Populating buildtime environment from dependencies
nomad: WARN No RUNTIME_ENVIRONMENT metadata file found for core/strace/4.19/20171011085951; falling back to PATH metadata
nomad: WARN Final generated PATH for /hab/pkgs/core/strace/4.19/20171011085951: /hab/pkgs/core/strace/4.19/20171011085951/bin:/hab/pkgs/core/glibc/2.22/20170513201042/bin
nomad: Layering runtime environment on top of system environment
nomad: Value of PATH is /hab/pkgs/core/hab-plan-build/0.55.0/20180321222204/bin:/hab/pkgs/core/xz/5.2.2/20170513214327/bin:/hab/pkgs/core/glibc/2.22/20170513201042/bin:/hab/pkgs/core/wget/1.19.1/20171024102323/bin:/hab/pkgs/core/openssl/1.0.2l/20171014213633/bin:/hab/pkgs/core/libidn/1.32/20170513215043/bin:/hab/pkgs/core/unzip/6.0/20180301000848/bin:/hab/pkgs/core/bzip2/1.0.6/20170513212938/bin:/hab/pkgs/core/tar/1.29/20170513213607/bin:/hab/pkgs/core/attr/2.4.47/20170513213059/bin:/hab/pkgs/core/acl/2.2.52/20170513213108/bin:/hab/pkgs/core/sed/4.2.2/20170513213123/bin:/hab/pkgs/core/rq/0.9.2/20170612005822/bin:/hab/pkgs/core/hab/0.55.0/20180321220925/bin:/hab/pkgs/core/gzip/1.6/20170513214605/bin:/hab/pkgs/core/pcre/8.38/20170513213423/bin:/hab/pkgs/core/ncurses/6.0/20170513213009/bin:/hab/pkgs/core/less/481/20170513213936/bin:/hab/pkgs/core/grep/2.22/20170513213444/bin:/hab/pkgs/core/gawk/4.1.3/20170513213646/bin:/hab/pkgs/core/findutils/4.4.2/20170513214305/bin:/hab/pkgs/core/file/5.24/20170513201915/bin:/hab/pkgs/core/coreutils/8.25/20170513213226/bin:/hab/pkgs/core/libcap/2.24/20170513213120/bin:/hab/pkgs/core/binutils/2.25.1/20170513201927/bin:/hab/pkgs/core/bash/4.3.42/20170513213519/bin
nomad: Layering buildtime environment on top of system environment
nomad: Found previous file 'nomad-0.8.3_linux_amd64.zip', attempting to re-use
nomad: Verifying nomad-0.8.3_linux_amd64.zip
nomad: Checksum verified for nomad-0.8.3_linux_amd64.zip
nomad: Using cached and verified 'nomad-0.8.3_linux_amd64.zip'
nomad: Verifying nomad-0.8.3_linux_amd64.zip
nomad: Checksum verified for nomad-0.8.3_linux_amd64.zip
nomad: Clean the cache
Archive: nomad-0.8.3_linux_amd64.zip
inflating: nomad-0.8.3/nomad
### Attaching to debugging session
From: /src/plan.sh @ line 38 :
28: # pkg_exposes=(port ssl-port)
29:
30: pkg_binds_optional=(
31: [consul]="host"
32: )
33:
34: # unzip our package into ${pkg_name}-${pkg_version} instead of just ${pkg_name}
35: do_unpack() {
36: cd "${HAB_CACHE_SRC_PATH}" || exit
37: unzip ${pkg_filename} -d "${pkg_name}-${pkg_version}"
=> 38: attach
39: }
40:
41:
42: # Since we have a precompiled binary, skip trying to compile it
43: do_build() {
44: return 0
45: }
46:
47: do_install() {
48: attach
[1] nomad(do_unpack)> ls
nomad-0.8.3 nomad-0.8.3_linux_amd64.zip
[2] nomad(do_unpack)> ./nomad-0.8.3/nomad
/hab/pkgs/core/hab-plan-build/0.55.0/20180321222204/bin/public.bash: line 240: ./nomad-0.8.3/nomad: No such file or directory
[3] nomad(do_unpack)> stat ./nomad-0.8.3/nomad
File: './nomad-0.8.3/nomad'
Size: 44838240 Blocks: 87576 IO Block: 4096 regular file
Device: fd00h/64768d Inode: 1463464 Links: 1
Access: (0755/-rwxr-xr-x) Uid: ( 0/ root) Gid: ( 0/ root)
Access: 2018-05-27 00:14:42.700624230 +0000
Modify: 2018-04-27 17:55:55.000000000 +0000
Change: 2018-05-27 00:14:31.153624043 +0000
Birth: -
[4] nomad(do_unpack)> file ./nomad-0.8.3/nomad
./nomad-0.8.3/nomad: ELF 64-bit LSB executable, x86-64, version 1 (SYSV), dynamically linked, interpreter /lib64/ld-linux-x86-64.so.2, for GNU/Linux 2.6.32, BuildID[sha1]=c55b7047bd313893aba2e9a880f1b3d8a89236a8, not stripped
[5] nomad(do_unpack)> ldd ./nomad-0.8.3/nomad
/hab/pkgs/core/glibc/2.22/20170513201042/bin/ldd: line 117: ./nomad-0.8.3/nomad: No such file or directory
[6] nomad(do_unpack)> sha256sum nomad-0.8.3/nomad
cab4e307bfd8d79091a503857ecde61ccc4ef0ecc54b88e8f9aa880a4f777720 nomad-0.8.3/nomad
Completing the build:
nomad: hab-plan-build cleanup
nomad:
nomad: Source Path: /hab/cache/src/nomad-0.8.3
nomad: Installed Path: /hab/pkgs/qubitrenegade/nomad/0.8.3/20180527001426
nomad: Artifact: /src/results/qubitrenegade-nomad-0.8.3-20180527001426-x86_64-linux.hart
nomad: Build Report: /src/results/last_build.env
nomad: SHA256 Checksum: ab84ffd66f889b524fbc9d62acce72d59ca6bece53528e4cfc1a1cd1e233d92c
nomad: Blake2b Checksum: da393dfb0bc27128de40c12ecf82d3720d0654a1e209ec804080a6860b8cfa9a
nomad:
nomad: I love it when a plan.sh comes together.
nomad:
nomad: Build time: 3m43s
and installing the resulting package allows me to execute it “natively”:
$ cat results/last_build.env
pkg_origin=qubitrenegade
pkg_name=nomad
pkg_version=0.8.3
pkg_release=20180527001426
pkg_ident=qubitrenegade/nomad/0.8.3/20180527001426
pkg_artifact=qubitrenegade-nomad-0.8.3-20180527001426-x86_64-linux.hart
pkg_sha256sum=ab84ffd66f889b524fbc9d62acce72d59ca6bece53528e4cfc1a1cd1e233d92c
pkg_blake2bsum=da393dfb0bc27128de40c12ecf82d3720d0654a1e209ec804080a6860b8cfa9a
$ source results/last_build.env
$ sudo hab pkg install results/$pkg_artifact
☛ Verifying qubitrenegade/nomad/0.8.3/20180527001426
→ Using core/glibc/2.22/20170513201042
→ Using core/linux-headers/4.3/20170513200956
→ Using core/linux/4.11.1/20170807154945
✓ Installed qubitrenegade/nomad/0.8.3/20180527001426
★ Install of qubitrenegade/nomad/0.8.3/20180527001426 complete with 1 new packages installed.
$ /hab/pkgs/$pkg_ident/bin/nomad -version
Nomad v0.8.3 (c85483da3471f4bd3a7c3de112e95f551071769f)
$ file /hab/pkgs/$pkg_ident/bin/nomad
/hab/pkgs/qubitrenegade/nomad/0.8.3/20180527001426/bin/nomad: ELF 64-bit LSB executable, x86-64, version 1 (SYSV), dynamically linked, interpreter /lib64/ld-linux-x86-64.so.2, for GNU/Linux 2.6.32, BuildID[sha1]=c55b7047bd313893aba2e9a880f1b3d8a89236a8, stripped
$ stat /hab/pkgs/$pkg_ident/bin/nomad
File: /hab/pkgs/qubitrenegade/nomad/0.8.3/20180527001426/bin/nomad
Size: 27963864 Blocks: 54624 IO Block: 4096 regular file
Device: fd00h/64768d Inode: 1245079 Links: 1
Access: (0755/-rwxr-xr-x) Uid: ( 0/ root) Gid: ( 0/ root)
Context: unconfined_u:object_r:default_t:s0
Access: 2018-05-26 18:23:38.517632870 -0600
Modify: 2018-05-26 18:17:47.000000000 -0600
Change: 2018-05-26 18:23:23.381632626 -0600
Birth: -
$ ldd /hab/pkgs/$pkg_ident/bin/nomad
linux-vdso.so.1 (0x00007ffe9f9b8000)
libpthread.so.0 => /lib64/libpthread.so.0 (0x00007f285e044000)
libc.so.6 => /lib64/libc.so.6 (0x00007f285dc8e000)
/lib64/ld-linux-x86-64.so.2 (0x00007f285e262000)
$ sha256sum /hab/pkgs/$pkg_ident/bin/nomad
474ee8364a368481297b628a7c6b480a7e22e380bf07697a24a55f0fe5b78beb /hab/pkgs/qubitrenegade/nomad/0.8.3/20180527001426/bin/nomad
Interestingly the nomad binary seems to have a different sha256sum when in the hab studio?
This also extends to running in docker:
$ cat results/last_docker_export.env
id=b925dab81524
name=qubitrenegade/nomad
tags=0.8.3-20180527001426,0.8.3,latest
name_tags=qubitrenegade/nomad:0.8.3-20180527001426,qubitrenegade/nomad:0.8.3,qubitrenegade/nomad:latest
$ source results/last_docker_export.env
$ docker run $name
hab-sup(MR): Supervisor Member-ID 07dd3a22398c416ba260dfd1cc7da89d
hab-sup(MR): Starting qubitrenegade/nomad
nomad.default(UCW): Watching user.toml
hab-sup(MR): Starting gossip-listener on 0.0.0.0:9638
hab-sup(MR): Starting http-gateway on 0.0.0.0:9631
nomad.default(HK): run, compiled to /hab/svc/nomad/hooks/run
nomad.default(HK): Hooks compiled
nomad.default(SR): Hooks recompiled
default(CF): Updated server.hcl 497f08225e5cd30e481b21ece767c4d09d0eaafc12640bcbe563a8d3a9e04cb6
nomad.default(SR): Configuration recompiled
nomad.default(SR): Initializing
nomad.default(SV): Starting service as user=root, group=root
nomad.default(O): + exec nomad agent -config-file=/hab/svc/nomad/config/server.hcl
nomad.default(O): /hab/svc/nomad/run: exec: line 1: nomad: not found
nomad.default(E): + exec
hab-launch(SV): Child for service 'nomad.default' with PID 153 exited with code exit code: 2
...
$ docker exec -it inspiring_aryabhata /bin/bash
/ # which nomad
/bin/nomad
# stat /bin/nomad
File: '/bin/nomad' -> '/hab/pkgs/qubitrenegade/nomad/0.8.3/20180527001426/bin/nomad'
Size: 60 Blocks: 8 IO Block: 4096 symbolic link
Device: 9ch/156d Inode: 2021518 Links: 1
Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root)
Access: 2018-05-27 00:29:54.000000000
Modify: 2018-05-27 00:29:54.000000000
Change: 2018-05-27 00:29:58.000000000
/ # file /bin/nomad
/bin/bash: file: not found
/ # ldd /bin/nomad
/bin/ldd: eval: line 1: /bin/nomad: not found
I’m kinda stumped as there’s nothing obvious to me that would indicate that the file doesn’t actually exist…
Could it be one of the shared libraries that’s missing? Most of the other Hashicorp products are static binaries, so I’m wondering if that has something to do with it.