Issue compiling with gfortran/gcc

mike10010100 · August 21, 2018, 9:23pm

Hi all,

When trying to compile OpenBLAS, I ran into this issue:

gfortran -pthread -O2 -Wall -m64 -fPIC -lm -lpthread -lgfortran -o testslamch slamch.o lsame.o slamchtst.o
gfortran -pthread -O2 -Wall -m64 -fPIC -lm -lpthread -lgfortran -o testdlamch dlamch.o lsame.o dlamchtst.o
/hab/pkgs/core/binutils/2.30/20180608050633/bin/ld.bfd.real: cannot find crt1.o: No such file or directory
/hab/pkgs/core/binutils/2.30/20180608050633/bin/ld.bfd.real: cannot find crti.o: No such file or directory
/hab/pkgs/core/binutils/2.30/20180608050633/bin/ld.bfd.real: cannot find -lm
/hab/pkgs/core/binutils/2.30/20180608050633/bin/ld.bfd.real: cannot find -lpthread
/hab/pkgs/core/binutils/2.30/20180608050633/bin/ld.bfd.real: cannot find -lm
/hab/pkgs/core/binutils/2.30/20180608050633/bin/ld.bfd.real: cannot find -lm
collect2: error: ld returned 1 exit status
make[2]: *** [Makefile:13: testdlamch] Error 1
make[2]: *** Waiting for unfinished jobs....
/hab/pkgs/core/binutils/2.30/20180608050633/bin/ld.bfd.real: cannot find crt1.o: No such file or directory
/hab/pkgs/core/binutils/2.30/20180608050633/bin/ld.bfd.real: cannot find crti.o: No such file or directory
/hab/pkgs/core/binutils/2.30/20180608050633/bin/ld.bfd.real: cannot find -lm
/hab/pkgs/core/binutils/2.30/20180608050633/bin/ld.bfd.real: cannot find -lpthread
/hab/pkgs/core/binutils/2.30/20180608050633/bin/ld.bfd.real: cannot find -lm
/hab/pkgs/core/binutils/2.30/20180608050633/bin/ld.bfd.real: cannot find -lm
collect2: error: ld returned 1 exit status
make[2]: *** [Makefile:7: testlsame] Error 1
/hab/pkgs/core/binutils/2.30/20180608050633/bin/ld.bfd.real: cannot find crt1.o: No such file or directory
/hab/pkgs/core/binutils/2.30/20180608050633/bin/ld.bfd.real: cannot find crti.o: No such file or directory
/hab/pkgs/core/binutils/2.30/20180608050633/bin/ld.bfd.real: cannot find -lm
/hab/pkgs/core/binutils/2.30/20180608050633/bin/ld.bfd.real: cannot find -lpthread
/hab/pkgs/core/binutils/2.30/20180608050633/bin/ld.bfd.real: cannot find -lm
/hab/pkgs/core/binutils/2.30/20180608050633/bin/ld.bfd.real: cannot find -lm
collect2: error: ld returned 1 exit status
make[2]: *** [Makefile:10: testslamch] Error 1
/hab/pkgs/core/binutils/2.30/20180608050633/bin/ld.bfd.real: cannot find crt1.o: No such file or directory
/hab/pkgs/core/binutils/2.30/20180608050633/bin/ld.bfd.real: cannot find crti.o: No such file or directory
/hab/pkgs/core/binutils/2.30/20180608050633/bin/ld.bfd.real: cannot find -lm
/hab/pkgs/core/binutils/2.30/20180608050633/bin/ld.bfd.real: cannot find -lpthread
/hab/pkgs/core/binutils/2.30/20180608050633/bin/ld.bfd.real: cannot find -lm
/hab/pkgs/core/binutils/2.30/20180608050633/bin/ld.bfd.real: cannot find -lm
collect2: error: ld returned 1 exit status
make[2]: *** [Makefile:17: testsecond] Error 1
make[2]: Leaving directory '/hab/cache/src/OpenBLAS-0.2.20/lapack-netlib/INSTALL'
make[1]: *** [Makefile:17: lapack_install] Error 2
make[1]: Leaving directory '/hab/cache/src/OpenBLAS-0.2.20/lapack-netlib'
make: *** [Makefile:215: netlib] Error 2

The following is my current plan.sh:

pkg_name=openblas

pkg_origin=mike

pkg_version=0.2.20

pkg_description="an optimized BLAS library based on GotoBLAS2 1.13 BSD version"

pkg_upstream_url=https://www.openblas.net/

pkg_license=('BSD')

pkg_maintainer="mike10010100 <mike10010100@gmail.com>"

pkg_source="https://github.com/xianyi/OpenBLAS/archive/v${pkg_version}.tar.gz"

pkg_filename="${pkg_name}-${pkg_version}.tar.gz"

pkg_dirname="OpenBLAS-${pkg_version}"

pkg_shasum=5ef38b15d9c652985774869efd548b8e3e972e1e99475c673b25537ed7bcf394

pkg_include_dirs=(include)

pkg_lib_dirs=(lib)

pkg_pconfig_dirs=(lib/pkgconfig)

pkg_build_deps=(

core/make

core/gcc

core/perl

core/ccache

)

pkg_deps=(

core/glibc

)

# do_prepare(){

# export HAB_ENV_LD_LIBRARY_PATH_TYPE=aggregate

# export LIBRARY_PATH=$(pkg_path_for core/glibc/2.27)/lib:$LIBRARY_PATH

# export LD_LIBRARY_PATH=$(pkg_path_for core/glibc/2.27)/lib:$LD_LIBRARY_PATH

# }

do_build() {

#stupid hack from https://github.com/xianyi/OpenBLAS/issues/818

CF="${CFLAGS}"

unset CFLAGS

make FC=gfortran CFLAGS="${CF}" NUM_THREADS=64

}

do_install() {

make PREFIX="$pkg_prefix" install

}

Note the commented out do_prepare section. This originally allowed the compilation to work. However, if I uncomment that section, it results in the following error:

Makefile.system:958: Makefile.: No such file or directory
make: *** No rule to make target 'Makefile.'.  Stop.

Thanks for all your help!

elliott-davis · August 22, 2018, 2:01pm

Is this related? "recompile with -fPIC" errors

baumanj · August 22, 2018, 2:08pm

@scotthain suggested the issue may be searching for stuff in /lib64/ that doesn’t exist in hab. That would explain why commenting out do_prepare fixes the issue. If the libraries are present on the system, but not available as habitat packages, you can’t build a habitat package that depends upon them.

Off the top of my head, I’m not sure what crt1.o and crti.o, but figuring out where they come from and whether they’re part of an existing core-plans package is a good place to start.

mike10010100 · August 22, 2018, 3:34pm

So both crt1.o and crti.o exist in glibc:

openblas(do_build)> find / -name crt1.o
/hab/pkgs/core/glibc/2.22/20170513201042/lib/crt1.o
/hab/pkgs/core/glibc/2.27/20180608041157/lib/crt1.o
[4] openblas(do_build)> find / -name crti.o
/hab/pkgs/core/glibc/2.22/20170513201042/lib/crti.o
/hab/pkgs/core/glibc/2.27/20180608041157/lib/crti.o

These are all, by default, contained in LDFLAGS and CFLAGS:
LDFLAGS=-L/hab/pkgs/core/glibc/2.27/20180608041157/lib -L/hab/pkgs/core/gcc/7.3.0/20180608051919/lib -L/hab/pkgs/core/perl/5.26.1/20180608094208/lib

I’m still stumped…

mike10010100 · August 22, 2018, 3:35pm

Unfortunately it doesn’t seem to be related. Thanks for your help though!

nellshamrell · August 22, 2018, 4:27pm

Hi Mike! I notice that you are using both glibc version 2.22 and glibc version 2.27 - is there a reason for that? Will try compiling your plan today and see what I can find

mike10010100 · August 22, 2018, 5:08pm

I’m not explicitly using two versions, for some reason the earlier version of glibc is included despite my best efforts… I’ve wiped the studio and attempted to ensure that nothing is piggybacking on an earlier version.

smacfarlane · August 22, 2018, 5:09pm

Hi @mike10010100, I think there are two things going on here.

The first is that the Makefile for lapack-netlib doesn’t appear to pass through LDFLAGS so gfortran doesn’t know where to look for libraries. I would expect the gfortran lines above to include

-L/hab/pkgs/core/glibc/2.27/20180608041157/lib -L/hab/pkgs/core/gcc/7.3.0/20180608051919/lib -L/hab/pkgs/core/perl/5.26.1/20180608094208/lib

You can test this by attaching in your plan.sh before the make, run make as it exists in your plan.sh and after you get an error do the following:

[3] openblas(do_build)> cd lapack-netlib/INSTALL
[4] openblas(do_build)> gfortran -pthread -O2 -Wall -m64 -fPIC -lm -lpthread -lgfortran -o testlsame lsame.o lsametst.o
/hab/pkgs/core/binutils/2.30/20180608050633/bin/ld.bfd.real: cannot find crt1.o: No such file or directory
/hab/pkgs/core/binutils/2.30/20180608050633/bin/ld.bfd.real: cannot find crti.o: No such file or directory
/hab/pkgs/core/binutils/2.30/20180608050633/bin/ld.bfd.real: cannot find -lm
/hab/pkgs/core/binutils/2.30/20180608050633/bin/ld.bfd.real: cannot find -lpthread
/hab/pkgs/core/binutils/2.30/20180608050633/bin/ld.bfd.real: cannot find -lm
/hab/pkgs/core/binutils/2.30/20180608050633/bin/ld.bfd.real: cannot find -lm
collect2: error: ld returned 1 exit status
[5] openblas(do_build)> gfortran -pthread -O2 -Wall -m64 -fPIC -lm -lpthread -lgfortran -o testlsame lsame.o lsametst.o $LDFLAGS
/hab/pkgs/core/binutils/2.30/20180608050633/bin/ld.bfd.real: cannot find crt1.o: No such file or directory
/hab/pkgs/core/binutils/2.30/20180608050633/bin/ld.bfd.real: cannot find crti.o: No such file or directory
collect2: error: ld returned 1 exit status

Unfortunately that still doesn’t fix the crt1.o,crtl.o errors, which is the second thing. I’m at the edge of my knowledge on how linking works, but I suspect that the Makefiles of lapack-netlib/INSTALL are masking something else, since everything else built/linked up to this point.

I suspect you’ll have to write a patch file and apply it during the do_prepare() phase since there isn’t any pre-make configuration to be run.

mike10010100 · August 22, 2018, 5:29pm

Thank you so much for your insight! I’m not really sure what the right way to go forward here is, however…

Scott_Hain · August 22, 2018, 7:44pm

I think there’s a few things going on - @smacfarlane was super close to the answer. crtl.o and crt1.o are the base c runtime libraries, which are part of the core/gcc-lib package (and gcc itself but we don’t need a compiler as a runtime dep, I think) - the reason this isn’t being automatically added is because the OpenBLAS compilation process itself doesn’t respect LDFLAGS from the environment. (Not sure if they’ve fixed it, have you tried a later version? Looks like 0.2.20 is a bit old)

Long story short - this should work: (notice we don’t set the LD_LIBRARY_PATH, this is on purpose, as we don’t want to confuse the build system runtime). I don’t know if the CFLAGS hack is needed anymore? I can’t guarantee this will work but it does build!

pkg_name=openblas
pkg_origin=mike
pkg_version=0.2.20
pkg_description="an optimized BLAS library based on GotoBLAS2 1.13 BSD version"
pkg_upstream_url=https://www.openblas.net/
pkg_license=('BSD')
pkg_maintainer="mike10010100 <mike10010100@gmail.com>"
pkg_source="https://github.com/xianyi/OpenBLAS/archive/v${pkg_version}.tar.gz"
pkg_filename="${pkg_name}-${pkg_version}.tar.gz"
pkg_dirname="OpenBLAS-${pkg_version}"
pkg_shasum=5ef38b15d9c652985774869efd548b8e3e972e1e99475c673b25537ed7bcf394
pkg_include_dirs=(include)
pkg_lib_dirs=(lib)
pkg_pconfig_dirs=(lib/pkgconfig)
pkg_build_deps=(
  core/make
  core/gcc
  core/perl
  core/ccache
)
pkg_deps=(
  core/gcc-libs
  core/glibc
)

do_prepare(){
  export LIBRARY_PATH=$(pkg_path_for core/glibc)/lib:$(pkg_path_for core/gcc-libs)/lib:$LIBRARY_PATH
}

do_build() {
  make FC=gfortran NUM_THREADS=64
}

do_install() {
  make PREFIX="$pkg_prefix" install
}

mike10010100 · August 22, 2018, 7:49pm

OMG so it was a red herring that those existed in glibc? Thank you so much for all your help! This is fantastic!

(and, as a side note, I did upgrade to OpenBLAS 0.3.2 and this solution worked perfectly for that as well!)

Thanks again for everything!

Scott_Hain · August 22, 2018, 7:59pm

@mike10010100 not a red herring actually - I was unclear in my last post - the gcc-libs was not to resolve the crtl.o issue, that was just using LIBRARY_PATH and not LD_LIBRARY_PATH. The gcc-libs were added because it needs libgfortran.so.4 in order to work, which is in gcc-libs.

mike10010100 · August 22, 2018, 8:01pm

@scotthain Ahhhh, I see now! Thank you so much for your insight!