Using Habitat with COTS

Habitat Wiki
1

While Habitat provides the best behavior for applications that can be compiled from source into the Habitat package format, it can also bring wonderful management benefits to applications distributed in binary-only form. For example, let’s say you’ve got some commercial off-the-shelf software that you regularly deploy as part of an application stack and you’d rather not deal with the cognitive overhead of 2 entirely different Lifecycle management strategies?

Let’s look at how to get the management features Habitat provides, even if you don’t have access to raw source code. First, I’d suggest you head out to our docs on creating binary wrapper packages. This covers most use-cases, but still doesn’t necessarily cover the situation in which maybe you’ve only got an .rpm or .deb file that you need to turn turn into a binary-wrapper package.

Getting Started

I’ve created a short dummy plan to help us walk through the couple of pieces we need to make this work. Let’s look at the plan options briefly:

pkg_name=patch
pkg_origin=core
pkg_version="2.7.6-3"
pkg_maintainer="The Habitat Maintainers <humans@habitat.sh>"
pkg_license=("Apache-2.0")
pkg_source="https://rpmfind.net/linux/fedora/linux/development/rawhide/Everything/x86_64/os/Packages/p/${pkg_name}-${pkg_version}.fc28.x86_64.rpm"
pkg_shasum="575268cecae12a7750d8e36e2ec60557dec6f99f9727497ad89005300cc2938f"
pkg_build_deps=(core/busybox-static core/cpio)
pkg_bin_dirs=(bin)
pkg_lib_dirs=(lib)
pkg_description="Some description."
pkg_upstream_url="http://example.com/project-name"

This should look familiar, but let’s take a look at a couple important options. First you’ll notice that pkg_source is currently set to the location of an .rpm package out on the web. For simplicity sake we’re going to pull a package directly from rpmfind.net , something easily extracted without too many moving parts. We’ve added in the corresponding shasum so that the build knows we’ve got the right .rpm, and we’ve actually added a couple of strange build deps and no runtime deps.

pkg_build_deps=(core/busybox-static core/cpio)

The reason we’ve not included any runtime deps is that in the case of our COTS, we don’ need the guarantees of portability and we’re not compiling from source, so we’re opting out of adding any runtime dependencies to be deployed alongside our service.

The two pkg_build_deps referenced in our plan required if you’re planning to redistribute an rpm. We’ll get to why in a moment.

The last thing we should take note of are the pkg_bin_dirs and pkg_lib_dirs options. These are set so that if we wanted to provide lifecycle hooks, our binary could be called without having to prefix those binary calls with hab pkg exec <origin>/<pkg_name>.

What this means

  • This software, will only run on machines that have been appropriately configured for the software we’re packaging. Thus any libraries your binary might need must exist on the underlying machine in order for this package to function.
  • Your binary will only link against underlying system libraries. As such, If you need your COTS to function on other OSes/platforms, you’ll want to discover what underlying libraries are necessary for linking before again referring back to the binary wrapper package docs paying careful attention to the sections Relocate Hard-Coded Library Dependencies If Possible and/or Fix Hardcoded Interpreters

Build Time

When our package doesn’t need to take into account the configuration of the underlying system that we intend to run on (e.g. we’re always going to run on RHEL v.FOO configured in BAR way) the operation of repacking an .rpm as a hartfile is pretty trivial. We’re going to lean on our build deps pkg_build_deps=(core/busybox-static core/cpio) to unpack the .rpm we’ve specified and move it into the package artifact.

Let’s look at the remainder of the example plan.sh.

do_unpack() {
  return 0
}

do_build() {
  rpm2cpio $HAB_CACHE_SRC_PATH/${pkg_name}-${pkg_version}.fc28.x86_64.rpm | cpio -idmv
}

do_install() {
  mv ./usr/* ${pkg_prefix}
}

To start we want to override the do_unpack phase to avoid the default behavior from being executed. This is a quick catch to effectively dump the unmolested .rpm file into $HAB_CACHE_SRC_PATH.

Moving on to the do_build phase, we’re going to use the rpm2cpio binary that is shipped inside of core/busybox-static to output the cpio archive from the RPM package to stdout. rpm2cpio $HAB_CACHE_SRC_PATH/${pkg_name}-${pkg_version}.fc28.x86_64.rpm. In order to extract the package files we’ll use the output from rpm2cpio and pipe them into the cpio command to extract and create the files we need. The cpio command copies files to and from archives. A quick note on the flags being used -i extracts the files from the archive, -d creates any leading directories where needed, -m preserves the file modification times when creating the files and -v (as usual) makes the command verbose.

The result is that we will now have a /usr directory containing the files from the rpm package in our present working directory. As we move onto the do_install phase, the only activity left to do is to copy our newly extracted files into the $pkg_prefix! We don’t want to move the usr dir, only the actual contents of the package which in our case consists of ./usr/bin, ./usr/share, and ./usr/lib.

At this point you have a build that repackages your RPM as a Habitat package which means you should also be theoretically able to add in your own lifecycle hooks and get all of the joy of the habitat supervisor and management layer while bypassing the determinism of the build system.

Below I’ve attached the example plan in whole:

pkg_name=patch
pkg_origin=core
pkg_version="2.7.6-3"
pkg_maintainer="The Habitat Maintainers <humans@habitat.sh>"
pkg_license=("Apache-2.0")
pkg_source="https://rpmfind.net/linux/fedora/linux/development/rawhide/Everything/x86_64/os/Packages/p/${pkg_name}-${pkg_version}.fc28.x86_64.rpm"
pkg_shasum="575268cecae12a7750d8e36e2ec60557dec6f99f9727497ad89005300cc2938f"
pkg_build_deps=(core/busybox-static core/cpio)
pkg_bin_dirs=(bin)
pkg_lib_dirs=(lib)
pkg_description="Some description."
pkg_upstream_url="http://example.com/project-name"

do_unpack() {
  return 0
}

do_build() {
  rpm2cpio $HAB_CACHE_SRC_PATH/${pkg_name}-${pkg_version}.fc28.x86_64.rpm | cpio -idmv
}

do_install() {
  cp -r ./usr/* ${pkg_prefix}
}
2 Likes