Changing out the Operating system underneath Habitat

If any of you are like me, you’ve read all of the Habitat marketing and have bought into this idea of decoupling your app from your Operating System. It’s a great goal and a lofty claim.
So I set out to test it.

Using Terraform, I built out a three node postgresql cluster on Ubuntu 16.04. The key here was I created a separate datastore, and mounted it under /hab, so I have my OS and everything underneath /hab is effectively isolated.

Next I built a CentOS 7.2 VM. Note that they have different Kernels:

root@hab-psql-tf-stable-1:~# uname -a
Linux hab-psql-tf-stable-1 4.4.0-92-generic #115-Ubuntu SMP Thu Aug 10 09:04:33 UTC 2017 x86_64 x86_64 x86_64 GNU/Linux
root@hab-psql-tf-stable-1:~# cat /etc/*release*
DISTRIB_ID=Ubuntu
DISTRIB_RELEASE=16.04
[root@hab-psql1-pt2 ~]# uname -a
Linux hab-psq1-pt2 3.10.0-327.el7.x86_64 #1 SMP Thu Nov 19 22:10:57 UTC 2015 x86_64 x86_64 x86_64 GNU/Linux
[root@hab-psql1-pt2 ~]# cat /etc/*release*
CentOS Linux release 7.2.1511 (Core)
Derived from Red Hat Enterprise Linux 7.2 (Source)

I chose ubuntu 16.04 and CentOS 7.2 for no particular reason other than it’s what I clicked on in OpenStack.

So I unmounted then mounted my volume on my new CentOS 72 image, created my systemd init script aaaand:

Mar  8 17:45:21 hab-psql1-pt2 systemd: Starting Habitat Supervisor...
Mar  8 17:45:21 hab-psql1-pt2 systemd: Failed at step EXEC spawning /bin/hab: No such file or directory

Doh. Oh, link /bin/hab to /hab/pkgs/core/hab/0.77.0/20190301212334/bin/hab, problem solved, right?

Well… sort of:

hab-sup(MR): Can't create directory /hab/svc/postgresql: hab-sup(ER)[components/sup/src/error.rs:460:46]: Package requires user hab to exist, but it doesn't
hab-sup(MR): If this service is running as non-root, you'll need to create /hab/svc/postgresql and give the current user write access to it

Ok, easy fix, create the hab user and group, start my service again, and

[root@hab-psq1-pt2 ~]# hab svc status
package                                type        desired  state  elapsed (s)  pid   group
core/postgresql/9.6.11/20190305213451  standalone  up       up     3            3012  postgresql.default

And, we can confirm on our master node:

Before the change

postgres=# SELECT client_addr, state FROM pg_stat_replication;
 client_addr  |   state
--------------+-----------
 172.16.10.11 | streaming
 172.16.10.37 | streaming

During the change:

postgres=# SELECT client_addr, state FROM pg_stat_replication;
 client_addr  |   state
--------------+-----------
 172.16.10.37 | streaming

After the change:

postgres=# SELECT client_addr, state FROM pg_stat_replication;
 client_addr  |   state
--------------+-----------
 172.16.10.37 | streaming
 172.16.10.18 | streaming

Once I mounted my /hab partition, created my systemd init script, linked /bin/hab to the right location, and created my hab user, I was off and cooking with gas having successfully changed out the OS underneath the application.

Awesome! This is definitely nice to see!

I love how when I build my apps, I am making a /hab filesystem. Everything that my app needs to run is located under there versus everywhere like how yum and apt package managers allow.

This is a super cool test and great to see!

Amazing work here! Thank you for testing/proving this out, and for sharing it back.