Reviewing Core Plans

Habitat Wiki
1

Ready to review some core plans? This can seem a bit scary, but it’s certainly doable- here is how!

Reviewing a New Plan

If someone is adding a new plan to github.com/habitat-sh/core-plans…

  • Check whether the new plan fits the requirements outlined in CONTRIBUTING.md
  • Check the plan out locally and ensure that it builds in a local studio. If it does not build, put the error in a comment in the PR and ask the person submitting the PR to look into it.
  • If the plan builds, go ahead and merge the PR.
  • Next, head to habitat.sh, login, navigate to Builder, find the core origin, and click the “Connect a plan file” button
  • Select habitat-sh, then core plans, then paste in the path to the new core-plan’s plan.sh file (probably core-plan-name/plan.sh)
  • Click “Build Latest Version”
  • After the build completes, promote the build to the stable branch.

Reviewing a Change to an Existing Plan

NOTE - these steps will change when we have completed adding in CI/CD tests, as outlined here. But you can still test them manually until that is ready!

First, identify whether a change is a low risk or high risk change. Generally, I consider a minor version bump of the software being packaged to be a low risk change.

Reviewing a Low Risk Change

If the PR is a low risk change - still check out the PR locally and ensure that it builds. Then merge the pull request and navigate to the page for the core plan on Builder. Merging the pull request should automatically kick off a new build in Builder. If for some reason it does not, click the “Build Latest Version” button.

If the new version builds successfully, you should be good to bulk promote everything built in that builder job to stable (NOTE - you will need to do more extensive testing with a high risk or hell dive change).

Reviewing a Medium Risk Change

I consider any change which adds new behavior to a plan to be a medium risk change. The testing process for these is more extensive. If the person who opened the pull request has not including testing steps for the changes, I always ask them to add those in a comment on the PR. When they do, check out the branch locally, do a local build, and then run the tests (I often spin up an instance of the new plan in a local Docker container).

When these tests are successfully complete, go ahead and merge the PR, then make sure a build gets kicked off on Builder. When the build completes, I usually check out the new package locally and test the behavior again to vet it. If all looks good, I will go ahead and promote it to stable. Then follow the same process as for the low risk change in order to build promote dependencies.

Reviewing a Major Version Update Change

To be added

Reviewing a Hell Dive Change

A “Hell Dive” change is anyone that will affect a significant (i.e. more than 20) other plans.

More to come on how to review/merge/promote these soon.

Useful Sample Apps for testing

If you are testing packages related to node, ruby, or go, here are some useful sample apps to use for testing (please feel free to add more as you discover them!)

For examples of reviewing different types of core plans, check out the articles labeled “Reviewing Core Plans example”, including

Promoting packages

When promoting packages in the core origin, it’s important to promote the entire build group to prevent dependency resolution errors in downstream packages. For the time being, this requires the use of the CLI, as the web ui doesn’t support promoting job groups. It is, however, an excellent resource to visualize the group, and examine logs when necessary.

It’s highly recommended you stay on a recent (preferably current) release of the hab cli.

To view recent jobs and their status you can run:

> hab bldr job status -o core
☁ Determining status of job groups in core origin

CREATED AT                        GROUP ID             STATUS    IDENT
2018-11-08T15:56:45.353609+00:00  1110873278867054592  Complete  core/hab-pkg-export-docker
2018-11-08T15:32:20.227732+00:00  1110860988516130816  Complete  core/hab-studio

You can then use the GROUP ID corresponding to the package that you reviewed to drill into the status of any downstream packages. It’s recommended to use grep to filter the results to just the core origin.

> hab bldr job status 1106167215905996800 -s |grep '^core'
core/sentinel                           Success  1106173282441224192  core/sentinel/0.1.0/20181102042732
core/opam                               Failure  1106173283481436160
core/scaffolding-go                     Success  1106173283137495040  core/scaffolding-go/0.1.0/20181102042733
core/gocd-server                        Success  1106173284169318400  core/gocd-server/18.9.0/20181102042834
core/img                                Success  1106173283825377280  core/img/0.4.6/20181102042804
...

In the above example, we can see that core/opam failed to build. We will need to look in the build logs to determine if it is related to the change that created this build group, or if it’s unrelated. This will vary based on the type of error, but as a general guide if the errors look like unable to find command/library or other execution errors related to the changed package, then this group shouldn’t be promoted. In this case, please file an issue on the upstream package and, time permitting, debug and open a PR with a fix. Sometimes, such as the opam package above, it’s a known issue that can be ignored, for example: https://github.com/habitat-sh/builder/issues/365 . In this case, please verify that that package has an issue filed against it, linking to the issue with the root cause or describe the cause so that we can prioritize and fix. In the case of transient errors, such as networking hiccups, upstream sites being offline, etc. you can ignore the error and rebuild the package through the Builder UI once this group is promoted.

There can be a lot of nuance to troubleshooting the above, especially if you are new to reviewing Habitat package builds, or are unfamiliar with the language or tooling used in building that package. Please don’t hesitate to reach out to other maintainers if you’re unsure.

If everything looks good to proceed, you can then use hab bldr job promote to promote the group to stable. My preference is to use the --origin core flag to filter packages, and the --interactive flag to view the list of packages to be promoted. You should also ensure you have the EDITOR env var set to your favorite text editor.

> env EDITOR=vim hab bldr job promote 1106167215905996800 stable --origin core --interactive

If the list looks good, save and quit your editor and answer Yes to the question. All packages will be promoted to the stable channel at this point.

Useful commands for promoting packages

Visual of a large tree of jobs

$ hab bldr job status -s <id> | egrep "(Failure|InProgress|NotStarted)"

Watch the above command’s progress

$ watch -n 10 hab bldr job status -s <id> | egrep "(Failure|InProgress|NotStarted)"