Packer InSpec provisioner fails with "Error opening/processing"

Chef InSpec
1

Hi Guys

I am using the Hashicorp Packer InSpec provisioner to run a hardening profile against AWS EC2 images which Packer is building.

I am storing the hardening profile in a self-hosted Gitlab instance but am having difficulty getting InSpec to correctly download the profile from Gitlab.

SSL connectivity was my initial problem but that has been solved by concatenating my local Root CA certificate to the cacert.pem in /opt/inspec/embedded/ssl/certs

I can successfully download and run the dev-sec profile at "https://github.com/dev-sec/linux-baseline" but when I try to use the same URL structure to download from gitlab InSpec errors out.

Here is the debug run of the InSpec provisioner

 ==> hardening-paradigm-base-partitioned-server: Provisioning with Inspec...
 ==> hardening-paradigm-base-partitioned-server: Executing Inspec: inspec exec https://gitlab01.paradigm.local/paradigmsuite/PFMHardening --backend ssh --host 127.0.0.1 --key-files /tmp/packer-provisioner-inspec.596799465.key --user paradigmuser --port 41839 --input-file /tmp/packer-provisioner-inspec.197024820.yml -l debug
 hardening-paradigm-base-partitioned-server: D, [2020-05-13T12:32:46.490098 #2176] DEBUG -- : [SSH] opening connection to paradigmuser@127.0.0.1
 hardening-paradigm-base-partitioned-server: D, [2020-05-13T12:32:46.490167 #2176] DEBUG -- : [SSH] using options {:user_known_hosts_file=>"/dev/null", :port=>41839, :compression=>false, :compression_level=>0, :keepalive=>true, :keepalive_interval=>60, :timeout=>15, :auth_methods=>["none", "publickey"], :keys_only=>true, :keys=>["/tmp/packer-provisioner-inspec.596799465.key"], :password=>"<hidden>", :forward_agent=>nil, :non_interactive=>false, :verify_host_key=>false}
 hardening-paradigm-base-partitioned-server: D, [2020-05-13T12:32:46.667158 #2176] DEBUG -- : [SSH] paradigmuser@127.0.0.1 cmd = cmd.exe /c ver
 hardening-paradigm-base-partitioned-server: D, [2020-05-13T12:32:46.775177 #2176] DEBUG -- : [SSH] paradigmuser@127.0.0.1 cmd = Get-WmiObject Win32_OperatingSystem | Select Caption,Version | ConvertTo-Json
 hardening-paradigm-base-partitioned-server: D, [2020-05-13T12:32:46.893801 #2176] DEBUG -- : [SSH] paradigmuser@127.0.0.1 cmd = uname -s
 hardening-paradigm-base-partitioned-server: D, [2020-05-13T12:32:46.991013 #2176] DEBUG -- : [SSH] paradigmuser@127.0.0.1 cmd = uname -m
 hardening-paradigm-base-partitioned-server: D, [2020-05-13T12:32:47.196134 #2176] DEBUG -- : [SSH] paradigmuser@127.0.0.1 cmd = test -f /etc/debian_version
 hardening-paradigm-base-partitioned-server: D, [2020-05-13T12:32:47.301559 #2176] DEBUG -- : [SSH] paradigmuser@127.0.0.1 cmd = test -f /etc/lsb-release && cat /etc/lsb-release
 hardening-paradigm-base-partitioned-server: [2020-05-13T12:32:47+10:00] DEBUG: Option backend_cache is enabled
 hardening-paradigm-base-partitioned-server: [2020-05-13T12:32:47+10:00] DEBUG: Resolve https://gitlab01.paradigm.local/paradigmsuite/PFMHardening into cache /home/paradigmuser/.inspec/cache
 hardening-paradigm-base-partitioned-server: [2020-05-13T12:32:47+10:00] DEBUG: Fetching URL: https://gitlab01.paradigm.local/paradigmsuite/PFMHardening
 hardening-paradigm-base-partitioned-server: [2020-05-13T12:32:48+10:00] WARN: Unrecognized content type: text/html; charset=utf-8. Assuming tar.gz
 hardening-paradigm-base-partitioned-server: [2020-05-13T12:32:48+10:00] DEBUG: Archive stored at temporary location: /tmp/inspec-dl-20200513-2176-1271vgv.tar.gz
 hardening-paradigm-base-partitioned-server: Error opening/processing /home/paradigmuser/.inspec/cache/caf35a9cd1b7f534ac7aef70b0d027fd3d0cc6bd8bbca9e1cc0eb8cb217901bf.tar.gz: not in gzip format
 hardening-paradigm-base-partitioned-server: [2020-05-13T12:32:48+10:00] DEBUG: Dependency does not exist in the cache https://gitlab01.paradigm.local/paradigmsuite/PFMHardening
 hardening-paradigm-base-partitioned-server: [2020-05-13T12:32:48+10:00] DEBUG: Fetched archive moved to: /home/paradigmuser/.inspec/cache/caf35a9cd1b7f534ac7aef70b0d027fd3d0cc6bd8bbca9e1cc0eb8cb217901bf.tar.gz

I noticed the "assuming .tar.gz" so based on that I provided a link to a .tar.gz file of the repository and then a .zip file but both produced the same error.

Do any of you know what the correct form of the profile address should or is there something that needs to be implemented in the profile to make it an acceptable download target?

Many thanks in advance
Regards
Ian Carson

3

It looks like InSpec has some code that guesses a .tar.gz URL from a git repo URL

There's only currently code for Github and bitbucket. Cases could be added for gitlab.com repos but I'm not sure there's a way to always transform a privately hosted Gitlab URL.

You will need to give the direct URL to a .tar.gz file for the time being.

4

@stocksy

Thanks mate, I appreciate the quick response.

Your reply eliminates inspec as a blocking issue (if it can't do it, it can't do it - no worries). I am currently trying to provide the right URL to directly pass a .tar.gz of the repo to the InSpec provisioner

Thanks & Regards
Ian Carson

5

@stocksy

Just a quick followup to let you know that pointing at a .tar.gz of the profile is working. Of course now I'm running into bugs in the profile - but that's a positive!
Regards
ian

6

Awesome!