Chef 12 backup: shutdown required?

Chef Infra (archive)
1

Backup documentation suggests the server should be offline for backups, “or otherwise quiet”.

Most of the time, nodes are only checking into Chef. New nodes are added very rarely. Does my use case qualify as “otherwise quiet”?

I’d like to automate backups via cron, but not have the server inaccessible for the time it takes to shut down, back up, and start again. Doc:
https://docs.chef.io/server_backup_restore.html#backup

2

You don’t want any database writes to happen when you are backing up, so no changes to node objects and the like. So if you have a windows where the Chef server isn’t doing anything and no nodes are trying to check in it should be safe to back up.

3

I see. Will database writes impact pg_dumpall, or tar?

4

The impact is that database writes that happen during the dump won’t be backed up. I wouldn’t recommend taring the data directly though if that is the format you want pg_dump supports that.

5

Oh, by tar I meant following the backup steps of first pg_dumpall and then taring /etc/opscode and /var/opt/opscode.

If the cost is merely missing some node check-in records’ details, I believe I can live with that, as long as the backup doesn’t end up in an inconsistent state.

Is anything other than PostgreSQL impacted by writes? Need I keep the server offline while backing up /var/opt/opscode as well?

My services are local-time sensitive, and with splay to ease the load on Chef servers, and servers distributed geographically, I don’t have a defined maintenance window. Thus far I’ve backed up infrequently, and I’m looking to change that. I rarely add nodes.

6

Other than log files I don’t believe there is anything in /var/opt/opscode or /etc/opscode that would be impacted by a node check in during backup. The tar command that is documented leaves out the postgres directories anyway, and that process should be really fast. PG backups through pg_dump will be consistent even if writes are being made. Unless you want to end up like GitLab, however, I suggest that a backup is NO GOOD until you have tested a restore of it.

7

Thanks @YarNhoj!

From what I understood, my pg_dumpall will miss whatever writes occur during the PostgreSQL dump if it’s done while Chef server is online, but tar should remain unaffected.

I’ll keep your warning in mind :slight_smile: I have the restoration of Chef automated, and it shouldn’t be difficult to restore and check that it’s working on some regular cadence.