etckeeper: revision control for /etc/ ===================================== :author: der.hans :title: etckeeper: revision control for /etc/ :date: 2018Nov09 @ SeaGL :copyright: 2015-2018 der.hans --- CC BY-SA 4.0 unported :max-width: 45em :data-uri: :icons: :duration: 60 :website: https://www.LuftHans.com/talks/ :source-highlighter: pygments :imagesdir: resources etckeeper: revision control for /etc/ ------------------------------------- SeaGL 2018 + 2018Nov09 + der.hans + https://mastodon.social/@lufthans What does etckeeper do? ----------------------- puts /etc/ into revision control etckeeper author ---------------- Joey Hess // :note: if Joey Hess created it, it's worth checking out Other software from Joey Hess ----------------------------- * git-annex * ikiwiki * keysafe * github-backup * myrepos * debconf * alien * debhelper in short -------- [role="incremental"] * Joey Hess is awesome * etckeeper is awesome * use it fini ---- What is system configuration? ----------------------------- /etc/ holds system configuration Changes in /etc/ can affect system behavior and performance plain text config files // :note: luckily we've been managing text files for a long time // :note: no registry where misconfiguration of one application blocks reading configuration for other applications What is etckeeper? ------------------ Part of a nutritious backup solution // :note: show bowl of cereal What it does ------------ puts /etc/ into version control * safely * completely * consistently What is a VCS? -------------- tracks changes to files shows changes between versions works great on text files not so great on binary files allows retreiving a specific version the March 4th, 2014 networking configuration safely ------ don't want to accidentally expose some files * /etc/shadow // :note: Only the Shadow knows. ? drwx------ 8 root root /etc/.git aside, netsaint ---------------- Let me tell you a story ... completely ---------- // :note: VCS usually don't fully track these * permissions * ownership * empty directories consistently ------------ auto-checkins after package installs Who cares? ---------- [role="incremental"] * see what changed recently when getting a wake up page * see what changed months ago when a service restart finally rereads the config file * revert changes when package management stomps all over your finely handcrafted configuration * restore a file when an inadvertent rm hits production rather than the test server you thought you were on * see what has changed when multiple people ( and tools ) work on a system * investigate changes after a script kiddie gets in // :note: img:snowflake // :note: PS1 is our friend, same for mollyguard etckeeper setup --------------- ---- $ sudo etckeeper init $ sudo etckeeper vcs commit -m "Initial checkin" $ profit ---- now it's a repo --------------- normal repo // :note: well, repo with secrets do normal repo things spear and magic helmet ---------------------- well, there is some etckeeper magic // :note: we'll get to that // :note: maybe pixie dust rather than spear and magic helmet Add a file ---------- ---- $ sudo touch /etc/testfile $ sudo git -C /etc add testfile $ sudo git -C /etc commit -m "A test file" ---- Undo a typo ----------- ---- $ echo "10.10.10.1111 nextcloud" | sudo tee -a /etc/hosts >/dev/null $ sudo git -C /etc/ diff -U0 hosts diff --git a/hosts b/hosts index 97f1792..ecc187a 100644 --- a/hosts +++ b/hosts @@ -9,0 +10 @@ ff02::2 ip6-allrouters +10.10.10.1111 nextcloud $ sudo git -C /etc/ commit hosts -m "Added nextcloud box" ---- [role="incremental"] ---- $ sudo git -C /etc/ revert HEAD -m "oops, wrong IP" $ echo "10.10.10.111 nextcloud" | sudo tee -a /etc/hosts >/dev/null $ sudo git -C /etc/ commit hosts -m "Correctly added nextcloud box" ---- File recovery ------------- ---- $ sudo rm ls /etc/hosts.allow rm: cannot remove 'ls': No such file or directory ---- [role="incremental"] ---- $ sudo git -C /etc/ checkout hosts.allow ---- Supported VCSen --------------- * git * mercurial * bazaar * darcs etckeeperification ------------------ convenience wrapper Wrapper usage examples ---------------------- // :note: Why do we sudo? * sudo etckeeper commit -m "Some msg" // :note: records the user using sudo ** sudo git -C /etc/ commit -m "Some msg" * sudo etckeeper vcs diff ** sudo git -C /etc/ diff * sudo etckeeper vcs status ** sudo git -C /etc/ status * sudo etckeeper vcs -C /etc/apache2 status ** sudo git -C /etc/apache2 status Copying the repo ---------------- ---- $ ssh server 'mkdir /etc-clone; cd /etc-clone; chmod 700 .; git init --bare' $ git remote add backup ssh://server/etc-clone $ git push backup --all ---- // :note: chmod 700 Ignores ------- use normal VCS ignores * /etc/.gitignore All the files? -------------- // :note: one ring image // :note: not quite ephemeral files * /etc/mtab cache data * /etc/ld.so.cache Other tools ----------- configuration management package management // :note: neither tracks all the files in etc filesystem snapshots // :note: can get everything and every change // :note: usually don't keep all snapshots // :note: diffs are less useful Is some puppet mucking about with your files? --------------------------------------------- image::Beaker_Muppet.jpg[] Is some seattlish chef bork bork borking your /etc/? ---------------------------------------------------- image::Swedish_Chef_Muppet-borkborkbork.jpg[] Configuration management ------------------------ Sets files or parts of files to specific state Can log changes to a file Doesn't track all files in /etc/ CMS to set state, etckeeper to track changes Package management ------------------- Sets files to an initial state Might provide tools for automated configuration changes Doesn't track all files in /etc/ Pkg mgt to set initial state, etckeeper to track changes Filesystem snapshots -------------------- Tracks changes to entire filesystem, more than just /etc/ Usually don't keep all snapshots across time Snapshots for backups, etckeeper to track changes Package management hooks ------------------------ Automagic checkins before and after pkg mgt changes VCS hooks --------- ---- # cat /etc/.git/hooks/pre-commit #!/bin/sh # pre-commit hook for etckeeper, to store metadata and do sanity checks set -e etckeeper pre-commit -d /etc ---- The Point --------- // audio::etckeeper.I_Hate.gesch2.ogg[] // video::Albuquerque.ogg[] // [role="incremental"] I hate sauerk^Huncommitted configuration changes Summary ------- * etckeeper keeps system configuration in VCS * your choice of VCS, but use git * part of resiliency. backup and DR architectures * don't forget the other parts :) Finding Hans ------------ Thank you! * https://mastodon.social/@lufthans ** Mastodon * LuftHans on Freenode, usually in #SeaGL, #LOPSA and #PLUGaz ** IRC Resources --------- etckeeper home page * https://etckeeper.branchable.com/ // :note: add Joey Hess home page? my Opensource.com etckeeper article * https://opensource.com/article/17/3/etckeeper-version-control Credits ------- https://en.wikipedia.org/wiki/Beaker_(Muppet)#/media/File:Beaker_(Muppet).jpg[Beaker] - WikiPedia https://www.pinterest.com/mikeycooks/swedish-chef/[Swedish Chef] - mikeycooks on Pinterest