Automatic puppet syntax check and production update
Usually I using a subversion repository to manage my puppet modules and manifests.
This is really usefull to historicize the evolution of my installations and my modifications on the production servers I manage; and, this is more usefull if you have more than one user which manage the puppet configurations on your environments.
So, this is my typicall subversion repository architecture:
subversion puppet repository
|
|---- production environment checkout on /etc/puppet
|
|---- matteo environment checkout on /home/matteo/puppet
|
|---- johndoe environment checkout on /home/johndoe/puppet
|
.....
So, If I need to update my infrastructure definitions with puppet:
1. Login on the master with my user
2. svn up
3. Edit puppet code
4. Login to a testing server
5. Running puppet with --environment=matteo
6. If everything it's ok, I can commit to my repository
7. Becoming root on the master
8. svn up of the production checkout
But, there are some situation when I don't check my changes with an environment-specific run on a client, and I commit directly to the repository, either because little-modifcations or quickly-crazy updates.
As you can imagine, sometimes I've wrote some junk code in my .pp and .erb files, and the production update broke the entire running on the production servers (sometimes, only 'cause I wrote a dot, instead of a period).
So, I've wrote some pre-commit and post-commit scripts to make more secure the commits on the productions. Now my quick-edit procedure is like this:
1. Login on the master with my user
2. svn up
3. Edit puppet code
4. Commit
First of all, I've wrote this pre-commit script (based on the one provided puppet):
#!/bin/sh
# SVN pre-commit hook to check Puppet syntax for .pp files
REPOS="$1"
TXN="$2"
tmpfile=`mktemp`
export HOME=/
SVNLOOK=/usr/bin/svnlook
# Validate *.pp files
$SVNLOOK changed -t "$TXN" "$REPOS" | awk '/^[^D].*\.pp$/ {print $2}' | while read line
do
$SVNLOOK cat -t "$TXN" "$REPOS" "$line" > $tmpfile
if [ $? -ne 0 ]
then
echo "Warning: Failed to checkout $line" >&2
fi
# Validation
puppet parser validate --color=false --confdir=/tmp --vardir=/tmp --ignoreimport $tmpfile 2>/dev/null
if [ $? -ne 0 ]
then
echo "Puppet syntax error in $line." >&2
exit 2
fi
done
# Validate *.erb files
$SVNLOOK changed -t "$TXN" "$REPOS" | awk '/^[^D].*\.erb$/ {print $2}' | while read line
do
$SVNLOOK cat -t "$TXN" "$REPOS" "$line" > $tmpfile
if [ $? -ne 0 ]
then
echo "Warning: Failed to checkout $line" >&2
fi
# Validation
erb -x -T '-' $tmpfile | ruby -c 2>/dev/null
if [ $? -ne 0 ]
then
echo "Template syntax error in $line." >&2
exit 2
fi
done
res=$?
rm -f $tmpfile
if [ $res -ne 0 ]
then
exit $res
fi
I put this file, named "pre-commit", on the directory $SVNROOT/puppet/hooks, and make that executable.
This will activating when you do a commit (svn ci) command, and validate the syntax of all modified *.pp and *.erb files, before put the changes on the repository.
So I've created a really easy updater script for the production, saved on /usr/local/sbin/updatePuppetProduction.sh
#!/bin/bash
svn up --username=root --password=mysvnrootuserpw /etc/puppet
exit $?
And give to my user the ability to launch that with sudo:
matteo ALL = NOPASSWD: /usr/local/sbin/updatePuppetProduction.sh
In the end, I've wrote the $SVNROOT/puppet/hooks/post-commit script:
#!/bin/sh
REPOS="$1"
REV="$2"
sudo /usr/local/sbin/updatePuppetProduction.sh
exit $?
Done!!!
Now when I launch my "svn ci" commit, my master do those operations:
1. Get the list of all modified .pp and *.erb files
*2.** Check the syntax of those files
3. If syntax is ok, confirm commit and load the changes on the repository
4. Execute the updatePuppetProduction.sh script as root to update the production environment
5. Done.
Enjoy