I’m currently running a WordPress cluster on AWS for livemocha.com. This
has been interesting (especially overlapping it with the existing legacy
site), but one of the big problems that I’ve been facing is trying to keep
the themes/plugins/etc consistent across the EC2 nodes. WordPress gets
unhappy if you activate a new plugin on one node and then can’t find it on
another node. The plugin activation state is stored in the DB, not the
local fs, so when it finds it missing it de-activates it across the
board… not helpful.
So what I currently do is put the homepage into maintenance (by shoving all
relevant requests over to the login page for the actual site), point my
hosts file to a build host (that does double-duty as the cron host), do my
updates, build a new AMI, spin down the old instances and spin up new
instances. When everything is copacetic, I then take the site out of
maintenance. This is, needless to say, time consuming. The timeframe from
queuing an AMI build to out of maintenance is about 20 minutes, let alone
the time taken to update things and test them.
Updates are becoming frequent, so I want to trim the amount of time this
process takes by updating the nodes already in service and saving full AMI
rebuilds for things like major OS updates (kernel updates, etc). I’ve
thought of a few different ways to handle this, but the one that I’m
currently leaning towards is a mix of Git and Chef. Basically I would put
the site into maintenance, do my changes on the build host, and if it
passes testing, then I would commit the changes into Git. This would
essentially be everything underneath the WordPress wp-contents/ directory
(if WordPress core updates become a big enough issue as well, then I can
expand it to be the entire web root, but for right now I’ll avoid that).
Once everything is pushed up in Git, I’ll then use Knife SSH to trigger a
chef run across the cluster that will do a git pull and ideally bounce
Apache (or I can use Knife to do that as well; either way, not a big deal).
If it’s something like a new plugin being added, I don’t even have to put
it into maintenance at all, just commit it in Git and push away with Knife.
Has anyone done something along these lines before? Can you think of a
better way to handle it? I thought about using s3fs to update files
in-place, given that APC would (or at least, should…) cache the PHP files
so that the latency on s3fs’s io wouldn’t (or at least, shouldn’t) be a
concern. However, I’m not 100% confident that it would perform up to snuff,
and if s3fs errors out, then the website goes offline, which my boss and I
agree makes it a non-starter approach. I imagine that I could store all the
files on the Chef server and do a recursive file pull, but I’m thinking Git
is simpler and easier to manage. It’ll also make tracking changes from our
theme designer a bit easier, too, which is one of the reasons I thought
about this in the first place.
Assuming that folks don’t point out some idea why this is bad, is the a
specific cookbook/LWRP that I should use when setting up this recipe? Also,
is there an easy way for me to configure this so that the chef-client
service wouldn’t update on every git commit (like the theme dev pushing
stuff up to us) but allow me to kick it off specifically via Knife? If not,
then I’ll just use different branches… the dev will push to a testing
branch, and then I’ll push from there to a release branch that Chef is
pulling from. I just want to make sure that he doesn’t make a tweak that
suddenly appears on the site before it’s ready.
Thanks for all the help!
~~ StormeRider ~~
“Every world needs its heroes […] They inspire us to be better than we
are. And they protect from the darkness that’s just around the corner.”
(from Smallville Season 6x1: “Zod”)
On why I hate the phrase “that’s so lame”… http://bit.ly/Ps3uSS