Perform a command serially in Capistrano

Capistrano is a great tool for doing software deployments and other system maintenance tasks (although for anything larger, I’d recommend Chef).  One small Capistrano issue I ran into this week is that there’s no way to run a command in serial across multiple machines (commands are always run in parallel). You might want to do this if you need to make sure that at least one service in a group is available at all times, or if the service restarting is going to put some load on another resource such as a database.

Here’s the solution I came up with:

desc "Do a rolling restart of <service>"
deploy.task :restart do
  hosts = self.roles[:server].to_ary # change :server to your role
  num_hosts = hosts.size
  hosts.each_with_index do |host, i|
    puts "Restarting <service> on #{host} (#{i+1} of #{num_hosts})"
    sudo "/etc/init.d/<sevice> restart", :hosts => host
    if i < num_hosts-1
      puts "Waiting 3s before next host."
      sleep(3)
    end
  end
end

Email alerts for PHP errors via SEC

Production systems should always have PHP’s display_errors disabled and log_errors enabled so errors are logged instead of displayed to users. Of course, you’ll want a to be notified when errors happen and that’s where a nifty tool called SEC (Simple Event Correlator) comes in. It’s not very sexy, but its incredibly powerful and can be used for all sorts of log watching tasks.

Here are some helpful guides for getting started:

• Working with SEC- the Simple Event Correlator
• Monitoring with Simple Event Correlator
• sec man page

At HipChat we use a config like the following to to monitor Apache’s error_log for PHP errors and send us emails:

# Capture error lines and store them in apache-php-errors
type=Single
ptype=RegExp
pattern=^\[.+\] \[error\] \[client .+\] PHP .+$
desc=PHP error or warning
action=add apache-php-errors $0

# Report errors every minute if apache-php-errors is set
type=Calendar
time=* * * * *
desc=Mail web errors
context=apache-php-errors
action=report apache-php-errors /usr/bin/mail -s "PHP errors" [email protected]; delete apache-php-errors;

Assuming you put this config in /etc/sec/apache.conf you’d run:

$ sec --conf=/etc/sec/apache.conf --input /var/log/apache2/error_log

Try writing a few of your own rules (the –debug flag is very helpful) or the examples in the tutorials above. SEC is incredibly powerful and can make complex monitoring tasks very simple.

2010 ski movie trailers

Trailers for the 2010 ski movies are starting to land! Here’s what I’ve found so far:

Level 1 Productions - Eye Trip

Matchstick Productions - The Way I See It

Poor Boyz Productions - Revolver

Teton Gravity Research - Light the Wick

Warren Miller - Session #61