Allow some paths when your app is in maintenance

August 5th, 2008

Those of you who already follow Ezra's great guidance on setting up nginx as a front end to a mongrel cluster will know that it already includes a nice block to enable maintenance mode when you trigger it in capistrano (creates a /shared/system/maintenance.html page and rewrites all requests to it).

    if (-f $document_root/system/maintenance.html) {
      rewrite  ^(.*)$  /system/maintenance.html last;
      break;
    }

However, we had a customer the other day who wanted to put his application into maintenance mode but still get to the admin area. After much tinkering with nginx rewrite rules we came up with a solution that worked and also let you host your images and css from the same application without having those calls rewritten as well. Basically it just required a reordering of the rewrite rules within the location block to a more sane order.

See an example nginx config which implements this - based on Ezra's original to give the customer complete anonymity.

One important thing to note about this config is that if you serve images or css via processing in your mongrels this will skip processing them - indeed, anything that ends css, jpg, png or gif will never make it to mongrels and be served by nginx instead. Consider this and remove the relevant if statement if this applies to you. In particular this will apply to Mephisto which serves css via controller processing in order to then cache it out statically to disk.

0 comments »

MetaWebLog API returns to Mephisto

July 22nd, 2008

At long last it looks like the MetaWebLog API has made it back into Mephisto as a plugin. I had started working on this myself but it was just too low priority for me with all the various things I have going on at the moment.

I know it was the intention of the Mephisto core team to stop this API from depending on the now deprecated ActionWebService framework and have someone else move it onto its own independent xmlrpc code. But to be honest this has been too long coming to turn down the plugin, even if it is still relying on ActionWebService (there seems to be a newish port of it on GitHub that might keep up with the rest of Rails with a new committing team).

And so onto the plugin itself, just drop by the RailsHacks blog to pickup the code and instructions for use. If you can see this post then it definitely works as I’ve just written it in the awesome MarsEdit and posted directly to this blog using it.

0 comments »

Setting up Background Job (BJ) Properly

June 23rd, 2008

I dealt with a customer today who was a bit miffed that his Background Jobtasks were taking a while to kick off after the Rails app submitted them to the queue.

I dug a little deeper into the problem - he was using our standard configuration, moving control of the runner out to cron instead of being controlled from within the app itself.

1

Bj.config["production.no_tickle"] = true

and then a cronjob such as

*/2 * * * * bj run --forever --rails_env=production --rails_root=/data/user/current

Read the rest of this entry 0 comments »

Hpricot Case Sensitivity

April 24th, 2008

Recently we were trying to use the awesome Hpricot to do some HTML parsing. Problem is that Hpricot doesn't easily allow you to do case insensitive searches for elements.

This means that we're missing any element where someone has placed an uppercase character in the tag we're explcitly looking for.

1
2
3
4

>> h.at("meta[@name=description]")
=> {emptyelem meta name"description" content"David Smalley, Ruby hacker based in Leeds, UK"}
>> h.at("meta[@name=Description]")
=> nil

This problem has been talked about on the Mofo Mailing List and even on the Hpricot challenge page - but I didn't really like the look of those solutions as downcase-ing everything would distort the original HTML.

So I devised this (horrible and hacky) bit of code that loops through all element names and attributes, downcase-ing them all without touching any innerHTML.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19

  def normalize(element)
    element.children.each do |child|
      if child.respond_to?(:name=)
        child.name = child.name.downcase if child.name
      end
      if child.respond_to?(:raw_attributes=)
        attribs = {}
        child.raw_attributes.each_pair do |key,value|
          attribs[key.downcase] = value.downcase if value
        end
        child.raw_attributes = attribs
      end
      normalize(child) if child.respond_to?(:children)
    end
    return element
  end
>> h = Hpricot(open("http://davidsmalley.com"))
=> Hpricot.....<snip>
>> normalize(h)

Now all your element names and tags will be downcase-ed and all innerHTML will be left alone. Comments, feedback and suggestions very much welcome!

Exciting times

April 9th, 2008

This last weekend I was up at Scotland on Rails – met loads of great people and saw lots of interesting talks. Well done to the organisers for putting together a Ruby/Rails conference in the UK. It had been a long time coming since RailsConf Europe came (and went) to London a few years ago (as an aside, is this always going to happen in Berlin now? I had hoped it was going to do a grand tour of Europe).

At the conference I was offered a job with Engine Yard and I’m delighted to say that I accepted the offer and will be starting in May!

Big thanks to Jamie van Dyke for giving me the opportunity.

Extension-less Formats in ActiveResource

March 26th, 2008 When trying to integrate Litmus with Fetch recently we came across an API that whilst appearing RESTful, didn't quite map onto ActiveResource conventions because their URLs 404'd if we included .xml as the extension.

So I quickly whipped up a little plugin to let us specify new formats that don't include extensions. 

1
2
3
4
5
6
7

class Something < ActiveResource::Base
  
  self.site = "http://someone:password@an-app.com"
  self.prefix = "/api/"
  self.format = :xml_no_extension
  
end


Now, calls to 


Something.find(:all)
will generate http://an-app.com/somethings rather than http://an-app.com/somethings.xml

Perfect!

You can grab the code from the ExtensionlessFormat GitHub repository

New Async Observer Start/Stop scripts

March 6th, 2008

I've updated the Async observer start/stop scripts that I've been using extensively for pushing off long running tasks in Litmus. Combined with a few extra cap tasks this has been working perfectly for us for several weeks now.

Lots of the code is pulled out of the ferret_server and spec_server code from the Ferret and Rspec projects respectively. Download it here async_worker.zip

and get some Cap tasks from below... 
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33

namespace :beanstalk do
  
  task :start, :roles => :app do
    run "/usr/bin/beanstalkd -d -l 127.0.0.1"
  end
  
  task :stop, :roles => :app do
    run 'killall beanstalkd'
  end
  
  task :restart, :roles => :app do
    stop
    start
  end
  
  namespace :worker do
    
    task :start, :roles => :app do
      run "cd #{deploy_to}/current && RAILS_ENV=#{rails_env} script/async_worker start"
    end
    
    task :stop, :roles => :app do
      run "cd #{deploy_to}/current && RAILS_ENV=#{rails_env} script/async_worker stop"
    end
    
    task :restart, :roles => :app do
      stop
      start
    end
    
  end
  
end

A productive morning for Litmus

March 6th, 2008

This is a somewhat self-congratulatory post today, after a few weeks of hard slog and testing we finally put the Litmus 24 hour passes live!

We’ve received lots of requests from customers for a credits based system, but we felt this didn’t fit the model of how people would use Litmus. When fixing a template bug people want to fire off lots of tests as they iteratively work through a problem without fearing how much each test is costing them. So we decided that a pass to use Litmus for 24 hours for as many tests as you care to do works perfectly. From the response we are getting from customers we know this was the right thing to do. We even got a nod from John Gruber over on Daring Fireball

That’s not all though…

In light of the growing demand for Litmus and our booming API/Whitelabel deals the current database server was taking a beating. So at 7am this morning we managed to successfully move the database over to a much more powerful server with only 20 minutes of downtime. So far the load looks to have eased and we look forward to the increased response times for our customers.

If that wasn’t enough, Matt managed to launch IE 8 Beta testing which should be available to all our customers immediately!

So if you haven’t already, swing by and give us a try – there’s a 7 day trial included in the monthly plans, or the new 24 hour pass is only €12.

Ruby on Rails: Quick tip for Find :conditions

January 11th, 2008 I stumbled across a sneaky tip for conditions when using find. When you might normally be doing this: 

1
2

Account.find(:all, :conditions => ['name LIKE ? AND updated_at < ?', 
"aname", 3.days.ago])
You can just do this instead: 
1
2

Account.find(:all, :conditions => ['name LIKE :name AND updated_at < 
:date', {:name => "aname", :date => 3.days.ago}])

This isn't the best example of using this symbol placeholder method, but imagine using it in a situation where you're repeating the same search param a few times. I know I've had a few times that my conditions has been something like 
1
2

['created_at > ? AND updated_at < ? 
AND invoiced_at > ?'], Time.now, Time.now, Time.now].
Using the hash at the end you could just pass this in as :time => Time.now and reuse :time in multiple places in your conditions hash. Neat.

Update: See ActiveRecord::Base on Noobkit for more info

Async Observer start/stop scripts

January 11th, 2008

Recently I’ve been adding some filters to Litmus which are firing off calls to remote services based on actions performed by our customers. Unfortunately this started to bring some “Timeout::Error” exceptions bubbling up when the remote services didn’t respond in time.

The functions these remote services provide are easily hidden if we get a failure when trying to contact them, the problem for us is that this meant a long wait time for customers whilst we tried to contact the service, followed by a 500 error once the connection had timed out.

I read about beanstalkd – its ability to asynchronously process jobs from a memcached like queue would be the perfect solution to the problem, allowing us to take a shot at reaching the remote services but not tying up mongrels and making our users wait.

Using it with Rails was easy using the async-observer plugin – I managed to get the whole thing up and running in around 30 minutes.

When it came to deploy I noticed that the async observers were in need of scripts to start and stop them so I hacked up a couple of scripts you can stick in your scripts directory and chmod +x ready to use in Capistrano.

I’ve literally just finished these scripts and they’ve had about 5 minutes of testing, so all comments/suggestions/corrections and even your own modifications posted elsewhere are welcome. Just drop any links/suggestions in the comments.

async-start-stop.zip

Update: I’ve just noticed that the “puts” returned by the async observer is causing Capistrano to hang during deployment. I’ll have to have a look at it when I get a chance. Any suggestions welcome. Ideally the scripts will still output some text to let you see something is happening during deployment…

Update: I’ve updated the script and it now works really well for us… get it here

Ruby: Require everything in a directory

July 3rd, 2007 Recently I've been working a lot with the Liquid templating system and in our application initialisation we needed to quickly require all the files in a particular directory that aren't in the normal rails load path. Normally a good way to do this is to just add the new paths onto your Rails load path 



$LOAD_PATH << File.join(File.dirname(__FILE__), '../app/drops')

or, in environment.rb 


config.load_paths += %W( #{RAILS_ROOT}/app/drops #{RAILS_ROOT}/app/filters )

This particular app uses engines so most of the configuration resides in our core plugin in order not to dirty up environment.rb and make is easier to keep all the configuration centrally managed. This means we can't really touch environment.rb or anything in the Rails initializer block for that matter.
Then I discovered the power of .glob 



Dir.glob(File.join(File.dirname(__FILE__), '../app/drops/*.rb')).each {|f| require f }

Perfect.

Edited: Yes - that second snippet of code is from Mephisto - at the moment it's the best reference for how to implement a liquid based templating system. Thanks Rick, you've really helped us out!

Rails Documentation Project

December 18th, 2006

What has happened to the attempts to fully document Rails? A while back, caboo.se launched a drive to raise money for paying authors to write up the largely undocumented and #nodoc Rails API but after collecting $16000 we still don’t seem to be getting anywhere.

I will put my hands up and admit that I have yet to contribute any money, but the reason I am posting about this now is because I do keep checking to see if anything is happening with the intent of throwing in $50 or so once the project gets moving – so far I’m not seeing this.

Considering that the original article says they are looking for $5000 to get the project started, then what’s the hold up?

What we really need in Rails is a solid documentation project aimed at developers working with the language everyday, which is fast to lookup, contains well researched examples of how to use the api and which offers the content in a variety of different formats (XML, RSS, HTML) for use in things such as dashboard widgets/custom browsers/textmate bundles.

So far all that seems to exist is a Stikipad Wiki for the project – come on guys, we need to keep up with the Django book project