blog.mhartl | Michael Hartl's tech blog

2008-09-26

Using Rails to serve different content to humans and robots

Filed under: Insoshi, Ruby on Rails — mhartl @ 11:38

This post answers the question, How do you use Rails to do one thing for robots, and another thing for humans?

Why would you want to do this? In our case, the Insoshi home page forwards to a portal page that uses frames in order to have an interface that unifies the sites on the insoshi.com domain with those off-site, such as our GitHub repository and bug tracker. The frames page is horribly search-unfriendly, though, so we serve bots the actual content of http://insoshi.com/home/index (our routes map / to /home/index). (Note: The front of the portal page as seen by a human is the same as the index page served to bots; be careful about doing anything else, since bots can punish you if you use this technique for anything slimy.)

Our method is to use a before filter in the Home controller. Here’s the code (minus some irrelevant bits):

class HomeController < ApplicationController
  before_filter :forward_nonbots_to_portal, :only => "index"
  
  .
  .
  .

  private
  
    # Return true if the user agent is a bot.
    def robot?
      bot = /(Baidu|bot|Google|SiteUptime|Slurp|WordPress|ZIBB|ZyBorg)/i
      request.user_agent =~ bot
    end
    
    # Allow an explicit override of the forward_nonbots_to_portal.
    def no_redirect?
      params[:redirect] == 'false' or RAILS_ENV['ENV'] != 'production'
    end
    
    def forward_nonbots_to_portal
      redirect_to "http://portal.insoshi.com" unless robot? or no_redirect?
    end
end

The key here is the robot? method, which has a regex with a list of the most common bots user agents:

    # Return true if the user agent is a bot.
    def robot?
      bot = /(Baidu|bot|Google|SiteUptime|Slurp|WordPress|ZIBB|ZyBorg)/i
      request.user_agent =~ bot
    end

If the user isn’t a bot, we redirect them to the portal.

N.B. The second boolean, no_redirect?, prevents the forwarding in development mode and also allows us to override the redirect by passing a redirect=false parameter. This latter condition allows us to link directly to the home page, without a redirect, by using http://insoshi.com/?redirect=false. In particular, the portal menu home link itself uses this URL, because otherwise clicking on the home link repeatedly would cause a bunch of nested portal pages to appear.

About these ads

2 Comments

  1. ifmvooozomgcnlrkwell, hi admin adn people nice forum indeed. how’s life? hope it’s introduce branch ;)

    Comment by Bariareshohensully — 2008-12-29 @ 17:30

  2. “Interesting Article – adding this to RSS now :)

    I’m also interested, which theme do you have here? I know someone who would to know if it’s a free one.”

    Comment by ffxi gil — 2009-02-26 @ 16:54


RSS feed for comments on this post.

Sorry, the comment form is closed at this time.

Customized Shocking Blue Green Theme Blog at WordPress.com.

Follow

Get every new post delivered to your Inbox.

Join 28 other followers

%d bloggers like this: