blog.mhartl | Michael Hartl's tech blog

I’m happy to announce the release of a new edge branch of the Insoshi social networking platform, which is fully updated with Rails 2.1 support. (The Insoshi demo site is currently running off this edge branch.) The result is a good example of upgrading a real-life application to Rails 2.1, so I thought a blog post detailing the steps might be useful.

Most of the changes here are quite generic—updates to widely used plugins, for example—and are not specific to Insoshi. I’ve been especially careful to include error messages when applicable, so that search engines can index them; I’m sure I’m not the only programmer who follows the “Google the error message” algorithm when debugging. I’ve also included the Git commands I used, since I know a lot of Rails developers are working to learn Git and I thought some examples might be helpful.

It’s important to note at the outset that you do not need to follow these steps yourself to upgrade Insoshi. Insoshi contributors, and anyone else who wants the edge version of Insoshi, should follow the instructions at the Insoshi wiki (if they haven’t already) and then issue the following commands:

$ git fetch origin
$ git branch --track edge origin/edge
$ git checkout edge

This way, you’ll get all these changes for free. (Once hosting support for Rails 2.1 is more widespread (especially at Heroku), we’ll merge the Rails 2.1 Insoshi edge into the master branch.)  UPDATE: The merge has happened, and now the Insoshi master branch runs Rails 2.1.

And now, on with our show. Here’s what it took to get Insoshi running under Rails 2.1.

Update RubyGems

Upgrading Insoshi to Rails 2.1 involves updating a bunch of plugins, many of which are now hosted at GitHub. Unfortunately, older versions of RubyGems can’t install directly from GitHub sources; for example, trying to install will_paginate with RubyGems 1.1.0 gives you this error message:

$ gem --version
1.1.0
$ sudo gem install mislav-will_paginate -s http://gems.github.com/
ERROR:  could not find mislav-will_paginate locally or in a repository

The solution is to update the system gems (including RubyGems) as follows:

$ sudo gem update --system
$ gem --version
1.2.0

Install Rails 2.1

Installing Rails 2.1 itself is the easiest step in the entire upgrade. (I think when people wonder, “How hard could upgrading to Rails 2.1 possibly be?”, they have mainly this step in mind.)

$ sudo gem update rails

Things now get a little trickier, since we want to freeze Rails 2.1 in vendor/rails to follow the best practice for production Rails apps. The first part goes smoothly:

$ git rm -r vendor/rails
$ git commit -a -m "Cleared out vendor/rails"

There’s a hitch with the second step. If you happen to have only Rails 2.1 (but not 2.0.2) on your machine, there’s a bootstrapping problem due to a variable in environment.rb:

$ rake rails:freeze:gems
Missing the Rails 2.0.2 gem. Please `gem install -v=2.0.2 rails`, update your
RAILS_GEM_VERSION setting in config/environment.rb for the Rails version you
do have installed, or comment out RAILS_GEM_VERSION to use the latest version
installed.

The solution is to update environment.rb with the proper Rails gem version:

# Specifies gem version of Rails to use when vendor/rails is not present
RAILS_GEM_VERSION = '2.1.0' unless defined? RAILS_GEM_VERSION

Then the freeze works fine:

$ rake rails:freeze:gems
$ git add .
$ git commit -a -m "Updated vendor/rails to Rails 2.1"

Finally, following the advice froma Akita’s Rolling With Rails 2.1, we add a new file called config/initializers/new_defaults.rb:

# These settings change the behavior of Rails 2 apps and will be defaults
# for Rails 3. You can remove this initializer when Rails 3 is released.

# Only save the attributes that have changed since the record was loaded.
ActiveRecord::Base.partial_updates = true

# Include ActiveRecord class name as root for JSON serialized output.
ActiveRecord::Base.include_root_in_json = true

# Use ISO 8601 format for JSON serialized times and dates
ActiveSupport.use_standard_json_time_format = true

# Don't escape HTML entities in JSON, leave that for the #json_escape helper
# if you're including raw json in an HTML page.
ActiveSupport.escape_html_entities_in_json = false

$ git add config/initializers/new_defaults.rb
$ git commit -m "Added new defaults initializer"

Update RSpec for Rails 2.1

A necessary step in verifying that the Insoshi application is working under Rails 2.1 is to get the test suite to pass. Our tests are written using RSpec, but older versions of RSpec don’t work with Rails 2.1, so we can’t even run the test suite. D’oh!

Since the old plugins don’t work, first we remove them:

$ git rm -r vendor/plugins/rspec*
$ git commit -a -m "removed outdated RSpec plugins"

Then we need to install the most recent versions of the RSpec plugins from GitHub:

$ script/plugin install git://github.com/dchelimsky/rspec.git
$ script/plugin install git://github.com/dchelimsky/rspec-rails.git
$ script/generate rspec

In my case, I was careful not to overwrite spec_helper.rb when running script/generate rspec in the last step, as the current spec helper contains several custom modifications that I didn’t want to lose.

Then we need to add the changes:

$ git add .
$ git commit -a -m "Added latest RSpec plugins from GitHub"

By the way, running specs from the command line won’t work:

$ spec spec/models/person_spec.rb
Your RSpec on Rails plugin is incompatible with your installed RSpec.

RSpec          : 20080526202855
RSpec on Rails : 20080628203842

To fix this, you can use script/spec in place of spec, or you can upgrade to the latest RSpec gem using the RSpec source from GitHub:

$ cd ~/tmp
$ git clone git://github.com/dchelimsky/rspec.git
$ cd rspec
$ rake gem
$ sudo gem install pkg/rspec-1.4.1.gem

(All of Insoshi’s tests use RSpec, but if you use Test::Unit you should know that as of Rails 2.1 default tests can’t be run at the command line using the ruby executable; for tests generated by Rails 2.1, you now have to include the test directory explicitly using the -I test flag. For a hypothetical resource foobar, for example, you would get this:

$ script/generate scaffold foobar baz:string
$ ruby test/functional/foobars_controller_test.rb
test/functional/foobars_controller_test.rb:1:in `require':
no such file to load -- test_helper (LoadError)
        from test/functional/foobars_controller_test.rb:1

It passes if you tell ruby about the test directory:

$ ruby -I test test/functional/foobars_controller_test.rb
Loaded suite test/functional/foobars_controller_test
Started
.......
Finished in 0.24375 seconds.

7 tests, 13 assertions, 0 failures, 0 errors

(N.B. Using rake still works fine.) It’s unclear why the Rails Core team decided to make this change, but it’s a definite gotcha so I thought it deserved mention.)

With the new RSpec installed we can at least run the test suite, but unfortunately there are huge amounts of breakage. This is mainly due to several plugin incompatibilities and a slight Rails 2.1/Insoshi conflict (discussed below). Let’s get started fixing them.

Update obsolete helper specs

One source of breakage is the helper specs (in the spec/helpers/ directory). All these specs have obsolete code, resulting in errors such as

NoMethodError in 'TopicsHelper should include the TopicHelper'
undefined method `metaclass'

By going to a temp directory and running a sample rspec_scaffold with the updated RSpec to get a sample spec template file, you can discover that the line

included_modules = self.metaclass.send :included_modules

needs to be changed to

included_modules = (class << helper; self; end).send :included_modules

in each helper spec. You can use search-and-replace in your text editor to make all the changes, and then commit them:

$ git commit -a -m "Updated outdated spec helpers"

This won’t fix everything in the helper specs; there are still warning messages like this:

Modules will no longer be automatically included in RSpec version 1.1.4

This is because the ActivitiesHelper module is not explicitly included in the helper spec. The fix is to add the relevant include:

spec/helpers/activities_helper_spec.rb

require File.dirname(__FILE__) + '/../spec_helper'
include ActivitiesHelper
.
.
.

Update will_paginate for Rails 2.1

The old will_paginate plugin won’t work with Rails 2.1. You get tons of errors like

SystemStackError in 'PeopleController people pages should have a working show page'
stack level too deep

We need to remove the old plugin and install an update from GitHub. The current recommended method is to install it as a gem, and luckily Rails 2.1 has a slick new method for dealing with gem dependencies. In principle, we just need to add the following lines to environment.rb

Rails::Initializer.run do |config|
  .
  .
  .
  # Custom gem requirements
  config.gem 'mislav-will_paginate', :version => '~> 2.3.2',
                                     :lib => 'will_paginate',
                                     :source => 'http://gems.github.com'
end

This tells Rails that our application requires will_paginate version 2.3.2 or later, and that it can be found at GitHub. We can do the installation like this:

$ sudo rake gems:install

Unfortunately, in our case this won’t work, since will_paginate doesn’t work as a gem if the application has Rails in vendor/rails. (This gotcha is buried in the will_paginate wiki at GitHub.) We have to install will_paginate as a plugin after all:

$ git rm -r vendor/plugins/will_paginate
$ script/plugin install git://github.com/mislav/will_paginate.git
$ git add .
$ git commit -a -m "Updated will_paginate plugin"

(Unfortunately, this still didn’t fix the problem for Insoshi, because there were extra files in the lib/ directory:

$ git rm -r lib/will_paginate*
$ git commit -a -m "Removed will_paginate from lib"

I’m not sure how that happened, but it sure took a while to figure out…)

Update TextMate footnotes for Rails 2.1

Now that will_paginate is fixed (and the corresponding specs pass), you’d think the relevant pages would work. You’d be wrong. We were using the edge version of TextMate foototes, and it turns out that the footnotes-edge plugin breaks horribly in Rails 2.1. Basically, every page gives you something like this in the browser:

ActionController::RenderError in HomeController#index

You called render with invalid options : {:layout=>false, :action=>"index"}, nil

To fix this, we need to install an update from GitHub (are you seeing a pattern here?):

$ git rm -r vendor/plugins/footnotes-edge
$ script/plugin install git://github.com/drnic/rails-footnotes.git
$ mv vendor/plugins/rails-footnotes vendor/plugins/footnotes
$ git add .
$ git commit -a -m "Updated TextMate footnotes"

Update attachment_fu for Rails 2.1

We’re almost there. A few photo specs still fail, with messages like

NoMethodError in 'PhotosController when logged in should create photo'
undefined method `callbacks_for' for #<Photo:0x529a61c>

The solution is to update attachment_fu:

$ git rm -r vendor/plugins/attachment_fu/
$ script/plugin install http://svn.techno-weenie.net/projects/plugins/attachment_fu/
$ git add .
$ git commit -a -m "Updated attachment_fu"

Fix the broken verify action

This would seem to complete the update, but unfortunately this isn’t the end; there’s one more (Insoshi-specific) problem. Insoshi includes the option to verify the email addresses of new members, using a custom action called verify inside the People controller. Unfortunately, the specs that test the email verification fail with the error

No action responded to verify

This looked to be a general problem with custom actions in Rails 2.1 tests, but it turns out that the culprit is the word verify. For some reason, an action called verify causes problems in Rails 2.1, but only in tests. (This took a long time to figure out.) The (rather inelegant) fix is to rename the controller action to verify_email, and then add a line in the routes file so that old email verification links still work:

app/controllers/people_controller.rb

def verify_email
  .
  .
  .
end

config/routes.rb

map.resources :people, :member => { :verify_email => :get,
                                    :common_contacts => :get }
map.connect '/people/verify/:id', :controller => 'people',
                                  :action => 'verify_email'

Of course, we also need to change get :verify to get :verify_email in the spec file (spec/controllers/people_controller_spec.rb). Once that is done, all the specs pass, and the upgrade is complete:

$ rake spec
..............................................................................
..............................................................................
..............................................................................
..............................................................................
..............................

Finished in 12.337675 seconds

342 examples, 0 failures

Phew!

Postscript: Rails 2.1 migrations, schema_info, and schema_migration

One benefit of moving to Rails 2.1 is the new method for handling migrations. We’ve already run into instances with the Insoshi project where we needed to merge updates with conflicting migration numbers, and it’s really not any fun with Rails 2.0. There is a potential gotcha, though, since in order to perform the new migration cleverness Rails 2.1 uses a table called schema_migration in place of the old schema_info table. The change is supposed to happen automatically when you first migrate after installing Rails 2.1, but we ran into some difficulties…

When making a Rails 2.0 to 2.1 upgrade, you shouldn’t run into any problems if you start from a clean database and run

$ rake db:migrate

If you have an existing database (for instance, our demo site database), the migration should automatically convert the schema_info table used in 2.0 (which stores only a single integer value) to schema_migration (which has entries for all the migrations that have been performed).

While that worked for us in development, we ran into an issue on our staging server (where we test everything before installing it on our production servers): the expected table conversion didn’t happen. Instead, the migration tried to recreate the tables from scratch. We got around this by bootstrapping the conversion before the migration, as follows.

1. Create the schema_migrations table via SQL:

   $ mysql insoshi_production -u root
   mysql> CREATE TABLE schema_migrations (
          version VARCHAR(255) NOT NULL,
          UNIQUE KEY unique_schema_migrations (version)
          );
   

2. Find out the current migration version number:

   mysql> SELECT version FROM schema_info;
   +---------+
   | version |
   +---------+
   |      26 |
   +---------+
   1 row in set (0.07 sec)
   

3. Insert values from 1 to the current version as strings (i.e., as a VARCHAR array):

   mysql> INSERT INTO schema_migrations VALUES ('1'),('2'),('3'),('4'),('5'),
   ('6'),('7'),('8'),('9'),('10'),('11'),('12'),('13'),('14'),('15'),
   ('16'),('17'),('18'),('19'),('20'),('21'),('22'),('23'),('24'),
   ('25'),('26');
   

Of course, you shouldn’t have to do this—things should Just Work™—but then again, upgrading to Rails 2.1 wasn’t supposed to be difficult, either. :-)