I ran a few benchmarks under 1.8.7 MRI, 1.8.7 REE, 1.9.2 and Rubinius to see how they compare. I also benchmarked to_json to have an order of magnitude. My benchmarks consist of serializing/deserializing a relatively complex Hash (many nested levels). The YAML representation of the Hash is ~3kb.

Conclusions:

• YAML is really slow
• YAML is twice as fast under 1.9.2 vs 1.8.7
• ZAML is much faster than YAML under 1.8.7, but about the same in 1.9.2
• REE (no GC tweaks) isn’t faster in this situation
• Rubinius is ridiculously slow
• Psych under Ruby 1.9.2 is disappointing

Here are the results:

ruby 1.8.7 (2009-06-12 patchlevel 174) [universal-darwin10.0]
--------------------------------------------------------------------
YAML.load:	0.7924
Marshal.load:	0.0742
to_yaml:	5.1661
ZAML.dump:	1.4574
to_json:	0.1152
Marshal.dump:	0.0845

ruby 1.8.7 (2010-04-19 patchlevel 253) [i686-darwin10.4.0], MBARI 0x6770, Ruby Enterprise Edition 2010.02
--------------------------------------------------------------------
YAML.load:	0.7976
Marshal.load:	0.0864
to_yaml:	4.9012
ZAML.dump:	1.3508
to_json:	0.0940
Marshal.dump:	0.0812

ruby 1.9.2p0 (2010-08-18 revision 29036) [x86_64-darwin10.5.0]
--------------------------------------------------------------------
YAML.load:		0.7079
YAML.load (Psych):	2.6055
Marshal.load:		0.0928
to_yaml:		2.1682
to_load (Psych):	2.2924
ZAML.dump:		5.0259
to_json:		0.1133
Marshal.dump:		0.0908



rubinius 1.1.1 (1.8.7 4018ebda 2010-11-16 JI) [x86_64-apple-darwin10.5.0]
--------------------------------------------------------------------
YAML.load:	2.7777
Marshal.load:	0.8560
to_yaml:	23.5828
ZAML.dump:	4.0745
to_json:	3.6620
Marshal.dump:	3.3797

If you want to benchmark on your machine, here’s the source code. Sorry, I couldn’t include the YAML string as it is confidential data.

require 'rubygems'
require 'benchmark'
require 'yaml'
require 'zaml'
require 'json'

yaml_str = <<-eos
yaml goes here
eos

h = nil
GC.start
yaml_load_time = Benchmark::realtime {
1000.times { h = YAML.load(yaml_str) }
}

GC.start
to_yaml_time = Benchmark::realtime {
1000.times { h.to_yaml }
}

begin
  YAML::ENGINE.yamler = 'psych' if RUBY_VERSION == "1.9.2"
rescue
  puts "Cannot load Psych. Is libyaml installed? Install it and recompile Ruby."
  puts "On OS X with RVM, install it with MacPorts and run:"
  puts "rvm install ruby-1.9.2 -C --with-libyaml-dir=/opt/local"
  exit 1
end

GC.start
psych_load_time = Benchmark::realtime {
  1000.times { YAML.load(yaml_str) }
}

GC.start
to_yaml_psych_time  = Benchmark::realtime {
  1000.times { h.to_yaml }
}

GC.start
to_json_time = Benchmark::realtime {
1000.times { h.to_json }
}

GC.start
zaml_dump_time = Benchmark::realtime {
1000.times { ZAML.dump(h) }
}

marshaled = nil
GC.start
marshal_dump_time = Benchmark::realtime {
1000.times { marshaled = Marshal.dump(h) }
}

GC.start
marshall_load_time = Benchmark::realtime {
1000.times { h = Marshal.load(marshaled) }
}

puts `ruby -v`
puts "--------------------------------------------------------------------"
puts "YAML.load:\t\t#{'%.4f' % yaml_load_time}"
puts "YAML.load (Psych):\t#{'%.4f' % psych_load_time}" if RUBY_VERSION == "1.9.2"
puts "Marshal.load:\t\t#{'%.4f' % marshall_load_time}"
puts "to_yaml:\t\t#{'%.4f' % to_yaml_time}"
puts "to_load (Psych):\t#{'%.4f' % to_yaml_psych_time}" if RUBY_VERSION == "1.9.2"
puts "ZAML.dump:\t\t#{'%.4f' % zaml_dump_time}"
puts "to_json:\t\t#{'%.4f' % to_json_time}"
puts "Marshal.dump:\t\t#{'%.4f' % marshal_dump_time}"

Alex Coles (@myabc) volunteered and is now the new maintainer. He already shared with me a couple good ideas for the site, which he’ll hopefully implement soon.

Thank you, Alex, and congrats!

I need to give credit to this page, without which I’d still be fighting with this whole thing.

This tutorial uses JRuby 1.2, Glassfish 2.1, Merb 1.0.10 and Ubuntu 8.04 LTS. To make my life easy, I simply fired an EC2 instance based on ami-71fd1a18. I recommend you do the same. However, it should work from just any vanilla Ubuntu 8.04.

Some important comments are in the script below, make sure to read them. Enjoy!

Managing gems has been painful lately. Like most people, we’re freezing them, which helps tremendously. However, there’s a few problems with this approach:

• We can’t freeze C extensions because they need to be compiled for the target platform. We develop on OS X and Linux 64, but deploy to Linux i386. As a workaround, we currently freeze but recompile them on every deployment. Definitely not optimal.
• Our frozen gems directory is getting huge.
• It’s becoming complicated and quite a mess to manage multiple gem versions for different branches/tags.
• We never quite know which branch needs which gem.

Of course, we could manually install the gems we need on our servers. But the word “manually” generally leads to “epic fail”.

So I thought, why not install them at deployment time, just before restarting the Mongrels, with Capistrano? That seemed like a pretty good idea! The only problem is that Rubygems will always install a gem when asked to do so, even if it’s already installed.

That’s where install_gems.rb comes in. install_gems.rb takes a file name as an argument. This file contains a simple list of gems along with their version number. A custom installation command can also be specified.

Here’s an example:

# This is an example file. As you can see, it supports comments.
haml 2.0.3
eventmachine 0.12.2
# Notice the custom command specified after the --
termios 0.9.4 -- gem install termios -v 0.9.4 --no-rdoc --no-ri

To install the above gems, you’d simply run
./install_gems.rb input_file

install_gems.rb has a few advantages over vanilla rubygems:

• It will only install a gem if the specified version is not already installed. That’s obviously much faster than reinstalling everything every time.
• C extensions will be properly compiled.
• Using different gem versions in different branches/tags/releases is now very easy. You just have to maintain a list of required gems in your Git repository and make sure Capistrano runs install_gems.rb against this list before restarting the application servers.
• It forces us to keep an up-to-date list of the gems we need.

Of course, for this to work as expected, you’ll need to require specific gem versions in your application. Your production server will likely have multiple versions of the same gems installed and if you don’t specify which version of the gem you want to require, things might (or will) break.

I released install_gems.rb on GitHub and Daniel Haran has already submitted a patch. Feel free to do so as well, I’m a puller!

Investigating…


1) Failure:
test_new_articles(AnnounceArticleFunctionalTest)
[test/functional/announce_article_test.rb:74:in `test_new_articles'
/usr/lib/ruby/gems/1.8/gems/mocha-0.5.6/lib/mocha/test_case_adapter.rb:19:in `__send__'
/usr/lib/ruby/gems/1.8/gems/mocha-0.5.6/lib/mocha/test_case_adapter.rb:19:in `run']:
<Wed, 21 May 2008 15:12:50 +0000> expected but was
<Wed, 21 May 2008 15:12:50 +0000>.


Leopard’s Rails installation worked great until I updated Rubygems with sudo gem update –system. After I did that, most of my gems were broken, and reinstalling them didn’t help. According to Google, I was not alone.

The right thing to do for Apple would have been to ship Leopard with MacPorts and have Ruby and Rubygems installed through it. Unfortunately, they had to reinvent the wheel…

This tutorial will guide you through fixing that mess. We will be reinstalling Ruby and Rails with MacPorts and deleting the original Ruby installation to avoid conflicts.

You might want to backup your system before proceeding as I cannot be held responsible if something goes wrong (my lawyer made me write that). I tried it on 2 different machines and it worked fine for me.

So let’s get started already!

Step 1: Install MacPorts

Start by installing MacPorts if you don’t already have it on your machine. It’s an awesome must-have piece of software anyways! It’s super easy to install with the Leopard package.

Step 2: Install XCode 3.0

You must also have XCode 3.0 installed. It’s a huge download so you might want to install it from your Leopard DVD. It’s in /Optional Installs/Xcode Tools/XcodeTools.mpkg

Installation will take some time, so go read Digg, Reddit, or install Defensio on your blog!

Step 3: List your gems

Get a list of your installed gems and save it to a text file somewhere. You will need to reinstall them! [source:php]gem list > ~/Desktop/installed_gems.txt[/source]

Step 4: Clean up your Mac

In order to avoid conflicts with your original Ruby installation, I recommend you simply delete it. To do so, run these commands (I know… scary stuff!):[source:php]sudo rm -r /System/Library/Frameworks/Ruby.framework/
sudo rm -r /Library/Ruby
sudo rm /usr/bin/ruby
sudo rm /usr/bin/gem
[/source]

Step 5: Install Ruby and Rubygems with MacPorts

If you don’t perform step 4, you will run into problems at step 6.

Now go ahead and install Rubygems through MacPorts. It will automatically install Ruby (and many other things) as it’s a dependency. The command is: [source:php]sudo port install rb-rubygems[/source]

This one will also take a while… When it’s done, update Rubygems: [source:php]sudo gem update –system[/source]

Step 6: Install Rails

At this point, you should have a brand new Ruby and Rubygems installation working! That wasn’t too hard was it?

Now let’s install Rails.

• For Rails 1.2.6, run [source:php]sudo gem install rails -v 1.2.6[/source]
• For Rails 2 (aka latest-and-greatest), run [source:php]sudo gem install rails[/source]

Note that recent versions of Rubygems no longer requires the –include-dependencies parameter.

Step 7: Reinstall your gems

Now’s the time to reinstall your original gems.

To reinstall the gems that shipped with Leopard, execute this command: [source:php]sudo gem install RedCloth acts_as_ferret mongrel fcgi capistrano ruby-openid ferret dnssd hpricot sqlite3-ruby libxml-ruby termios[/source]

Then, you might want to reinstall the other gems you had. The list is on your desktop in installed_gems.txt.

That’s it! You now have a more standard and less prone to problems Rails stack on your Mac!

As you know, restarting your a Mongrel cluster is a matter of issuing this command: /etc/init.d/mongrel_cluster restart. Here’s what happens for a cluster of 4 Mongrels:

For one of our application, the “stop” time is about 2 seconds and the “start” time is somewhere around 10 seconds for each Mongrel process. Which means that for a cluster of 15, we have a window of about 3 minutes where at least 1 and often many Mongrels are innaccessible. Worse, there’s even a point where there is no Mongrel running at all. In a high traffic production environment, this is not acceptable.

With some hacking, I managed to modify mongrel_cluster_ctl (the script called by /etc/init.d/mongrel_cluster) to avoid the above scenario. My hack also makes sure that at most 1 Mongrel will be down at any given time. Here’s a graphical representation of my hack:

I believe this is much more efficient. Moreover, when implemented with Swiftiply, downtime is reduced to zero since Swiftiply will detect a “dead” Mongrel and route requests to one that is alive.

Links:

• Download my hacked mongrel_cluster_ctl file
• How to configure Swiftiply with Nginx
• Seesaw, an alternative approach to zero-downtime Mongrel restarts

In a nutshell, I had:

URL: /families/1/people/5
[source:ruby]
family_id = params[:family_id]
person_id = params[:id]
[/source]

However, for the sake of simplicity and to be able to use the method in question from other controllers (ex: /families/1/people/5/friends/7) without any modifications, I wanted to do person_id = params[:person_id] regardless of which controller was being accessed.

This was fairly easy to fix and I rolled out a little helper method I call smart_params:

[source:ruby]
def smart_params
  if params.key?(:id)
    param_name = params[:controller].singularize + ‘_id’
    params[param_name.to_sym] = params[:id]
  end
end
[/source]

I put it in /app/controllers/application.rb and added before_filter :smart_params to the class. Voila! Unified and consistent params for your IDs!

I don’t know if you’ll find this useful for anybody else, but it’s saving me a lot of time.

EDITS:

March 3, 2007: modified smart_params slightly to avoid creating a new nil param when params[:id] does not exist.