gem-testers: Great QA Justice in a Gem

2011-01-18 19:00:00 -0500

If you're my age (I had a MC Hammer cassette tape in elementary school) and a rubyist, chances are you've done a dash of Perl in your time. Whatever the reason may be, we are now rubyists, and writing our web applications, database interaction and daemons in Ruby.

When it comes to Perl vs. Ruby, two things immediately stand out heavily weighted as superior in Perl's court:

• Documentation
• Quality Assurance

Today, I'm focusing on the latter. In fact, it has been my full-time focus for the last two months outside of work, and a healthy chunk of the last two years.

"Anything your QA infrastructure can do, mine can do better" - anonymous, possibly fictional Perl programmer

And they're right. Compared to the swath of tools available to the eager Perl tester, Ruby's testing facilities look weak in comparison. Granted, compared to the programming spectrum as a whole, Ruby's not so bad. Perl, however, has an amazingly great testing credo in which several things happen even if you don't care:

1. When you install a module from CPAN, it is tested before it can be installed.
2. Day in day out, results from testing are reported to a site where they can be reviewed by library authors. This is SERIOUS BUSINESS.
3. Good Perl programmers simply do not use libraries that aren't tested or documented. This is fact.

Note that I am not saying use Perl; here. Perl is a fine system, but we're attracted to Ruby for a reason, right? Instead, let's make Ruby better. We need testing on a variety of environments to fix bugs in our gems. Gem authors frequently do not have the resources that, say, the whole freakin' internet has.

Gem Testers and it's companion gem, rubygems-test are an attempt to provide these things. The gem testers system is pretty crude at this point, but should be a good start. Here's what it does when you run gem test rubygems-test:

• Runs 'rake test' for the provided gem.
• Captures the output of the system and records any exit status.
• Reports this to the Gem Testers Website
• Returns a happy little URL you can give to people when you write that angry trouble ticket about their gem.

Additionally, gem users are unfortunately more and more frequently finding themselves confronted with gems that are inadequately suited to their needs. This isn't necessarily because they are bad; maybe they haven't been maintained, or simply don't work on their preferred platform. Gem Testers is not just for gem authors; users can quickly work with the results to adequately vet the stability of a particular gem (or even a particular version). Evaluating libraries can be one of the more important things you do when starting a new application or library feature.

Josiah Kiehl and I have made a herculean effort with the help of people on the rubygems and various ruby platform teams to support a wide range of ruby platforms. All of these rubies run rubygems-test on Windows, Mac, and other POSIX systems:

• MRI 1.8
• MRI 1.9
• Rubinius 1.2
• JRuby 1.5 and 1.6beta
• Windows versions include both Luis Lavena's RubyInstaller and the 'garbagecollect' Visual C builds.

You're out of excuses

rubygems-test is easy to integrate, most likely supports your preferred platform, and gives you testing that you simply don't have time or resources to produce on your own. Heck, even gem building systems like Hoe and Ore are adding support for it. Why not take the plunge? I dare you to!

Want to contribute?

The entire project is open source, down to the Capfile. You can help us! rubygems-test and gem-testers have all the fixings for you to help!

blog comments powered by Disqus