As you've probably gathered by now, we're passionate about testing. Over the course of the past year, we've implemented various improvements to rcov — the Ruby code coverage analysis tool — to help satisfy our testing appetite.

To date, we've squashed numerous bugs, simplified the report-generation logic (making way for better template support in the future), and achieved experimental support for Ruby 1.9. While working out bugs reported by developers using our fork of rcov (relevance-rcov), one thing has become painfully obvious to us: the bug reports lack the context we need to properly troubleshoot the issues. Fortunately, rcov is bundled with a handy tool that can help mitigate this problem, and we'd like to take a moment to share the details.

Let's say you're having an issue with rcov and you want to report a bug. First, simply locate the test that produces the bug. Then, use rcov's report-cov-bug flag to produce the bug report. From your console run:

  rcov --report-cov-bug='Class#method' path/to/testfile

As a practical example, let's say that we have a test for a Rails ActiveRecord class called Project and an offending method called #archived?. This class has Micronaut-based tests located in examples/models/project_example.rb. In order to produce an rcov bug report you would run:

  rcov --rails --report-cov-bug='Project#archived?' examples/models/project_example.rb

This will produce a nicely-formatted bug report that tells us what version of Ruby you are running as well as your version of rcov, the arguments you passed in, and the method in question. Along with this information, it will provide additional details about how the code was analyzed. The end result will look something like the following:

  Bug report generated on Wed Jul 22 13:17:35 -0400 2009

  Ruby version:              1.8.6 (2008-08-11)
  Platform:                  universal-darwin9.0
  rcov version:              0.8.3.4
  rcovrt loaded?             true
  using RubyGems?            true
  Command-line arguments:    ["--rails", "--report-cov-bug=Project#archived?", "examples/models/project_example.rb"]
  Coverage analysis bug in:  Project#archived?

  Line(s) 46 should be green.
        
  Raw coverage information (feel free to remove useless data, but please leave
  some context around the faulty lines):

        3:   45:  def archived?
        2:   46:    archived_at
        2:   47:  end
        0:   48:

Once you have that information, you're ready to log the issue at github.com/relevance/rcov/issues.

Known Issues with report-cov-bug

As helpful as the report-cov-bug tool is, it does fall down in a few places:

• ActiveRecord "declarative" methods (e.g., named_scope, default_scope) report the actual ActiveRecord code, not the model code.
• The C version of #callsite does not accurately report executed methods for Ruby 1.9.

Supported Environments

We use rcov on Linux and OS X for all our Ruby 1.8.x projects. If you run into an issue using rcov in this environment, please let us know.

What about other runtimes and platforms?

• For JRuby projects, be sure to check out jruby-rcov. (relevance-rcov does not currently support JRuby.)
• The Ruby 1.9 support is not ready for prime time. We consider it an experiment at this point.
• We are not Windows users day-to-day, so Windows is not currently supported. However, you're welcome to fork the repository if you have the urge to scratch that itch.

You Can Help Build a Better rcov

Providing the report-cov-bug output with your bug report gives us a much better chance of tracking down the root cause of the issue. Take relevance-rcov for a spin on your project, and if you see any hiccups, let us know.