This article is part of a series describing a port of the samples from Practical Common Lisp (PCL) to Clojure. You will probably want to read the intro first.
This article covers Chapter 9, Practical: Building a Unit Test Framework.
To build a minimal testing library, I need nothing more than tests and results. To keep reporting as simple as possible, I will start with console output. The report-result function tests a result, and prints pass or FAIL, plus a form with supporting detail:
(defn report-result [result form]
(println (format "%s: %s" (if result "pass" "FAIL") (pr-str form))))Now any function can be a test. The detail message can often be the same form that caused the error, so I will pass the same form twice: once for evaluation, and again (quoted!) for use in the detail message:
(defn test-+ []
(report-result (= (+ 1 2) 3) '(= (+ 1 2) 3))
(report-result (= (+ 1 2 3) 6) '(= (+ 1 2 3) 6))
(report-result (= (+ -1 -3) -4) '(= (+ -1 -3) -4)))The console output for test-+ looks like this:
user=> (test-+)
pass: (= (+ 1 2) 3)
FAIL: (= (+ 1 2 3) 7)
pass: (= (+ -1 -3) -4)The fact that I want to pass the same form twice, but with different evaluation semantics, just screams macro. Sure enough, I can clean up the code with a macro:
(defmacro check [form]
`(report-result ~form '~form))The macro expands the form twice, once for evaluation and once quoted for the detail message. Now I can replace calls to report-result with simpler calls to check:
(defn test-* []
(check (= (* 1 2) 3))
(check (= (* 1 2 3) 6))
(check (= (* -1 -3) -4)))Hmm. The calls to check are cleaner than the calls to report-result in the earlier example, but the check itself still looks repetitive. Solution: a better check macro that can handle multiple forms:
(defmacro check [& forms]
`(do
~@(map (fn [f] `(report-result ~f '~f)) forms)))The quoting and unquoting is a little more complex–play around with macroexpand-1 to see how it works.
With the better check in place, test functions are quite simple:
(defn test-rem []
(check (= (rem 10 3) 1)
(= (rem 6 2) 0)
(= (rem 7 4) 3)))So far I have tests and console output. Next, I need some way to aggregate a set of checks into a single, top-level "checks passed" or "checks failed".
I would like to simply and together all the individual checks, but that does not quite work. As in many languages, Clojure's and short-circuits and stops evaluating when it encounters a logical false. That's no good here: Even if one test fails, I still want all the tests to run.
Since it is a question of optional evaluation, a macro is appropriate. The combine-results macro works like and, but it always evaluates all the forms:
(defmacro combine-results [& forms]
`(every? identity (list ~@forms)))Now check can use combine-results instead of do.
(defmacro check [& forms]
`(combine-results
~@(map (fn [f] `(report-result ~f '~f)) forms)))All existing functionality still works, and now I can see a useful return value from a test.
user=> (test-*)
pass: (= (* 2 4) 8)
pass: (= (* 3 3) 9)
trueTests ought to have names. In fact, tests ought to support multiple names. You can imagine a test detail report saying:
Check math->addition->associative passed: ...
Where associative is the name of a check, addition is the name of a function, and math is the name of another function that called addition.
First, I need a variable to store a sequence of names:
(def *test-name* [])Printing the variable as part of a result is easy:
(defn report-result [result form]
(println (format "%s: %s %s"
(if result "pass" "fail")
(pr-str *test-name*)
(pr-str form)))
result)Now for the hard part: populating the collection of names. For this, I will introduce a deftest macro:
(defmacro deftest [name & forms]
`(defn ~name []
(binding [*test-name* (conj *test-name* (str '~name))]
~@forms)))The macro expansion perfomed by deftest is nothing new: deftest turns around and defns a new function named name. The interesting part is the call to binding, which rebinds *test-name* to a new collection built from the old *test-name* plus the name of the current test.
The new binding of *test-name* is visible anywhere inside the dynamic scope of the binding form. The dynamic scope includes any function calls made inside the binding, and their function calls, and so on ad infinitum … or until another binding performs the same trick again. This gives exactly the semantics we want:
test-name an an argument all over the place. Nested functions "remember" a stack of their caller's names through *test-name*.*test-name* outside a binding. Code after the binding will never see the values *test-name* takes during the binding.With deftest in place, I can defined a hierarchy of nested tests:
(deftest test-*
(check (= (* 2 4) 8)
(= (* 3 3) 9)))
(deftest test-math
; TODO: test rest of math
(test-*))
(deftest test-all-of-nature
; TODO: test rest of nature
(test-math))Calling test-all-of-nature will demonstrate multiple levels of nested name in a test report:
user=> (test-all-of-nature)
pass: ["test-all-of-nature" "test-math" "test-*"] (= (* 2 4) 8)
pass: ["test-all-of-nature" "test-math" "test-*"] (= (* 3 3) 9)
true From here, better formatting of the console message is just mopping up.
When I first read Practical Common Lisp, this was my favorite chapter. The testing library evolves quickly and naturally to a substantial feature set. (In case you didn't keep count, the entire "framework" is less than twenty lines of code.)
Try implementing the unit-testing example in your language of choice. Don't just implement the finished design. Work through each of the iterations described above:
I would love to hear about your results, and I will link to them here.