Opie thought he would find a kewl looking girl cat on the computer
Creative Commons License photo credit: turtlemom4bacon

UPDATE: Jeff Brown, one of the architects of MbUnit and Gallio, responded to my post suggesting other (=better) ways of checking assertions. I’ve added his suggestions at the bottom of the post.

We’re developing an acceptance test framework (which I will write more about when it reaches some maturity state) which will use MbUnit and Gallio to execute the test code. We developed some utility assertions of our own (which in turn use MbUnit Assert* methods) and we wanted to test them using MbUnit. So basically we wanted to unit test the unit test code ;) .

An interesting problem occurred when we wanted to test that one of these assertions actually fails under certain conditions. MbUnit throws AssertionException when an assertion fails, but this gets eaten by test runners as an indicator that the test case has failed (obviously). Of course, we didn’t want the test to fail, since we expect our assertion method to fail… OK I know it sound complicated, so let me show you the code instead of blabbering too much:

try
{
    runner.AssertSmsReceived("incorrect sms");
    Assert.Fail("Exception should have been thrown here");
}
catch (AssertionException ex)
{
    // this is to filter out an assertion for wrong SMS received.
    Assert.IsFalse(ex.Message.Contains("Exception should have been thrown here"));
}

Explanation: we expect runner.AssertSmsReceived() to throw AssertException. That’s why we catch this exception afterwards. But if the method has not failed, we force the failure with Assert.Fail(). Since both conditions throw the same type of an exception (AssertionException), we check its message contents to find out if the proper condition was met.

There’s probably a better way to do this, but I haven’t found it (other than throwing a different type of exception instead of calling Assert.Fail(), but I wanted to avoid this because Assert.Fail() gives a cleaner test result output). Or it’s just too late in the day for me to think…

Yes, there’s a better way (thanks Jeff):

   // check if received message is correct
    AssertionFailure[] failures = AssertionHelper.Eval(()
         => runner.AssertSmsReceived("incorrect sms"));
    Assert.AreEqual(1, failures.Length);
    Assert.IsTrue(failures[0].Message.Contains("did not receive an expected SMS message"));

Jeff also posted a helper class which Gallio guys use for testing MbUnit v3 asserts:

[TestFrameworkInternal]
public static AssertionFailure[] Capture(Gallio.Action action)
{
    AssertionFailure[] failures = AssertionHelper.Eval(action);

    if (failures.Length != 0)
    {
        using (TestLog.BeginSection(”Captured Assertion Failures”))
        {
            foreach (AssertionFailure failure in failures)
                failure.WriteTo(TestLog.Default);
        }
    }

    return failures;
}