Using `check` functions to protect your tests from code change


Userlevel 1
Badge +2

Aleksy Kladov wrote a really interesting article on his view on testing, which you can find here.  One thing that really stood out to me was the idea of using a check function to wrap the SUT, protecting you from data type changes.

 

Here’s an example of what he proposes.  Say we’re testing a method presenceOf() which checks whether a collection has an object and, if so, returns true.  Our tests look like this:

presenceOf tests:

// This is psuedocode, but we're just checking if "dog"
// is present in the array ["cat", "dog")

test ArrayWithStringPresent {
assert( presenceOf( "dog" in ["cat", "dog"] ) == true );
}

test ArrayWithStringAbsent {
assert( presenceOf( "fish" in ["cat", "dog"] ) == false );
}

OK, great… but what if we change the method to return the position of the item in the array, and -1 if absent?  What if we change it so it throws an exception if the element is present?  We could change all our tests… But we might have dozens of them, one for each Collection type. 

 

Instead, Aleksy proposes introducing a check function that we call instead of making assertions directly, so we can just alter that.  Basically, it’s a custom assertion that knows how to consume our input and desired output:

 

presenceOf tests with check function:

method checkPresenceOf(Value v, Collection c) {
Boolean returnValue = false;
try {
Integer presence = presenceOf(c, v);
if presence > 0, returnValue = true;
}
return returnValue;
}

test ArrayWithStringPresent {
assert( checkPresenceOf( "dog" in ["cat", "dog"] ) == true );
}

test ArrayWithStringAbsent {
assert( checkPresenceOf( "fish" in ["cat", "dog""] ) == false );
}

 

I like this idea…. But I’m torn

I gave a talk at SauceCon where I did some live test refactoring.  One of the things I did during that talk was create a custom assertion to simplify my test cases.  It enhances test code because it makes it clearer what’s actually under test, rather than duplicating how you’re going to test… But I’m not entirely sure about Alexsy’s usecase.

 

How do you know whether it’s worth writing the wrapper function or not.  Sure, if you’re having to update a bunch of tests now and you think you might do it in the future it’s a good idea, but his article makes it sound like it’s something you should be doing in advance.  Is it just a matter of experience, when you learn what methods might be prone to change?  Is it worth boiling the ocean to wrap every single repeat assertion in a custom wrapper?


0 replies

Be the first to reply!

Reply