Better Unit Test Data For An Infinite Range

Imagine you come across the listing below. It’s for a parameterised unit test verifying that a ShoppingCart’s Add() method throws an InvalidQuantity exception when the quantity argument is either 0 or a negative integer value. The quantity will only ever be an integer. 

The unit test code is OK. However, the same cannot be said for the test data values: -4, -6, -2, and -5. 

I have two questions for you:

1. What problems can you see with the test data values?
2. And, what test data values would you choose for this unit test?

The above unit test asserts that a ShoppingCart instance will throw an exception when we are adding a quantity containing zero or negative (integer) value. Yet, the test data, -4, -6, -2, and -5, doesn’t do a great job at covering that range. On the contrary, the supplied data gives the misleading impression of us testing for merely a very narrow and finite range—namely from -2 to -6. The name of this unit test tells a somewhat different story from the test data, which is not what we want. We like to convey an accurate and congruous account of what is being tested to the uninitiated reader. Everything about the test, its name, structure, and data, should all be consistent so unambiguously highlight the test’s purpose.

Let’s improve the test data. 

First, we are not verifying the finite boundary—the test data does not include 0. Positive values will not throw an exception, but 0 will. Next, what about the first negative value, -1? I believe that is worth having too. Including -1 will indicate a continuity of values. If we test for 0 and then from -5 onwards, someone might wonder whether there is a gap where the test does not apply; i.e. between -1 and -4. Let’s make sure we include -1 in our test data.

So far, we have 0 and -1. Should we include the next few values, i.e. -2, -3, -4, -5? I believe that such a selection of values would, once again, give the (wrong) impression that we want to confine the test to a narrow finite range: 0 to -5.

How can we better communicate via our test data that the test covers, if not an infinite, then at least a vast range of negative integers? My preferred method is to increase the data values by half powers of 10: 

• 10^0.5 = 3 (approx.)
• 10^1 = 10
• 10^1.5 = 30 (approx.)
• 10^2 = 100

What would our test data look like now? We will have 0, -1, -3, -10, -30, -100. 

So far, so good. Why not whole powers of 10, e.g. 10, 100, 1000, and so on? Those are decent values too, but I believe they give the incorrect impression that only powers of 10 will work for our test. 

Our test data looks great, and often, I will leave it at what we have so far. However, we could do with one more value, an extreme negative quantity. We could choose the integer data type’s limit of -2,147,483,648. However, that strikes me as a bit misleading: We are specifying these test data values to exercise the invalid quantity limits of the product we might legitimately come across when adding quantities into the ShoppingCart, not to test the limits of the integer data type. Given that our ShoppingCart implementation is unlikely to ever have -2,147,483,648 product items attempted to put into it, I prefer to employ as a real-world example a more realistic figure of maybe around -10,000 or even as much as a negative million. Or test data becomes: 0, -1, -3, -10, -30, -100, -1000000.

Here’s our test again, now with a much broader range of test data values:

In my opinion, our new set of test data is consistent with the name of the test and the narrative of testing for 0 and negative values. Unlike the original data set, our values are ordered, further adding to the clarity of the unit test.

Some people may consider this too much test data, and this will hit the performance of our unit test suite. No, it won’t. Not if the tests are proper unit tests, and we can run thousands of them inside a few seconds.