Unit Testing Irritants in Visual Studio 2008
I have been delving into the unit testing capabilities of Visual Studio (sans NUnit), otherwise known as MSUnit. In truth, MSUnit is a name the community came up with. The entire contents of the unit testing framework are in this namespace:
Microsoft.VisualStudio.TestTools.UnitTesting
Overall, there is more to the unit testing capabilities baked into the IDE than I can grok in a day. It is an incredibly rich environment and frankly that may be part of my issue with it.
The UI
I find the UI confusing, even after several hours of use on my part. Maybe it’s just me, but having several windows to run my tests just seems a bit much. The windows involved in the unit testing functionality of Visual Studio are:
- Test View
- Test List Editor
- Test Results
- Test Runs
This screen shot shows the access menu to get at all those windows. I get the point of each of them, and each feature has been thought through very carefully. Maybe a bit too thought through. Is it me? All this just seems a bit, well, MUCH. You know, like all those MS Word features we joke about not using? Maybe the team just over thought the problem a bit?
To be fair, what MS has done here is incorporate Unit Testing as a functionality of the overall test experience. Many of these features are meant to drive other types of testing as well as unit tests. That may mean that unit testing got a short straw.
Test Results Window Behavior
Tonight, my real beef is with the Test Results window and the way it manages tests as you work with it to build out your code. After a test passes, it is removed from the list of tests to be executed on the next run. Here is the workflow I am describing:
- Write TestMethodA.
- Execute tests.
- TestMethodA fails. No implementation.
- Implement code to fill in for TestMethodA.
- Execute tests.
- TestMethodA passes. Excellent.
- Move to TestMethodB
- Execute tests,
- TestMethodB fails. No implementation. So TestMethodA must have still passed, right? Wrong.
TestMethodA didn’t get run because the default behavior is to not run a test which passed previously. The following screen shot shows how the Test Results window has reacted after the bottom 5 tests passed. The check marks next to the TestMethods indicate the tests will be run the next time you click the Run button. The passing tests are not checked.
The worst part of this default behavior is that I can run the 2nd round of testing, modify a previously passing test method so that it fails, and it will not get picked up in this test session. Yikes!
A work around exists. Instead of using the Run button, 
use the key combination CNTRL+R, D. This will run all the tests whether they are checked or not.
All that I have to say is two letters (OK I have to actualy say more than that because I need to explain that my reply is very short):
R#
I am not using R# for this exercise on purpose even though I love the user experience of unit testing (not to mentio all the other stuff) using R#.
Firstly, I using the MSUnit testing capabilites instead of NUnit and while R# has an open extension point to hook into MSUnit, I don’t know that anyone has actually done that yet.
Secondly, there are compelling reasons to stick to a standard, vendor-issued tool in a large scale environment. Open source licensing on your unit testing framework can actually end up being problematic in distribution or commercial environment. At $300 a pop, a 100 person dev shop starts to wonder whether resharper is really the best option.
And laslty, I just want to see how the other half lives :).
If you have the project with the tests set as the Startup project then Ctrl-F5 (or F5) will run all the tests.
Richard,
I owe you a beer. That was the single best tip I have received in weeks. Thank you! Thank you! Thank you!
CTRL+F5 executes without the debugger.
You are the man! THE MAN, I say!
Is there a good tutorial article on how to do this in VS 2008? I’ve found this one but it is for VS 2005:
http://msdn2.microsoft.com/en-us/library/ms379625(VS.80).aspx