Monday, September 26, 2011

Microsoft Test Manager Preview (MTM11)

About 18 months ago Microsoft’s released Microsoft Test Manager (MTM), a new test tool with focus on agile teams and collaboration. MTM introduced a lot of features to simplify and improve common test tasks and improve team collaboration. A week ago, at the build conference Microsoft release a Preview of the next version of Visual Studio including MTM.

Missing feature set in MTM2010
Even if MTM brought many new features to the tester, MTM do have some short comings and lack some features most testers expects. Amongst the most commonly wanted is the possibility to use Multi Line and Rich Text Formatting in Test Steps. The edit test case experience only uses a fraction of the screen to edit/view test steps. Testers have found workarounds for some issues like editing in excel and pasting back to MTM or using third party tools like the Test steps editor (http://teststepeditor.codeplex.com). Of course the list of wanted features is a lot longer, but I think those mentioned is amongst the most commonly wanted.

MTM11 addresses most commonly wanted features
The big story in MTM11 is focused around Exploratory testing , but a quick look at the preview of MTM11 it looks like Microsoft also focus on addressing the the wanted features in the next release. MTM11 now supports Multi Line, and Rich Text Formatting (even if the formatting toolbar is pretty limited). Even the test case editor is improved and a splitter is introduced so you can adjust and use all available space for viewing /editing test steps.

MTM11 Backward compatibility?
With the most commonly asked for features fixed I was curious, is MTM11 backward compatible with TFS 2010 and MTM 2010? Could I be using the MTM11 on a TFS 2010 project side by side with MTM2010 and use the new features?
After taking a test drive I found that Microsoft seems to have backward compatibility on its mind. MTM11 works directly an existing tfs2010 team project. I can view/edit/run test cases with both MTM11 and MTM2010. Using MTM11 against an old TFS2010 team project gives me a better test case editor with Multi Line editing. If I open a test case created in MTM2010 and adds line breaks in test steps they render correctly in MTM2010
However, using MTM11 against an old team project disables the rich text editing is however disabled. To enable the rich text editing you need to connect to a tfs11 project or probably do some kind of process template upgrade.

Test step editor and MTM/TFS 11?
As many teams already worked around the limitations of MTM with tools like test steps editor, I was also curious how the next generation of TFS and MTM11 react to test cases edited with the test steps editor. The questions are, does MTM11 acknowledge line breaks made by test steps editor and how will they be treated by TFS11? From my test drive it looks promising, MTM11 seems to handle line breaks made by test steps editor in an old TFS2010 team projects. Upgrading the project to TFS11 works fine, and the line breaks made by test steps editor works fine in MTM11 after the upgrade.

MTM11 Preview is a good start
This post is just based on my findings after a short test drive with a preview of the next generation of MTM, but it looks promising. If Microsoft makes MTM11 backward compatible it will make it a lot easier to implement and use the version of MTM in real life projects. Hopefully Microsoft will continue and address commonly requested features like real pause/resume of test cases (new status), edit test cases during execution, ability to bind data bound test cases against SQL commands or other data sources.