5/11/2005 -- In my last column I told you about my enthusiasm for the new simulation questions that Microsoft was introducing. Now that I’ve taken a Microsoft exam containing simulation questions, I wanted to share my experience with you.

There were a handful of simulation questions contained on the 70-290 exam I recently took. When preparing, I actually anticipated that there would be more on the exam, but now that I’ve taken it I see how time consuming these questions are to complete -- having more than a handful could easily have been overkill, detracting from the remainder of the exam content. Windows Server 2003 is a huge product, so there’s a great breadth of material to test candidates on.

Speaking of content, you’ll find no shortage of administration coverage on this exam. Aside from the simulations, there were a number of drag-and-drop and other questions where candidates are expected to use error messages or a properties page from the GUI to deliver the correct outcome. By my reckoning, approximately a quarter of the exam makes candidates work directly with the Windows Server 2003 GUI performing administration tasks -- this should go a long way to further refute the accusation that these exams lead to paper certifications and don’t test real-world skills.

The simulation questions themselves were straightforward, although they do take more time than a regular multiple choice question to complete, as mentioned above (you will still have plenty of time, though). The exam objectives suggest the types of exam questions you may be asked, so in that regard none of the simulation areas I encountered were completely unexpected.

As for the simulation technology itself, I thought it ran well, but I did run into a few things. For one, I found that the relevant admin tools for the simulation can only be launched from the normal Windows menus. When I attempted to launch a tool by name from the Run menu, I got a VBScript error. This isn't a big deal, but it is something to be aware of.

I also discovered an interesting side effect of the simulation software. When answering one of the simulation questions, I misread it and attempted to do something outside of what was being asked. Because I was working in a simulation environment and not the real Windows GUI, the simulation GUI didn’t respond to my action. This lack of response made me realize I’d read the question wrong: When nothing happened, I went back, read it again, and selected the correct option. In this situation, the limits of the simulation helped me; however, I would imagine that as Microsoft gets more experience in setting these types of questions, this “safety net” may not be something you can count on in the future.

Overall, I feel these new types of simulation exam questions are a good addition, but I don’t want to overstate their importance. They’re another welcome development in improving Microsoft’s certification testing, which was already pretty solid.

Before you take a Microsoft exam containing simulations, make sure you check out Microsoft’s demo that also functions as a tutorial here. Two of the current exams (70-290 and 70-291) currently contain simulations, with another two (70-293 and 70-294) scheduled to be updated by June. For those of you who have taken exams these new exams with simulations, let me know what you thought by posting your thoughts below. I’d also be interested in your general thoughts on how you think these questions will (or won’t) improve the validity of Microsoft testing.