Whoever heard of a product launch without rigorous pilot testing? And yet when it comes to our eLearning effort, we sometimes roll it out without pilot testing it and getting valuable feedback from the people who matter most – the learners themselves.
Having spent all our energy and effort into development and having reviewed our courses till our eye balls bubble, we might think that we have covered the program from all angles. And that now it’s just a matter of rolling it out. Not really. Pilot programs help in many ways. They help evaluate reactions and get feedback (provided inputs are taken seriously- a thick skin helps!). Your test group is also likely to spot errors that you were blind to and help you improve your course. What is more, they also help in promoting eLearning. How? Success from the pilot tests can be used to market the training in the organization because your pilot group can really sell the program to others and be your eLearning champions.
So who is the best audience for a pilot test? When it comes to pilot testing our eLearning course, especially for first time implementation, it’s very tempting to select a bunch of people who are likely to be comfortable and enthusiastic about eLearning. The reason is obvious- they are less likely to give critical feedback, and being more comfortable with technology, will have fewer issues in orientation to the new mode- and hence more likely to give glowing reports. And hey, who doesn’t like a few compliments at the end of all that hard work trying to put together an engaging learner experience?
However, some practitioners feel that pilot testing is likely to be more effective if run on someone who has no particular interest in eLearning and who is even a little technically challenged. (Think of it as something similar to stress testing for automobiles. Under ideal road conditions, results can be misleading.) You would want ‘lay people’ to come and test and give their candid feedback on all the issues they faced. For all you know, that slick course which you thought was interactive and engaging was probably your lay person’s navigational nightmare!
Everyone has their own preferred method for capturing feedback from pilot programs. More formal methods could include giving the selected group of learners’ questionnaires for feedback on various dimensions of the learning experience including content, comfort level, challenges with the new mode etc. My favorite is to watch people test it out live (but do ensure that your presence is not making them more uncomfortable than they are already). You get to feel their pulse and see their reactions to the entire program (ranging from ‘What? Is this for real?! to ‘Wow! This is fun!’). When you observe learners taking their first course, you can also gauge what the problem areas are. Maybe some learners are having trouble with navigation, or some are not sure of the consequences of a given click and are hesitant to proceed (this is very common when asked to submit an answer). Maybe some of them want to revisit a certain section, but are not sure how the course will respond (‘Will it throw me out and will I have to start all over again?’).
To conclude, pilot programs help – in more ways than one.