You've finished recording your course and are preparing to launch. You've asked a few trusted members of your network to preview the course, and they have notes. In fact, they've found some problems with your course. And not just simple stuff like typos on your slides. Gaps in coverage. Spots where your explanations aren't 100% clear. A few have even suggested your course is too difficult for the target audience.
I've seen this scenario before, and it's not pretty. Revising a course after production is incredibly painful. All that time and effort spent polishing the course, and with the finish line in sight... I've even seen authors give up and never finish their course because the weight of making revisions is too daunting.
But there's a way to prevent this pain: pilot testing your course content prior to production.
What is Pilot Testing?
Pilot testing means workshopping a lo-fi version of your course with students as early as possible in the development process. Essentially, as soon as you have a workable version of your course materials, you organize a way to deliver this early version of your course to students to validate that you're on the right track. Think of it as a test drive of the curriculum before investing effort to record and edit video.
The best pilot test is lightweight. You choose a weekend (or evenings over the course of one week). You recruit 10 students who fit your target student profile to participate. And you deliver the course to the students, simulating the experience they'll have with your on-demand course materials. Where they'd watch a video, you deliver a real-time lecture. Where they'd complete an exercise or take a quiz, you wait while they do the work. And if there's a final exam or project for students to complete - give them a fixed amount of time to complete it.
Benefits of Pilot Testing
Ensure that your course delivers the target learning outcomes
Pilot testing will help you figure out if your course works. No matter how good you are at planning, the only way to know that your course actually teaches students new skills is to watch students go through the course material and assess their abilities afterwards. It's much better to discover any shortcomings in your curriculum during testing rather than after production, or worse, in the form of requests for refunds.
Save yourself from wasted effort during production
No matter how good you are at planning and how well you know your audience, you'll discover better ways to teach your students. Would you rather learn that your curriculum needs tweaking before or after spending 60-80 hours (maybe longer) recording and editing your content?
By the time you finish post-production, you want the revisions to be minimal and mainly focused on tightening up the course media, not reworking the content. Pilot testing gives you a chance to test and revise your content without the added cost of needing to redo large amounts of production and postproduction work.
Validate your audience demand
If you struggle to find 10 people to take a lo-fi workshop of your course, you may be in for a long battle finding paying customers. Additionally, pilot students will give you feedback on how well the expectations you're setting around the course match their own objectives. You'll be able to better position your course for the right target audience, with messaging that resonates.
How to Run a Pilot Test
Prepare the lo-fi version of your course materials. Remember, for your pilot, your goal is to simulate the future experience students will have with your course. You're validating that your course teaches new skills effectively and provides a great student experience. For that, you'll want to have usable versions of any quizzes, exercises, and projects that your students need to complete in the course. Make sure those are bug-free. Plan your lecture content in detail, and make sure you have plenty of examples. But draft versions of your slides will be fine.
Recruit a small group of students; 10 students is plenty.
Pick a time to take them through your course content. Schedule over a few days (like an hour a day) or in a couple longer sessions over a weekend, depending on the length of the course and what works for your schedule. Since you're trying to test quickly and get rapid feedback from students, don’t spread the schedule over more than a week.
You can run the pilot test in person or remotely. Things will probably move more smoothly in person, and in my experience, engagement will be higher, but if your course will be delivered online, you should strongly consider delivering the pilot course over Zoom in order to better simulate the future student experience.
Recruit a friend or colleague to help you administer the pilot program, and take note of any areas where students run into problems, have clarifying questions, etc.
Survey your students at the end of each lesson, and at the end of the course. You're looking for feedback on course quality, learning outcomes, and areas where they feel they may have gaps in understanding.
Make sure as many students as possible complete the course and make it through all of the assignments. Dig in and provide extra help where necessary, and make note of the problem areas. This will help you evaluate whether additional practice or coverage may be necessary.
That's all there is to it. You're now ready to pilot test your course content.
Have you pilot tested your courses before? I'd love to hear about your experience.