Designing effective assessments is a key principle of instructional design. You want to have confidence that successful completion (a.k.a. passing) of the test(s) in your course is a true and reliable indicator that the learner has mastered the material. One way to improve the reliability of tests is to present different questions to each learner every time the test is taken. Randomizing your questions reduces the likelihood that the learner has, at best, memorized answers, is using process of elimination on subsequent attempts to improve his or her score or, at worst, has been given the answers from other learners. Lectora® and Lectora® Online support this instructional tenet by allowing you to randomize tests or portions of tests.
To set up a randomized test, the first thing you’ll need to do is develop all of the questions for your test. Within Lectora and Lectora Online you can create tests that include nine gradable question types:
Note: You may also include non-gradable (unscored) questions, if you’d like.
For each question you can choose to include feedback to your learners, letting them know if they have gotten each question correct or incorrect. However depending on the purpose of your test (i.e., review or assessment), you may elect to disable feedback.
Randomizing a Full Test
To randomize the entire test, select Randomize on the Test Behavior ribbon. (Note that randomized tests can also be timed, if desired.)
After selecting Randomize, you’ll need to indicate how many pages the test should include.
"Standard" tests in Lectora may include as many questions on a page as you’d like, including having your entire test contained on one long, scrolling page. However, when creating a randomized test, you will probably want to place each question on its own, separate page. The reason for this is that the randomization function doesn’t actually select questions. It randomly selects pages.
For example, if your 100-question test contains 50 pages and you set the randomization number to 10, Lectora will select ten pages from the set—not 10 questions:
- Pages in Test: 50
- Questions per page: 2
- Total Number of Questions in “pool”: 50 x 2 = 100
- Random Pages: 10
- Questions student will see: 10 x 2 = 20
Therefore, if your goal is to randomly select individual questions, you’ll want to put only one question on each page:
- Pages in Test: 100
- Questions per page: 1
- Total Number of Questions in “pool”: 100 x 1 = 100
- Random Pages: 10
- Questions student will see: 10 x 1 = 10
In standard tests you need to include the Process Test/Survey action, typically on the last page attached to a “Done” or “Submit” button. This is not necessary for a fully randomized test. Since Lectora is selecting the desired number of pages randomly from within the test, there is no way of knowing which page will be selected last. Therefore, Lectora will automatically process the test after the desired number of pages has been selected; e.g., when the learner leaves the 10th page in the example above.
Randomizing a Portion of a Test
Lectora and Lectora Online allow you to divide your test into Test Sections. This is very useful for separating and organizing questions, perhaps by learning objective, topic or difficulty level.
When this is the case, you have the option of randomizing the entire test as described above, or randomizing any section(s). If you elect to randomize the entire test (on the Test Behavior ribbon), Lectora will choose pages from among all of the pages in the test, regardless of the section containing them. In other words, the sections are treated as organizational only; the 10 pages for the test are pulled from any section.
Alternatively, you can choose instead to not randomize the entire test, but rather select the specific section(s) from which to randomly choose pages. In this way you can ensure that a certain number of questions are included from each section (i.e., topic or difficulty level).
You would use this same approach to create a test in which some pages/questions are mandatory and others are randomly chosen.
Important: If the last section of the test is NOT randomized you will need to include the Process Test/Survey action. Otherwise, Lectora will not know that the test is finished and calculate the score. For example, you may have a page or a section at the beginning of the test which is not randomized (i.e., required to be seen by all learners), followed by one or more randomized test sections and concluding with required (i.e., non-randomized) instructions to submit the test:
Finally, remember that in all of these cases the test score is based on the point values of the questions presented. Therefore, if the point values vary by question, the possible number of points totaling 100% will vary for each learner every time the test is taken. You will want to keep this in mind as you determine the Mastery% (i.e., the required minimum passing score) for the test.
Not a Lectora or Lectora Online user yet? You’re missing out on this great assessment feature! Sign up for a free trial today.
