Moodle offers a bewildering array of configuration settings for quizzes and quiz items. How is an instructor to know which options are best? There is no “one size fits all” answer. The options chosen should be based on the type and purpose of the assessment, when it is occurring in the learning process, and how much weight it will contribute to the overall course grade. In this article, we distinguish between “low-stakes” and “high-stakes” assessments, and discuss the best options available in Moodle for each. Sometimes the best type of assessment is not a quiz at all, but a different type of activity, such as a research paper, group project, forum discussion, or workshop.
Low-stakes assessments
Low-stakes assessments, sometimes called “formative assessments,” are intended for the student and instructor to be able to check on the student learning process as it is happening. These may be weekly quizzes, knowledge checks, self-checks, etc. Low-stakes assessments are usually not worth a large percentage of the overall course grade. The most important aspect of a low-stakes assessment is the feedback provided to the student. Automatic scoring, if used, needs to be able to identify common errors and point them out to the learner. The low percentage of grade weighting helps to make cheating less likely, even though the responses may be forced-choice, and easy to copy. Forum discussions are also often used as low-stakes assessments.
High-stakes assessments
High-stakes assessments, also called “summative assessments,” final exams, or even unit tests, are used after learning has (hopefully) taken place, to verify that the student is ready to move on to the next topic, course, etc. While feedback is still included, and is important, it is not meant to help the student improve further for the purpose of the current course. High-stakes assessments tend to be worth a large proportion of a course grade, and their accuracy and validity are very important. They should consist primarily of “constructed response” formats, e.g. essays, projects, or word problems or case studies in which the answer is a written paragraph, not a selected value or word. Requiring individually constructed student responses helps to minimize cheating, as identical wording will be more obvious to the instructor during grading, and improves the validity of measurements of higher-order learning objectives. Often a “capstone” project is a better summative assessment than any kind of quiz.
Moodle allows two levels of assessment configuration: Quiz Settings, and Item Types and Settings. Here are some best-practice guidelines for each:
Quiz Settings
|
Low-Stakes or “Formative” assessments |
High-Stakes or “Summative” assessments |
|
|
Restricted Access Dates |
Less Important |
More Important |
|
Time Limit |
Not Recommended |
Possibly Recommended, with caveats* |
|
Attempts Allowed |
Multiple |
1** |
|
Grading Method |
Highest Grade |
N/A |
|
Layout |
Any |
Shuffled Randomly |
|
New Page |
Few questions per page (for feedback) |
Few questions per page (for continuous saving*** |
|
Shuffle Within Questions |
Yes, especially if multiple attempts allowed |
Critical, if M/C used |
|
How Questions Behave |
Immediate Feedback or Deferred Feedback, or Interactive with Multiple Tries (medium-stakes), CBM optional |
Deferred Feedback (after Quiz is closed) |
|
Each Attempt Builds on Last (Advanced option) |
Yes |
N/A or No |
|
Review Options |
“During the attempt” or “Immediately after the attempt” |
“After the quiz is closed” |
|
Enforced delay |
May be useful to encourage students to review |
N/A |
|
Browser Security |
No – not ADA compliant |
Unreliable as an anti-cheating mechanism |
|
Overall feedback |
Can be very useful for students at this stage, especially if multiple attempts are allowed |
Individualized (manual) feedback is likely to be more valuable |
|
Restrict Access |
Useful to enforce prerequisites |
Not recommended unless you wish to force a review prior to the exam |
* We are sometimes asked about time limits in assessments. Unless you are emphasizing fluency (speed of response) in your assessment, time limits are probably not appropriate. They can cause severe problems for students with unstable or slow internet connections. Design your assessments with the understanding that students will be able to take extended time, and even consult references while taking the assessment. For higher order learning objectives (application of principles, evaluation, etc.), this should not be an issue. ** Be aware that allowing “one” attempt still enables the student to save work, log out, and continue the attempt later, provided questions are distributed across multiple pages. ***We advise that the number of questions per page be limited to 10-15 minutes of expected effort. This protects students from losing work in the event of a technical outage, as responses are saved whenever the student advances to a new page.
Item Types and Settings
|
Low-Stakes or “Formative” assessments |
High-Stakes or “Summative” assessments |
|
|
Multiple Choice |
Yes |
No |
|
Multiple Answers |
No – Confusing and difficult to validate |
No |
|
Matching |
Yes |
Possibly, especially for ordered steps |
|
True/False |
Possibly – but easily guessed |
No |
|
Calculated (Use Simple unless you need to share values, e.g. a Testlet) |
Yes |
Yes – consider “testlets” or Exhibit Based Item Sets |
|
Numerical |
Yes – But Calculated offers more flexibility |
No – too easy to cheat |
|
Cloze |
Possibly – time-consuming to set up |
No – primarily tests memory. |
|
Short Answer |
Yes, but limited automated feedback |
Yes – Grade manually |
|
Essay |
No – consider Workshop activity |
Yes – Grade manually. |