thoughts on multiple choice exams and Blackboard

Today we gave the midterm for the Soc. 1 class that Susan and I are teaching. 

Multiple choice tests have gotten somewhat of a bad name in higher education, a development that I think is unfortunate and counterproductive.  Done with some care I think that multiple choice can be an excellent tool to aid learning.  Blackboard, when combined with test banks offered by publishers in conjunctions with texts, can facilitate student learning and the transfer of curriculum into long-term memory.

Here are the ingredients:

1.  Load the testbanks or written multiple choice questions into Blackboard pools.

2.   Have weekly quizzes on each chapter drawn from the pools.  Allow students to take practice quizzes each week as many times as they like drawn from these pools.  Have the graded quizzes pull from the same question pools as the practice quizzes.  Encourage open book and student team collaboration for the quizzes.   Make sure that lecture and class material connects with the quizzes.   The principle is frequent, low-stakes assessment done close in time to when students work with the curriculum.

3.  For the midterm pull the same questions from the questions pools used for the weekly quizzes.  As with the quizzes, create a practice midterm test that matches the number of questions for the graded midterm.  For quizzes I have 20 questions, midterm is 50.  We covered chapters 1 to 7 so far, with all these chapters represented on the midterm.  

4.  The midterm is given through Blackboard during class period.  The test is open book, but not open note.  The goal is to get the students to spend some quality time with their textbooks and thinking about the curriculum.  Since students should have been working with the questions and answers all along, and have had the opportunity to see all the questions during their practice sessions, then the midterm should be less stressful.

5.  The emphasis is on learning, not on the tests to diagnose that learning.  Be clear that multiple choice tests largely test test taking ability.  But...the conditions can be set-up so that any student that diligently prepares and attends to preparation should do well without too much stress.  A good experience with the test will reinforce positive feelings towards the subject matter.

6.  Repeat process for the rest of the semester quizzes and the non-cumulative final exam.

The students may give some pushback in that these conditions should allow everyone to succeed.  Students, particularly students at competitive institutions, will equate the value of an assessment to the differentiation in grading.  In other words, they will value assessments and assignments more where a scarcity of top grades exist.  For an exam format such as described above the students will be uncomfortable with anything less then perfect grades.  This is a "teachable moment" - a chance to talk about your learning philosophy and if you believe that good grades are scarce (grading on a distribution) or if the conditions can be set-up for more widespread higher grades.

Ideally, quizzes and exams should not be the grade differentiator.  Students that demonstrate strong leadership in the course, who stand out in terms of helping the class and their fellow learners succeed, and who demonstrate consistently high quality work in projects should get the highest grades.  Exams are a good place for learning, a good place to get students to spend time with the curriculum, but a poor method to differentiate on grades.

Comments

[Anthony Helm]

Hey Josh,

Again I am right on board with this approach and like the way that you have methodically presented this. Multiple choice quizzes should not be used exclusively to evaluate learning and progress towards goals, but offer a great opportunity for self-quizzing and reinforcement if students take advantage of the opportunity. And, in the scenario you have highlighted, this is not an option, but a requirement and is a good way to get students to engage with the textbook readings.

Also, what is true for web design is true for quiz design as well...you gotta have the content, which can be the hardest or most time consuming part to put together. The publisher's test question banks save a lot of that time. However, the problems I ran into a few years ago were technical. The publisher's content pack for Bb was all invisible to students by default and was chunked in such a way that it took WAY too much time to click through and enable all the material that a faculty wanted to use for quizzes or otherwise. It sounds like some of that may have been remedied with newer versions of both Bb and publisher content.

I'm looking forward to how your approach turns out at the end of the term and to hearing about feedback from your students.

[Brian Reid]

Here's another example of using Blackboard quizzes with the emphasis on learning and not on grading. One of our Med School courses, On Doctoring, uses Blackboard quizzes as preparatory exercises for small group sessions. The quiz questions are constructed by the instructor and include multiple choice, multiple answer, matching, fill-in-the-blanks, and image hot spot questions. So it's a little more involved to get the quizzes set up (it's time consuming as Anthony points out), but it's all automatically graded. The students are required to take the quizzes, but their scores are not included in their course grade calculation. The students can take the quizzes as many times as possible using any resources, sometime before class, with a goal of getting at least 70% (most do a lot better). Their immediate reward is a copy of the quiz answer key (automatically made available using Bb adaptive release when the quiz score > 69). The reward for the course instructor and small group leaders is that the students are much better prepared for small group sessions.