Some writers claim that MCQ's are unsuitable for
testing higher level abilities, although Vernon (Vernon
P.E. 1964) stated that it is by no means certain that
conventional examinations can test higher level thinking
either.
It has been claimed that Multiple Choice Questions
(MCQ's) promote convergent thinking ie. that there is
only one correct answer to a question, and that answer is
known. This criticism is more substantial than the first,
especially if MCQ's are used to test higher order
cognitive skills (see Frederiksen 1984).
I believe that open ended questions may discourage an
answer, not because of any shortage of knowledge or
understanding, but because of the difficulty involved in
the construction of an answer. Mathews (Mathews 1967)
expressed similar opinions regarding open-ended questions
and, it is the validity of open-ended questions as an
assessment technique is being questioned this time.
The essay question is often the favoured 'traditional'
examination item at all levels. For those who advocate
essay questions above all other, I would direct them to a
comprehensive discussion of essay questions, and the
problems they raise regarding the nature of extended
writing, it's constructs (content, organisation and style
etc.), the construction of questions, marking issues,
marker error, analytic vs. holistic (impression) marking
and the influence of handwriting given by Wood (Wood
R.(1) 1993)
Wood (Wood R.(2) 1993) claims that it is now possible
to be prescriptive about a number of issues concerning
the construction of MCQ test items. These issues
include:-
1. Number
of alternatives.
The practical difficulties involved in writing
multiple distractors, and the results of research by a
variety of authors, suggests that a four choice, three
distractor format is amongst the most effective.
2. 'None
of These' responses.
The use of this as a response tends to increase the
level of complexity without , increasing reliability or
validity.
3. Scoring
Systems.
Students who do not know an answer will often
guess.
Guessing is the main concern by those who use MCQ's to
test students, as the guessed answer does not guarantee
knowledge of the subject. On balance, research suggests
that to instruct students not to guess answers is
inadvisable. Linn (Linn M.C. 1987), showed that females
are more reluctant to guess than males, and Grandy
(Grandy J. 1987) showed that females are more likely to
leave items blank.
Akeroyd's (Akeroyd F.M. 1982) "Dual
Response" system may be useful in rewarding partial
information. For a four-choice test item, candidates
score:
- 1 mark for identifying the correct answer with 1
response;
- .5 marks for 2 responses, 1 of which is
correct;
- .25 marks for an omission ie. no response;
- 0 mark for an incorrect answer OR combination of
2 incorrect responses.
The .25 mark for omissions is
said to eliminate fluctuations of scores due to random
guessing.
Akeroyd claims that this system is a good way of
learning about student response behaviour, but appears to
be untried to date. An adaptation of this, for computer
aided assessment use, may be possible.
4. Answer
until correct.
This gives immediate feedback to the student. The
scoring is dependent upon the number of alternatives ie.
if there are three, then a correct selection first time
scores 2, second time scores 1 and third scores 0.
5. Changing
Answers.
If a student is capable of recognising an
incorrect answer after reflection, then they should be
able to change it. If the student does not know the
answer in the first place, they are unlikely to attempt
to change the answer given.
6. Confidence
Weighting.
This system involves asking students to award a
weighting to their answer depending upon their confidence
in that answer being correct.
Research in this technique has failed to establish its
ability to provide useful data (Wood R. (3)1990).
7. Differential
Weighting.
This system involves weighting responses in a variety
of ways. Research in this area has failed to identify a
method that can offer reliable differentiation across all
levels of response. Echternacht (Echternacht G.J. 1976)
found that his specially constructed, subjectively
weighted test items were even less accurate than
number-right scores.
The
Cloze Testing Technique.
These tests are often "grouped together"
with multi choice tests as they appear similar. Cloze
tests involve the student inputting a word in order to
complete a meaningful sentence. Cloze tests may offer a
selection of correct words, correct words and additional
incorrect "distractors" (cloze-elide) for the
student to position correctly within the sentence(s), or
words may not be provided at all.
Criticism of cloze is that it is
"unauthentic", and Carroll (Carroll et.al.
1959) found that:-
"the ability to
restore texts is somewhat independent of competence
in a language at it is ordinarily defined. That is to
say, we observed many people who are perfectly
competent and literate in a language but who do not
show facility in a special task of guessing what a
missing letter or word in a text might be."
