This three-part white paper explores some possibilities and ideas in creating multiple choice questions (MCQs). In Part II, we look at:

• The types of knowledge (by Bloom’s taxonomy of cognitive objectives) that you can test for using MCQs
• How to create MCQs for each of the types of knowledge

Why are we using Bloom’s taxonomy as a starting point? Well, there are many misconceptions about multiple choice questions (including the idea that they can be used only for superficial testing, that MCQs are easy to answer, that MCQs are easy to write, and so on). We can clear such misconceptions by looking at what an examiner wants to test, and whether the MCQ format is suitable for the purpose.

By taking a systematic look at cognitive domains, we can look at how to create a balanced test or exam—for example, by allocating some questions to each domain.

 

Section I: Types Of Knowledge

Bloom’s taxonomy of cognitive objectives divides a learner’s knowledge of an area into six kinds (levels): Knowledge, Comprehension, Application, Analysis, Synthesis, and Evaluation. Here’s a quick, intuitive look at what these mean. Recall the children’s fairy tale of Red Riding Hood:

• To test Knowledge, one would ask: “Why did Red Riding Hood go into the forest?”
• To test Comprehension, one would ask: “Why did the wolf masquerade as Red Riding Hood’s grandmother?”
• To test Application: “What lesson do you derive from the story?”
• To test Analysis: “What might have happened if Red Riding Hood had quickly realised that it was the wolf and not her granny?”
• To test Synthesis: “Use the motif of the story of Red Riding Hood to construct one with the same moral, but set in modern times.”
• To test Evaluation: “Whom do you think the story is meant for?”

The “Knowledge” Domain

“Knowledge” means everything that has been explicitly presented to the learner. In chemistry, for example, a knowledge of the names of the elements; in history, dates of important events, and so on.

It involves remembering, memorizing, recognising, recalling identification, and information recall.

Testing for this category of your learner’s knowledge is straightforward. In fact, a very large percentage of all MCQs only test for Knowledge, under the misconception that they can only be used for this purpose.

Here’s a word list typical in such questions:

• Describe
• Find
• List
• Name
• Relate
• State
• Tell

An example question: “In which column of the periodic table is argon?” Or, “State the column of the periodic table in which argon lies.”

The “Comprehension” Domain

This category goes a step beyond “just knowledge.” It comprises what the learner should have understood from what has been presented, even if it wasn’t explicit.

Here’s a word list typical in such questions:

• Compare
• Discuss
• Distinguish
• Explain
• Interpret
• Outline
• Restate

An example question: “Why is argon in the first column of the periodic table?” Or, “Explain why argon is in the first column of the periodic table.”

The “Application” Domain

“Application” is what it sounds like—the learner’s ability to apply his knowledge in new situations, that is, situations that were not presented to him during the course.

Here’s a word list typical in such questions:

• Build
• Construct
• Examine
• Illustrate
• Solve
• Use

Here’s an example question: “Using the fact that argon is in the first column of the periodic table, figure out which other elements are in the first row.” Or, “Construct a list of elements in the first column of the periodic table, using the fact that argon is in that column.”

The “Analysis” Domain

This domain, again, is what it sounds like. Given a certain part of the learner’s knowledge, “Analysis” is about the learner taking that apart and coming to conclusions that were not earlier presented.

Here’s a word list typical in such questions:

• Analyse
• Categorise
• Compare
• Contrast
• Distinguish
• Examine
• Investigate

An example question: “Looking at all the elements in the first column of the periodic table, what would you say are the commonalities?” Or, “Examine the common properties of all the elements in the first column of the periodic table.”

The “Synthesis” Domain 

This refers to coming up with something new or novel based on information learnt from a course.

Here’s a word list typical in such questions:

• Compose
• Construct
• Create
• Design
• Devise
• Formulate
• Predict
• Propose

An example question: “Given the periodic table, come up with an imaginary one for elements with their valences changed according to a certain formula.” Or, “Devise an imaginary periodic table with the valences of the elements changed according to a certain formula.”

The “Evaluation” Domain 

The highest level of Bloom’s taxonomy tests whether the learner can critically appraise a work of art, a novel, etc. More simply, it tests the learner’s ability to compare and contrast course material with entities in the real world. This ability would indicate that he has grasped the material to a sufficient level.

Here’s a word list typical in such questions:

• Argue
• Assess
• Choose
• Decide
• Determine
• Judge
• Justify
• Rate
• Recommend
• Select

An example question: “What are the shortcomings of Mendeleev’s periodic table?” Or, “Assess Mendeleev’s periodic table of elements in terms of usefulness and shortcomings.”

 

 

Section II: Creating MCQs For The Knowledge Types

This section will let you explore questions phrased in different ways to test for the different types of knowledge. Here, we present a few question possibilities for learners who have just read Nobel laureate William Golding’s Lord of the Flies. The following will be easier to read if you’re familiar with the book; even otherwise, the explanatory notes below should suffice to clarify the examples.

The full text of the book is available at http://gv.pl/index.php/main/szkola/e-books/pdf/lord_of_the_flies.pdf . Brief chapter summaries are available at http://www.bookrags.com/notes/lof/ .

Here are very general steps to construct an MCQ for a question that seems “essay type” rather than “objective”:

Step1. From a given topic, think of a good question

Step 2. Think of a good answer

Step 3. Phrase the answer correctly

Step 4. Modify the phrase such that you can have distracters phrased similarly

Step 5. Decide upon how you will construct distracters, based upon factors like difficulty

Step 6. Choose plausible distracters

Step 7. Phrase the distracters appropriately

It helps to look at it from the point of view of the examinee, in the context of a topic or section of the course:

>> To come up with a question, think: What areas from this topic are relevant or necessary?

>> To come up with a correct answer, think: What exactly should the learner have understood?

>> To come up with distracters, think: What might he have understood? That is, what correct and incorrect ideas might he have?

 

Part 1 of 6: Questions To Test Knowledge

In this aspect, the questions that the examiner frames check whether the student can recall facts, figures, events, and the like. In the context of the book Lord of the Flies, here are six questions that come under the “Knowledge” aspect.

• What was the entity that “spoke” to Simon?
• Who is the central figure in the story?
• What did Ralph feel when he first saw Piggy?
• What happened to Simon when he heard the pig’s head speak to him?
• Was Jack more keen on hunting or on getting back home?
• Who initiated the ritual of the conch?

In all these cases, the answers are concise, factual, and well-defined. Here are example MCQs for the first three.

Question 1: Name the entity that “spoke” to Simon.

Answer Choices:

1. The monster
2. The volcano
3. The pig
4. The pig’s head

Question 2: Who is the central figure in the story?

Answer Choices:

1. Ralph
2. Jack
3. Piggy
4. Simon

Question 3: What emotion did Ralph primarily experience when he first saw Piggy?

Answer Choices:

1. He was taken aback
2. He was quite happy
3. He was jealous of sharing the island with Piggy
4. He was scornful because Piggy wore spectacles

Part 2 of 6: Questions To Test Comprehension

Comprehension goes beyond Knowledge, in the sense that we look at an understanding of the material, not a recall of the material itself. It is possible to test for Comprehension using MCQs, but this generally requires more effort on the part of the instructor (as opposed to testing for Knowledge).

In the context of our example novel, suppose you want the student to outline the most important insight of the story. You can word it appropriately, and create distracters such that the correct answer will most likely be chosen by someone who has understood the material.

Question: Which of the following is the most prominent moral of the story? (Correct answer: A)

Answer Choices:

1. It is assumed that men are Good by nature, but in reality, society imposes it
2. Evil can overcome Good anytime it wants to
3. There is no such thing as Good and Evil; it all depends on the situation
4. A person like Ralph will always lose in the fight

In this example, we have tried to keep the answer choices divergent, that is, the answer choices come from different themes.

For the same question, let’s try and keep the answer choices similar:

Question: Which of the following is the most important insight of the tale?

Answer Choices:

1. It is assumed that men are Good by nature, but society imposes it
2. It is assumed that men are Evil by nature, and society causes it
3. Good always loses in the fight with Evil
4. Men are Evil by nature, and when there are no women, it becomes obvious

Here are other examples of questions that test Comprehension:

• Write out a very brief outline of the story.
• What might have happened if Piggy had not died?
• Who represented Reason: Simon or Piggy?
• In the story, what does Ralph stand for?
• Distinguish between the characters of Ralph and Jack at a basic level.

Let’s look at question #3. There seem to be only two options, in which case it becomes a question that tests Knowledge instead of Comprehension. Also, it would be a trivial question with only two options. But we can frame a “Comprehension” MCQ for it in the “statement… reason” style:

Question: Who do you think represented Reason: Simon or Piggy? Why? (Correct answer: A)

Answer Choices:

1. Piggy, because he introduced the idea of the Conch
2. Simon, because the pig’s head told him what was really happening
3. Neither; Reason is represented by Ralph alone
4. Piggy, because his spectacles represent intelligence

Looking at #4, “What does Ralph stand for,” it might seem difficult to frame an MCQ. The examiner might initially wonder: If I use single words as answer choices, would it carry the proper meaning? If I use an objective-type question, can I really test Comprehension? And so on.

As a matter of fact, single-word answer choices in such cases often don’t make sense. One solution is to keep the answer choices descriptive and also plausible. Here’s an example:

Question: From among the following, what does Ralph stand for most? (Correct Answer: C)

Answer Choices:  

1. The strength of a group as opposed to the strength of the individual
2. The confusion of the human race
3. Good in the midst of evil
4. The power of judgment and discrimination

All the options seem plausible, that is, there are no humorous or random distracters. Option (A) has something to do with the story, but is incorrect. Option (B) is a tempting distracter. Option (C) is the correct answer, and (D) has nothing to do with the story. Constructed varied distracters in this way is suited for testing comprehension.

 

Part 3 of 6: Questions To Test Application

Let’s first list some Application areas – that is, questions to test application of knowledge – for the novel Lord of the Flies:

• Applying the story to one’s own life
• Applying it as a moral tale for kids
• Applying it to write a different story by imitation
• Applying the story to broaden one’s knowledge of the world
• Applying the story to modify one’s understanding about people

These can be framed as the following questions respectively:

• Can you think of an instance where you were faced by a situation similar to the one in the story?
• Use the story and modify it to teach young children a moral about life.
• Apply the storytelling technique in Lord of the Flies to a one-page story of your own.
• Develop a set of features of an oceanic island as depicted in Lord of the Flies.
• How does the story change your idea about human beings?

Some of these can be written as multiple-choice questions. We can phrase question #2 above, “Use the story to teach children a life moral,” as an MCQ:

Question: What can Lord of the Flies teach young children? (Correct answer: A)

Answer Choices:

1. That people can be good and bad at the same time
2. That boys with spectacles are usually the smartest
3. That a deserted island makes people bad
4. That having to take sides always makes things difficult

Here, we’ve created plausible distracters using factual correctness. That is, options B, C, and D have a factual connection with the story, but these options cannot be thought of as a lesson for children.

For question #5, “How does the story change your idea about human beings,” it might seem that no objective question is possible. But we can phrase the question such that it asks for a list: “List the characteristics of human beings as implied by the author in Lord of the Flies.” Then we construct a list of characteristics as the correct answer. For the distracters, we begin with the same list and change some words radically.

Question: Which of the following characteristics of human beings are closest to those implied in Lord of the Flies? (Correct answer: A)

Answer Choices:

1. Good and evil at the same time, dependent on society
2. Good but confused, independent
3. Social, evil at times
4. Bound by society, full of hatred

 

Part 4 of 6: Questions To Test Analysis

Here, we are looking at testing the student’s ability to analyse the subject matter. Multiple choice questions for this will check whether the student knows the subject matter sufficiently well to be able to analyse it. The idea is not so much the analysis but whether the student can pinpoint what he is about to analyse. In this sense, such questions can be framed as MCQs.

Here are some examples of what a student might be asked to analyse about Lord of the Flies:

• Did Simon change the course of the events in the story? If so, how?
• What exactly happened when Simon faced the pig’s head?
• Sam and Eric are twins, but are they similar?
• Why did Sam’n’eric join Jack?
• Does Ralph “lose” or “win” in the end?

Let’s look at #2 above. In an MCQ, “what exactly happened” cannot be explained completely by the examinee. So, for the correct answer, we can list out one aspect of “what happened,” and write the distracters as variations on that aspect.

Question: Which of the following best describes what happened when Simon faced the pig’s head? (Correct answer: A)

Answer Choices:

1. Simon understood the cause of the boys’ fear
2. Simon realised that Ralph and Piggy were headed in the right direction
3. Fear at what he saw made him faint
4. Simon got the courage to tell Jack not to fight Ralph

The distracters (B, C, D) here are phrased such that they relate to actual events in the story. This makes them plausible.

Question #3 above asks about how the twins Sam and Eric are similar. This Analysis question can involve a detailed write-up of the actions of the twins. But for an MCQ, we can present a few options, each summarising one line of analysis. Note that unless the student had analysed the event, he would not be able to arrive at the answer.

Question: Sam and Eric are twins, but are they similar?  (Correct answer: A)

Answer Choices:

1. Yes; they are almost identical
2. Yes, but their subtle differences are important to the story
3. No, they have nothing in common except that they are twins
4. No, because Eric joined Jack’s camp

 

Part 5 of 6: Questions To Test Synthesis

“Synthesis” in Bloom’s taxonomy is, by definition, not testable using MCQs. With considerable effort on the part of the exam creator, the student’s ability to perform a “synthesis” can indirectly be tested.

Essay-type questions that involve Synthesis would include:

• Write an essay that shows the author’s negative themes in a positive light.
• Write pages 20 to 23 of this story in verse form.
• Create your own dialogues for Piggy while he is being verbally abused.
• How would events have been different if Ralph had had an evil streak?
• Given the turning points in the tale, which ending(s) do you think are plausible?

Here is an example: “Create your own dialogues for Piggy while he is being verbally abused.” A question could be asked: “What would Piggy have said if he had needed a crutch and it was stolen?” Answer choices:

1. “You just bring that back!”
2. “My crutch! I can’t walk without it! Please…!”
3. “OK, let’s see what they do with it -- they don’t need it, you know!”
4. “Ralph! Simon! Bring back my crutch!”

Such questions cannot test whether the learner can “synthesise.” They can, however, test the learner’s mastery of the topic, as well as his imagination. The instructor must do the “Synthesis” and check how close behind him his learners are.

 

Part 6 of 6: Questions To Test Evaluation

“Evaluate” has a distinct meaning in Bloom’s terminology: If a learner can “Evaluate” a piece of knowledge or study material, his knowledge of it extends to the highest level. It is difficult to adequately test for the ability to evaluate, but it can be done to an extent. Here are example questions in the context of Lord of the Flies.

• What is the human value of the story?
• Can you defend the idea that Simon’s incident with the pig’s head is the most mystical in the story?
• In the light of this novel, do you think evil is necessary in the world?
• What changes would you recommend to make the story more readable for 13-year-old readers?
• What was your dominant emotion when you discovered that Jack was planning to kill Ralph?
• Do you agree or disagree that the author should have been awarded a Nobel prize for Lord of the Flies?

Question #2 can be framed as an MCQ. We phrase the question as an assertion, and put the answer choices as reasons. In other words: The essay-type question would ask the learner about whether the idea can be defended. An MCQ would say it can be defended, and then ask for the reason.

Question: Why might you say that Simon’s incident with the pig’s head is the most mystical in the story? (Correct answer: A)

Answer Options:

1. Because it is a revelation
2. Because it is not real
3. Because it is a turning point in the plot
4. Because it showed everyone that the Beast didn’t exist

Another example is #4, “What changes would you recommend to make the story more readable for 13-year-olds.” This can be phrased as a “Check all that apply” question.

Question: To make the story more readable for 13-year-old readers, you would… (check all that apply)

Answer Options:

1. Within the story, give clues about the symbolism
2. Simplify the language by shortening sentences and using more common words
3. Remove some of chapter 6 because there is very little action
4. State at the beginning what the characters stand for

To construct answer choices for such questions, the examiner must put himself in the place of the learner. The subject matter, the type of learner, the duration and depth of the course, all make a difference when it comes to testing for Evaluation. The examiner’s judgment is essential.

In this example: Options A and B are both sensible choices for most learners, though B is the best suggestion. If a learner were to choose option C, it would probably mean that he does not understand the novel. Choosing option D would probably mean that the learner does not understand symbolism at all. Thus, this question does not demand a critical appraisal of the novel, but it eliminates test-takers who would not be capable of it.

 

 

These general guidelines for creating multiple-choice questions will be continued in Part III, Other Types of Multiple Choice Questions, which discusses question types other than that with four options.

©Focalworks 2008

 

In the tips and ideas above, did you find something particularly insightful? Is there a point you disagree with? Let us know!