WeeklyMentorBanner

Homeschool and Testing, Part II

(The Huge Problem With Multiple Choice Exams)

by Oliver DeMille

(Note: if you haven’t read Homeschool and Testing Part I, please read it here,)

Question: What is the problem with multiple choice tests?

Answer: Well, it depends…

Child with learning difficultiesReally. That’s the best answer. Nearly always. “It depends…”

C.S. Lewis taught in The Abolition of Man that all schooling has a design, meaning that each curriculum and testing arrangement is designed to bring about certain results. And many great educational thinkers from all sides of the education debates have agreed with this idea—from Dewey to Montessori, and from Adler to Jacques Barzun, for example.

So, to get straight to the point, what are multiple choice exams designed to do to (and for) students? To explore this question, let’s use a little comparison.

How and What to Test

Oral exams naturally focus on what the student has learned, and how effectively he or she thinks. Oral examiners ask a question, listen to the answer, and then respond with further follow-up questions. In Socratic form, they notice an area of hesitation in the student’s response, a clear error, or a potential weakness. And so they follow up with additional questions to see if this is really a lack of knowledge and understanding, or if perhaps the student simply misspoke.

multiplechoice

Likewise, the examiners witness a strength, or even a talent, and ask the student to share more of it. In this way, oral exams can very effectively arrive at an overview and outline of what the student knows and doesn’t know, cares about or doesn’t, has prepared for or not. Most of all, this format allows the examiner to see if the student can think creatively and independently, and how well he can do both.

To a lesser extent (because it isn’t as interactive), an essay exam can discover a great deal about what the student knows about a topic. I used to schedule three-day exams for college students and ask essay questions like:

“Write everything you know about the Federalist Papers. List all 85 and comment on each. Take your time. Be thorough.”

Or,

“List every year from 1776 to the present year, and write at least one major event (what happened, why it was important, how it influenced later events, etc,) for each year.”

The results often ran to forty or more handwritten pages of solid, excellent summary, annotation, commentary, and thinking. Essay exams test knowledge where oral exams more effectively test a combination of knowledge, understanding, wisdom, and the ability to think.

Multiple Choices

think read 2Projects, reports, research papers, and other such assignments are other forms of examination (though not always considered so in the United States). Depending on the specifics of the project, they test different things. In general, such individual or even group assignments test the ability to understand what is expected, plan and work to deliver it, and then implement the plan and follow through. This naturally tests one’s initiative, innovation, work ethic, delivery, and tenacity. All vitally important skills.

In short, and this is my first major point in this article, the best education includes all of these modes of testing (especially between ages 15 and 25). Why not learn all these lessons? Why leave any out? In fact, we should include multiple choice tests in this overall model.

At their best, multiple choice exams are designed to assess precision, rote memorization, and the ability to dialectically narrow down options to a single correct response. These are essential skills.

Schooling, Experts, and Purpose

My second point is that while multiple choice exams are a good part of a youth’s overall educational experience, in modern education they are often overdone—at the expense of the other types of testing and their important lessons. Why not do fewer multiple choice exams and more of the others? Why not reach for a meaningful balance that increases excellence?

Rear view of class raising hands

The problem with this is that too many schools teach to the tests—too often. This is backed by legislatures, administrators, departments, experts, policies, and mandates, so it is difficult to reform. But it should be reformed, nonetheless. And the sooner, the better.

Teaching to the tests puts schooling ahead of learning, and this is literally the same as putting the cart before the horse. It simply doesn’t work very well. When the means become more important than the goals, something is awry, and quality always suffers under such circumstances. Our schools deliver mediocrity and failure much more often than they would if their focus was learning—rather than schooling.

Schooling is supposed to support learning, and it absolutely can. But on the issue of testing, particularly standardized multiple choice testing, schooling frequently harms learning. It harms teaching. It harms studying. It harms thinking.

None of this is good for education.

Flawed Logic

The Purpose of LearningThird, and this is my main thesis, while multiple choice exams do teach effectively in some fields, as a national standardized system they are a very bad idea—because their design is actually based on a flawed logic that is unknown to most people and deeply hurts our society.

What do I mean? Well, in fields where the answers to questions are very much a matter of thinking, not rote memorization, multiple choice exams pit the “expert” answer against the student’s natural tendency to think and reason. This may in fact separate the scores of students who have learned to please the experts (usually labeled “good” students) from those who don’t care what the experts think (often called “bad” students), but it leaves a third group: those who care, but disagree; those who are thinking on their own (e.g. the natural leaders).

In short, in many cases multiple choice exams—whatever academic subject they test and however the student scores—leave the following lessons imprinted on many test-takers:

  1. If you get the right answer, you’re smart
  2. If you get the wrong answer, you’re not smart
  3. The experts determine the answer — so your purpose isn’t to think, it’s to agree with the experts
  4. Smart people agree with the experts
  5. Your independent thinking isn’t smart

Are there logical leaps in this progression? Certainly. But the flawed lessons are there as well. And there is at least as much logic supporting any of these statements as rejecting them. How can a young mind not wonder: “Why else would the adults put so much value and weight on these tests? Obviously, agreeing with the experts is the path to success.”

Testing the Test

Let’s consider this same point from another angle, using different words and examples. Imagine you take the following test, in a highly emotional, competitive schooling environment where you are repeatedly told by those in authority that such tests will determine your success, status, progress, earning levels, and to a large extent your happiness in life. In this emotionally charged state, you sit and try to answer the following:

  1. How many truly excellent ways are there to test students?
    1. One
    2. Two
    3. Three
    4. More than Three
    5. None of the Above
    6. All of the Above
    7. It Depends
  2. Which is the best way to test students?
    1. Multiple Choice Questions
    2. Essay Questions
    3. Assignments/Projects/Reports
    4. Oral Questions
    5. None of the Above
    6. All of the Above
    7. Two of the Above
    8. It Depends

Of course, these exact questions aren’t really on any test that I know about. Thankfully. But they are implicit in every multiple choice test any student ever takes. Think about this for a moment. This is huge!

This form of testing is a good example of many actual exams in which, ironically, the only possible correct answer differs according to an unwritten code.

This is a serious problem. How is one to know when the “code” is in play, and when it isn’t? What, in fact, is the code?

Why is there an unwritten code in the first place?

Exploding the Code

To address these concerns, let’s back up to the first time a child takes a multiple choice exam. A number of things are going on when this happens. For example, the child has no idea there is a “code.” Nor does the child have any idea what the code actually is. Nor has the child ever pondered that the code is designed to reward those who conform to the views of the experts and simultaneously “put in their place” those who think in other ways. So, as a result, the child takes the exam, guided by nothing except his or her innate senses and any instructions given by an adult.

For some children, this turns out to be a victory. Their good memory helps, but ultimately it is their innate sense of pleasing authority that helps them mark the answers they’ve heard from the adults in their life. They are quickly labeled the “good” students. (Assuming, of course, that the parent or teacher uses an exam designed “of the experts, by the experts, and for the experts”—as most modern multiple choice exams are.)

Learning Styles Matter-Mentors failureBut for other children this seminal event is much less promising. Let’s say, for example, that the student’s innate senses are much more attuned to reasoning things out in his own mind, or taking in the evidence from her personal experiences and applying them to each question on the exam, or even doodling on the side of the exam because he innately feels happier and learns more when he activates what pop psychology would call his “right brain creativity.”

Such students will very likely get poor marks on the test. As a result, parent-teacher conferences will be held, adults will worry about and plan for these children, and the kids may even sense from a whisper, a glance, or a word that something has changed in how the adults now think of them.

Of course, the opposite will occur if the teacher or parent chooses, or by simple accident uses, an exam designed to test the child’s reasoning ability—not his attunement to the “right” answers so highly valued by the experts. In such a situation, the first group will feel the sense of loss while the latter groups will be told they are smart.

Certainly the reality is much more complex than this little example. But over time, test after test after test, grade after grade, the child learns to either run with his initial tendencies or find ways to suppress them. He is, inevitably, taught to accept the “expert” approach to learning if he wants to succeed in most schools.

And whether he comes from Group One or Group Two above, he learns that “good,” “smart,” “gold star” students take multiple choice tests the “expert” way, by turning off their thinking, creative, independent, innovative brains and focusing on the rote, accepted, repetitive, expert way of arriving at answers. Not every test is necessarily part of this system, but most are.

Questions and Failure

The great tragedy is that this choice bleeds into other aspects of the young person’s life. How could it not? If tests are so important—and all the adults are loudly affirming that this is inarguably the case—then certainly the world is made in the image of such tests. Right?

beware institutionalism-beware 2The pathway to success is obvious in such a world: rote, repetitive, risk averse, non-creative, dependent but safe, trusting without really challenging assumptions; and, most importantly, truly applying yourself, working very hard to excel, and doing these things better than anyone else does them! Get the rote, safe, accepted answer more frequently than your peers. In fact, get it every time. Get it without fail. Be a truly great student.

Never fail. Failure is…failure.

Don’t learn from failure, just don’t fail. Don’t risk; just do better at what the experts want. This is the way to good grades, scholarships, acceptance, success, advancement, promotion. Even in programs that promote multi-lane thinking, like law schools, such thinking is still expected to conform to clearly delineated norms established by teams of experts.

The fact that such paths usually lead to what has been called “high-class drone work” or “highly-paid repetitive careers” isn’t the point. Or, more accurately: it’s the whole point! The goal. The objective. The plan. The definition of a successful life.

The difficulty is that the answer to these test questions should be “It Depends.” Every time. On every multiple choice test.

That’s a bold statement. But just consider:

  1. How many truly excellent ways are there to test students?
    1. One
    2. Two
    3. Three
    4. It Depends
  2. Which is the best way to test students?
    1. Multiple Choice Questions
    2. Essay Questions
    3. Assignments/Projects/Reports
    4. Oral Questions
    5. None of the Above
    6. All of the Above
    7. It Depends
  3. Solve the problem: 1 + 1 = _________. Mark the correct answer:
    1. 2
    2. 3
    3. 1
    4. All of the Above
    5. None of the Above
    6. x
    7. It Depends

Now, to repeat, here’s the kicker: The best answer to all three of these exam questions is “It Depends.”

Why?

Not for the reasons you might think.

“It Depends”

First of all, “it depends” is the best answer because it is the most interesting answer. In fact, it’s the only interesting answer of those provided. And that’s the point: When the answer has to be one of those provided by the experts, the creative and independent part of the student’s thinking automatically shuts down—at least a little. And frequently a lot.

connectionsYet the best answers, the great answers, any groundbreaking or truly innovative answers, will always come when no solution is provided and you have to use your initiative, innovation, and thinking to come up with something excellent.

Think about it. The student who sincerely answers these questions with “It Depends” is thinking.

In other words, in giving the answer “It Depends” to a question like 1 + 1 = _____, the respondent is telling us something amazing about himself. He is “out the box.” He can see other alternatives. (Too many people trained in the multiple choice era simply don’t see any alternatives, or are fully trained at dismissing them.)

He isn’t dependent on someone’s list of options. He can innovate. He isn’t bound by the so-called “right” answer provided by the experts. In all the years of training and schooling and being told the “right” answers and the “right ways” to find the right answers, and even to “show his work” to prove how he got the right answers in the “approved” way, he has kept a bit of himself apart from the conveyor belt, aloof from pleasing the experts, and independent of doing things just to fit in.

Moreover, by not just marking “2” he is saying that he does not, in fact, care about impressing you—whoever you are who is administering the test—more than about thinking, pondering, considering, and looking beneath the surface. To the world “of experts, by experts, and for experts,” he may be seen as a rebel.

Good. Where thinking and innovating are concerned, we need more rebels. By definition, if you are doing things the accepted, normal way—the way sanctioned and outlined by the experts—you aren’t actually innovating. We need more who think outside the box. More leaders, more trailblazers, more inventors, more artists, more who deeply challenge assumptions and seek for better ways—our world hungers for these.Schoolhouse meme

Excellent Sheep

Our current societal decline is, at least in part, a result of our sad deficiency of originality, initiative, innovation, independent thinking, and chutzpah. And multiple choice tests are designed to decrease such traits even more broadly in the general population. They naturally train “excellent sheep” as William Deresiewicz wrote in his challenging book by that title. Allan Bloom referred to this very trend as “the closing of the American mind.”

The educational battle has in most circles become a competition to win the contest of The Top Rote Learner, The Best Expert Pleaser, and The Superior Elite-Drone Careerist. And multiple choice tests are essential to this model. They aren’t its only feature, to be sure, but without them this system would dwindle. They are part of its glue, its ether, its essence.

They may, in fact, be its essence. The self-chosen name of this system is “the meritocracy,” and what would it be without standardized, nationalized, multiple choice exams? Literally: nothing. It would collapse.

This isn’t to say that a meritocratic essay or oral exam system would be any better. These would simply wend their way directly back to aristocracy. Any one-size-fits-all system of schooling will always underperform in genuine learning and create a funnel to credentialist elitism at the top.

That’s not the answer. The ideal is something much better, a truly educated populace, where all are independent, creative, deep learners and thinkers. Adults as well as students.

Too idealistic? Maybe. But this goal is worth fighting for. And the solution to the current “rote-conveyor-belt/drone-work-careers/excellent-sheep-meritocracy” model is true educational diversity: Let every family ponder, study, consider and choose wisely the best educational path for each individual child and youth—depending on various different, and constantly changing, needs. Let the market respond accordingly.

1 + 1 = ?

A huge surprise-Freedom WorksFreedom actually works. We should give it a chance in the field of learning. To the extent that we do, learning will overtake schooling—to the huge benefit of all society.

But,