Homeschool and Testing Part II: The Weekly Mentor by Oliver DeMille

Homeschool and Testing, Part II

(The Huge Problem With Multiple Choice Exams)

by Oliver DeMille

(Note: if you haven’t read Homeschool and Testing Part I, please read it here,)

Question: What is the problem with multiple choice tests?

Answer: Well, it depends…

Really. That’s the best answer. Nearly always. “It depends…”

C.S. Lewis taught in The Abolition of Man that all schooling has a design, meaning that each curriculum and testing arrangement is designed to bring about certain results. And many great educational thinkers from all sides of the education debates have agreed with this idea—from Dewey to Montessori, and from Adler to Jacques Barzun, for example.

So, to get straight to the point, what are multiple choice exams designed to do to (and for) students? To explore this question, let’s use a little comparison.

How and What to Test

Oral exams naturally focus on what the student has learned, and how effectively he or she thinks. Oral examiners ask a question, listen to the answer, and then respond with further follow-up questions. In Socratic form, they notice an area of hesitation in the student’s response, a clear error, or a potential weakness. And so they follow up with additional questions to see if this is really a lack of knowledge and understanding, or if perhaps the student simply misspoke.

Likewise, the examiners witness a strength, or even a talent, and ask the student to share more of it. In this way, oral exams can very effectively arrive at an overview and outline of what the student knows and doesn’t know, cares about or doesn’t, has prepared for or not. Most of all, this format allows the examiner to see if the student can think creatively and independently, and how well he can do both.

To a lesser extent (because it isn’t as interactive), an essay exam can discover a great deal about what the student knows about a topic. I used to schedule three-day exams for college students and ask essay questions like:

“Write everything you know about the Federalist Papers. List all 85 and comment on each. Take your time. Be thorough.”

Or,

“List every year from 1776 to the present year, and write at least one major event (what happened, why it was important, how it influenced later events, etc,) for each year.”

The results often ran to forty or more handwritten pages of solid, excellent summary, annotation, commentary, and thinking. Essay exams test knowledge where oral exams more effectively test a combination of knowledge, understanding, wisdom, and the ability to think.

Multiple Choices

Projects, reports, research papers, and other such assignments are other forms of examination (though not always considered so in the United States). Depending on the specifics of the project, they test different things. In general, such individual or even group assignments test the ability to understand what is expected, plan and work to deliver it, and then implement the plan and follow through. This naturally tests one’s initiative, innovation, work ethic, delivery, and tenacity. All vitally important skills.

In short, and this is my first major point in this article, the best education includes all of these modes of testing (especially between ages 15 and 25). Why not learn all these lessons? Why leave any out? In fact, we should include multiple choice tests in this overall model.

At their best, multiple choice exams are designed to assess precision, rote memorization, and the ability to dialectically narrow down options to a single correct response. These are essential skills.

Schooling, Experts, and Purpose

My second point is that while multiple choice exams are a good part of a youth’s overall educational experience, in modern education they are often overdone—at the expense of the other types of testing and their important lessons. Why not do fewer multiple choice exams and more of the others? Why not reach for a meaningful balance that increases excellence?

The problem with this is that too many schools teach to the tests—too often. This is backed by legislatures, administrators, departments, experts, policies, and mandates, so it is difficult to reform. But it should be reformed, nonetheless. And the sooner, the better.

Teaching to the tests puts schooling ahead of learning, and this is literally the same as putting the cart before the horse. It simply doesn’t work very well. When the means become more important than the goals, something is awry, and quality always suffers under such circumstances. Our schools deliver mediocrity and failure much more often than they would if their focus was learning—rather than schooling.

Schooling is supposed to support learning, and it absolutely can. But on the issue of testing, particularly standardized multiple choice testing, schooling frequently harms learning. It harms teaching. It harms studying. It harms thinking.

None of this is good for education.

Flawed Logic

Third, and this is my main thesis, while multiple choice exams do teach effectively in some fields, as a national standardized system they are a very bad idea—because their design is actually based on a flawed logic that is unknown to most people and deeply hurts our society.

What do I mean? Well, in fields where the answers to questions are very much a matter of thinking, not rote memorization, multiple choice exams pit the “expert” answer against the student’s natural tendency to think and reason. This may in fact separate the scores of students who have learned to please the experts (usually labeled “good” students) from those who don’t care what the experts think (often called “bad” students), but it leaves a third group: those who care, but disagree; those who are thinking on their own (e.g. the natural leaders).

In short, in many cases multiple choice exams—whatever academic subject they test and however the student scores—leave the following lessons imprinted on many test-takers:

1. If you get the right answer, you’re smart
2. If you get the wrong answer, you’re not smart
3. The experts determine the answer — so your purpose isn’t to think, it’s to agree with the experts
4. Smart people agree with the experts
5. Your independent thinking isn’t smart

Are there logical leaps in this progression? Certainly. But the flawed lessons are there as well. And there is at least as much logic supporting any of these statements as rejecting them. How can a young mind not wonder: “Why else would the adults put so much value and weight on these tests? Obviously, agreeing with the experts is the path to success.”

Testing the Test

Let’s consider this same point from another angle, using different words and examples. Imagine you take the following test, in a highly emotional, competitive schooling environment where you are repeatedly told by those in authority that such tests will determine your success, status, progress, earning levels, and to a large extent your happiness in life. In this emotionally charged state, you sit and try to answer the following:

1. How many truly excellent ways are there to test students?
   1. One
   2. Two
   3. Three
   4. More than Three
   5. None of the Above
   6. All of the Above
   7. It Depends
2. Which is the best way to test students?
   1. Multiple Choice Questions
   2. Essay Questions
   3. Assignments/Projects/Reports
   4. Oral Questions
   5. None of the Above
   6. All of the Above
   7. Two of the Above
   8. It Depends

Of course, these exact questions aren’t really on any test that I know about. Thankfully. But they are implicit in every multiple choice test any student ever takes. Think about this for a moment. This is huge!

This form of testing is a good example of many actual exams in which, ironically, the only possible correct answer differs according to an unwritten code.

This is a serious problem. How is one to know when the “code” is in play, and when it isn’t? What, in fact, is the code?

Why is there an unwritten code in the first place?

Exploding the Code

To address these concerns, let’s back up to the first time a child takes a multiple choice exam. A number of things are going on when this happens. For example, the child has no idea there is a “code.” Nor does the child have any idea what the code actually is. Nor has the child ever pondered that the code is designed to reward those who conform to the views of the experts and simultaneously “put in their place” those who think in other ways. So, as a result, the child takes the exam, guided by nothing except his or her innate senses and any instructions given by an adult.

For some children, this turns out to be a victory. Their good memory helps, but ultimately it is their innate sense of pleasing authority that helps them mark the answers they’ve heard from the adults in their life. They are quickly labeled the “good” students. (Assuming, of course, that the parent or teacher uses an exam designed “of the experts, by the experts, and for the experts”—as most modern multiple choice exams are.)

But for other children this seminal event is much less promising. Let’s say, for example, that the student’s innate senses are much more attuned to reasoning things out in his own mind, or taking in the evidence from her personal experiences and applying them to each question on the exam, or even doodling on the side of the exam because he innately feels happier and learns more when he activates what pop psychology would call his “right brain creativity.”

Such students will very likely get poor marks on the test. As a result, parent-teacher conferences will be held, adults will worry about and plan for these children, and the kids may even sense from a whisper, a glance, or a word that something has changed in how the adults now think of them.

Of course, the opposite will occur if the teacher or parent chooses, or by simple accident uses, an exam designed to test the child’s reasoning ability—not his attunement to the “right” answers so highly valued by the experts. In such a situation, the first group will feel the sense of loss while the latter groups will be told they are smart.

Certainly the reality is much more complex than this little example. But over time, test after test after test, grade after grade, the child learns to either run with his initial tendencies or find ways to suppress them. He is, inevitably, taught to accept the “expert” approach to learning if he wants to succeed in most schools.

And whether he comes from Group One or Group Two above, he learns that “good,” “smart,” “gold star” students take multiple choice tests the “expert” way, by turning off their thinking, creative, independent, innovative brains and focusing on the rote, accepted, repetitive, expert way of arriving at answers. Not every test is necessarily part of this system, but most are.

Questions and Failure

The great tragedy is that this choice bleeds into other aspects of the young person’s life. How could it not? If tests are so important—and all the adults are loudly affirming that this is inarguably the case—then certainly the world is made in the image of such tests. Right?

The pathway to success is obvious in such a world: rote, repetitive, risk averse, non-creative, dependent but safe, trusting without really challenging assumptions; and, most importantly, truly applying yourself, working very hard to excel, and doing these things better than anyone else does them! Get the rote, safe, accepted answer more frequently than your peers. In fact, get it every time. Get it without fail. Be a truly great student.

Never fail. Failure is…failure.

Don’t learn from failure, just don’t fail. Don’t risk; just do better at what the experts want. This is the way to good grades, scholarships, acceptance, success, advancement, promotion. Even in programs that promote multi-lane thinking, like law schools, such thinking is still expected to conform to clearly delineated norms established by teams of experts.

The fact that such paths usually lead to what has been called “high-class drone work” or “highly-paid repetitive careers” isn’t the point. Or, more accurately: it’s the whole point! The goal. The objective. The plan. The definition of a successful life.

The difficulty is that the answer to these test questions should be “It Depends.” Every time. On every multiple choice test.

That’s a bold statement. But just consider:

1. How many truly excellent ways are there to test students?
   1. One
   2. Two
   3. Three
   4. It Depends
2. Which is the best way to test students?
   1. Multiple Choice Questions
   2. Essay Questions
   3. Assignments/Projects/Reports
   4. Oral Questions
   5. None of the Above
   6. All of the Above
   7. It Depends
3. Solve the problem: 1 + 1 = _________. Mark the correct answer:
   1. 2
   2. 3
   3. 1
   4. All of the Above
   5. None of the Above
   6. x
   7. It Depends

Now, to repeat, here’s the kicker: The best answer to all three of these exam questions is “It Depends.”

Why?

Not for the reasons you might think.

“It Depends”

First of all, “it depends” is the best answer because it is the most interesting answer. In fact, it’s the only interesting answer of those provided. And that’s the point: When the answer has to be one of those provided by the experts, the creative and independent part of the student’s thinking automatically shuts down—at least a little. And frequently a lot.

Yet the best answers, the great answers, any groundbreaking or truly innovative answers, will always come when no solution is provided and you have to use your initiative, innovation, and thinking to come up with something excellent.

Think about it. The student who sincerely answers these questions with “It Depends” is thinking.

In other words, in giving the answer “It Depends” to a question like 1 + 1 = _____, the respondent is telling us something amazing about himself. He is “out the box.” He can see other alternatives. (Too many people trained in the multiple choice era simply don’t see any alternatives, or are fully trained at dismissing them.)

He isn’t dependent on someone’s list of options. He can innovate. He isn’t bound by the so-called “right” answer provided by the experts. In all the years of training and schooling and being told the “right” answers and the “right ways” to find the right answers, and even to “show his work” to prove how he got the right answers in the “approved” way, he has kept a bit of himself apart from the conveyor belt, aloof from pleasing the experts, and independent of doing things just to fit in.

Moreover, by not just marking “2” he is saying that he does not, in fact, care about impressing you—whoever you are who is administering the test—more than about thinking, pondering, considering, and looking beneath the surface. To the world “of experts, by experts, and for experts,” he may be seen as a rebel.

Good. Where thinking and innovating are concerned, we need more rebels. By definition, if you are doing things the accepted, normal way—the way sanctioned and outlined by the experts—you aren’t actually innovating. We need more who think outside the box. More leaders, more trailblazers, more inventors, more artists, more who deeply challenge assumptions and seek for better ways—our world hungers for these.

Excellent Sheep

Our current societal decline is, at least in part, a result of our sad deficiency of originality, initiative, innovation, independent thinking, and chutzpah. And multiple choice tests are designed to decrease such traits even more broadly in the general population. They naturally train “excellent sheep” as William Deresiewicz wrote in his challenging book by that title. Allan Bloom referred to this very trend as “the closing of the American mind.”

The educational battle has in most circles become a competition to win the contest of The Top Rote Learner, The Best Expert Pleaser, and The Superior Elite-Drone Careerist. And multiple choice tests are essential to this model. They aren’t its only feature, to be sure, but without them this system would dwindle. They are part of its glue, its ether, its essence.

They may, in fact, be its essence. The self-chosen name of this system is “the meritocracy,” and what would it be without standardized, nationalized, multiple choice exams? Literally: nothing. It would collapse.

This isn’t to say that a meritocratic essay or oral exam system would be any better. These would simply wend their way directly back to aristocracy. Any one-size-fits-all system of schooling will always underperform in genuine learning and create a funnel to credentialist elitism at the top.

That’s not the answer. The ideal is something much better, a truly educated populace, where all are independent, creative, deep learners and thinkers. Adults as well as students.

Too idealistic? Maybe. But this goal is worth fighting for. And the solution to the current “rote-conveyor-belt/drone-work-careers/excellent-sheep-meritocracy” model is true educational diversity: Let every family ponder, study, consider and choose wisely the best educational path for each individual child and youth—depending on various different, and constantly changing, needs. Let the market respond accordingly.

1 + 1 = ?

Freedom actually works. We should give it a chance in the field of learning. To the extent that we do, learning will overtake schooling—to the huge benefit of all society.

But, before we go on, there’s one more thing. There’s a second reason that the most accurate solution to 1 + 1 = ________ is “it depends.” What is it? Well, in fact, “it depends” is actually the most accurate sum. It really is.

What does this mean? For most people trained on the conveyor belt system of modern education, the rote method has trained them to think that 1 + 1 is 2. And it can be, under certain circumstances. But what the large majority of people educated in this era of rote factory-style learning were never taught is that 1 + 1 = 2 has never been proven. Why? Because no mathematician has ever been able to prove that 1 = 1.

In fact, 1 = 1 is an unproven axiom, meaning that it is an assumption that mathematicians just accept even though they can’t prove it. There is a long history of great mathematicians and scientists who tried to prove that 1 = 1, and that therefore 1 + 1 = 2, but the best they could come up with is to start their mathematics with the postulate:

“Let” 1 = 1

Then they built everything else, including 1 + 1 = 2, on top of this foundational assumption. But the foundation is still unproven.

In fact, it has been widely disproven. For example, if you are talking about apples, 1 never equals 1. There are no 2 apples that are truly the same.

The only way they are identical is numerically, meaning in our mathematical minds or what Plato called “The Ideal.” Not in physical reality. So, yes, if you are only working a math problem, then 1 + 1 might equal 2 because humans long ago decided to “Let 1 + 1.”

But there are many times in the real world where 1 doesn’t equal 1. In fact, the only place 1 = 1 is in theoretical mathematics. So, if the question is posed as it was above (“Solve the problem: 1 + 1 = _________”), the truly most accurate answer, mathematically and otherwise, is “it depends.” If you are using theoretical math and you have defined your terms based on “Let 1 = 1”, the answer is 2. If you are not, the answer is “it depends.”

But since the question above didn’t specify, then it depends. Period.

In short, the student who marked “it depends” got it right while all those who marked “2” got it wrong. The ones who chose “2” weren’t just trying to please the experts; most of them actually don’t understand the mathematics. Thus the sheep aren’t all that “excellent” after all. And that’s a huge problem in our educational system, because many multiple choice questions on our nationalized exams (in all subjects) are this way. Many!

The Power of Many

Again, we need more than one kind of testing to get it right, because not every student is the same—any more than your 1 apple is the same as any other 1 apple. And while the number 1 can be used numerically in theoretical mathematics, when we are talking about children we need something much more serviceable. We need to help each one of them get a superb education, and that means that one size doesn’t fit everyone.

Let there be many kinds of schools, and many kinds of tests. Some will prove beneficial to great learning, while others will not. And over time those most effective will flourish and grow. But all will be available to the individual student who needs one or two of them to really flourish.

This kind of educational buffet only thrives in a free system. And right now, homeschooling is a leader in this charge. Let there be as many kinds of testing as there are children with different educational needs, goals, talents and dreams. Let the variety of tests grow so there is a truly excellent testing model to match each student.

That’s great education. Nothing else comes close.

Is this kind of truly personalized education and testing for every single important student even possible at a national level? Perhaps. Perhaps not. But it is certainly possible in most homes and families.

Starting with yours.

******

Are you ready for an upgrade to your family education culture? Check out the TJEd Implementation course, Mentoring in the Classics!