Who taught you to write tests?

Can a test capture how well you understand something?

I’m not talking about the massive standardized tests that are making us crazy right now. Just the humble classroom test, written by a teacher to evaluate how well a student is doing in class. The kind of test that forms a grade, then a transcript. Then a GPA and a college decision.

Most of the tests I remember from my school days were absurd.

The consumer education final, which we also took as a pre-test. That test was so easy I earned a B+ based on common sense, and I couldn’t believe my high school still made me take the course. And yet, I did learn things in the class.

Every physics test. Those tests were ridiculously hard, and they had almost nothing to do with what we had learned in class. I think my teacher actually forgot what he had taught us when he wrote them.

And the English tests that expected us to have the “right” interpretation of symbolism and allusions? I still don’t believe there are right answers to those questions. There shouldn’t be.

Writing tests that capture the essence of what students know is an art form, and we are not well prepared to do it. The thoughtful conversations about test-writing when I got my license? Non-existent. The guidance from my supervising teacher? Similarly lacking.

Most teachers I knew when I was starting out just took questions from the textbook test bank. Sometimes they aligned to what happened in class, and sometimes not. Sometimes the wording was clear, and sometimes not.

It wasn’t until a few years ago, when I was trained in “Understanding by Design,” that I started thinking seriously about test design. The tests we write count for a lot. Tests are the summative feedback that tell our students and their parents whether they “get it” or not. They are part of the signal that tells others — like colleges, scholarship sponsors and even insurance companies — what to expect from a student. It is crucial that we do this well.  

What should we know about test-writing that we don’t learn in school?

1. No test is a perfect instrument. Your best bet is to diversify, like picking a stock portfolio. Use some multiple choice and also some essay questions. Let students use notes on some tests but not on others. Make some tests oral, if you can. Students who perform well on all of these definitely know their stuff. Students who struggle with one form of testing will have other opportunities to shine.
2. Backwards design makes sense. Think about what students need to know and what you will expect them to do. Have your test in mind while you develop your curriculum. This sounds like “teach to the test,” but that’s an false simplification. If I want students to know how to use measures of central tendency (mean, median, mode), then I should write questions that deeply probe their understanding of these concepts. And I should teach in a way that will help them answer all of those kinds of questions.
3. Write questions at different levels of Bloom’s taxonomy. Tests should include simple questions that nearly everyone can answer (like definitions) as well as challenging questions that signal deep understanding. “What is a median?” is a very different question from “In what circumstances is the median a more accurate measure than a mean?” or “Which measure of central tendency should be used when evaluating the typical household income, and why?”
4. Every question is important. Think about what you are asking and how it is phrased, and never ask a throwaway question (or include a throwaway distracter). Emphasize the most important concepts, not just the ones that are easiest to test, and be sure the emphasis of your test aligns with what is emphasized in the class.
5. Use neutral language. Don’t write test questions that unfairly assume prior knowledge. If you want to see how completely unfair and culture-specific tests can be, try this World War I IQ test.
6. Use test performance to evaluate yourself. If my students do badly on a test, it reflects on them. Maybe they didn’t study. But it also reflects on me, and I need to use the test data to figure out how to teach better. What are they missing? Did I explain it well enough? Does the test use terminology they can understand or reason out?

Writing tests that truly reflect our students’ understanding is difficult, but if we can do it well, it helps build trust between teachers, students and parents. Clear expectations and clear measurement signal students’ what they need to do, and they are more likely to buy in.