Monthly Archives: October 2012

Think like a test writer II

It is a bit harder to write your own good quantitative question stems (trust me, I once wrote a big chunk of a test bank for my old university’s quantitative methods subject — it took forever!) but there is still some useful thinking like a test writer to do for your normal multiple-choice problem solving questions on the GMAT, GRE or SAT.  In particular, if you are a test writer, you want various percentages of people to pick the wrong answer over the right answer.  So how do you get them to do that?

Well, harder problems are harder (generally) because they require more steps.  Many non-TestSmashing test takers will forget where they are heading on a question and just kind of stop when they feel they have taken enough steps — or stop when they have a number that matches an answer choice.  So how do you gull these poor souls into choosing a wrong answer?  Make sure that each step’s answer appears in the answer choices!  You, of course, are going to remember what the test really tests, and make sure you actually answer the question being asked, right?

And, while it breaks my heart to say it, waaaay too many people who feel helpless on a question will just randomly combine the numbers in a question.  (Are you one of those people?  For the love of God and all that is good, resolve to STOP TODAY and never, ever do that again!)  Fill in the remaining wrong answer slots with such random combinations.

If you want active practice thinking like a test writer for problem solving questions, pick up an algebra text book (or search the internet) and find some open-ended word problems.  Solve them, and note along the way the intermediate steps.  Come up with 5 answer choices.  Post in comments if you want!

Advertisement

Think like a test writer

One great way to start smashing test questions is to start thinking like a test writer, instead of a test taker.  You’d better believe that the test writers spend tons of time studying how you think,* so don’t you think you should return the favor?

If you were assigned to write questions for your choice of test, what would your objective be?  Well, this is a standardized test, and the standardization means that the people who take the test this year have to have basically the same experience as last year and the year before, only the questions have to be completely different.  Not only does it have to feel the same, but the same person (who hasn’t done any test smashing) should get the same score every time she takes the test, within some narrow band.  And in order for that to happen, you have to know as precisely as possible, how many people — and of what “sort” — will get your question correct or incorrect.  Of those who get the question incorrect, you should have a pretty good idea of the distribution of which wrong answer they will choose.  The hopeless test takers will go for one wrong answer, and the almost-there test takers will, perhaps, fall for a different trap.

Every time you take an official test, you are contributing your lab rat skills on those experimental sections.  (Does that make you feel good?  You are contributing to science!  For free!)  That is where they test and calibrate their questions.  But you’d better believe that they aren’t writing random questions that they then learn about on the experimental section: they already have a pretty good idea what is going to happen by the time the question shows up on your test.

So, if you are the manager of a bunch of question writers, what are you going to do?  Well, there have to be templates that writers share, so that Sarah’s questions look the same as Karim’s questions.  And, dear TestSmasher, if you can figure out the templates, it is like having a master key to the test — or a sledge hammer!

This could work for any question type, but I think arguments are particularly well suited for this exercise, which is to sit down and write (at least) 3 questions in the style of your test:

  1. Pick one stem (maybe a question type you hate?) and use it for all three.
  2. Find a couple of official questions of the same type.
  3. Analyze the best you can the argument structure: number of sentences, the role they serve, etc.
  4. Choose 3 totally unrelated topics
  5. Craft an argument on each topic that conforms to the structure
  6. Write at least three answer choices (five might get tedious)

Most of the art of question writing lies in writing the answer choices, so let’s drill down into that one.

The correct answer has to be:

  1. Indisputably correct.  (Think about aggrieved test takers and their lawyers.)
  2. Not obvious.

How do you do both at once?  The way I would go about it is write an obvious and correct answer choice and then revise by picking synonyms or other phrases that mean the same thing but aren’t the way you would think to characterize the correct answer if you were a normal person instead of a test writer.

The wrong answers have to be:

  1. Indisputably incorrect.  (Think lawyers again!)
  2. Attractive.

And how do you do that?  Well, you say things that are correct but not answering the question, of course!  Or things that are in response to the question but are slightly, subtly, but fatally wrong.   Write at least one of each.

Once you have done this exercise, first, post your fake questions in the comments — I’d love to see what you come up with.  Then, go do a set of real questions with the same stem.  Do you approach the questions differently?  Identify the wrong (or right) answers more consistently?  Do you catch yourself before you fall into a trap that used to trip you up?

*There is an entire scientific field, psychometrics, that studies how you think and respond to questions designed to measure something about your cognition, psychology, or character.

My current project

I’ve been neglecting the website lately, as I have been spending most of my TestSmasher time working on my new book, Leaving Low Scores Behind: LSAT Edition.  It is a three week self-study guide to walk you through how to get past a low score, re-establish a much better relationship with the test, and develop study skills and a strategy to launch you into a successful period of study and test taking where you actually get the score you need to go to the law school of your dreams.  The book is based on insights from neuroscience, social psychology, and my years of teaching struggling students.  I’ve now finished a draft and am starting to think about the best way to publish, market, and distribute the book.

If this sounds intriguing to you, please consider signing up to be my beta tester!  I would love to work with a few more students who are looking to improve their scores significantly so that I can understand how well it speaks to different people and figure out ways to help you and it kick ass even harder.  I’m still offering my “Low Score Recovery” package, which is a steeply discounted opportunity to work with me as you go through Leaving Low Scores Behind.   If you are interested in the book but not the tutoring, please shoot me an email, and we can work something out.

Finally, if you want to walk proudly away from an old low score and on to a much higher score on the GMAT or the GRE, please also contact me!  My plan is to adapt this to each of the graduate-level tests, and I’d love to help you too.