Category Archives: The REAL test

The Study Cycle

What do you do when you study?  Have you thought at all about how studying for the LSAT/GMAT/GRE/SAT is different from a biology test or a French exam?  Remember that standardized tests test how you think far more than testing what you know.  How do you study for something so meta?

The only way to change how you think is to think differently.  Sounds like a tautology, right?  But if you approach every question the same way, you are going to get basically the same results each time.  Remember, test writers have an entire scientific field dedicated to ensuring this is true.  So you have to break your brain out of its box, and no, I don’t mean by banging it against a brick wall.  Instead, you want to design a study cycle that constantly tries new perspectives and then knits them into the basic test taking experience.

A study cycle involves these steps:

  1. Take a baseline measure.  This might be a timed section, timed test, or just a sequence of 10 questions of the type you are targeting.
  2. Analyze your results.  What questions did you get wrong?  Why did you get them wrong?  What patterns are there?  If you could stop doing one thing wrong, what would it be?
  3. Brainstorm some exercises that will attack that one thing you want most to fix.  I suggested some already, and will continue to do so.  You want to look for exercises that take you out of your comfort zone, such as not reading the RC passage.  Some are clearly not what you’d actually do on a test, such as the question writing exercise, but they give you a new perspective.
  4. Practice your exercise enough to give it a real go — don’t stop after one attempt because it is a disaster or it scares the shit out of you — spend real time on it.
  5. Take a post-exercise measurement.  How did your experience change?  Did you get more questions correct?  Were your mistakes of a different nature?  Any progress on that one thing wrong?
  6. Repeat.

In general, you want these cycles to be pretty short, maybe spanning two or three days of work, one or two pomodoro units a day.  Some will be bigger efforts, though you might want to mix in study cycles of a totally different sort to keep your brain nimble and your interest piqued.

A lot of what I, and other tutors, can do for you is suggest creative changes to your process — we have a lot of experience that way.  But it is also important that you start to own your own study cycle.  Be a little playful, and enjoy the creativity involved in coming up with something new.

Advertisement

Think like a test writer II

It is a bit harder to write your own good quantitative question stems (trust me, I once wrote a big chunk of a test bank for my old university’s quantitative methods subject — it took forever!) but there is still some useful thinking like a test writer to do for your normal multiple-choice problem solving questions on the GMAT, GRE or SAT.  In particular, if you are a test writer, you want various percentages of people to pick the wrong answer over the right answer.  So how do you get them to do that?

Well, harder problems are harder (generally) because they require more steps.  Many non-TestSmashing test takers will forget where they are heading on a question and just kind of stop when they feel they have taken enough steps — or stop when they have a number that matches an answer choice.  So how do you gull these poor souls into choosing a wrong answer?  Make sure that each step’s answer appears in the answer choices!  You, of course, are going to remember what the test really tests, and make sure you actually answer the question being asked, right?

And, while it breaks my heart to say it, waaaay too many people who feel helpless on a question will just randomly combine the numbers in a question.  (Are you one of those people?  For the love of God and all that is good, resolve to STOP TODAY and never, ever do that again!)  Fill in the remaining wrong answer slots with such random combinations.

If you want active practice thinking like a test writer for problem solving questions, pick up an algebra text book (or search the internet) and find some open-ended word problems.  Solve them, and note along the way the intermediate steps.  Come up with 5 answer choices.  Post in comments if you want!

Think like a test writer

One great way to start smashing test questions is to start thinking like a test writer, instead of a test taker.  You’d better believe that the test writers spend tons of time studying how you think,* so don’t you think you should return the favor?

If you were assigned to write questions for your choice of test, what would your objective be?  Well, this is a standardized test, and the standardization means that the people who take the test this year have to have basically the same experience as last year and the year before, only the questions have to be completely different.  Not only does it have to feel the same, but the same person (who hasn’t done any test smashing) should get the same score every time she takes the test, within some narrow band.  And in order for that to happen, you have to know as precisely as possible, how many people — and of what “sort” — will get your question correct or incorrect.  Of those who get the question incorrect, you should have a pretty good idea of the distribution of which wrong answer they will choose.  The hopeless test takers will go for one wrong answer, and the almost-there test takers will, perhaps, fall for a different trap.

Every time you take an official test, you are contributing your lab rat skills on those experimental sections.  (Does that make you feel good?  You are contributing to science!  For free!)  That is where they test and calibrate their questions.  But you’d better believe that they aren’t writing random questions that they then learn about on the experimental section: they already have a pretty good idea what is going to happen by the time the question shows up on your test.

So, if you are the manager of a bunch of question writers, what are you going to do?  Well, there have to be templates that writers share, so that Sarah’s questions look the same as Karim’s questions.  And, dear TestSmasher, if you can figure out the templates, it is like having a master key to the test — or a sledge hammer!

This could work for any question type, but I think arguments are particularly well suited for this exercise, which is to sit down and write (at least) 3 questions in the style of your test:

  1. Pick one stem (maybe a question type you hate?) and use it for all three.
  2. Find a couple of official questions of the same type.
  3. Analyze the best you can the argument structure: number of sentences, the role they serve, etc.
  4. Choose 3 totally unrelated topics
  5. Craft an argument on each topic that conforms to the structure
  6. Write at least three answer choices (five might get tedious)

Most of the art of question writing lies in writing the answer choices, so let’s drill down into that one.

The correct answer has to be:

  1. Indisputably correct.  (Think about aggrieved test takers and their lawyers.)
  2. Not obvious.

How do you do both at once?  The way I would go about it is write an obvious and correct answer choice and then revise by picking synonyms or other phrases that mean the same thing but aren’t the way you would think to characterize the correct answer if you were a normal person instead of a test writer.

The wrong answers have to be:

  1. Indisputably incorrect.  (Think lawyers again!)
  2. Attractive.

And how do you do that?  Well, you say things that are correct but not answering the question, of course!  Or things that are in response to the question but are slightly, subtly, but fatally wrong.   Write at least one of each.

Once you have done this exercise, first, post your fake questions in the comments — I’d love to see what you come up with.  Then, go do a set of real questions with the same stem.  Do you approach the questions differently?  Identify the wrong (or right) answers more consistently?  Do you catch yourself before you fall into a trap that used to trip you up?

*There is an entire scientific field, psychometrics, that studies how you think and respond to questions designed to measure something about your cognition, psychology, or character.

A note on question types

In my last post, I was a bit snarky about the tendency of test prep companies to develop complicated taxonomies of, say, argument question types.  I’d like to be a bit more nuanced about that.  I like simple taxonomies, ergo my DESTROY, SUPPORT, DESCRIBE for arguments, and I do think that all of the arguments fit into one of those three broad categories.

HOWEVER.  In the course of your studying, if you start to notice that it helps you to think about “inference” questions differently than “conclusion” questions, that is fantastic.  If it identifies for you a difference that is meaningful to you in what approach you take/traps to look out for, then you know you are studying and learning well.  My only quarrel is when you spend a ton of time (and money) learning someone else’s taxonomy, with nuances that don’t mean a whole lot to you.  Then you stress out a bunch over your inability to see any difference between an inference or a conclusion question, when at the end of the day, there isn’t a difference, and it shouldn’t cause you any stress, because if you weren’t so fucking stressed out, you’d get the right answer.

This applies to other question types as well (RC, LSAT games, math questions on the various math-y tests).

What are you studying for?

I want to post some more worked examples for arguments, but I’ve been chewing over this question for quite a while now, and want to provide some thoughts on what the bulk of test-prep study time should be focused on.

There is a little bit of substantive stuff that is worth making sure you know for each of the tests.  Some math maybe, or logic rules for the LSAT.  This has to be done.  It is also possible to create some things to study for, such as finely grained categories of argument questions.  This is usually a waste of time.  Or you could just do practice questions until the cows come home, hoping for improved results.  The tests are pretty resistant to magical thinking, though.

So what are you studying for?

I like to think of it as taking the decisions out of the test.  You want to absolutely minimize (i.e. get to zero) the moments on the test where you are looking at a question and don’t have a plan for solving it.  This is different from knowing what the right answer is off the bat.  Instead, it is having a process that you know and trust and FOLLOW EVERY SINGLE TIME.  My suspicion for the students who take a test prep class, feel like they are improving, and then get the same fucking score as they have gotten at the start, is that they get into the exam and wing it, rather than relying on a set process.

When you are nervous, you make DUMB decisions.  Big time DUMB.  You are not dumb, but when you are thinking about “ohmygodIhopeIdowellbecause IreallyreallywannagetintoHarvard” you are not at your tip top in thinking through the ins and outs of your approach.  It’s just the way the brain works.  So the way to cope is not to start thinking “ohmygodIbetternotfreakoutbecausethen I’mgoingtomakebaddecisionsandnotgetintoHarvard” but instead to not make decisions when it isn’t safe.  Sort of like not driving drunk.

When you turn the page/click to the next screen, and see a reading comprehension passage, you want to have a tried-and-true process that you don’t question and don’t change on test day.  You are going to read the first question stem, skim for an answer, prove every fucking word of your answer choice is addressed in the passage, rinse and repeat.  You are not suddenly going to decide that maybe reading the passage first would be a good idea.  You aren’t going to go all magical thinking and pick (C) because it just “sounds” like the right answer.  You aren’t going to panic about the time and pick the first plausible answer because “whatifI’mtooslowanddon’tfinishontime bettergofasterorImightnotgetintoHarvard.”

You want to walk into the test with a very clear, realistic idea of what your score is going to be and how you are going to earn it.  Of course there will be some variation and some moments that may call for a bit of creativity or insight, but that is to be kept to a minimum.  If you don’t already know what your score will be, you shouldn’t be taking the test.  If you think you know what your score will be but hope it will be lots better, you DEFINITELY aren’t ready to take the test.  Instead, you want to work systematically through the question types on your test, and figure out your approach, practice it consistently until you would NEVER do it differently, then move on to the next question type.  This is the heart of Test Smashing.  The test should be little smithereens before you walk into the test center.

What do they REALLY test?

I said earlier that the tests don’t really test math or verbal skills.  This is why I have an ethical problem with these tests: they do their thing by trying to deceive and distract you.  Not cool, ETS.  On the other hand, the skills they do test?  They are handy skills to have, and will help you in school and in your job and in life.  People who cultivate these skills definitely have a leg up in life.  This is good news, ’cause you are about to start cultivating them, and I can promise you, it won’t be a waste of time.  Hard, maybe, but worth it.

So what are these secret skills?

  1. Following directions
  2. Paying attention to details
  3. Organizing your information effectively

“Say whaa?”  you say.  “That’s so easy.  Why aren’t I already at 800/180?”  Well, my friend, chances are you suck at this stuff.  Most people do.  Following directions?  We are, as a human race, catastrophically bad at following written directions.  Catastrophic because we assume that we can follow said directions, which means we try to give direction by writing.  (Have you ever tried to follow prescription directions?  How to exit a building safely in the case of fire?  Can you even follow a recipe?  Can you fill out your tax forms?  Correctly??)

Paying attention to details?  Even more hopeless.  People work in approximations.  They don’t want to think through things really carefully, and read every single word.  Hvea yuo sene tohse eaxmlpes wrhee teh ltetres aer all jmubeld?  We can read them anyway!  Which is cool, but it means we aren’t sweating the small stuff.  Great for lots of life, bad for the SAT/GMAT/LSAT/GRE.

Organizing your information effectively is a subtler skill than the other two.  It is what helps you overcome your instincts to gloss over important details or answer a question differently than the one being asked.  It is also, of course, what will crack open the analytical reasoning section on the LSAT (i.e. the LSAT games).  It also gets you most of the distance on data sufficiency questions (GMAT) and quantitative comparison questions (GRE and SAT).

I actually think that my previous career as a musician was instrumental to my being good at these tests.  When you are reading music, it is all about following directions and paying attention to detail.  Every dot on the page has lots of meaning.  You also learn to structure that mess of data into phrases, movements, musical forms, etc.  You see the structure beneath the variations.

So how do you study these skills?  There are no formulas to memorize.

I would first approach this as a practice, rather than as a finite set of skills.  Think yoga or meditation, not cramming for your calculus exam.  Awareness is the first step.  If you start studying for the GRE, say, by memorizing a bunch of geometry formulas, you are heading for a long, slow, unproductive slog.  If instead, you take some practice questions (written by the real test writers, please!) observe yourself as you go through the questions.  If you get a question wrong, was it because you “just made a careless mistake”?  ETS test writers lure you into carelessness.  That didn’t happen by accident.  Which piece of information necessary to solving the question did you gloss over?  Did you rely on your gut when you chose your answer?  Start making a list, and study it for common themes.

You can develop your practice in other ways.  Start working your way through intricate recipes, Julie & Julia -style.  When a recipe doesn’t come out the way it should, is that because of something you glossed over or an ambiguity in the recipe?  Annotate your experiences, look for patterns.  Play Sudoku and develop a process for solving the puzzles quickly that works for you.  Do LSAT logic games, even if you aren’t studying for the LSAT.  If you play an instrument or sing, learn new, rhythmically complicated pieces.

You are building a set of cognitive skills that are hard.  Be patient (think of the baby falling to his butt, time and time again, as he learns to take his first step) and stay focused on these skills everywhere you can.  Track the clarity that starts to emerge when you do those practice questions.