Some Things Can't Be Measured

With public education required to “prove” student progress through test scores and number data, it has become more and more common to track the reading progress of MG students through Lexile numbers and other measures of text difficulty, such as Fountas and Pinnell and Accelerated Reader. I have seen these numbers cropping up on Individualized Education Plans (IEPs) for both Gifted and Special Ed students as an official measure of student progress.

The problem is – how do you measure the difficulty of a book? Sometimes, it seems obvious. Take Gary Paulson’s book Masters of Disaster (102 pgs), which is about three boys trying to get into a record book. Compare it to Jane Austen’s Pride and Prejudice. Which one do you think is harder to read?

Pride and Prejudice has a Lexile rating between 900 and 1070, depending on the edition.

Masters of Disaster has a Lexile rating of 1100.

Yes, you read that right.

According to the Lexile website, ratings are measured by a program that looks at word frequency and sentence length.  Hmmm …

First sentence of Pride and Prejudice: It is a truth universally acknowledged that a single man in possession of a good fortune must be in want of a wife.

First sentence of Masters of Disaster: I’ve called you here today, men, because I have an important announcement.

Nope, I’m not seeing it. But there are always flukes. Let’s look at some more.

The Princess Bride – 870. 

Harry Potter and the Deathly Hallows – 980. 

The Hound of the Baskervilles – 1090. 

Vordak the Incomprehensible – 1140.  (Vordak is the “Diary of a Wimpy Kid” of Supervillains.)

I’ve spent a lot of time on the Lexile website, because my daughters are frequently required to select independent reading titles based on a Lexile level assigned to them by a MAP test (which is a whole ‘nother story – don’t get me started!). In sixth grade, one of my daughters was required to select books with a Lexile over 1100 for a reading project every single month. I was ready to pull my hair out, trying to find books she wanted to read – that were actually a challenge for her – and also topped 1100 on the Lexile chart. I didn’t want to have to give her Moby Dick!

The bottom line is: None of these measures – Lexile, F&P, AR – take into account theme, genre, and story complexity. They are all based on formulas, word frequency, and sentence structure (although the results still mystify me). 

Based on my experiences teaching MG readers for a quarter of a century, I can say that – regardless of text difficulty – a book is harder to read if the student is unfamiliar with the setting because it is historical fiction or set in a foreign land. Conversely, fantasy and science fiction books often get a more difficult rating than they deserve because of the made-up names for people, places, and things.  The computer programs don’t know how to handle words like Vordak.

A biography of John F. Kennedy is likely to be harder than a biography of Harry Houdini, even if they are written on the same “level” – because one includes a great deal of politics while the other includes a great deal of escape tricks. Books that include sarcasm and irony harder are harder than books that involve mostly bathroom humor. And finally, a boy who desperately wants to read The Hunger Games because all his friends are reading it will probably have less trouble than a boy handed Little House on the Prairie as a class assignment by his teacher.

Some things ought not be measured by a computer program or mandated in the name of accountability – especially when the measures are superficial and look good on paper while being utterly meaningless. 

The quality and difficulty of a book is one of those things.