Behind The Rankings

To the editor, RE "Behind The Rankings" (June 8, 2009):

If Newsweek published a ranking of car seats based solely on the number of people who bought different brands (but without regard to the safety of the seats or the types of children they???re designed for), parents would be up in arms. Certainly, few would bother buying the magazine. Yet Newsweek publishes the Challenge Index, based on similarly faulty measurements, every other year.

The National Association of Independent Schools (NAIS) believes that ranking schools is a disservice to families. Rankings like the Challenge Index imply that school quality can be measured by one narrow criterion. We know otherwise. The best school is the school that most closely meets the needs of an individual child.

The process of finding the right school for a child requires parents to evaluate what each child needs from his or her school. The family must then investigate the schools available to them to determine how well each school meets those needs. NAIS has developed a series of tips and questions to help families navigate the process. The questions can be found at www.nais.org/go/parents.

Patrick F. Bassett
President
National Association of Independent Schools
Washington, DC

Newsweek's rankings have a HUGE serious flaw. Let's ignore for a moment that they focus on a single metric (AP exams taken), and let's ignore that even for that single metric that they ignore whether the AP courses themselves are even taken, or whether the tests are passed, there's another flaw with the measure that (at least for some subset of schools) renders it even more meaningless.

Here's how Newsweek describes what they do: "We take the total number of Advanced Placement, International Baccalaureate or Cambridge tests given at a school in May, and divide by the number of seniors graduating in May or June."

If a school has a stable population, such a metric (again ignoring its value) at least is internally consistent. But if a school does not, the metric begins to completely break down and make no sense. This calculation gives a tremendous advantage to schools whose populations dwindle over the four years (as often happens at many charters). Since the metric used is AP exams divided by graduating seniors, having younger students who take an exam (and are counted in the numerator), but do not graduate (so are not counted in the denominator) can dramatically over-inflate the metric.

For example, in Massachusetts, three charters made the top 100.
* Sturgis Charter Public (64 seniors v. 108 freshmen)
* MATCH Charter (34 seniors v. 94 freshmen)
* Mystic Valley Regional Charter (41 seniors v. 97 freshmen)

It was difficult to find the same data for schools outside of Massachusetts, but I did find one other: Peak to Peak in Colorado had 97 seniors and 159 freshmen. It does not appear that such population dwindling (which should be a HUGE negative in evaluating a school, not a positive) is accounted for in the metric. All four of these schools would likely drop from the top 100 with an appropriate adjustment to their data. And I'm sure the same is true for many others.

For Mr. Mathews and Newsweek,

I find it ironic that Mr. Mathews' own alma mater, Scarsdale High School in Scarsdale, New York, is dropping AP courses. The school will thus fall off your list, but not off the list of any discerning parents in the New York City area looking for a really top public school.

On the other hand, the inclusion of Holmes High School in Covington Kentucky on your list offers perhaps one of the very best arguments of all against your highly contrived rating system.

Holmes is a perfect example of a school that caters to an elite few while very poorly serving all the rest.

Once you add the rest of those kids to the mix, however, here is what happens.

Kentucky now tests all (every one of them) 11th grade students with the ACT college entrance test. Holmes only outscored 25 of the 232 Kentucky high schools that had scores reported in 2008. That???s all.

Only one high school scored lower than Holmes on Kentucky???s state assessment program in 2008. In fact, the school is in the very worst assessment category in Kentucky. Things are so bad that the Kentucky Board of Education has a special monitoring program for the school and district. The latest hearings were just held during the board???s June 11, 2009 meeting.

The new Education Week ???Graduations Counts??? on line search tool shows Holmes had a graduation rate in 2006 of only 48.1 percent, far below the US average of 69.2 percent. It???s not hard to see how your contrived ranking might make Holmes look good once you consider that.

To be sure, the AP tests are important, but using them as you do does a great disservice to many students and educators. The fact that one of Kentucky???s poorest performing high schools wound up on your list is ample evidence of the problems in your approach, and I urge you to discontinue this highly inaccurate rating program.

Richard Innes

Recommendations to Jay Mathews- 1) If schools have an entrance exam or qualifying standards to attend that school, they are not on the same playing field as true public facilities. Put these schools in a secondary list & note as such. 2) Check that the schools TEACH the subject; many schools offer the test but do NOT offer the class... that reflects more on the determination of the student, not the quality of the school. 3) I do not agree that the # of AP tests given per year/# of seniors at that school is a good measure of what makes a top school or is even a good measure of how many students are in AP courses/student population. Most high level students take multiple AP tests/AP courses any given year(local valedictorian took 14 in 3 years); therefore, you are measuring the success of a few not the masses. Also, you mentioned a school where only 1 out of 65 students taking the exam passed the AP- that school failed those students and should not be considered a top high school. 4) Measuring if a high school is a good one IS a complicated affair. One data point is never good for a scientific review. Investigate how local regions rate their schools to get ideas (try asking Business First in western NY for example). Choose the, say, top 5 common quantifying measures and rate the schools according to those.

How many of the top-ranked schools have a significant population fo non-English speaking students/ Of disabled students? #8 on the list is nearly 100% white, has almost no disabled or non-English students. Disabled students are sent elsewhere. Students who are performing poorly are also sent elsewhere - usually to an alternative school in another town. Newsweek needs to take a harder look at how many students TAKE the AP exams and how they score. Just taking an AP class serves NO purpose if the students do not take the exams to get the advanced placement in college. Corbett has few students who do so. I seriously doubt they are #8 in the US or even #8 in Oregon!

Research shows that there is a correlation between SAT/ACT test scores and parental income. Additionally, there is no correlation between these test scores and college success for many. Standardized test scores are a biased method for gauging success.

Why do you use # of graduating seniors to show relative school size and not number of students? A school on your list is one of two high schools in my district, and while it has a strong IB program, it also has a very high number of dropouts. Going by graduating seniors makes the school seem smaller than it is, and skews it up the rankings. The other high school in our district has a well-regarded AP program, but did not make the list. But it has a much higher graduation rate.

Stories from Newsweek have as much credibility as Star magazine articles with ???Elvis sightings???. This story and list is not different.
Here is a quote from the newsweek bar napkin: ???The high-performing schools we have excluded from the list all have great teachers, but research indicates that high SAT and ACT averages are much more an indication of the affluence of the students' parents.???
REALLY? Did the parents take the test? This is class warfare. Socialist garbage.
This list isn???t worth the paper it was printed on. This is an insult to those on the list. People want to be recognized for real achievment, not given recognition based on mediocroty.

I just hope people do not get fooled by this list and buy their homes based on this obviously flawed ranking. For example, Montgomery county, MD has many high schools in the top 100 or top 1000. If a parent is stupid enough to send their kids to lousy school such as the Richard Montgomery high school (#38) instead of the Walt whitman High school in Bethesda, MD or Churchill High school of Potomac, MD, that's too bad. If the parents believe that Gaithersburg high school (#876?) is America's top high school, they should be glad that their kids did not become a criminal or get killed or bitten at school. When I tried to buy investment properties, my Realtor told me not to buy any in the Gaithersburg high school area. One piece of advice for anyone who are looking for a place to buy or rent and a good school for their children, DO NOT use this list. At a minimum, use sites such as www.schoolmatters.com.

I just hope people do not get fooled by this list and buy their homes based on this obviously flawed ranking. For example, Montgomery county, MD has many high schools in the top 100 or top 1000. If a parent is stupid enough to send their kids to lousy school such as the Richard Montgomery high school (#38) instead of the Walt whitman High school in Bethesda, MD or Churchill High school of Potomac, MD, that's too bad. If the parents believe that Gaithersburg high school (#876?) is America's top high school, they should be glad that their kids did not become a criminal or get killed or bitten at school. When I tried to buy investment properties, my Realtor told me not to buy any in the Gaithersburg high school area. One piece of advice for anyone who are looking for a place to buy or rent and a good school for their children, DO NOT use this list. At a minimum, use sites such as www.schoolmatters.com.

Mr. Matthews, if you happen to read this comment, I suggest that you tweak the way that you are ranking schools. I happen to teach at one of the schools in your list. You promote all students taking AP tests. All students aren't advanced. AP tests may help with college but they are not necessary. I am a college graduate who never took an AP class because they did not exist. I made it without problems. I suspect that you are young, and have grown up believing that AP classes and tests are a valid determination of the quality of a curriculum and of the students' performance. They are not. If the students were tracked after they left a high school, a determination of how well that school prepared those students for life could be made. Then you would have a real picture of how good a school really is. College is not the only way students can have successful lives. Measure the overall success of the students leaving a school, and then rank them. Tough to do, I confess.

i WENT TO MULTIPLE OF THESE SCHOOLS AND THE ONE THAT WAS THE BEST IN MY MIND (QUALITY OF TEACHERS, QUALITY OF MATERIALS USED IN CLASS, QUALITY OF LEARNING) IS NOT ON THIS LIST. I ONLY GRADUATED TWO YEARS AGO...ALSO, SOME OF THESE SCHOOL ACTUALLY PAY FOR STUDENTS TO TAKE THE AP TESTS. OBVIOUSLY THOSE SCHOOLS HAVE MORE KIDS WHO TAKE THEM SINCE THE 85 DOLLAR FEE ISN'T PASSED ON TO THE FAMILIES. THIS LIST HAS SOME SERIOUS FLAWS AND SHOULDN'T BE RANKED ONLY BASED ON TEST SCORES.

I have a suggestion for the challenge index. How about take the total number of Advanced Placement, International Baccalaureate or Cambridge tests "with passing grade" given at a school in May, and divide by the number of seniors graduating in May or June. In this way, schools pay the student test fees won't be able to arbitarily boost their indices if the students can't get pass grades.

I have a suggestion for the challenge index. How about take the total number of Advanced Placement, International Baccalaureate or Cambridge tests "with passing grade" given at a school in May, and divide by the number of seniors graduating in May or June. In this way, schools pay the student test fees won't be able to arbitarily boost their indices if the students can't get pass grades.

I liked the way that Jay Mathews wrote this explanation in the first person, avoiding some royal, editorial we; and that he made tough choices and defended them. To be around writers, or high school teachers, or presidents who do this helps us learn to think more rigorously by osmosis.

It would be interesting to see this list after being revised to include the median of parental income for each school.

I would be interested to see this list revised to include the median parental income for each school.

The proper name for this ranking should be "AP participation and social equalization rating of average public high schools in America". There is a big socialist ideology behind this "ranking" joke.