Behind The Rankings

6. Why don't I see on the NEWSWEEK list famous public high schools like Stuyvesant in New York City or Thomas Jefferson in Fairfax County, Va., or the Illinois Mathematics and Science Academy in Aurora, Ill., or Whitney High in Cerritos, Calif.?
We do not include any magnet or charter high school that draws such a high concentration of top students that its average SAT or ACT score significantly exceeds the highest average for any normal-enrollment school in the country. This year that meant such schools had to have an average SAT score below 1950 or an average ACT score below 29 to be included on the list.

The schools you name are terrific places with some of the highest average test scores in the country, but it would be deceptive for us to put them on this list. The Challenge Index is designed to honor schools that have done the best job in persuading average students to take college-level courses and tests. It does not work with schools that have no, or almost no, average students. The idea is to create a list that measures how good schools are in challenging all students and not just how high their students' test scores are. The high-performing schools we have excluded from the list all have great teachers, but research indicates that high SAT and ACT averages are much more an indication of the affluence of the students' parents.

Using average SAT or ACT scores is a change from the previous system we used, which excluded schools that admitted more than half of their student based on grades and test scores. That system penalized some inner-city magnet schools that had high Challenge Index ratings but whose average SAT or ACT scores were below those of some normal-enrollment suburban schools, so we switched to a system that we consider fairer and clearer.

We do, however, acknowledge on our Public Elites list the schools that did not make the list because their average SAT or ACT scores were too high. This year there are 16 of them.

7. Aren't all the schools on the list doing very well with AP or IB? So why rank them and make some feel badly that they are on the lower end of the scale?
You make a very important point. These are all exceptional schools. Every one is in the top 6 percent of American high schools measured this way. They have all shown remarkable AP and IB strength. I am mildly ashamed of my reason for ranking, but I do it anyway. I want people to pay attention to this issue, because I think it is vitally important for the improvement of American high schools. Like most journalists, I learned long ago that we humans are tribal primates with a deep commitment to pecking orders. We cannot resist looking at ranked lists. It doesn't matter what it is—SUVS, ice-cream stores, football teams, fertilizer dispensers. We want to see who is on top and who is not. So I rank to get attention, with the hope that people will argue about the list and in the process think about the issues it raises.

8. Is it not true that school districts who pay the AP or IB exam fees for their students skew the results of your Challenge Index? Should not an asterisk be attached to schools in districts that do that?
If I thought that those districts who pay for the test and require that students take it were somehow cheating, and giving themselves an unfair advantage that made their programs look stronger than they were, I would add that asterisk or discount them in some way. But I think the opposite is true. Districts who spend money to increase the likelihood that their students take AP or IB tests are adding value to the education of their students. Taking the test is good. It gives students a necessary taste of what college demands. It is bad that many students in AP courses avoid taking the tests just because they prefer to spend May of their senior year sunning themselves on the beach or buying their prom garb. (Since AP and IB tests must be graded by human beings, the results arrive long after June report cards, They usually do not count as part of the class grade. Most schools allow students to skip the AP test if they wish. IB is organized differently, and few IB students miss those exams.)

If paying test fees persuades students, indeed forces them, to take the test, that is good, just as it is good if a school spends money to hire more AP teachers or makes it difficult for students to drop out of AP without a good reason. I was happy when the state of Arkansas and most districts in northern Virginia began to pay the test fees and require that the tests be taken. I hope many other districts follow suit.

9. Why don't you count the college exams that high school students take at local colleges?
I would like to. NEWSWEEK has tried to count what are often called dual-enrollment exams, those given to high-school students who have taken local college courses. But it proved to be too difficult. The problem is that we want to make sure that the dual-enrollment final exams are comparable to the AP, IB and Cambridge exams that define the index. We tried to set a standard—WE would only count dual-enrollment final exams that were at least two hours long and had some free-response questions that required thought and analysis, just as the AP, IB and Cambridge exams do. And we wanted to be sure that the exams were written and scored by people who were not employed by the high school so that, like AP, IB and Cambridge exams, they could not be dumbed down to make the school or the teacher look good. Some high schools provided us with the necessary information, but most could not. It was too difficult for them to persuade the colleges managing the exams to help them, or they did not have the staff to gather the data we required. We did not want to be counting extra exams only for those schools that could afford extra staff, so we decided to stay with AP, IB and Cambridge, while we thought about better ways to count dual enrollment.

10. Why do some states have so many schools on your list and others so few?
The more schools I have examined, the more I have come to believe in the power of high-school cultures, which are different in different parts of the country for reasons that often have little to do with the usual keys to high-school performance—The incomes and educations of the parents.

California, New York, Texas and Florida lead the nation in number of schools on the list. That is no surprise. But it is more difficult to explain why much-less-populous Virginia and Maryland come right after those megastates in the number of challenging high schools, and why Iowa, with some of the highest test scores in the country, has only a handful of high schools that met the criteria.

My tentative explanation is that some areas have had the good fortune to get school boards and superintendents who see that they serve their students better by opening up AP, IB and Cambridge to everyone who wants to work hard. Once a few districts in a state do that, others follow. And once a state has success with wide-open programs, its neighboring states begin to wonder why they aren't doing the same.

11. Why limit your list to public high schools? Don't you think those of us who pay tens of thousands of dollars to educate our children at private schools are also interested in how our schools measures up?
My children attended both public and private high schools. I share your interest in rating both varieties. The public schools are very quick to give NEWSWEEK and The Washington Post the data we need. They are, after all, tax-supported institutions. Many private schools, sadly, have resisted this and most other attempts to quantify what they are doing so that parents could compare one private school to another. The National Association of Independent Schools has even warned its members against cooperating with reporters like me who might be trying to help what they call consumer-conscious parents like you. They say that parents should reject such numerical comparisons and instead visit each private school to soak up its ambiance. I am all for visits, but I think those private schools are essentially saying that parents like you and me are too stupid to read a list in a magazine or newspaper and reach our own sensible conclusions about its worth.

A few private schools have shared their data with me, but since the majority are resisting, any list of private schools would be too incomplete to be very useful.

12. Shouldn't I worry if my child's high school has dropped in rank since the last NEWSWEEK list?
No. Keep in mind, as I said before, that every school on the list is within the top 6 percent of all American high schools measured in this way. If you want to gauge a school's progress, look at its rating, not its ranking. Many schools drop in rank each year because there is more competition to be on the list, but at the same time improve their ratio of tests to graduating seniors. That means they are getting better, and the rank is even less significant. Also, almost all schools on the list drop in rank in the updated Web site version of the list a few weeks after the list first appears in NEWSWEEK, because we add schools that get their data to us after the deadline.

I realize it is my fault that people put too much emphasis on the rankings. If I didn't rank, this would not happen. I was startled that people even remembered what their school's rank was in previous years. The important thing is that your school is on the list, not where on the list it is.