According to the National Center for Education Statistics, as of the 2017-2018 school year there were over 4,000 degree-granting postsecondary institutions in the United States.
To sift and sort through all of these school options, prospective students and their families often turn towards college ranking sites to gauge how “good” one school is compared to another. But how does one define school quality in the first place, especially when everyone is looking for different characteristics in a school? Can a numerical rank actually capture all the variability in student experiences and learning outcomes between schools, let alone within a single school community?
The short answer is no. Since everyone values different aspects of a college, it’s important to arm yourself with the knowledge of what factors contribute to ranking formulas so you can steer clear of sites that provide misleading information and use college rankings as a jumping off point and nothing more.
For this post, we’re going to focus on the ranking methodology of one of the most widely cited school ranking sites — U.S. News & World Report (USNWR) — and how you can save yourself from absorbing arbitrary data from their site.
First, here’s how USNWR describes their ranking process:
Each school is sorted into a category based on its mission (research university, liberal arts college, regional university, or regional college) and, in the case of regional institutions, its location (North, South, Midwest, and West).
Sixteen data points related to academic excellence are collected about each school. Each indicator is weighted differently, with measures deemed more important assigned a greater weight. The greater the weight, the more it impacts each school’s final calculated rank.
The indicators include both input measures, which reflect the quality of students, faculty and other resources used in education, and outcome measures, which capture the results of the education an individual receives.
Each school’s ranking is calculated based on the formula crafted by USNWR and the list is published.
Let’s zoom in on ranking step #2: the sixteen data points that are collected about each school. This data is the core of the ranking formula and — as anybody who’s taken a statistics class knows — the quality of any data-driven formula relies on the quality of the data used. Now, time for some real talk: many of the data points USNWR relies on are either subjective, easily manipulated by colleges, completely arbitrary, biased towards the schools with the most wealth, don’t take into account institutional impact, or… all the above. It’s also challenging to trust any year-to-year changes in a school’s ranking because USNWR often changes their methodology, so the data points they use and weights they assign are often the real drivers of up-and-down shifts.
Here are those sixteen data points the USNWR uses, organized into eight categories, and their assigned weight (expressed as a percentage) in the ranking formula:
To USNWR’s credit, they are fairly transparent about how each data point is collected (it’s worth a read!), although — not surprisingly — their explanations lack self-critique.
As you’re digesting this information, I encourage you to reflect on these questions:
How do I define academic “quality”?
Which USNWR measures matter to me and my college search? Which measures don’t matter to me? What measures are missing altogether?
How would I create my own ranking formula based on what I’m looking for in my future college or university? What measures would I include in my formula, how would I collect the data, and how would each factor be weighted?
When I think about all of the students I’ve worked with over the years, I can confidently say that each of their personal ranking formulas would look at least slightly — if not entirely — different from that of USNWR and any of their college-going peers. As you reflect on your own personal ranking formula and use it to thoughtfully guide your college search, you’ll find your personal top 10 and ultimately land at the top-ranking school for you.