A critical look at international city rankings

“Well, big deal,” the Montreal Gazette sneered in Montreal and its place in the world, its editorial response to a recent international survey on urban quality-of-life. Montreal was behind Toronto, Vancouver and Calgary. As a native Montrealer, I have to concur with the Gazette’s summary:

…rankings tend to favour an ideal, cleanly scrubbed and tidily tended city – which is essentially a suburb.

The editorial consoled readers, throwing in that New York City came 56th on the list.

So how accurate is the measuring stick for the wide range of surveys which rank cities?

This is the question that Toronto’s Intergovernmental Committee on Economic and Labour Force Development (ICE Committee) asked when it commissioned a review of the various urban ranking surveys last year.

As expected, the final report found methodological weaknesses in the comparisons and poor interpretations of the findings by the media and public creates more confusion than clarity when it came to grading the world’s cities. The report author reviewed forty-four rankings and identified seven key lessons:

  • Audience and purpose matter
  • Beware of over-simplification
  • Look at the scores, not the rankings
  • Be wary of data that has been overly manipulated and processed.
  • Longitudinal data are more useful than one-off “snapshot” studied, but watch out for iterative studied that change the rules as they go.
  • Stale source data may leave a false impression.
  • Make sure that apples are being compared to apples.
Probably the fairest explanation for why these studies continue to pop up in the media is attributed to Joel Garreau:
 “These lists are journalistic catnip. Fun to read and look at the pictures but I find the liveable cities lists intellectually on a par with People magazine’s ‘sexiest people’ lists.”

(Still, if you lean towards parochialism, patriotism, or partisan, if you believe Toronto is the centre of the world, you will be glad to know that Toronto generally does well on these international scorecards.)

Related articles
Advertisements

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s

%d bloggers like this: