Data and statistics: Objectivity versus selling the soap
I break out in a wide grin at every new twist in the controversy over the World Bank’s Ease of Doing Business survey. This report was started quite a while ago to assess how hard it was to start and run a business in countries around the world. I’m not sure how many businesses made decisions using it but as was recounted in a recent Economist article, countries started to take it seriously and worked to raise their ranking, which was a good thing.
Now a new report has found that World Bank officials tweaked the methodology after being pressured by the Chinese government. It has become a big controversy with many countries including the United States calling for the head of the IMF (who was in charge of the EODB survey before) to be fired. It’s another example of China’s increasingly brazen behaviour, yadda yadda yadda.
Why I chuckle about this is my recollection of KPMG’s Competitive Alternatives report which was started way back in the 1990s, funded by the Canadian government and others. Long timers will remember that report. It compared the cost of doing business across multiple industries in something like 150 countries and conveniently, most Canadian cities (especially New Brunswick!) looked real good.
The first year that report came out, Quebec cities didn’t fair particularly well.
The second year the report came out, a number of Quebec cities were at or near the top of the pile.
What had happened? KMPG tweaked the methodology and, presto, Quebec was highly competitive for business investment.
I suspect no high level official at Industry Canada was fired. There was no global controversy. A number of economic developers, like myself, grumbled about it but to no avail.
The use of data and statistics to support economic development is my core business. It has animated my career since I first walked into the 5th Floor of the Centennial Building in Fredericton in 1993 to take a 3-month job building business cost models meant to position New Brunswick as a more cost effective location to do business. If I had turned right instead of left that day who knows what might have been?
The bottom line is there is ‘objectivity’ and there is objectivity. When I am working with a community or industry to position it better for investment and growth, I use factual statistical data.
I’ve never made up a statistic in my life (except to plug gaps in data and then I am clear in the assumptions).
But I am not particularly objective. In the case of economic development promotion, we use statistics that help our story. In the case of economic development policy, we are using statistics to help strengthen our value proposition. The two audiences are completely different.
So the World Bank EODB survey should not have changed its methodology or at the very least should have been very clear with the readers they did and the reason for the changes.
The KPMG Competitive Alternatives? That one is a little more tricky. On the one hand, it is paid for by the Canadian government and on the other government wants to use the KPMG brand to establish credibility. In the end, this one is caveat emptor.