Mahalanobis in the era of Big Data and AI

Contact Counsellor

Last Updated30/06/2023

With the global shift in both the nature of data and statistics, the enduring legacy of P.C. Mahalanobis may have given India and the world many answers

Professor P.C. Mahalanobis introduced statistics to India
Mahalanobis’s lifelong courtship with statistics, his unwavering and fearless leadership to advance a statistics and survey culture in India, the founding of the Indian Statistical Institute and his nurturing of a generation of outstanding academicians have all left behind an enduring legacy.
Today, in the midst of the shifting socio-economic dynamics in post-pandemic India, he is greatly missed.

Over the past two decades, there has been a global shift in both the nature of data and statistics.
With the advent of the Internet and virtually everything confined to the Internet of Things, there has been a flood of data, most of it being junk.
We now have much more data than what available technology can leverage.
This is widely perceived as the era of Big Data.
Another significant yet related issue is how artificial intelligence (AI) is transforming our lives and lifestyles.

Mahalanobis was “a physicist by training, a statistician by instinct and an economist by conviction”.
He had an uncanny knack for embracing technology for human welfare.
He even built some simple machines to facilitate his surveys and measurements. Thus, one may safely perceive that Mahalanobis would have embraced the power of AI in enhancing human productivity, such as Big Data analysis and perhaps in a way that is far more effective than how AI is currently applied to that goal.
One recent Big Data foible, for instance, involved numerous contradictory projections during the COVID-19 era.
One could argue that if Mahalanobis was alive today, the country’s COVID-19 response could have been much stronger.
If he was in the lead, our data might be beyond question, and the analyses might be far more accurate.
And India’s “Plan Man” could be the best person for planning to build optimal health-related infrastructures for combating future disasters.

Around seven decades ago, from the perspective of the newly independent nation, planning - with the aid of extensive technocratic exercises with democratic participation - moved from the realm of politics, primarily due to Mahalanobis.Now, we are at crossroads.
India’s upcoming census will be a digital exercise. The dynamics of other surveys are also bound to change in the new normal setup.
That is how statistics is evolving.
We would miss the leadership of an expert such as Mahalanobis from this changed statistical perspective.

As AI is threatening to replace millions of jobs without creating alternatives and is also aiding in spreading disinformation, there is a substantial global attempt to clip its wings.