Big data is fundamentally changing our lives now – Indians helped make it happen: John A. Deighton

John A. Deighton is professor emeritus at Harvard Business School. Speaking to Srijana Mitra Das, he explains how data is reshaping society:

What is the core of your research?

I study how data has gone from being important to becoming absolutely fundamental and disruptive online. Today, we can simply enter a topic into a search engine — in a second, everything on that topic appears in front of us. It is an incredible agglomeration of data, sorted according to our needs. My work studies its evolution.

A set of innovations at the turn of the 20th century completely changed the way data could be used – previously a “needle in a haystack” principle applied to masses of data, which meant that it took a long time to find what we wanted. But two innovations, both from Google, changed that. The first was Hadoop which allowed storage of unlimited data in multiple places on inexpensive servers – today we call it the Cloud. The second was an innovation by Sergey Brin, a founder of Google, who created trawling or ways to search haystacks of data in an incredibly short time.

Can we quantify the scale of big data online now?

Consider how each individual is profiled over 3,000 times in databases run by data brokers. These collections of information about you include facts such as whether you have a pet, what your political preferences are, your friends’ hobbies, and more. – with a little algebra, they can also estimate your income, property status, net worth, etc.

The contribution of these data practices in the United States is currently around 10% of GDP. We do a study every four years listing the value of the internet and its services – we’ve found that these are increasing in size by 20% a year, an evolutionary dislocation driven by the fact that every few years a whole new range of data-dependent activities emerge. The last big one was the introduction of streaming video – we are now looking at the data enabling the rise of freelancers, from influencers to ride drivers, musicians and more.

How has big data changed advertising?

It pushed him in two contradictory directions. On the one hand, methods such as tracking, first-party and third-party cookies and mobile fingerprinting can be called surveillance. There is a bill in the US and Europe to limit this for privacy. Yet little has actually interfered with the ability of digital advertising to reach very specific targets – first-party information that a company possesses by virtue of its relationship with a customer is considered its property. Meanwhile, big data allows us to be perpetually on stage, speaking out on politics, social concerns, and more. We don’t mind being watched in this context, the personal identification data being the aim of this exercise, not the price. This desire to express our identity is an obstacle to regulations favoring privacy.

Advertising itself has been completely reshaped by data. Nothing reveals its intrusiveness more than how if you search for a product online you quickly receive suggestions about similar products – clearly you are being watched. However, the same practices also occur offline. Recently, it was discovered that a major US retailer had developed a model to predict the parenthood of its customers. Becoming a parent is a huge retail event in the United States because it changes a lot of shopping habits. This chain has developed an algorithm around expected parenthood, based on first-party data, drawing on things people have purchased, like unscented cosmetics, and other sources like credit cards. This revealed how offline retailers also engage in surveillance.

Is data changing us beyond our shopping habits?

It alters our perception of ourselves. The digital infrastructure allows us to befriend or date people, express opinions and form allegiances. Big data allows us to get married now without depending on family or community networks – the algorithm practically takes the place of a parent, associating us with partners ranging from books, food and music to friends. This is a fundamental overhaul of society. Starting just 20 years ago, it’s also incredibly fast – we’re already approaching a huge change when what we think of as ‘big data’ will now appear quite insignificant.

Why do you write that society is either “delusional” or “naive” about data?

The “illusion” manifests itself in lawmakers imagining that if they simply dismantle large tech conglomerates, they would take us back to a more innocent time. But the upcoming increase in the amount of data will undermine existing monopolies – it’s easy to imagine a new Google waiting to be launched, capable of doing things on a much larger scale. This preoccupation with curbing the biggest tech companies is simply closing the stable door after the horse has run away. On the other hand, we are naïve if we do not consider the uses of the data. We need a much greater societal concern about what big data means to us.

Have Indian IT professionals contributed to the global data boom?

Absolutely. The growth of data also deeply reflects the talent of people in the Indian subcontinent. It is very difficult to imagine the big data revolution without the contribution of Indians.

Comments are closed.