Month: February 2019

Active aggression and passive aggression

For the record I’m most often actively aggressive. I believe passive aggression is a waste of energy since not only do you end up fighting but you also end up trying to second guess the other party, which leads to suboptimal outcomes. This post is a justification of that.

Let’s say you and I are trying to decide the price of something I want to sell you. There are two ways we can go about it. One way is for us to have a negotiation. I can name my asking price. You call your bid. And if the two meet, well and good. Most often they won’t meet. So one of us will have to budge. We start budging slowly, in steps, until a time when the bid and ask are close together. And then we have a deal.

In most situations (except exceptional cases where there are very few buyers and sellers – read the first chapter of my book. This is within the Kindle sample), this will lead to an efficient outcome. Even if the final price were a little too close to the bid or to the ask, both parties know that under the circumstances they couldn’t get better. And the transaction takes place and the parties move on.

The other situation is where one party publicly states that they are unwilling to negotiate and will do the deal if and only if the counterparty comes up with a good enough offer. If the offer is not good enough, there is no deal. This is similar to the ultimatum game popular in behavioural economics. In this case you are also required to guess (and you have exactly one guess) what the counterparty’s hurdle rate is.

When there is a liquid market, there is no issue with this kind of a game – you simply have your own hurdle rate and you bid that. And irrespective of whether it gets accepted or not, you get the optimal outcome – since the market is liquid, it is likely that your quote will get accepted somewhere.

In a highly illiquid market, with only one buyer and one seller, the ultimatum setup can lead to highly suboptimal outcomes. I mean if you’re desperate to do the sale, you might bring your price “all the way to zero” to ensure you do the deal, but the thing is that irrespective of whether you get a deal or not, you are bound to feel disappointed.

If your ask got accepted, you start wondering if you could’ve charged more. If you didn’t get your deal, you start wondering if reducing a price “just a little” would have gotten the deal done. It is endless headache, something that’s not there when there is an active negotiation process.

Now to build the analogy – instead of a sale, think of the situation when you have a disagreement with someone and need to resolve it. You can either confront them about it and solve it “using negotiation” or you can be passive aggressive, letting them know you’re “not happy”. Notice that in this case the disagreement is with one specific party, the market is as illiquid as it can get – no negotiations with any third party will have any impact (ignore snitching here).

When you express your disagreement and you talk/fight it out, you know that irrespective of the outcome (whether it was resolved or not), you have done what you could. Either it has been resolved, which has happened with you telling what exactly your position is, or you have given it all to explain yourself and things remain bad (in this case, whatever happened there would have been “no deal” or an “unhappy deal”).

And that is why active aggression is always better than passive aggression. By expressing your disagreement, even if that means you’re being aggressive, you are stating the exact extent of the problem and the solution will be to your satisfaction. When you’re passive aggressive, nobody is the winner.

PS: I realise that by writing this post I’m violating this own advice, since this post itself can be seen as a form of passive aggression! Mea culpa.

Glass Houses

When I was in middle or high school, I learnt about the greenhouse effect, and learnt from my textbook that “glass houses” are an example of greenhouses. These glass houses are used to control temperature inside, I read, and this helps to grow a specific kind of plants. While all this sounded good in theory, the problem was that it didn’t really fit the example I had seen in real life.

I’ve known glass houses from the time I was very young, thanks to the one in Lalbagh in Bangalore, which was erected in 1889 to commemorate the visit of Prince Albert Victor (Queen Victoria’s husband) to Bangalore. Lalbagh being located close to home,  I would go there every other weekend, and the routine consisted of walking through the glass house and sitting on one of the bull statues in front of it.

From memory (I’ve hardly gone to Lalbagh in adulthood), the glass house was always empty, except for exhibition times in August and January every year, when it would be full of flowering plants. And the glass house being largely open, the temperature and humidity insight wasn’t noticeably different from that outside. And that meant that I couldn’t particularly appreciate my science lesson that glass houses are greenhouses.

All that changed yesterday when we visited the Palm House in Kew Gardens in London. It was an unusually warm day for February in London, but even then the difference between the inside and outside of the glass house was rather noticeable. The Palm House houses tropical plants from Asia, Africa and the Americas, and consequently a tropical weather is maintained. And this is achieved by dint of it being a glass house (i.e. a grenhouse), and also frequent waterings to increase humidity in the house. And this meant that while it’s still winter in London, tropical plants were blooming and buzzing in the Palm House!

I don’t know what it will cost Lalbagh to maintain a permanent collection of plants in the glass house. Also Bangalore can get warm in summer and maintaining a different temperature inside may not be desirable. Nevertheless, thinking back, it would have immensely helped me in high school had the glass house in Bangalore actually functioned as a Glass House!

Surveying Income

For a long time now, I’ve been sceptical of the practice of finding out the average income in a country or state or city or locality by doing a random survey. The argument I’ve made is “whether you keep Mukesh Ambani in the sample or not makes a huge difference in your estimate”. So far, though, I hadn’t been able to make a proper mathematical argument.

In the course of writing a piece for Bloomberg Quint (my first for that publication), I figured out a precise mathematical argument. Basically, incomes are distributed according to a power law distribution, and the exponent of the power law means that variance is not defined. And hence the Central Limit Theorem isn’t applicable.

OK let me explain that in English. The reason sample surveys work is due to a result known as the Central Limit Theorem. This states that for a distribution with finite mean and variance, the average of a random sample of data points is not very far from the average of the population, and the difference follows a normal distribution with zero mean and variance that is inversely proportional to the number of points surveyed.

So if you want to find out the average height of the population of adults in an area, you can simply take a random sample, find out their heights and you can estimate the distribution of the average height of people in that area. It is similar with voting intention – as long as the sample of people you survey is random (and without bias), the average of their voting intention can tell you with high confidence the voting intention of the population.

This, however, doesn’t work for income. Based on data from the Indian Income Tax department, I could confirm (what theory states) that income in India follows a power law distribution. As I wrote in my piece:

The basic feature of a power law distribution is that it is self-similar – where a part of the distribution looks like the entire distribution.

Based on the income tax returns data, the number of taxpayers earning more than Rs 50 lakh is 40 times the number of taxpayers earning over Rs 5 crore.
The ratio of the number of people earning more than Rs 1 crore to the number of people earning over Rs 10 crore is 38.
About 36 times as many people earn more than Rs 5 crore as do people earning more than Rs 50 crore.

In other words, if you increase the income limit by a factor of 10, the number of people who earn over that limit falls by a factor between 35 and 40. This translates to a power law exponent between 1.55 and 1.6 (log 35 to base 10 and log 40 to base 10 respectively).

Now power laws have a quirk – their mean and variance are not always defined. If the exponent of the power law is less than 1, the mean is not defined. If the exponent is less than 2, then the distribution doesn’t have a defined variance. So in this case, with an exponent around 1.6, the distribution of income in India has a well-defined mean but no well-defined variance.

To recall, the central limit theorem states that the population mean follows a normal distribution with the mean centred at the sample mean, and a variance of \frac{\sigma^2}{n} where \sigma is the standard deviation of the underlying distribution. And when the underlying distribution itself is a power law distribution with an exponent less than 2 (as the case is in India), \sigma itself is not defined.

Which means the distribution of population mean around sample mean has infinite variance. Which means the sample mean tells you absolutely nothing!

And hence, surveying is not a good way to find the average income of a population.

Mass marketing and objective journalism

This is a fascinating essay by Antonio García Martinez on the history and future of journalism (possibly paywalled). The money paragraph is this:

The bigger switch happened as a national market for consumer goods opened after the Civil War, when purveyors like department stores wanted to reach large urban audiences. Newspapers responded by increasing the number of ads relative to content, and switched to models that went light on the political partisanship in the interest of expanding circulation. This move was driven not exclusively by lofty ideals but also by mercenary greed. And it worked. Newspapers used to make lots of money. Mountains of money.

Basically, the move to objective journalism came in the late 1800s when advertisers such as Macy’s wanted to take out full page ads, and wanted to do so in newspapers that served the largest sections of the market. And when a newspaper had to reach a large section of the market, it inevitably had to tone down the partisanship, and become more objective.

Over the last decade, we have been witnessing (across the world) the decline of objective media. All media is “#paidmedia” based on which side of the political spectrum you stand on. There aren’t that many truly objective papers around, and social media is bombarded left and right by extremely politicised reporting that goes as “news”.

It is perhaps no coincidence that this period has coincided with a time when print circulation has been dropping steadily (in the developed world at least), and where online advertising can be highly targeted.

In theory, mass marketing is inefficient. When you pay to put up a hoarding somewhere, you’re possibly paying a small amount for each person who sees the hoarding, but not all of them might find it interesting. Consequently, this reflects in a depressed per-person price of the hoarding implying the owner of that real estate can’t make as much as she could if the hoarding were to be more “targeted”.

When you can target your advertisements more precisely, everybody wins. You as the marketer know that your advertisement is only being shown to your intended audience. The owner of the real estate where you put your advertisement can thus charge you more for your advertisement. Even the customer will be less pained by the advertisement if it is highly relevant to her.

Another way of seeing it is – an advertisement shown to a customer who doesn’t want to see it is wasted. The monetary cost of this waste are borne by the owner of the real estate and the advertiser, and the non-monetary cost is borne by the customer (being forced to see something she didn’t want to see). And so one of the biggest technological problems of today is on how we can target advertisements better so that we can minimise such costs – and in the last decade and half, we’ve made significant progress on that front.

The problem with greater efficiency, however, is that it comes with the side-effect of biased media. When Nike knows that it can precisely target an advertisement at American leftwingers, it makes an ad with Colin Kaepernick and shows them to American leftwingers to sell them more shoes.

This doesn’t however, mean that Nike only sells to left-wingers. The same company can make another advertisement targeted precisely at right-wingers and use it to sell shoes to them!

So now that you can make left-wing and right-wing ads, and you have the ability to target them, you want to cut the waste and place the ads so that you can target as best as possible. In other words, you want to place your left-wing ads in places that only left-wingers want to see, and right-wing ads only in places that right-wingers will see. And so you prefer to advertise in CNN and Fox rather than in a hypothetical “broad market” media outlet.

And the reason you created the politically charged ads in the first place was because there were some outlets (Facebook, for example) where you could precisely target people based on their political orientation. And so you see the vicious cycle – that you can target in some places means you want other places where you can target and that creates demand for more polarised media.

It was the opposite cycle that took effect in the late 1800s and early 1900s. There was no way brands could target (also, when you make physical advertisements, with 1900s technology, each advertisement is costly and you don’t want to make one per segment) too effectively, and so they went mass market in their communication.

And this meant advertising in the outlets that could get them the maximum number of eyeballs. When you can’t discriminate between a “right” and a “wrong” eyeball, you pay based on the number of eyeballs. And the way for media organisations to grow then was to cater to everyone. Which meant less less bias and more objectivity and more “features”.

Sadly that cycle is now behind us.

Vlogging!

The first seed was sown in my head by Harish “the Psycho” J, who told me a few months back that nobody reads blogs any more, and I should start making “analytics videos” to increase my reach and hopefully hit a new kind of audience with my work.

While the idea was great, I wasn’t sure for a long time what videos I could make. After all, I’m not the most technical guy around, and I had no patience for making videos on “how to use regression” and stuff like that. I needed a topic that would be both potentially catchy and something where I could add value. So the idea remained an idea.

For the last four or five years, my most common lunchtime activity has been to watch chess videos. I subscribe to the Youtube channels of Daniel King and Agadmator, and most days when I eat lunch alone at home are spent watching their analyses of games. Usually this routine gets disrupted on Fridays when the wife works from home (she positively hates these videos), but one Friday a couple of months back I decided to ignore her anyway and watch the videos (she was in her room working).

She had come out to serve herself to another serving of whatever she had made that day and saw me watching the videos. And suddenly asked me why I couldn’t make such videos as well. She has seen me work over the last seven years to build what I think is a fairly cool cricket visualisation, and said that I should use it to make little videos analysing cricket matches.

And since then my constant “background process” has been to prepare for these videos. Earlier, Stephen Rushe of Cricsheet used to unfailingly upload ball by ball data of all cricket matches as soon as they were done. However, two years back he went into “maintenance mode” and has stopped updating the data. And so I needed a method to get data as well.

Here, I must acknowledge the contributions of Joe Harris of White Ball Analytics, who not only showed me the APIs to get ball by ball data of cricket matches, but also gave very helpful inputs on how to make the visualisation more intuitive, and palatable to the normal cricket fan who hasn’t seen such a thing before. Joe has his own win probability model based on ball by ball data, which I think is possibly superior to mine in a lot of scenarios (my model does badly in high-scoring run chases), though I’ve continued to use my own model.

So finally the data is ready, and I have a much improved visualisation to what I had during the IPL last year, and I’ve created what I think is a nice app using the Shiny package that you can check out for yourself here. This covers all T20 international games, and you can use the app to see the “story of each game”.

And this is where the vlogging comes in – in order to explain how the model works and how to use it, I’ve created a short video. You can watch it here:

While I still have a long way to go in terms of my delivery, you can see that the video has come out rather well. There are no sync issues, and you see my face also in one corner. This was possible due to my school friend Sunil Kowlgi‘s Outklip app. It’s a pretty easy to use Chrome app, and the videos are immediately available on the platform. There is quick YouTube integration as well, for you to upload them.

And this is not a one time effort – going forward I’ll be making videos of limited overs games analysing them using my app, and posting them on my Youtube channel (or maybe I’ll make a new channel for these videos. I’ll keep you updated). I hope to become a regular Vlogger!

So in the meantime, watch the above video. And give my app a spin. Soon I’ll be releasing versions covering One Day Internationals and franchise T20s as well.

 

Volatility and price differentiation

In a rather surreal interview to the rather fantastically named Aurangzeb Naqshbandi and Hindustan Times editor Sukumar Ranganathan, Congress president Rahul Gandhi has made a stunning statement in the context of agricultural markets:

Markets are far more volatile in terms of rapid price differentiation, than they were before.

I find this sentence rather surreal, in that I don’t really know what Gandhi is talking about. As a markets guy and a quant, there is only one way in which I interpret this statement.

It is about how market volatility is calculated. While it might be standard to use standard deviation as a measure of market volatility, quants prefer to use a method called “quadratic variation” (when the market price movement follows a random walk, quadratic variation equals the variance).

To calculate quadratic variation, you take market returns at a succession of very small intervals, square these returns and then sum them up. And thinking about it mathematically, calculating returns at short time intervals is similar to taking the derivative of the price, and you can call it “price differentiation”.

So when Gandhi says “markets are far more volatile in terms of rapid price differentiation”, he is basically quoting the formula for quadratic variation – when the derivative of the price time series goes up, the market volatility increases by definition.

This is what you have, ladies and gentlemen – the president of the principal opposition party in India has quoted the formula that quants use for market volatility in an interview with a popular newspaper! Yet, some people continue to call him “pappu”.

Tigers and Bullwhips

Over three years ago, well before our daughter was born, my wife’s cousin had told us that she likes to watch her daughter’s TV shows because they contained “morals”, which were often useful to her at work. While we never took to the “moral” TV show she mentioned (Daniel Tiger – it is bloody boring), I have begun to notice that there are important management lessons in other popular children’s stories.

So I hereby begin this blog series on what I call the “Kiddie MBA” – basically business lessons from kids’s stories. And we will start with that all-time classic, The Tiger Who Came To Tea, by Judith Kerr. 

The basic premise of this story that remains a classic fifty years after being published is what operations managers call the “bullwhip effect“. Sometimes a business, possibly in trading, can be subject to a sudden demand, which the business will not be able to fulfil given its current inventories.

As a result of this sudden one-time spurt in demand, the business increases its future forecasts of demand, and starts keeping more inventory. This business’s supplier sees this increased demand and increases its own forecasts upward, and increases its own inventory. Thus, this one-time demand “shock” percolates up the supply chain, giving the illusion of higher demand and with each layer in the chain keeping higher and higher inventory.

And then one day the retailer will realise that this demand shock is not replicable and moves forecasts downwards, and this triggers a downward edge in the forecasts up the value chain, and demand at the source comes crashing down.

Being a children’s book, The Tiger Who Came To Tea eschews the complexity of the supply chain and instead keeps the story at one level – at the level of the household of the protagonist Sophie (not to be confused with Sophie the Giraffe).

The premise of the story is the demand shock for supplies in Sophie’s home – a tiger comes home for tea and eats up everything that’s at home, drinks up all that’s there to be drunk (including “all the water in the tap”) and leaves, leaving nothing for Sophie and her family.

Assuming that the tiger will return the next day, Sophie’s family stocks up heavily, including “lots of tiger food”. And the tiger never arrives.

My guess is that the rest of the supply chain is left as an exercise to the reader – how the retailer who sold Sophie the tiger food will react to the suddenly higher demand for food (and for tiger food), how this retailer’s supplier will react, whether the tiger visits some other household for tea the next day (making this demand “regular” at the retailer’s level), and so forth.

Perhaps this is what makes this such as great book, and an all-time classic!