Analytics and complexity

I recently learnt that a number of people think that the more the number of variables you use in your model, the better your model is! What has surprised me is that I’ve met a lot of people who think so, and recommendations for simple models haven’t been taken too kindly.

The conversation usually goes like this

“so what variables have you considered for your analysis of ______ ?”
“A,B,C”
“Why don’t you consider D,E,F,… X,Y,Z also? These variables matter for these reasons. You should keep all of them and build a more complete model”
“Well I considered them but they were not significant so my model didn’t pick them up”
“No but I think your model is too simplistic if it uses only three variables”

This is a conversation i’ve had with so many people that i wonder what kind of conceptions people have about analytics. Now I wonder if this is because of the difference in the way I communicate compared to other “analytics professionals”.

When you do analytics, there are two ways to communicate – to simplify and to complicate (for lack of a better word). Based on my experience, what I find is that a majority of analytics professionals and modelers prefer to complicate – they talk about complicated statistical techniques they use for solving the problem (usually with fancy names) and bulldoze the counterparty into thinking they are indeed doing something hi-funda.

The other approach, followed by (in my opinion) a smaller number of people, is to simplify. You try and explain your model in simple terms that the counterparty will understand. So if your final model contains only three explanatory variables, you tell them that only three variables are used, and you show how each of these variables (and combinations thereof) contribute to the model. You draw analogies to models the counterparty can appreciate, and use that to explain.

Now, like analytics professionals can be divided into two kinds (as above), I think consumers of analytics can also be divided into two kinds. There are those that like to understand the model, and those that simply want to get into the insights. The former are better served by the complicating type analytics professionals, and the latter by the simplifying type. The other two combinations lead to disaster.

Like a good management consultant, I represent this problem using the following two-by-two:

analytics2by2

 As a principle, I like to explain models in a simplified fashion, so that the consumer can completely understand it and use it in a way he sees appropriate. The more pragmatic among you, however, can take a guess on what type the consumer is and tweak your communication accordingly.

 

 

Kabaddi and Jesus Navas

I’ve always talked about the Kabaddi style of solving a problem. In Kabaddi, when you are defending, six out of the seven players in the team form a chain in order to encircle the attacker. The seventh defender, however, strikes it alone, in a different direction, trying to draw the attacker into a position where he can be effectively surrounded.

Now there is a footballing analogy to this – the Jesus Navas style. Those of you who watched either Spain’s game with Honduras or the second half of their loss to Switzerland would’ve noticed that Spain effectively followed two lines of attack. The first was the traditional way – attack down the middle in a series of slow passes and build-up. Five of Spain’s front six players would get involved in this attack down the centre, almost rendering their game one-dimensional. And then there was Navas.

I haven’t confirmed this stat but in the game and half that he has played Navas has completed more crosses than anyone else in the tournament. He would strike it on his own down the Spanish right flank, hug the touchline, beat the full back and put in crosses. Minute in and minute out. Sometimes with a little help from full back Sergio Ramos, but mostly alone. It was fantastic to watch.

What this ended up doing was to divert the attention of the opposing defenders to cover Navas. If everyone were to have been attacking down the centre, the defending team could’ve just parked their bus in front of their centre and prevented any scoring. Spain letting free this one guy to take a different route meant that the opposition needed to cover that also leading to insufficient cover in the centre (it is another matter that Spain failed to score against Switzerland. But they did get so many more chances after Navas came on).

I’ve always been fascinated by such strategies at work, in business. You have a bunch of guys who try to attack the problem front-on, in the conventional way, working together, passing to each other frequently. And then there is this one guy who has been left out of this clique who attacks the problem “from the flank”. In his own way, without fear of failure. He knows that he is only an auxiliary solver, that he has nothing to lose (Navas lost his place in the XI after the Honduras match but I don’t think he had expected to ever play at all), and he can just go for it. The option value of letting one guy in the team loose in order to search for alternate solutions while everyone else is building up down the middle is immense, I think.

This is similar to Nassim Taleb’s “barbell investment strategy”. Acccording to that, he parks some 90% of his assets in ultra-risk government securities. They don’t give spectacular returns but his money is safe. And the rest of the 10% he uses to punt by buying stuff like out-of-the-money options. If they expire worthlessly, he hasn’t lost much of his wealth. The optionality (here, literally) of that additional 10% is, however, immense, and there is potential for spectacular returns from this strategy. with losses being capped.

Relationships and Prisoner’s Dilemma

So I ws thinking about this car analogy for relationships. I was thinking about how when you start your car, you will need to drive in first gear, with full engine power, slowly releasing the clutch, using a lot of fuel. However, after you have gathered certain speed, it is wasteful and unstable to go on in first gear. It is time for you to take your foot off the gas pedal, hold down the clutch and change gears, and shift the car to lower engine power.

I think it is similar with romantic relationships. Once you’ve reached a certain level and gotten past the initial phase, it is wasteful to continue in the same full throttle. Once both of you understand that the other is firmly in the basket, there is no need to waste much time just assuring and reassuring each other of the other’s presence. It is simply a wastage of fuel. Also, if there is too much torque at too much speed, there is a good chance that the car will spin out of control, so that needs to be avoided.

A relationship is like a car with two control systems. It is important that both of you coordinate the gear change, else there is a danger that the axle might snap. Let us move out of the analogy for the rest of the post.

So there are two of you and both of you have the choice of whether to change gear or not. Now, the ideal thing to do would be to change gears together, since that will ensure the relationship is at the same level but you’ll both be spending lesser energy on it. The worst case is if exactly one of you changes gears. If one of you suddenly slows down while the other is still at full throttle, it is likely that the other will suddenly feel insecure that the one has stopped responding, and this is likely to lead to some sort of breakdown in the relationship, even if temporary. And in order to get things back on track, you’ll need to go full throttle, thus leading to wastage of energy.

So basically, exactly one party deciding to scale down can prove to be disastrous for both of them, because of which the dominant strategy is to stay where you are – at full power. Let me draw the 2 by 2.

———————————————————————

|                        |    Scale down           |  Remain at full blast     |

———————————————————————

| Scale down|    0                               |  -100                                  |

———————————————————————

|Remain at   |  – 100                         |  -50 |

| full blast     |                                      |                                              |

——————————————————————–

You will notice that the players start off at a Nash equilibrium! Of both of them remaining at full blast. And thus neither has the incentive to scale down, unless he/she is sure that the other will also scale down simultaneously! And if the couple is not communicative enough, they will continue in this suboptimal state for too long, and end up burning way too much energy and willpower, which could’ve been otherwise put to good use.

Hence it is important that the couple communicates about matters such as these, and coordinates the shift in gears, and saves valuable energy!

Intellectual Property

A blog post earlier this month on Econlog finished off with a very strong quote by Friedrich Hayek:

One of the forms of private property that people cherish most is their ideas. If you convince them that their ideas are wrong, you have caused them to suffer a capital loss.

I ended up liking it so much that I added it to my work email signature. Thinking about it further, why is it that some people are more open to debate than others? Why do some people admit to their mistakes easily while others are dogmatic about them? Why do some people simply refuse to discuss their ideas with other people? I think Hayek’s observation offers a clue.

Let us consider two people – Mr. Brown and Mr. Green. Mr. Brown believes in diversification, and his investments are spread across several financial instruments, belonging to different categories, with a relatively small amount of money in each of them. For purposes of this analogy, let us assume that no two instruments in his portfolio are strongly correlated with each other (what is strong correlation? I don’t know. I can’t put a number on it. But I suppose you get the drift)

Mr. Green on the other hand has chosen a few instruments and has put a large amount of money on each of them. It is just to do with his investment philosophy, which we shall not go into, as this is just an analogy.

Let us suppose that both Mr. Brown and Mr. Green held Satyam stock on 6th January 2009. They were both invested in Satyam according to their respective philosophies – and the weightage of Satyam in their respective portfolios was also in line with their philosophies. The next day, 7th of January, the Satyam fraud came out. The stock crashed to a tenth of its value. Almost went to zero. How would our friends react to this situation?

Mr. Green obviously doesn’t like it. A large part of his investments has been wiped out. He has become a significantly poorer man. For a while he will be in denial about this. He will refuse to accept that such a thing could happen to one of his chosen stocks. He will try to convince himself that this fall (a 90% fall, no less) is transient, and the stock will go back to where it once was. As days go by, he realizes that his investments have been lost for ever. He is significantly poorer.

Mr. Brown will also be disappointed by the fall – after all, he too has lost money in the fall. However, his disappointment is mitigated by the fact that the loss is small compared to his portfolio. There have been other stocks in his portfolio which have been doing well, and their performance will probably absorb the Satyam losses. Some of the stocks in his portfolio may also be fundamentally negatively correlated with Satyam, which means they will now gain. There is also the possibility that the Satyam fall has opened up some new possible areas of investment for Mr. Brown, and he might put money into them. It is much easier for Mr. Brown to accept the fall of Satyam compared to Mr. Green.

So you replace stocks by ideas, and I suppose you konw what I am gettting at. The degree of openness that people show with respect to an idea they have varies inversely with the share of this particular idea in their “idea portfolio”. The smaller the proportion of this idea, the lesser will be the “capital cost” of their losing the idea. And hence, they will be more open to debate, to discussion, to letting someone critically examine their ideas. If the proportion of this particular idea in their overall portfolio is large, there will obviously be resistancce.

A corrolary of this is that when someone possesses a small number of ideas they are more likely to be dogmatic about them (I am using the indefinitive “more likely” here because even when you have a small number of securities in your portfolio, your exposure to some of them will be really small and so you’ll be less unwilling to lose them. Though I must point out that people with small ideas portfolios become so used to madly defending the big ideas in the portfolio that they start adopting the same tactic for the smaller ideas in their portfolio and become dogmatic about them – which is irrational).

I just hope I didn’t cause you a capital loss by writing this. For me, on the other hand, this was a bonus stock.