Pertinent Observations

Premier League Sub-Tables

Ahead of the Chelsea-Liverpool game on Sunday, the pre-match show showed a “sub-table” of the premier league of how the top 8 teams had fared against each other. While by definition, the Premier League is played among 20 teams, and your result against Chelsea is as important as that against Crystal Palace, looking at sub-leagues like this one can help us gauge what the overall points table is trying to hide.

In this post I’ll just show the sub-league of the top of the league (top 3 to top 7) and bottom of the league (bottom 3 to bottom 7). Offered without further comment.

Silencing the temple

The temple across the road from my house has really started annoying me. The priest has one tape, of supposedly “devotional” songs and this morning he thought it appropriate to play it on the loudspeaker at 5:30, startling me and waking me up.

This is not the first time he has done so, either. He has been a consistent offender. Earlier, the tape would go on at around 7:30, after we had woken up so it didn’t really affect me so much – I’d put on my own music to drown the offending noise and all would be fine. Of late, though, the bugger seems to be getting to his temple early, and he considers it his sacred duty to wake up the locality with his noise.

Polite attempts (by the wife) to ask him to turn down the volume have had no impact. I’m told over ten years back my grandmother-in-law (known in her time to have been an extremely strong and clever woman) had tried her own methods to silence him, but had failed and given up (one of her rare failures, according to the wife). One of the things she had apparently tried was to threaten to call in the cops. It didn’t work.

As we lay tossing and turning in bed this morning having been rudely woken up by the temple noise, we thought of strategies. One was to write out a police complaint, get neighbours to weigh in with their support and go to the cops. Another was to get in touch with the local politicians (corporator, MLA, etc.) and see if they can do something about it.

One thing bothers me about either approach, though – no there is no risk per se, but I don’t think any of this will really work. The problem is the Indian definition of “secularism” – which is not “each citizen will practice his/her own religion in private and the state will not interfere” but instead is “each citizen can make a big loud show of practicing his/her religion and the state will not interfere”.

And so if I go to the cops or the politicians asking them to intervene, one question that will invariably come up is why the temple priest should shut up when there are no restrictions whatsoever on the muezzin’s call. And if you go to the muezzin and ask him to turn down the volume, he’ll agree on the condition that the loudspeakers at the Ganesh pandals be turned down. And thus we will set off on an infinite loop.

This is the sad story with religion in India. Anything goes in the name of religion. If you oppose something done in the name of religion itself, you are being anti-religion, and that is blasphemous.

Anyway, I still have the conundrum of how to deal with the hooligan priest in the temple across the road from my house!

Premier League – Home and Away

So we are halfway through what is easily the most competitive English Premier League in recent times. To illustrate, Liverpool were on top of the table at Christmas, and after two successive defeats, lie fifth at the New Year. Anything can happen this season and the top seven or eight teams are all still in contention.

One interesting factor this season, however, has been the fixture list. For example, only Manchester United among the top eight has played at Anfield (Liverpool’s home ground) this season. Liverpool has played every other top eight team away so far – which means they will face them all at home in the second half of the season.

On the other hand, Manchester City has hosted all top eight teams bar Chelsea so far! Which means they will be playing all these teams away this season! Sunderland is currently bottom. However, they have hosted mostly top-half teams (which are significantly superior to them) and played away to other bottom half teams which are of comparable strength.

The following table illustrates who has hosted whom. The table is to be read row-wise. “H” and a red cell means that the team in the corresponding row has hosted the team in the corresponding column. A while cell and “A” implies otherwise.Notice that this is the only point of time in the season when we can do this analysis for now everyone has played everyone else exactly once. Teams in this table are ordered in descending order of points.

Can we have a metric of who has had the best set of home fixtures this season so far? For this, let us define a “home index”. For each team, this is calculated as the difference between the average points of the teams played at home and the average points of the teams played away.

Let me illustrate. Arsenal, for example have so far hosted Chelsea, Everton, Liverpool, Spurs, Southampton, Hull, Stoke, Villa and Norwich. These teams have an average of 28.6 points as of now. Arsenal has so far visited the rest of the clubs, viz. Manchester City, Manchester United, Newcastle, Swansea, Cardiff, West Brom, Crystal Palace, Fulham, West Ham and Sunderland. These teams average 22.6 points as of now. So Arsenal’s home index is 28.6 – 22.6 = 6.0

A positive home index implies a team has played more strong teams at home and weak teams away. It is not easy to say, however, whether this implies an easier second half of the season. Arsenal, for example, would be happy to host the weaker teams they have traveled to, but the fixture list means they will be traveling to more strong teams, which means the potential for dropping points is higher.

Among the relegation-threatened teams, Sunderland has the highest “home index”, and for them the season is likely to become better – in the second half, they will get to host the weaker teams whom they can reasonably expect to beat while traveling to stronger teams (who they’ve lost to anyway) won’t change much. Thus, a high home index is a positive for lower-ranked clubs.

The following graph shows the home-away index for all clubs:

Notice that both Arsenal and Manchester City have had an easier run so far – hosting the better teams. it will be interesting to see how they perform in the second half of the season when they travel to the better clubs. Especially given that Liverpool and Chelsea have had a bad run of fixtures in the first half and are likely to improve in the second half.

Finally, the first part of the post assumes that teams are better off playing at home rather than away. Is this really true? To check this, let us look at the average points scored by teams in home and away games. This number should be taken with a pinch of salt, though – teams with a high home index are less likely to perform significantly better at home compared to away.

Based on the performance so far, the average points in home games is 0.28 more than the average points in an away game. However, we also need to take into account the home index. When we regress the difference between home and away points against the home index, we find that for every one point higher in home index, the difference between home and away performance comes down by 0.04 (the R-square, for those that are interested in such things, is 15%).

The following table shows the difference in home and away performances of different teams:

Most teams, you can see from this table, have done better home than away. The significant exceptions are Aston Villa, Manchester United and Spurs. Villa and Manchester United have a high Home Index which possibly explains this. There is no explanation of spurs’ home form, though.

Spending on Indian Players in IPL Auctions

In the first IPL Auction in 2008, teams spent an average (median) of 47% of their overall spend on Indian players, the rest going to foreign players. By the time of the auction in 2011, however, they had wisened up to the fact that only four foreigners can play in the eleven, and the average (median) spend on Indian players went up to 65%.

How did different teams fare on this count? The following graph describes this (I’m generally not a big fan of “dodged” bar graphs but couldn’t think of a better way of representing this data. If you have any ideas, do let me know).

As you can see in this graph, most teams significantly increased their spending on Indian players. The only teams that failed to do so were Deccan Chargers (who performed really badly and then dropped out of the IPL), Kings XI Punjab (performed badly all three seasons) and Rajasthan Royals (who built their team around “uncapped” Indian players who were not part of the auction).

It will be interesting to see what this ratio is like in the following auction.

Why I became a Liverpool fan

In mid-April 2005, I was on the District Line train from Mansion House to South Kensington, in London, and in the Victoria station, a huge number of people got on to the train. They were all dressed in red, and carrying Liverpool scarves and cans of Carlsberg beer. They were on their way to Stamford Bridge, to watch Liverpool take on Chelski in the Champions League semis at Stamford Bridge. And they started singing.

It was magical, as they first sang “you never walk alone”, and then followed it up with personalized songs for each of the players, and for the coach Rafa Benitez. I remember one going “Steve Gerrard Gerrard, pass the ball forty yards .. ” . And another, to the tune of “La Bamba”, going “Rarararararafa Benitez, Xabi Alonso, Garcia and Nunez” (honouring all the Spaniards in the team). I was sold.

Till then, I hadn’t been much of a football fan, though I would watch the odd World Cup or Euro game. I had never really followed club football, and never supported any team. That day, things changed. I went to a crowded pub in Kensington to watch the game, perhaps I was the only Red fan there. I got to know the names of the Liverpool players (I’d heard of Gerrard and Milan Baros thanks to their exploits in Euro ’04, and I knew Alonso, Garcia and Nunez (never saw him play) thanks to the song). And quietly cheered for Liverpool in that semi final.

It has been a roller coaster ride for the last eight odd years, with more downs than ups. The undoubted high came just a month after I’d declared myself a Liverpool fan, when they came back from 0-3 down to beat Milan in the Champions League finals in Istanbul. There have been several low points, the one that hurts the most is them failing to win the Premiership in 2008-09, when they came a close second. And then, they were to sell Xabi Alonso, who had been my favourite player.

The kind of passion I feel when I watch Liverpool play is unmatched, even by what I feel when I watch the Indian Test cricket team. There is a kind of tension that develops that I seldom feel otherwise. The disappointment when they lose (or fail to win) is the kind that I normally reserve for personal debacles.

And to think it all started with a random train ride with a bunch of loud drunks.

Baptists and Bootleggers: Karnataka Edition

“Baptists and bootleggers” is a popular concept in economics. It is used to illustrate that in the absence of sound economic thinking, good intentions don’t count for much. According to this concept, baptists want to ban the sale of alcohol on Sundays because it is the day of the lord, and they don’t want people to be drinking that day. And this plays out directly into the hands of bootleggers – who make a living supplying people their booze on Sundays.

So by calling for the sale of liquor to be banned on Sundays, baptists are essentially encouraging an illegal activity and an illegal trade. If not for the baptists, people would be able to buy their liquor legally on Sundays, and bootleggers would be out of business.

There is also a social cost to policies like this – by pushing an activity (such as the sale of liquor on Sundays) underground, you encourage nefarious elements to get into business, rather than keeping it in clean hands. And this is likely to increase the overall rate of crime.

Thus, by their supposedly moral position that alcohol should not be sold on Sundays, baptists actually end up unintentionally encouraging crime!

A similar story to this has been playing out in Karnataka in the last twenty years. For whatever reason, in 1993, the government of Karnataka decided to freeze the total number of liquor licenses in the state. Since 1993, if you want to open a bar or a liquor shop, you need to purchase a license from the secondary market. Effectively, for every new liquor outlet, some outlet somewhere in the state has to close down (whether such closure is usually voluntary or not is left as an exercise to the reader). This increases the cost of liquor intermediation in the state and leads to higher prices for the consumer.

While higher prices may be desirable for “sin goods” such as liquor, there is a better way for the government to increase consumer prices – by levying higher taxes, which ensures that the additional money thus paid by the consumer flows into the government coffers. By limiting the number of licenses, however, the government doesn’t get extra revenue.

Instead what this encourages is illegal sale of liquor! That there is a limit on the number of liquor licenses doesn’t push down people’s need for liquor. And they end up buying liquor from illegal sources and bootleggers, and it becomes difficult to maintain quality and hygiene standards on such sales. And with a bar having to close down for every new one that needs to open, you might imagine the kind of characters that might get involved in the process.

Back in 2008, a friend was trying to start a lounge bar, and he mentioned that he had to pay up to the tune of Rs. 30 lakh to get his license, while the official price is about a tenth of the amount. It is obvious that not all the money he paid for his license went to the government’s coffers.

Where do the baptists come here? Because every time there is a proposal to increase the number of liquor licenses, you will have a wave of morality which protests this decision. They are the baptists who keep Karnataka’s bootleggers in business.

Also read this piece on the funny rules of Karnataka’s liquor licensing regime.

Analyzing the IPL Auction Rules – 1

So finally after a really long delay the rules for the IPL Auction 2014 are out. Each franchise has the option of retaining up to five players, with additional “trump cards” that allow them to match the price of a winning bid in the auction for players that were part of their teams in the earlier IPLs.

At the outset, the rules of the auction look loaded towards teams that already have strong squads and want to retain as many players as they can – for example, given the rules of the auction, a team can retain up to 6 players from their existing squads, and this significantly biases the auction in favour of teams that want to retain players.

Looking a bit deeper, though, it is clear that this luxury of retention comes at a price. For example, irrespective of what the team negotiates with its number one player, Rs. 12.5 Crore (125 million), or a little more than 20% of the cumulative salary cap will be debited from the team’s account. For the next player, Rs. 9.5 Crore (95 million) will be debited. There is a sliding scale and the fifth player a team retains will cost them Rs. 4 Crore in terms of their budget.

The question is if this pricing is appropriate – is charging 20% of the team budget for the number one player enough compensation for the benefit the team gets by way of retention? Is charging two thirds of the total salary cap (Rs. 39 Crore) enough for retention of five players?

At first look, this pricing looks appropriate – after all, why would someone want to forego two thirds of their auction kitty for keeping just five players, when the total squad size is 16 to 27? However, looking at the previous auctions tells a different story.

The two graphs here shows the proportion of total auction money spent by each team on each player in the last two auctions. The graph might appear complicated so let me explain. For each team, I ordered players bought in the auction in the descending order of price. Then I looked at how much the top player cost as a proportion of the total money spent at the auction. Then, how much the top two players cost and so on.

ipl11

(click on images for full size. For the 2008 auction, marquee players have been included in the analysis)

In the 2008 auction, teams spent between 60 and 85% of their budgets in order to select their five most expensive players, with a median of 72%. In the 2011 auction, teams spent between 65 and 90% of their budgets for their top five players (takes into account retained players), and the median spend was 71%.

Given that the “top 5” players for each team cost them upwards of 70% of their total budgets in the last two auctions, charging teams only Rs. 39 Crore (65%) for retaining five players is blatantly unfair, and biased towards the teams that want to retain. Also, considering that retained players are “known devils”, there is more value for money for teams from retained players. So in the ideal case, the fee for retaining 5 players should have been definitely upwards of 75% of the total budget (Rs. 45 Crore).

The following table helps to show the undervaluation of each retained player:

The second and third columns in the above table shows the median percentage of total budget teams spent in order to buy their top N players. The last column shows what percentage of their budget they would have to spend if they are to retain players in the auction.

The message for teams is clear: retain as much as you can. It is cheaper to retain your top players rather than building a new team from the available pool. The challenge, however, is to negotiate a good price with these players.

PS: I have a solution that can help teams plan their auction strategy. If you are an IPL team and you are interested in this, contact me through the contact form.

The Question About Adarsh No One Is Asking

Nikhil Service Station is one of the more popular petrol bunks in South Bangalore. If you wonder why you have never heard of it, however, it is because nobody refers to the petrol bunk by its real name. The bunk is owned by Anitha Kumaraswamy, wife of former Karnataka Chief Minister HD Kumaraswamy and daughter-in-law of former Prime Minister HD Deve Gowda. The service station came into service in the early 2000s. It had been allotted by Indian Oil in the “Kargil martyrs” quota. It is now a landmark in South Bangalore, and popularly known as “Deve Gowda Petrol Bunk”.

The reason I’m bringing up the issue of the petrol bunk is to draw a parallel with the scam-ridden Adarsh Cooperative Housing Society in Mumbai, which was again ostensibly built for “serving and retired army personnel”. The Adarsh scam is in the news once again due to the rejection of the report by the Maharashtra cabinet and the connection with arrested diplomat Devyani Khobragade.

The question about Adarsh that nobody is asking is this – why is it the government’s business to construct housing for “serving and retired army personnel”? Why is it that the government should compensate families of martyrs with petrol bunks and LPG dealerships and not cash? Aren’t these structures designed to be scammed?

Nobody argues that the army must be paid well. Nobody argues, either, that army personnel should be generously insured and compensated, given the hazardous nature of their jobs. My argument, however, is that this insurance and compensation should be universal and standardized. Allotments such as housing and LPG dealerships are discretionary by nature, and that makes them prone to abuse.

Consider for example, a housing society the government constructs for “serving and retired army personnel”. Let us say that the society has 500 apartments. How does the government choose who gets these apartments? And in what way are the 500 such chosen personnel different from those that did not get the allotment? Does this discretionary allotment not leave the system to abuse? Does this not lead to unhealthy competition among the “serving and retired army personnel”? Do we want that in our armed forces?

On a similar note, after each railway accident, we have the railway minister announcing a discretionary compensation for the dead and injured. The question is why this should be discretionary. Cannot the railways simply buy group insurance for all its passengers, which is automatically paid out upon an accident?

The argument I’m making is that some of the processes we follow are designed to be scammed. In the time of tragedy, either in an accident or in battle, what we need is a standardized and predictable response on behalf of the government agencies. By not putting that in place, the system is prone to abuse.

On finding the right signal

It is not necessary that every problem yields a “signal”. It is well possible that sometimes you try to solve a problem using data and you are simply unable to find any signal. This does not mean that you have failed in your quest – the fact that you have found the absence of a signal means is valuable information and needs to be appreciated.

Sometimes, however, clients and consumers of analytics fail to appreciate this. In their opinion, if you fail to find an answer to a particular problem, you as an analyst have failed in your quest. They think that with a better analyst or better analysis it is possible to get a superior signal.

This failure by consumers of analytics to appreciate that sometimes there need not be a signal can sometimes lead to fudging. Let us say you have a data set where there is a very weak signal – let us say that all your explanatory variables explain about 1% of the variance in the dependent variable. In most cases (unless you are trading – in which case a 1% signal has some value), there is little value to be gleaned from this, and you are better off without applying a model. However, the fact that the client may not appreciate you if you give “no” as an answer can lead you to propose this 1% explanatory model as truth.

What one needs to recognize is that a bad model can sometimes subtract value. One of my clients once was using this model that had been put in place by an earlier consultant. This model had prescribed certain criteria they had to follow in recruitment, and I was asked to take a look at it. What I found was that the model showed absolutely no “signal” – based on my analysis, people with a high score as per that model were no more likely to do better than those that scored low based on that model!

You might ask what the problem with such a model is. The problem is that by recommending a certain set of scores on a certain set of parameters, the model was filtering out a large number of candidates, and without any basis. Thus, using a poor model, the company was trying to recruit out of a much smaller pool, which led to lesser choice for the hiring managers which led to suboptimal decisions. I remember closing that case with a recommendation to dismantle the model (since it wasn’t giving much of a signal anyway) and to instead simply empower the hiring manager!

Essentially companies need to recognize two things. Firstly, not having a model is better than having a poor model, for a poor model can subtract value and lead to suboptimal decision-making. Secondly, not every problem has a quantitative solution. It is very well possible that there is absolutely no signal in the data. So if no signal exists, the analyst is not at fault if she doesn’t find a signal! In fact, she would be dishonest if she were to report a signal when none existed!

It is important that companies keep these two things in mind while hiring a consultant to solve a problem using data.

Maratha Names

Going by Uday Kulkarni’s book Solstice at Panipat which I’ve been reading over the last few days, there was a Maratha chieftain named Balwant Rao Mehendale. He was supposed to be a brilliant warrior and led one of the divisions of the Maratha army. However, in a skirmish about a month before the big battle, he was hit by a bullet and perished.

What I found amusing is his son’s name – “Appa Balwant”, which translates into Kannada as “father Balwant”. The name amuses me because it is as if the child wasn’t given a name. Whenever someone would ask for his name, he would simply say “Appa Balwant” (“my father is Balwant”). Wonder how it affected him!

Thinking more about it, though, it is possible that the kid’s given name was “Appa” (since Marathis use father’s given names as middle names) and so his full name was “Appa Balwant Mehendale” or something of the sort.

It continues to amaze, however, that someone would name their kid “Appa”.