Author: SK

The problem with spider charts

On FiveThirtyEight, Nate Silver has a piece looking ahead to the Democratic primaries ahead of the presidential elections in the US next year. I don’t know enough about US politics to comment on the piece itself, but what caught my eye is the spider chart describing the various Democratic nominees.

This is a standard spider chart that people who read business news should recognise, so the appearance of such a chart isn’t big news. What bothers me, though, is that a respected data journalist like Nate Silver is publishing such charts, especially in an article under his own name. For spider charts do a lousy job of conveying information.

Implicitly, you might think that the area of the pentagon (in this case) thus formed conveys the strength of a particular candidate. Leaving aside the fact that the human eye can judge areas less well than lengths, the area of a spider chart accurately shows “strength” only in one corner case – where the values along all five axes are the same.

In all other cases, such as in the spider charts  above, the area of the pentagon (or whatever-gon) thus formed depends on the order in which the factors are placed. For example, in this chart, why should black voters be placed between the asian/hispanic and millennials? Why should party loyalists lie between the asian/hispanics and the left?

I may not have that much insight into US politics, but it should be fairly clear that the ordering of the factors in this case has no particular sanctity. You should be able to jumble up the order of the axes and the information in the chart should remain the same.

The spider chart doesn’t work this way. If lengths of the “semidiagonals” (the five axes on which we are measuring) are l_1, l_2, ... l_n, the area of the polygon thus formed equals \frac{1}{2} sin \frac{360}{n}  (l_1.l_2 + l_2.l_3 + ... + l_n.l_1). It is not hard to see that for any value of n \ge 4, the ordering of the “axes” makes a material difference in the area of the chart.

Moreover, in this particular case, with the legend being shown only with one politician, you need to keep looking back and forth to analyse where a particular candidate lies in terms of support among the five big democrat bases. Also, the representation suggests that these five bases have equal strength in the Democrat support base, while the reality may be far from it (again I don’t have domain knowledge).

Spider charts can look pretty, which might make them attractive for graphic designers. They are just not so good in conveying information.

PS: for this particular data set, I would just go with bars with small multiples (call me boring if you may). One set of bar graphs for each candidate, with consistent colour coding and ordering among the bars so that candidates can be compared easily.

High command and higher command

Last week a friend messaged me early in the evening and asked if we could meet for drinks later in the evening. I almost replied to him saying that I’ll get back to him once I’d “checked with High Command”, but then realised that I now have not one but two high commands.

Just the previous day, I had watched Liverpool FC play at Manchester City. This was the first time I was watching live football this season, primarily because we don’t have cable at home. Since I didn’t want to “jinx Liverpool“, I waited for a game where they were not the favourites to make my watching debut, and Manchester City away seemed like the best bet for that.

When I told the senior high command that I wanted to go watch this game, and if she could take care of the junior high command during the duration of it, she had no objections. Just as I was about to leave for the game, though, the junior high command (or maybe I should call her the higher command) decided to throw a tantrum. Some distractions and the promise to watch endless episodes of Bing and Pablo were required to allow me to step out.

The next day, I was pretty sure that the high command would be cool with me going out for evening drinks. We had discussed our lives at the turn of the year, and had agreed that we should meet more people, and I had specifically made it a point that I’d go out more on evenings and weekends. Still being early in the year, I knew the high command would be okay with me going out. I wasn’t sure if the higher command would be okay though.

The story played out the same way as the previous day. I picked up higher command from the nursery. High command then returned from office, and it was time for me to step out. And higher command wasn’t pleased at all. Once again it took distractions to allow me to step out. And it turned out that the price of the bribe was rather high that evening. When I returned, the higher command was watching Bing, and didn’t even turn around to look at me (normally she’s all over me when I come home from anywhere).

Another friend just asked me if we can meet on Wednesday evening. Again I was about to type “let me check with the high command”, and then decided it is prudent to make that plural. Once again I know the high command will be okay with this but the higher command may not be.

Training to be a quizzer

Eleven days before our daughter was born in September 2016, the wife and I attended the annual Family Quiz organised by the Karnataka Quiz Association. We ended up doing fairly well in the quiz, and placed third.

Unfortunately, despite the quiz having been described as “for teams of three and under from the same family”, we only got two book coupons as a prize. If they had given us three prizes instead, I could’ve claimed that our daughter won her first quiz even before she was born.

Two years and four months on, I’ve greatly disappointed my wife in terms of how much I’ve taught our daughter. According to some sort of an agreement we had come to ages back (maybe even before we were married), I was supposed to have taught our daughter calculus by now. As it happens, she can barely count to twenty, and still hasn’t fully got the concept of counting objects.

However, there are other areas of development where, despite me not putting any sort of effort whatsoever, she has done rather well in terms of her learning. I had written last month that she had proved adept at showing off her Quantum Physics for Babies in front of visitors. And before that she had shown promise by reading bus number boards.

Now, an anecdote from last night suggests she is already gearing up to be a quizzer.

Back in May, a friend and business associate gifted her this illustrated book of nursery rhymes (I don’t have the copy with me as I write this, but it was possibly this one). Since each page contains a full rhyme and maybe one or two illustrations, we don’t use the book too much – the daughter prefers books that have a higher picture-to-word ratio.

In fact, the fact that I’m not even sure what the book precisely looks like should tell you that we don’t use the book too often. Once in a while, my wife reads out poems from that just before bedtime, but I normally don’t read from it.

Anyways, last night as I was putting the daughter to bed, she picked out this book and asked me to read it to her before she fell asleep. And then she showed off her prowess as a quizzer.

Being two and a third years old, she can’t yet read (she knows the numbers and a few letters of the English and Kannada alphabets, but not much more). However, the way she was recognising the poems from the illustrations suggested that she was actually reading!

Guessing “Humpty Dumpty” by looking at an egghead sitting on a wall was rather easy. Some of the other poems she guessed correctly were, however, stuff I surely wouldn’t have  gotten from the illustrations. In fact, this included poems whose existence even I wasn’t aware of before I saw them in the book. I was so impressed that I didn’t really mind that she didn’t go to bed until it was eleven o’clock last night!

Now, this might be a false alarm. In the past she seems to have answered arithmetic questions correctly which later turned out to be a fluke. The sheer proportion of poems she got correct last night suggested this is not the case. The other doubt is that she might have seen the book elsewhere, and thus mugged up the picture-poem associations.

The way she was guessing, however, suggested to me that she was simply recognising the objects in the pictures and the actions they were exhibiting, and then working out the name of the poem from these figures and actions (obviously she knew it’s a book of rhymes, so the sample space was finite). And that is exactly how your mental process goes when you’re attempting a (good) quiz.

Now I don’t mind so much that she still has a long way to go before she can learn calculus.

Kader Khan and Slippery Fish

Until I read his obituaries, I didn’t know that the just-deceased actor Kader Khan was also a dialogue writer, having written the dialogues for several iconic Amitabh Bachchan movies such as Laawaris, Muqaddar Ka Sikander and Agneepath (I highly recommend his obituaries by GreatBong and by The Economist. Both are brilliantly written and highly informative).

When I found that Kader Khan wrote the dialogues for Agneepath, I was reminded of this old piece written by Nitin Pai, Director of the Takshashila Institution, that referenced a particular dialogue from Agneepath. In that, he talks about international relations and the “law of the jungle” that operates there. In fact, I recommend you see that dialogue from Agneepath. It’s on youtube:

Major level up in terms of my respect for Kader Khan after having re-watched this. I only knew him as a masterful comedian and actor as I mentioned. Now I want to go back and watch more of the movies for which he wrote dialogues.

In any case, if you watched the above video from Agneepath, you see that Amitabh Bachchan talks about the law of the jungle in the form of a food chain. I don’t know the Hindi names of animals precisely, but he talks about a frog being eaten by a snake, and a wolf being killed by lions, etc.

The talk of the food chain reminded me of another song which is my daughter’s favourite nowadays.  This talks about the marine food chain. Watch the song here:

I had no idea about this song until the daughter started singing it. In fact, since she liked the third stanza of the song best (“tuna fish, tuna fish”), it had also caused us a bit of a problem one weekend when she kept requesting for it but we couldn’t find it on any streaming service (you should ask for “slippery fish”).

I’m pleasantly surprised that they teach about the food chain to children as young as two years old, since it introduces to them the concept of death, and the fact that animals eat other animals to survive. Somehow I had thought that kids are told that all animals co-exist like they do in Peppa Pig (as an aside, I wonder what will happen to Peppa Pig and friends when it’s time for them to grow up and start dating), and that the concept of death is also taboo in some circles.

Anyway I’m glad my daughter likes the song. Maybe it’s time to let her graduate to the Amitabh Bachchan dialogue? But that would involve her learning Hindi, which is my wife and my “secret language” (when we’ve to say something we don’t want the daughter to understand) !

Podcasts to replace books

Yesterday I listened to an excellent podcast episode with Steven Pinker at Amit Varma’s Seen and Unseen podcast. In this, they discuss concepts from Pinker’s latest book Enlightenment Now.

Pinker is an author I’ve found difficult to read. Based on glowing recommendations, I bought his books The Stuff of Thought  and The Language Instinct a decade ago, but couldn’t get beyond the first ten pages of both, despite trying several times. As a consequence, I’ve declared that his writing style is not suited for me, and I won’t bother reading his books any more.

However, since I’ve heard good things about the book, listening to a podcast episode which covered the major concepts in the book was damn useful.

It is similar with poker player and author of Thinking in Bets Annie Duke. She’s highly regarded by the “finance twitter circlejerk” that I follow (I follow her as well), and she appears on several podcasts with members of this circlejerk. However, a friend whose opinions I trust told me that the book itself isn’t particularly great, and that there wasn’t that much in the book about thinking in bets per se.

A quick reading of the Kindle sample confirmed this hypothesis, so I didn’t bother with the rest of the book. Instead, I substituted for it by listening to a couple of podcasts that Duke has recorded, to get the best of her insights. This combined with following her on twitter, I don’t think I’ve done too badly.

I’ve found this “podcast trumping book” concept work in other cases and other ways as well. Beyond the first chapter, I found Ray Dalio’s Principles unreadable, but then I realised that I had got most of the concepts in the book from his podcast recording with Shane Parrish of Farnam Street.

Jordan Peterson’s 12 rules for life again is extremely insightful, but a very boring read. So for someone who doesn’t have the patience to plough through all his philosophy and religion stuff (which are weak compared to his psychology stuff which is incredible), I would just recommend that they listen to his podcast recording with Russ Roberts.

In some ways this takes me back to my old concept of how a lot of non-fiction books simply don’t have that much information content and just keep repeating the same points over and over. The antidote to this, I’ve argued, is to pack the book with a sufficient number of “side stories” so that there is more information packed in it. I think, and hope, that I’ve done this with my own book.

When an author records a podcast to promote a book, the intent is to get a potential reader to be attracted to the book, and hence the best concepts in the book get put out there. Also, as long as the podcast interviewer is good (Amit Varma, Russ Roberts and Shane Parrish are all very good podcast hosts), the podcast will never be boring and you’ll be able to get the information content in an easy way. So unless you want depth (I’m glad I ploughed through Jordan Peterson’s book since I found it has some depth, but not everyone would feel the same), just listening to the book-related podcasts should serve you well.

Oh, and I’ve recorded some five or six podcast episodes with Amit Varma’s seen and unseen podcast to discuss the book. I guess a lot of those listeners thought like me, so they didn’t bother buying my book!

Premier League Points Efficiency

It would be tautological to say that you win in football by scoring more goals than your opponent. What is interesting is that scoring more goals and letting in fewer works across games in a season as well, as data from the English Premier League shows.

We had seen an inkling of this last year, when I had showed that points in the Premier League were highly correlated with goal difference (96% R square for those that are interested). A little past the midway point of the current season and the correlation holds – 96% again.

In other words, a team’s goal difference (number of goals scored minus goals let in) can explain 96% of the variance in the number of points gained by the team in the season so far. The point of this post is to focus on the rest.

In the above image, the blue line is the line of best fit (or regression line). This line predicts the number of points scored by a team given their goal difference. Teams located above this line have been more efficient or lucky – they have got more points than their goal different would suggest. Teams below this line have been less efficient or unlucky – their goal difference has been distributed badly across games, leading to fewer points than the team should have got.

Manchester City seem to be extremely unlucky this season, in that they have scored about five fewer points than what their goal difference suggests. The other teams close to the top of the league are all above the line – showing they’ve been more efficient in the way their goals have been distributed (Spurs and Arsenal have been luckier than ManYoo, Chelski and Liverpool).

At the other end of the table, Huddersfield Town have been unlucky – their goal difference suggests they should have had four more points – a big difference for a relegation threatened team. Southampton, Newcastle and Crystal Palace are also in the same boat.

Finally, the use of goal difference is used to break ties in league tables is an attempt to undo the luck (or lack of it) that would have resulted in teams under- or over-performing in terms of points given the number of goals they’ve scored and let in. Some teams would have gotten much more (or less) points than deserved by sheer dint of their goals having been distributed better across matches (big losses and narrow wins). The use of goal difference is a small attempt to set that right.

The Old Shoe Theory of Relationships

When our daughter was young, some friends saw uncanny resemblances between her and me, and remarked that “Karthik could have married an old shoe and still produced a child that looks like this”, essentially remarking that at least as far as looks were concerned, the wife hadn’t contributed much (Bambi eyes apart).

Over time, the daughter has shown certain other traits that make her seem rather similar to me. For example, she has the practice of sticking her tongue out when performing tasks that require some degree of concentration. She laughs like me. Screeches like me. And makes a “burl-burl” noise with her fingers and lips like I do (admittedly the last one is taught). I’ve already written a fuller list of ways in which the daughter is similar to me.

If you are single and looking to get into a long-term gene propagating relationship, you inevitably ask yourself the question of whether someone is “the one” for you. We have discussed this topic multiple times on this blog.

For example, we have discussed that as far as men are concerned, one thing they look for in potential partners is “consistent fuckability“. We have also discussed that whether someone is “the one” is not a symmetric question, and when you ask yourself the question, you either get “no” or “maybe” as an answer, implying that you need to use Monte Carlo algorithms. Being married to the Marriage Broker Auntie, I’m pretty sure I’ve discussed this topic on this blog several other times.

This is a tubelight post – at least two years too late (the “old shoe” comment came that long ago), but this is yet another framework you can use to determine if you want someone as your long-term gene-propagating partner. Basically you replace yourself by an old shoe.

In other words, assume that the genes that you will propagate along with this person will result in kids who look like them, talk like them, act like them, and rather than a “next best thing”, might just be a superior version of them. Ask yourself if you are okay with having a child who is like this, and who you will be proud of.

This is another Monte Carlo type question, but if the answer in this case is no (you may not be particularly proud of a progeny who is exactly like the person under consideration – for whatever reason), you don’t want to risk propagating genes with this person. In case the answer is yes – that you are willing to parent a child who is exactly like this counterparty, then you can seriously consider this long term relationship.

Again, this applies if and only if you’re looking for a gene propagating relationship. If that isn’t an issue (no pun intended), then you don’t need to worry about old shoes of any kind.

City Shutdown

Piccadilly Circus on Christmas Day. By Justin Bramwell of the BBC

I wouldn’t have imagined that it would be possible for the capital city of a supposedly liberal Western democracy to completely shut down, even if it were for a day. The extent to which London shut down yesterday on account of Christmas proved me wrong, and the only parallels I could think of were Bangalore on the day after Rajkumar died in 2006, and Gurgaon on the day of Holi in 2009.

Considering that the latter two instances were essentially a response and a pre-emptive response to hooliganism, I was surprised to see London reach the same levels of shut down on an otherwise peaceful day.

There were no shops open through the day. A handful of restaurants were open, but only for those with reservations. A pub (short for “public house”) close to home had a signboard saying “for reservations only. Not open for public”. Most other pubs and restaurants were shut.

Even the small versions of large chain-stores that open from early morning to late night on most days were shut. So were the mom-and-pop stores that are “off-license” and hence not subject to Sunday trading restrictions (another irritating thing I find about Europe, coming from India where Sunday is the busiest day for shopping). Public transport wasn’t running. There wasn’t much private transport either – the streets were mostly empty. I didn’t pass by many medical shops but those too weren’t open.

Thankfully we realised that the shutdown was impending – I don’t really know how we realised, but by Monday it was clear that yesterday wouldn’t be a “normal day”. We duly stocked up on all essentials and non-essentials, especially given we have houseguests. The kids wanted to go to the park in the afternoon, and we weren’t sure if that would be open either (it was). A lone Starbucks in the area was open and it did brisk business.

I don’t know if there’s regulation that states that stores ought to close on Christmas in London (given “sunday trading rules”, I wouldn’t rule that out). To me this shut down illustrates the fragility of a city with one very dominant culture (yes, London is multi-racial and multi-national, but curiously everyone shuts down on Christmas). While Bangalore, where I’m from, has few foreigners and is majority Hindu, there is some low-level multiculturalism in the city that means that the whole city doesn’t shut down on the same day (unless there are riots, that is).

It’s possibly because in India we have so many festivals that there is no one festival that is the “major festival” for everyone. So while people for whom the day is the major festival go on holiday, others for whom it isn’t that major a festival remain open for business and profit from the reduced competition. In some sense, establishments “take turns to go on holiday”. Public transport runs (albeit at reduced levels) on these holidays.

Again, all of India isn’t like this. As I mentioned at the top of the post, Delhi virtually shuts down on the day of Holi – the result of one dominant culture in that city. Similarly, the memetic fitness of the Ganesha drowning event in Mumbai is so strong that that city shuts down on the drowning days as well  (again there’s an element of hooliganism present) – though not to the extent to which London shut down yesterday.

There are a few odd people in London who don’t shut down for Christmas, though. This one photographer from the BBC goes out every Christmas to chronicle the empty streets of London.

Oxford Circus. Source: BBC

 

Somerset wanderings

So we went for a road trip. To be precise, four adults and three children rented a car and drove down to a relative’s place in Cornwall, and all the way back. And on the way back, we saw Stonehenge. Rather, tried to see it, failed, and then stumbled upon it. For more details, read on.

The premise is that my wife’s cousin and her family are visiting us, and on Friday all of us set out to my wife’s uncle’s (same “side” as the visiting cousin) house in Cornwall. The idea of driving there was that on the way, or the way back, we could “cover” some tourist attractions that were hard to do by public transport, such as Stonehenge.

There are times when I pride myself on my planning. Such as this afternoon when we were driving from Cornwall towards Stonehenge, on our way back to London. On the way to Cornwall on Friday, I had noticed that the rest stops on the “M” motorways were much better equipped than those on the “A” highways (admittedly based on one fully sampled data point each, along with signboards). Our journey from Cornwall to Stonehenge had a short stretch on an M motorway sandwiched between two A highways. And I announced a slightly early lunch break so we could take advantage of the better facilities.

A post-prandial double espresso relieved me of the severe headache caused by caffeine withdrawal symptoms, and I presently took the wheel. Ten minutes later, I had taken a wrong turn at a roundabout which meant we were back on our motorway rather than motoring way towards Stonehenge. The wife, who was sitting next to me and navigating, proudly announced that the estimated time of arrival hadn’t changed due to my mistake.

I don’t know if the estimated time of arrival changed during the next hour and three-quarters, but most of that time was spent driving through the country roads of Somerset and some surrounding counties. There were hills and valleys and grasslands and sheep. We frequently passed through beautiful forests, which retained a tinge of green despite it being winter. The roads were mostly two-lane (one in each direction), and the sceneries kept changing.

Sometimes I like to describe my wife as being my “conscience keeper”, for she quickly pulls me back when I make the sort of mistakes I normally caution people against. For example, for the last six years I’ve been lecturing about cognitive biases, and I fell right into one of them when I said “I guess missing the exit wasn’t too bad after all, since we’ve been rewarded with such beautiful scenery”. “Well”, she replied, “you can’t say that because you don’t know what you really missed (in the road not taken)”. I quickly complimented her on how smart she is and drove on.

It was windy. Occasionally it was foggy. It even rained a fair bit. And the kids were screaming in the back of the car. But it was a most pleasurable journey. By the time we got close to Stonehenge, I thought to myself that it wouldn’t matter if we couldn’t see Stonehenge – the journey itself had been worth it (I’m not sure the adults in the back of the car shared this view).

And then it turned out that I had wished for too much. I have mentioned earlier about how I pride myself on my planning abilities, such as optimisation of lunch breaks. One thing I had failed to plan on, though, was Stonehenge’s opening hours. I had only seen that the place is open till 5, not that the last entry is at 3 pm. And when we happily drove past meadows of sheep and signs warning us that tanks might be crossing the road to finally reach Stonehenge, we were politely asked to turn back by security personnel.

Not having a place to park suitably as the wife tried to find directions home, I just drove round and round a roundabout. Directions found, the next order of business was to give some tired arses a rest, and to comfort the screaming kids strapped into their child seats. I quickly pulled into the first available hard shoulder on the A303, without waiting for a designated “service area” (with toilets and restaurants and fuel). By the time we had reoriented ourselves and pulled out of there, there was a traffic backlog ahead of us.

The road dipped and then rose again. Until the dip, there was bumper-to-bumper traffic. Beyond it, I saw cars go freely. It was similar in the other direction – there was bumper-to-bumper traffic leading up to the dip. After that, there was free movement of cars.

My first thought was that there was possibly an accident there. I soon dismissed that and thought there were sheep on the road (there were plenty in the meadows around). And then someone in the back figured out why the traffic had backed up from the dip from both directions – Stonehenge was clearly visible from there, and people had been slowing down to take pictures!

So here is one such picture taken from our car, along with a few others from our trip over the last few days.

Cousin-in-law-in-law drove the “home stretch”, which I didn’t mind at all since it was mostly along motorways which I find boring. I absolutely enjoyed driving around Cornwall yesterday (though we didn’t see that much of the famed Cornish coast), and the unexpected roads of rural Somerset today.

 

Football Elo Application

This morning, I discovered the Club Elo Ratings, and promptly proceeded to analyse Liverpool FC’s performance over the years based on these ratings, and then correlated the performance by manager.

Then, playing around with the data of different clubs, I realised that there are plenty more stories to be told using this data, and they are best told by people who are passionate about their respective clubs. So the best thing I could do is to put the data out there (in a form similar to what I did for Liverpool), so that people can analyse how their clubs have performed over the years, and under different managers.

Sitting beside me as I was doing this analysis, my wife popped in with a pertinent observation. Now, she doesn’t watch football. She hates it that I watch so much football. Nevertheless, she has a strong eye for metrics. And watching me analyse club performance by manager, she asked me if I can analyse manager performance by club!

And so I’ve added that as well to the Shiny app that I’ve built. It might look a bit clunky, with two unrelate graphs, one on top of the other, but since the two are strongly related, it makes sense to have both in the same app. The managers listed in the bottom dropdown are those who have managed at least two clubs in the Premier League.

If you’re interested in Premier League football, you should definitely check out the app. I think there are some interesting insights to be gleaned (such as what I presented in this morning’s post).