Tag: CAT

Speed, Accuracy and Shannon’s Channel Coding Theorem

I was probably the CAT topper in my year (2004) (they don’t give out ranks, only percentiles (to two digits of precision), so this is a stochastic measure). I was also perhaps the only (or one of the very few) person to get into IIMs that year despite getting 20 questions wrong.

It had just happened that I had attempted far more questions than most other people. And so even though my accuracy was rather poor, my speed more than made up for it, and I ended up doing rather well.

I remember this time during my CAT prep, where the guy who was leading my CAT factory once suggested that I was making too many errors so I should possibly slow down and make fewer mistakes. I did that in a few mock exams. I ended up attempting far fewer questions. My accuracy (measured as % of answers I got wrong) didn’t change by much. So it was an easy decision to forget above accuracy and focus on speed and that served me well.

However, what serves you well in an entrance exam need not necessarily serve you well in life. An exam is, by definition, an artificial space. It is usually bounded by certain norms (of the format). And so, you can make blanket decisions such as “let me just go for speed”, and you can get away with it. In a way, an exam is a predictable space. It is a caricature of the world. So your learnings from there don’t extend to life.

In real life, you can’t “get away with 20 wrong answers”. If you have done something wrong, you are (most likely) expected to correct it. Which means, in real life, if you are inaccurate in your work, you will end up making further iterations.

Observing myself, and people around me (literally and figuratively at work), I sometimes wonder if there is a sort of efficient frontier in terms of speed and accuracy. For a given level of speed and accuracy, can we determine an “ideal gradient” – on which way a person needs to move in order to make the maximum impact?

Once in a while, I take book recommendations from academics, and end up reading (rather, trying to read) academic books. Recently, someone had recommended a book that combined information theory and machine learning, and I started reading it. Needless to say, within half a chapter, I was lost, and I had abandoned the book. Yet, the little I read performed the useful purpose of reminding me of Shannon’s channel coding theorem.

Paraphrasing, what it states is that irrespective of how noisy a channel is, using the right kind of encoding and redundancy, we will be able to predictably send across information at a certain maximum speed. The noisier the channel, the more the redundancy we will need, and the lower the speed of transmission.

In my opinion (and in the opinions of several others, I’m sure), this is a rather profound observation, and has significant impact on various aspects of life. In fact, I’m prone to abusing it in inexact manners (no wonder I never tried to become an academic).

So while thinking of the tradeoff between speed and accuracy, I started thinking of the channel coding theorem. You can think of a person’s work (or “working mind”) as a communication channel. The speed is the raw speed of transmission. The accuracy (rather, the lack of it) is a measure of noise in the channel.

So the less accurate someone is, the more the redundancy they require in communication (or in work). For example, if you are especially prone to mistakes (like I am sometimes), you might need to redo your work (or at least a part of it) several times. If you are the more accurate types, you need to redo less often.

And different people have different speed-accuracy trade-offs.

I don’t have a perfect way to quantify this, but maybe we can think of “true speed of work” by dividing the actual speed in which someone does a piece of work by the number of iterations they need to get it right.  OK it is not so straightforward (there might be other ways to build redundancy – like getting two independent people to do the same thing and then tally the numbers), but I suppose you get the drift.

The interesting thing here is that the speed and accuracy is not only depend on the person but the nature of work itself. For me, a piece of work that on average takes 1 hour has a different speed-accuracy tradeoff compared to a piece of work that on average takes a day (usually, the more complicated and involved a piece of analysis, the more the error rate for me).

In any case, the point to be noted is that the speed-accuracy tradeoff is different for different people, and in different contexts. For some people, in some contexts, there is no point at all in expecting highly accurate work – you know they will make mistakes anyways, so you might as well get the work done quickly (to allow for more time to iterate).

And in a way, figuring out speed-accuracy tradeoffs of the people who work for you is an important step in getting the best out of them.

 

The Business Standard is innumerate

I guess there is not that much information in the headline here – claiming that a bunch of journalists and editors are innumerate is like saying that the sky is blue. You would be hard-pressed to find journalists and editors who can actually parse numbers, though I must mention that I’ve been lucky enough to work with a few editors who actually understand arithmetic!

So what happened today? Basically in today’s front page, BS journalists (one Vinay Umarji in particular) and editors have displayed an utter lack of understanding on how relative grading and percentiles work. The context is CAT results, which came out yesterday.

(I’ve put a scan since the online version is behind a paywall).

There is information in saying that “number of candidates scoring 100 percentile is lowest in six years”, and the information I take out of that is that the number of test takers this year is the lowest in six years.

And for four of those six years, the numbers were inflated, since double the number of people who were supposed to get 100 percentile actually got 100 percentile. Since CAT percentiles are given to two decimal places, you get 100 percentile if you are in the top 0.005% of all candidates who took the exam. Or – if your “percentile” is higher than 99.995, it gets rounded up to 100.

For three years in the middle, the CAT administrators (usually they’re Quantitative Methods professors at IIMs), for whatever reason, rounded up everyone who got a percentile higher than 99.990 to 100. I’d written about that in my article for Mint three years back.

Coming back, CAT is an exam that follows relative grading. All that someone  has got “100 percentile” means is that they are within the top 0.005% of all candidates who wrote the exam. So if more candidates write the exam, more people will get “100 percentile”. In my time, for example (CAT 2003-4) some 1.3 lakh people had written the exam, so 7 of us got “100 percentile”. Nowadays the number of test takers has gone up, so more people get that score.

And then I found the rest of the article funny in a way as well, trying to do some sort of sociological analysis of the backgrounds of the people who had scored highly in the exam.

PS: The graph doesn’t give out much information (and I don’t know why the 2019 data point is missing there), but I guess it’s been put in there to make the journalists and editors seem more numerate than they are.

 

vaDe for meat and tithi ooTas

The story goes that the humble medu vaDe was invented a couple of millennia back when Brahmins went veggie (to compete with Buddhism and Jainism), and needed a source of protein to replace meat. The vaDe, packed with urad dal and deep fried, can perhaps be described as the perfect keto snack, especially considering that it’s eaten with coconut chutney.

So the humble vaDe is a fixture at lunch during death ceremonies. A standard feature of Kannadiga Brahmin death ceremonies is the “feeding of the brahmins”. These are no ordinary brahmins – they are special brahmins who are part of the ceremony where one represents God and the other represents the deceased in whose name the death ceremony (colloquially called ‘tithi’) is being performed.

Given that these brahmins have fasted before the meal and will fast the rest of the day (this is all in theory, of course), they need to be fed nutritious meals, and what is a better source of long-lasting nutrition than the humble vaDe? The vaDe has become so synonymous with tithis that in Karnataka it is symbolic of death ceremonies, and not prepared on auspicious occasions. The phrase “I’ll eat vaDe in your name” can be considered as a mild death threat, for example.

Right from childhood I’ve always wanted some crunchy stuff to eat with my rice. Back then, my parents would ensure that our house was well-stocked with crunchies such as Congress peanuts, nippaT, mixture, etc., which I would eat along with my rice. Occasionally my mother would make happaLa (fried paapaD). Back when was at IITM, I would make the decision on whether to eat chapati or rice for lunch based on the availability of happaLa – I’m such a sucker for crunchies with rice.

Death ceremonies being solemn occasions, however, crunchies aren’t made. It’s taboo to serve happaLa during these kind of ceremonies (despite the protein that packs, too). The occasional lunch can be eaten without crunchies, but if you have to eat tithi ooTa on a regular basis, some “adjustment” has to be made?

The epiphany happened on the 13th of April 2007, at Paschimavaahini near Mysore. My father had passed away two days earlier after a prolonged illness, and after having cremated his remains, we had gone to Paschimavaahini to dunk his ashes in the Kaveri. This was my first exposure to performing death ceremonies, and I found it so unpleasant that I only performed a limited subset of them when my mother passed away in 2009, and gave up altogether on performing my parents’ annual death ceremonies in 2012 after a series of unpleasant experiences.

That day in 2007, however, was when I discovered the utility of the vaDe as the crunchy during tithi ooTas (ooTa is Kannada for meal). Chutney had also been served, and some vaDes were served at the beginning of the meal along with the rice. You break off a piece of vaDe, dip it in the chutney, and then pick it up with a morsel of huLianna (sambar rice) or saaranna (rasam rice), and you get both crunchies and enhanced taste. And that has formed my template for tithi ooTas (which I’m forced to occasionally attend, though I don’t perform tithis myself) ever since.

Yet another epiphany happened last month, when I was at one such tithi ooTa (in memory of my cousin’s grandmother). Sometime between the initial epiphany and this, I had started eating meat, and this was a key component going into this epiphany.

As I was polishing off huLianna with vaDe and chutney at my cousin’s grandmother’s tithi, the process seemed rather familiar. Considering that I don’t eat too many tithi ooTas, this was surprising. And then it struck me that the way I was eating was exactly the same as the way one eats meat with rice (while eating with fingers in South Indian style). You break off the piece of meat, and pick it up with a morsel of rice (mixed with whatever), and put them together into your mouth.

That was when I got reminded of the vaDe replacing meat in the Brahmin diet. It all seemed to fit in now. Even the way it is traditionally consumed (nothing gets more traditional than a tithi ooTa) is the same!

Tailpiece: Speaking of tithi ooTas, there’s a saying that goes “tie up the cat and perform the tithi”. So I was quite amused when I saw a cat polish off a rather large mound of rice outside a “tithi hotel” yesterday. The rice had been put out on a plantain leaf, evidently deliberately for the cat. From getting tied up during tithis to getting mounds of rice, the cat has come a long way.

Analyzing IIMA Admissions

In response to an RTI query, IIM Ahmedabad has disclosed the cutoff percentiles across various categories for getting a seat in IIMA. Before we analyze further, there are two points to be noted. Firstly, what has been disclosed is the “minimum cutoff percentile”, which means that at least one student with that percentile score was admitted to IIMA in that year. It gives us no information on the “average percentile score” for admitted students belonging to that particular category. Secondly, CAT percentile is only one of the criteria used for admission into IIMs. A response by IIM Bangalore a few years back to an RTI query showed that the CAT percentile has only a 15% weight in the entire admission process (the rest going to 10th and 12th standard board exam scores, college CGPA, performance in interviews and the like). Given these two conditions, we should look at the following analysis with a bit of salt.

First up, here is a graph showing the minimum percentile among admitted students of various categories over the years:

Rplot

 

There are a few things that stand out from this graph:

1. The cutoff percentage for general category students has been consistently high. Despite a comprehensive set of factors being used for admissions, if you belong to the general category, a high CAT percentile is a necessary condition to join IIMA

2. Reservations for Other Backward Classes (OBCs) happened in a phased manner. In the first year (2008) only about 5% of the seats were reserved for students from these classes. This has been gradually ramped up to the statutory 27.5%. In the initial years, after reservation for OBCs was imposed, commentators mentioned that their cutoff was not much lower than that for general category students and so there would be no dilution in quality. However, the data above shows that it was a function of the extent of reservation that the cutoffs were similar. If CAT percentile is to be taken as a general statement of an MBA student’s quality, reservation for OBCs has definitely led to dilution.

3. There is massive volatility in cutoffs for SC/STs. It must be noted here that the percentile scores are national – percentiles for students from different categories are not disclosed separately. It seems like the quality of applicants belonging to SC/ST categories has been varying significantly over the years. One year (2008/09) SC/ST students need to be in top 10% of all applicants to gain admission into IIMA. In another (2013) students belonging to ST category need to beat only 40% of all applicants to get in! This is bizarre, and it brings us to..

4. Students from ST category getting admission with 40 percentile in CAT in 2013 is plain absurd. What makes it more absurd is that more than half the students who attempted CAT in 2013 got ZERO or less (remember that CAT has negative marking). Maybe there was a real dearth of applicants from the ST category last year but what this tells us is that someone who got an overall negative score in CAT got admission into IIMA last year. Actually this is beyond bizarre.

5. Time for a personal anecdote. Close to 20 out of the 180 odd people who started at IIM Bangalore with me (2004-06) did not make it to the second year, based on their performance in the first year. About half of those were put on a “slow track programme” and finished their MBA in three years. The other ten did so badly they were asked to repeat the first year in full, without concessions. From what I remember all of them eventually dropped out. A large proportion of these twenty who did not make it past the first year belonged to SC/ST categories. I must also mention here that there was a significant number of students from these categories that did rather well and finished close to the top of the batch.

While it might be seen as an act of nobility to give admission in a premier college to someone with a low score but from a historically underprivileged background, the impact on their careers must also be taken into account. All said and done, the flagship course in IIMs is a rather tough course, and it is not difficult to fall behind. What is the use of giving someone admission only for him to fail and eventually drop out? Would he not have been better off continuing in his pre-MBA job rather than having his career disrupted by admission to a premier institute and subsequent failure?

All this said, it would make sense for someone in an IIM (a professor involved in admissions, perhaps) to do an analysis of correlation of CAT scores with performance at IIMs (I understand that one of the reasons the weight of CAT score was  reduced was that one such study revealed CAT score was less of a predictor of IIM performance than high school and undergraduate scores). An analysis such as that might reveal that there is an absolute lower cutoff in terms of performance in CAT such that students scoring lower are extremely unlikely to do well. It might give a case for reassessment of the affirmative action policies.