Category: IIT

The Second Great Wall (of programming)

Back in 2000, I entered the Computer Science undergrad program at IIT Madras thinking I was a fairly competent coder. In my high school, I had a pretty good reputation in terms of my programming skills and had built a whole bunch of games.

By the time half the course was done I had completely fallen out of love with programming, deciding a career in Computer Science was not for me. I even ignored Kama (current diro)’s advice and went on to do an MBA.

What had happened? Basically it was a sudden increase in the steepness of the learning curve. Or that I’m a massive sucker for user experience, which the Computer Science program didn’t care for.

Back in school, my IDE of choice (rather the only one available) was TurboC, a DOS-based program. You would write your code, and then hit Ctrl+F9 to run the program. And it would just run. I didn’t have to deal with any technical issues. Looking back, we had built some fairly complex programs just using TurboC.

And then I went to IIT and found that there was no TurboC, no DOS. Most computers there had an ancient version of Unix (or worse, Solaris). These didn’t come with nice IDEs such as TurboC. Instead, you had to use vi (some of the computers were so old they didn’t even have vim) to write the code, and then compile it from outside.

Difficulties in coming to terms with vi meant that my speed of typing dropped. I couldn’t “code at the speed of thought” any more. This was the first give up moment.

Then, I discovered that C++ had now got this new set of “standard template libraries” (STL) with vectors and stuff. This was very alien to the way I had learnt C++ in school. Also I found that some of my classmates were very proficient with this, and I just couldn’t keep up with this. The effort seemed too much (and the general workload of the program was so high that I couldn’t get much time for “learning by myself”), so I gave up once  again.

Next, I figured that a lot of my professors were suckers for graphic UIs (though they denied us good UX by denying us good computers). This, circa 2001-2, meant programming in Java and writing applets. It was a massive degree of complexity (and “boringness”) compared to the crisp C/C++ code I was used to writing. I gave up yet again.

I wasn’t done with giving up yet. Beyond all of this, there was “systems programming”. You had to write some network layers and stuff. You had to go deep into the workings of the computer system to get your code to run. This came rather intuitively to most of my engineering-minded classmates. It didn’t to me (programming in C was the “deepest” I could grok). And I gave up even more.

A decade after I graduated from IIT Madras, I visited IIM Calcutta to deliver a lecture. And got educated.

I did my B.Tech. project in “theoretical computer science”, managed to graduate and went on to do an MBA. Just before my MBA, I was helping my father with some work, and he figured I sucked at Excel. “What is the use of completing a B.Tech. in computer science if you can’t even do simple Excel stuff?”, he thundered.

In IIMB, all of us bought computers with pirated Windows and Office. I started using Excel. It was an absolute joy. It was a decade before I started using Apple products, but the UX of Windows was such a massive upgrade compared to what I’d seen in my more technical life.

In my first job (where I didn’t last long) I learnt the absolute joy of Visual Basic macros for Excel. This was another level of unlock. I did some insane gymnastics in that. I pissed off a lot of people in my second job by replicating what they thought was a complex model on an Excel sheet. In my third job, I replaced a guy on my team with an Excel macro. My programming mojo was back.

Goldman Sachs’s SLANG was even better. By the time I left from there, I had learnt R as well. And then I became a “data scientist”. People asked me to use Python. I struggled with it. After the user experience of R, this was too complex. This brought back bad memories of all the systems programming and dealing with bad UX I had encountered in my undergrad. This time I was in control (I was a freelancer) so I didn’t need to give up – I was able to get all my work done in R.

The second giving up

I’ve happily used R for most of my data work in the last decade. Recently at work I started using Databricks (still write my code in R there, using sparklyr), and I’m quite liking that as well. However, in the last 3-4 months there has been a lot of developments in “AI”, which I’ve wanted to explore.

The unfortunate thing is that most of this is available only in Python. And the bad UX problem is back again.

Initially I got excited, and managed to install Stable Diffusion on my personal Mac. I started writing some OpenAI code as well (largely using R). I started tracking developments in artificial intelligence, and trying some of them out.

And now, in the last 2-3 weeks, I’ve been struggling with “virtual environments”. Each newfangled open-source AI that is released comes with its own codebase and package requirements. They are all mutually incompatible. You install one package, and you break another package.

The “solution” to this, from what I could gather, is to use virtual environments – basically a sandbox for each of these things that I’ve downloaded. That, I find, is grossly inadequate. One of the points of using open source software is to experiment with connecting up two or more of them. And if each needs to be in its own sandbox, how is one supposed to do this? And how are all other data scientists and software engineers okay with this?

This whole virtual environment mess means that I’m giving up on programming once again. I won’t fully give up – I’ll continue to use R for most of my data work (including my job), but I’m on the verge of giving up in terms of these “complex AI”.

It’s the UX thing all over again. I simply can’t handle bad UX. Maybe it’s my ADHD. But when something is hard to use, I simply don’t want to use it.

And so I’m giving up once again. Tagore was right.

Compression Stereotypes

One of the most mindblowing things I learnt while I was doing my undergrad in Computer Science and Engineering was Lempel-Ziv-Welch (LZW) compression. It’s one of the standard compression algorithms used everywhere nowadays.

The reason I remember this is twofold – firstly, I remember implementing this as part of an assignment (our CSE program at IITM was full of those), and feeling happy to be coding in C rather than in the dreaded Java (which we had to use for most other assignments).

The other is that this is one of those algorithms that I “internalised” while doing something totally different – in this case I was having coffee/ tea with a classmate in our hostel mess.

I won’t go into the algorithm here. However, the basic concept is that as and when we see a new pattern, we give it a code, and every subsequent occurrence of that pattern is replaced by its corresponding code. And the beauty of it is that you don’t need to ship a separate dictionary -the compressed code itself encapsulates it.

Anyway, in practical terms, the more the same kind of patterns are repeated in the original file, the more the file can be compressed. In some sense, the more the repetition of patterns, the less the overall “information” that the original file can carry – but that discussion is for another day.

I’ve been thinking of compression in general and LZW compression in particular when I think of stereotyping. The whole idea of stereotyping is that we are fundamentally lazy, and want to “classify” or categorise or pigeon-hole people using the fewest number of bits necessary.

And so, we use lazy heuristics – gender, caste, race, degrees, employers, height, even names, etc. to make our assumptions of what people are going to be like. This is fundamentally lazy, but also effective – in a sense, we have evolved to stereotype people (and objects and animals) because that allows our brain to be efficient; to internalise more data by using fewer bits. And for this precise reason, to some extent, stereotyping is rational.

However, the problem with stereotypes is that they can frequently be wrong. We might see a name and assume something about a person, and they might turn out to be completely different. The rational response to this is not to beat oneself for stereotyping in the first place – it is to update one’s priors with the new information that one has learnt about this person.

So, you might have used a combination of pre-known features of a person to categorise him/her. The moment you realise that this categorisation is wrong, you ought to invest additional bits in your brain to classify this person so that the stereotype doesn’t remain any more.

The more idiosyncratic and interesting you are, the more the number of bits that will be required to describe you. You are very very different from any of the stereotypes that can possibly be used to describe you, and this means people will need to make that effort to try and understand you.

One of the downsides of being idiosyncratic, though, is that most people are lazy and won’t make the effort to use the additional bits required to know you, and so will grossly mischaracterise you using one of the standard stereotypes.

On yet another tangential note, getting to know someone is a Bayesian process. You make your first impressions of them based on whatever you find out about them, and go on building a picture of them incrementally based on the information you find out about them. It is like loading a picture on a website using a bad internet connection – first the picture appears grainy, and then the more idiosyncratic features can be seen.

The problem with refusing to use stereotypes, or demonising stereotypes, is that you fail to use the grainy pictures when that is the best available, and instead infinitely wait to get better pictures. On the other hand, failing to see beyond stereotypes means that you end up using grainy pictures when more clear ones are available.

And both of these approaches is suboptimal.

PS: I’ve sometimes wondered why I find it so hard to remember certain people’s faces. And I realise that it’s usually because they are highly idiosyncratic and not easy to stereotype / compress (both are the same thing). And so it takes more effort to remember them, and if I don’t really need to remember them so much, I just don’t bother.

Ranga and Big Data

There are some meeting stories that are worth retelling and retelling. Sometimes you think it should be included in some movie (or at least a TV show). And you never tire of telling the stories.

The way I met Ranga can qualify as one such story. At the outset, there was nothing special about it – both of us had joined IIT Madras at the same time, to do a B.Tech. in Computer Science. But the first conversation itself was epic, and something worth telling again and again.

During our orientation, one of the planned events was “a visit to the facilities”, where a professor would take us around to see the library, the workshops, a few prominent labs and other things.

I remember that the gathering point for Computer Science students was right behind the Central Lecture Theatre. This was the second day of orientation and I’d already met a few classmates by then. And that’s where I found Ranga.

The conversation went somewhat like this:

“Hi I’m Karthik. I’m from Bangalore”.
“Hi I’m Ranga. I’m from Madras. What are your hobbies?”
“I play the violin, I play chess…. ”
“Oh, you play chess? Me too. Why don’t we play a blindfold game right now?”
“Er. What? What do you want to do? Now?”
“Yeah. Let’s start. e4”.
(I finally managed to gather my senses) “c5”

And so we played for the next two hours. I clearly remember playing a Sicilian Dragon. It was a hard fought game until we ended up in an endgame with opposite coloured bishops. Coincidentally, by that time the tour of the facilities had ended. And we called it a draw.

We kept playing through our B.Techs., mostly blindfold in the backbenches of classrooms. Most of the time I would get soundly thrashed. One time I remember going from our class, with the half-played game in our heads, setting it up on a board in Ranga’s room, and continued to play.

In any case, chess apart, we’ve also had a lot of nice conversations over the last 21 years. Ranga runs a big data and AI company called TheDataTeam, so I thought it would be good to record one of our conversations and share it with the world.

And so I present to you the second episode of my new “Data Chatter” podcast. Ranga and I talk about all things “big data”, data architectures, warehousing, data engineering and all that.

As usual, the podcast is available on all podcasting platforms (though, curiously, each episode takes much longer to appear on Google Podcasts after it has released. So this second episode is already there on Spotify, Apple Podcasts, CastBox, etc. but not on Google yet).

Give it a listen. Share it with whoever you think might like it. Subscribe to my podcast. And let me know what you think of it.

Social Skills Decelerator

This post is a consequence of a conversation I had with my wife this morning.

I was telling her about how a friend, who also went to IIT Madras with me, recently said something like “I’m surprised XXXXX (another mutual friend) has so much confidence in himself even though he went to <local college>. A lot of my self-confidence comes from IIT”.

“It is precisely because XXXXX went to <local college> that he has so much confidence”, my wife countered (she studied in an engineering college that is very similar to the aforementioned <local college>). And then we started talking about respective lives, and experiences, and all that.

The big insight (I’m writing about this because I think my wife is too lazy to blog about this – though it’s her idea) is that in different schools, you build reputation based on different things. In 11th and 12th standards in schools where people are mugging for JEE, for example, the primary currency for reputation is academics (both my wife and I went to such schools – different branches of the same school, to be precise).

In other schools, it’s due to extracurricular achievement. Some schools place a premium on sport. Others on gadgets and shoes. Yet others simply on good looks. And some on who you know. It’s not a finite list, except that each school has its own “portfolio” of things that help build its students’ reputations.

Coming back to my wife’s big insight, it’s that in IIT, at the prime ages of 17/18 to 21-22, the primary currency to build reputation still remains academics, and that is not the case in other colleges. And in some sense, in terms of building social skills and generally increasing one’s confidence in life, these are the prime years.

And the other thing is that academics is a zero-sum status game – there might be infinitely many “nine pointers” but there can only be one class topper. And if reputation is based on academics, it is likely to be based on relative grading. So at the prime years of your life, when you are looking to build social skills (social skills are positive sum), you are playing a zero-sum academic game for marks. You are not only not building your social skills at an age when you should, but you are actively destroying them by indulging in too many zero-sum status games.

The other corollary, of course, is that by indulging in status games where there can only be one winner, we are all (most of us at least) effectively turning ourselves into losers.  By being in an environment where reputation is built only on one (zero sum) dimension, our confidence is shattered.

Some of us come back from this shattered confidence, and at some point in life (albeit belatedly) start building social skills. Others, unfortunately, never recover.

My wife is right – people are not “confident despite not going to IIT”. They are “confident BECAUSE THEY DIDN’T GO TO IIT”!

 

Does alma matter?

I just spent the holiday afternoon massively triggering myself by watching the just-released Netflix documentary Alma Matters, about life in IIT Kharagpur. Based on the trailer itself, I thought I could relate to it, thanks to my four years at IIT Madras. And so my wife and I sat, and spent three hours on the documentary. Our daughter was with us for the first half hour, and then disappeared for reasons mentioned below.

I have too many random thoughts in my head right now, so let me do this post in bullet points.

  • I have always had mixed feelings about my time at IIT Madras. On the one hand, I found it incredibly depressing. Even now, the very thought of going to Chennai depresses me. On the other, I have a lot of great memories from there, and built a strong network.

    Now that I think of it, having watched the documentary, a lot of those “great memories” were simply about me making the best use of a bad situation I was in. I don’t think I want to put myself, or my daughter, through that kind of an experience again

  • My basic problem at IIT was that I just couldn’t connect with most people there. I sometimes joke that I couldn’t connect with 80% of the people there, but remain in touch with the remaining 20%. And that is possibly right.

    The problem is that most people there were either “too fighter”, always worried about and doing academics, or “too given up”, not caring about anything at all in life. I couldn’t empathise with either and ended up having a not so great time.

  • My wife intently watched the show with me, even though she got bored by the end of the first episode. “It’s all so depressing”, she kept saying. “Yes, this is how life was”, I kept countering.

    And then I think she caught the point. “Take out the cigarettes and alcohol, and this is just like school. Not like college at all”, she said. And I think that quite sums up IIT for me. We were adults (most of us for most of the time – I turned eighteen a few months after I joined), but were treated like children for the most part. And led our lives like children in some ways, either being too regimented, or massively rebelling.

    “Now I can see why people don’t grow up when they go to IIT”, my wife said. After I had agreed, she went on, “this applies to you as well. You also haven’t grown up”. I couldn’t counter.

  • The “maleness” of the place wasn’t easy to notice. After one scene, my wife mentioned that we spend such a long time in the prime years of our lives dealing only with other men, that it is impossible to have normal relationships later on. It’s only a few who have come from more liberal backgrounds, or who manage to unlearn the IIT stuff, who manage to have reasonably normal long-term relationships.
  • The maleness of IITs was also given sort of ironic treatment by the show. There is a segment in the first episode about elections, which shows a female candidate, about how girls have a really bad time at IIT due to the massively warped sex ratio (in my time it was 16:1), and so it is difficult for girls to get respect.

    And then that turned out to be the “token female segment” in the show, as girls were all but absent in the rest of the three hours. That girls hardly made it to the show sort of self-reinforced the concept that girls aren’t treated well at IITs,

  • After intently watching for half an hour, my daughter asked, “if this is a movie about IIT, why aren’t you in it?”. I told her that it’s about a different IIT. “OK fine. I’ll watch it when they make a movie about your IIT”, she said and disappeared.
  • While the second and third episodes of the show were too long-drawn and sort of boring, I did manage to finish the show end-to-end in one sitting, which has to say something about it being gripping (no doubt to someone like me who could empathise with parts of it).
  • Finally, watch this trailer to the show. Watch what the guy says about people with different CGPA ranges.
  • He talks about “respect 8 pointers and don’t like 9 pointers”. That sort of made me happy since I finished (I THINK) with a CGPA of 8.9

If you’re from an IIT, you are likely to empathise with the show. If you are close to someone from IIT, you might appreciate them better when you watch it. Overall three episodes is too long-drawn. The first episode is a good enough gist of life at IIT.

And yeah, trigger warnings apply.

Not all minutes are equal

I seem to be on a bit of a self-reflection roll today. Last night I had this insight about my first ever job (which I’ve  said I’ll write about sometime). This morning, I wrote about how in my 15 years of professional life I’ve become more positive sum, and stopped seeing everything as a competition.

This blogpost is about an insight I realised a long time back, but haven’t been able to quantify until today. The basic concept, which I might have written about in other ways, is that “not all minutes are created equal”.

Back when I was in IIT, I wasn’t particularly happy. With the benefit of hindsight, I think my mental illness troubles started around that time. One of the mindsets I had got into then (maybe thanks to the insecurity of having just taken a highly competitive, and status-seeking, exam) was that I “need to earn the right to relax”.

In the two years prior to going to IIT, it had been drilled into my head that it was wrong to relax or have fun until I had “achieved my goals”, which at that point in time was basically about getting into IIT. I did have some fun in that period, but it usually came with a heavy dose of guilt – that I was straying from my goal.

In any case, I got into IIT and the attitude continued. I felt that I couldn’t relax until I had “finished my work”. And since IIT was this constant treadmill of tests and exams and assignments and grades, this meant that this kind of “achievement” of finishing work didn’t come easily. And so I went about my life without chilling. And was unhappy.

The problem with IIT was that it was full of “puritan toppers“. Maybe because the exam selected for extreme fighters, people at IIT largely belonged to one of two categories – those that continued to put extreme fight, and those who completely gave up. And thanks to this, the opinion formed in my head that if I were to “have fun before finishing my work” I would join the ranks of the latter.

IIMB was different – the entrance exam itself selected for studness, and the process that included essays and interviews meant that people who were not necessary insane fighters made it. You had a rather large cohort of people who managed to do well academically without studying much (a cohort I happily joined. It was definitely a good thing that there were at least two others in my hostel wing who did rather well without studying at all).

And since you had a significant number of people who both had fun and did well academically, it impacted me massively in terms of my attitude. I realised that it was actually okay to have fun without “having finished one’s work”. The campus parties every Saturday night contributed in no small measure in driving this attitude.

That is an attitude I have carried with me since. And if I were to describe it simply, I would say “not all minutes are created equal”. Let me explain with a metaphor, again from IIMB.

The favourite phrase of Dr. Prem Chander, a visiting professor who taught us Mergers and Acquisitions, was “you can never eliminate risk. You can only transfer it to someone who can handle it better”. In terms of personal life and work, it can be translated to “you can never eliminate work. However, you can transfer it to a time when you can do it better”.

Earlier this evening I was staring at the huge pile of vessels in my sink (we need to get some civil work done before we can buy a dishwasher, so we’ve been putting off that decision). I was already feeling tired, and in our domestic lockdown time division of household chores, doing the dishes falls under my remit.

My instinct was “ok let me just finish this off first. I can chill later”. This was the 2002 me speaking. And then a minute later I decided “no, but I’m feeling insanely tired now having just cooked dinner and <… > and <….. >. So I might as well chill now, and do this when I’m in a better frame of mind”.

The minute when I had this thought is not the same as the minute an hour from now (when I’ll actually get down to doing this work). In the intervening time, I’ve would’ve had a few drinks,  had dinner,  written this blogpost, hung out with my daughter as she’s going to bed, and might have also caught some IPL action. And I foresee that I will be in a far better frame of mind when I finally go out to do the dishes, than I was when I saw the pile in the sink.

It is important to be able to make this distinction easily. It is important to recognise that in “real life” (unlike in entrance exam life) it is seldom that “all work will be done”. It is important to realise that not all minutes are made equal. And some minutes are better for working than others, and to optimise life accordingly.

If you’ve gotten this far, you might think this is all rather obvious stuff, but having been on the other side, let me assure you that it isn’t. And some people can take it to an extreme extreme, like the protagonists of Ganesha Subramanya who decide that they will not interact with women until they’ve achieved something!

Unbundling Higher Education

In July 2004, I went to Madras, wore fancy clothes and collected a laminated piece of paper. The piece of paper, formally called “Bachelor of Technology”, certified three things.

First, it said that I had (very likely) got a very good rank in IIT JEE, which enabled me to enrol in the Computer Science B.Tech. program at IIT Madras.  Then, it certified that I had attended a certain number of lectures and laboratories (equivalent to “180 credits”) at IIT Madras. Finally, it certified that I had completed assignments and passed tests administered by IIT Madras to a sufficient degree that qualified me to get the piece of paper.

Note that all these three were necessary conditions to my getting my degree from IIT Madras. Not passing IIT JEE with a fancy enough rank would have precluded me from the other two steps in the first place. Either not attending lectures and labs, or not doing the assignments and exams, would  have meant that my “coursework would be incomplete”, leaving me ineligible to get the degree.

In other words, my higher education was bundled. There is no reason that should be so.

There is no reason that a single entity should have been responsible for entry testing (which is what IIT-JEE essentially is), teaching and exit testing. Each of these three could have been done by an independent entity.

For example, you could have “credentialing entities” or “entry testing entities”, whose job is to test you on things that admissions departments of colleges might test you on. This could include subject tests such as IIT-JEE, or aptitude tests such as GRE, or even evaluations of extra-curricular activities, recommendation letters and essays as practiced in American universities.

Then, you could have “teaching entities”. This is like the MOOCs we already have. The job of these teaching entities is to teach a subject or set of subjects, and make sure you understood your stuff. Testing whether you had learnt the stuff, however, is not the job of the teaching entities. It is also likely that unless there are superstar teachers, the value of these teaching entities comes down, on account of marginal cost pricing, close to zero.

To test whether you learnt your stuff, you have the testing entities. Their job is to test whether your level of knowledge is sufficient to certify that you have learnt a particular subject.  It is well possible that some testing entities might demand that you cleared a particular cutoff on entry tests before you are eligible to get their exit test certificates, but many others may not.

The only downside of this unbundling is that independent evaluation becomes difficult. What do you make of a person who has cleared entry tests  mandated by a certain set of institutions, and exit tests traditionally associated with a completely different set of institutions? Is the entry test certificate (and associated rank or percentile) enough to give you a particular credential or should it be associated with an exit test as well?

These complications are possibly why higher education hasn’t experimented with any such unbundling so far (though MOOCs have taken the teaching bit outside the traditional classroom).

However, there is an opportunity now. Covid-19 means that several universities have decided to have online-only classes in 2019-20. Without the peer learning aspect, people are wondering if it is worth paying the traditional amount for these schools. People are also calling for top universities to expand their programs since the marginal cost is slipping further, with the backlash being that this will “dilute” the degrees.

This is where unbundling comes into play. Essentially anyone should be able to attend the Harvard lectures, and maybe even do the Harvard exams (if this can be done with a sufficiently low marginal cost). However, you get a Harvard degree if and only if you have cleared the traditional Harvard admission criteria (maybe the rest get a diploma or something?).

Some other people might decide upon clearing the traditional Harvard admission criteria that this credential itself is sufficient for them and not bother getting the full degree. The possibilities are endless.

Old-time readers of this blog might remember that I had almost experimented with something like this. Highly disillusioned during my first year at IIT, I had considered dropping out, reasoning that my JEE rank was credential enough. Finally, I turned out to be too much of a chicken to do that.

Night trains

In anticipation of tonight’s Merseyside Derby, I was thinking of previous instances of this fixture at Goodison Park. My mind first went back to the game in the 2013-14 season, which was a see-saw 3-3 draw, with the Liverpool backline being incredibly troubled by Romelu Lukaku, and Daniel Sturridge scoring with a header immediately after coming on to make it 3-3 (and Joe Allen had missed a sitter earlier when Liverpool were 2-1 up).

I remember my wife coming back home from work in the middle of that game, and I didn’t pay attention to her until it was over. She wasn’t particularly happy about that, but the intense nature of the game gave me a fever (that used to happen often in the 2013-14 and 2008-9 seasons).

Then I remember Everton winning 3-0 once, though I don’t remember when that was (googling tells me that was in the 2006-7 season, when I was already a Liverpool fan, but not watching regularly).

And then I started thinking about what happened to this game last season, and then remembered that it was a 0-0 draw. Incidentally, it was on the same day that I travelled to Liverpool – I had a ticket for an Anfield Tour the next morning.

I now see that I had written about getting to Liverpool after I got to my hotel that night. However, I haven’t written about what happened before that. My train from Euston was around 8:00 pm. I remember leaving home (which was in Ealing) at around 6 or so, and then taking two tubes (Central changing to Victoria at Oxford Circus) to get to Euston. And then buying chewing gum and a bottle of water at Marks and Spencer while waiting for my train.

I also remember that while leaving home that evening, I was scared. I was psyched out. It wasn’t supposed to be that way. This was a trip to Liverpool I had been wanting to make for the best part of 14 years. I had kept putting it off during my stay in London until I knew that I was going to move out of London in two weeks’ time. Liverpool were having a great season (they would go on to win the Champions League, and only narrowly lose the Premiser League title).

I was supposed to be excited. Instead I was nervous. My nerve possibly settled only after I was seated in the train that evening.

Thinking about it, I basically hate night trains (well, this wasn’t an overnight train, but it started late in the evening). I hate night buses as well. And this only applies to night trains and buses that take me away from my normal place of residence – starting towards “home” late in the night never worries me.

This anxiety possibly started when I was in IIT Madras. I remember clearly then that I used to sleep comfortably without fail while travelling from Madras to Bangalore, but almost always never slept or only slept fitfully when travelling in the opposite direction. While in hindsight it all appears fine, I never felt particularly settled when I was at IITM.

And consequently, anything that reminds me of travelling to IITM psyches me out. I always took the night train while travelling there, and the anxiety would start on the drive to the railway station. Even now, sometimes, I get anxious while taking that road late in the evening.

Then, taking night trains has been indelibly linked to travelling to Madras, and something that I’ve come to fear as well. While I haven’t taken a train in India since 2012, my experience with the trip to Liverpool last year tells me that even non-overnight night trains have that effect on me.

And then, of course, there is the city of Chennai as well. The smells of the city after the train crosses Basin Bridge trigger the first wave of anxiety. Stepping out of the railway station and the thought of finding an autorickshaw trigger the next wave (things might be different now with Uber/Ola, but I haven’t experienced that).

The last time I went to Chennai was for a close friend’s wedding in 2012. I remember waking up early on the day of the wedding and then having a massive panic attack. I spent long enough time staring at the ceiling of my hotel room that I ended up missing the muhurtham.

I’ve made up my mind that the next time I have to go to Chennai, I’ll just drive there. And for sure, I’m not going to take a train leaving Bangalore in the night.

Cookbooks, Code and College

Why Is This Interesting, a fascinating daily newsletter I subscribe to, has this edition on code and cookbooks. The basic crib here is that most coding books teach you to code as if you were trying to become a professional coder, rather than trying to teach you to code as an additional life skill.

This, the author Noah Brier remarks, is quite unlike how most cookbooks teach you to cook, where there is absolutely no pretence of trying to turn you into a professional cook. Cookbooks know that most people who want to learn to cook simply want to cook for themselves or their families, so professional level learning is not required. This, however, is not the case with books on coding.

In fact, this pretty much explains why I completely fell out of love with coding during my undergrad in Computer Science. I remember being rather excited in 2000, when I got an entrance exam score good enough to get admitted to the Computer Science program at IIT Madras. I had learnt to code only two years before, but I’d taken on to it rather well, and had quickly built a reputation of being one of the better coders in school.

And then the four year program in Computer Science sucked out all the love I had for coding. This cooking-code post reminded me why – basically most professors in my department assumed that all of us wanted to be academics and taught us that way. This wasn’t an unfair assumption, since 17 of the 22 of us who graduated in 2004 either immediately or in a couple of years went to grad school.

However, the approach of teaching that assumed that you would be an expert or an academic meant a paradigm that made it incredibly hard to learn unless you were insanely motivated.

For example, the fourth year B.Tech. project was almost always supposed to be a “work of research” that would turn into a paper (or dozen). There was a lot of theory all round (I didn’t mind parts of it, like some bits of algorithm analysis, but most of it was boring). The course was heavy in terms of assignments, which you can argue was a practical concept, but the way the assignments were done by most people meant that the bar was rather academic.

And that meant that someone like me, who didn’t want to be “an engineer” to begin with, but had entered with a love for coding, quickly fell out of love with the field itself. In hindsight, given the way I was taught, I’m not surprised that my first option upon exit was to go to business school, and it would be at least five years later that I would begin to appreciate that I had an aptitude for code.

(Interestingly, business school was different. Nobody assumed anybody would become an academic, so the teaching was far more palatable.)

The Base Rate in Hitting on People

Last week I met a single friend at a bar. He remarked that had I been late, or not turned up at all, he would have seriously considered chatting up a couple of women at the table next to ours.

This friend has spent considerable time in several cities. The conversation moved to how conducive these cities are for chatting up people, and what occasions are appropriate for chatting up. In Delhi, for example, he mentioned that you never try and chat up a strange woman – you are likely to be greeted with a swap.

In Bombay, he said, it depends on where you chat up. What caught my attention was when he mentioned that in hipster cafes, the ones that offer quinoa bowls and vegan smoothies, it is rather normal to chat up strangers, whether you are doing so with a romantic intent or not. One factor he mentioned was the price of real estate in Bombay which means most of these places have large “communal tables” that encourage conversation.

The other thing we spoke about how the sort of food and drink such places serve create a sort of “brotherhood” (ok not appropriate analogy when talking about chatting up women), and that automatically “qualifies” you as not being a creep, and your chatting up being taken up seriously.

This got me thinking about the concept of “base rates” or “priors”. I spent the prime years of my youth in IIT Madras, which is by most accounts a great college, but where for some inexplicable reason, not too many women apply to get in. That results in a rather lopsided ratio that you would more associate with a dating app in India rather than a co-educational college.

In marketing you have the concept of a “qualified lead”. When you randomly call a customer to pitch your product there is the high chance that she will hang up on you. So you need to “prime” the customer to expect your call and respond positively. Building your brand helps. Also, doing something that gauges the customer’s interest before the call, and calling only once the interest is established, helps.

What you are playing on here in marketing is is the “base rate” or the “prior” that the customer has in her head. By building your brand, you automatically place yourself in a better place in the customer’s mind, so she is more likely to respond positively. If, before the call, the customer expects to have a better experience with you, that increases the likelihood of a positive outcome from the call.

And this applies to chatting up women as well. The lopsided ratio at IIT Madras, where I spent the prime of my youth, meant that you started with a rather low base rate (the analogy with dating apps in India is appropriate). Consequently, chatting up women meant that you had to give an early signal that you were not a creep, or that you were a nice guy (the lopsided ratio also turns most guys there into misogynists, and not particularly nice. This is a rather vicious cycle). Of course, you could build your brand with grades or other things, but it wasn’t easy.

Coming from that prior, it took me a while to adjust to situations with better base rates, and made me hesitant for a long time, and for whatever reason I assumed I was a “low base rate” guy (I’m really glad, in hindsight, that my wife “approached” me (on Orkut) and said the first few words. Of course, once we’d chatted for a while, I moved swiftly to put her in my “basket”).

Essentially, when we lack information, we stereotype someone with the best information we have about them. When the best information we have about them is not much, we start with a rather low prior, and it is upon them to impress us soon enough to upgrade them. And upgrading yourself in someone’s eyes is not an easy business. And so you should rather start from a position where the base rate is high enough.

And this “upgrade” is not necessarily linear – you can also use this to brand yourself in the axes that you want to be upgraded. Hipster cafes provide a good base rate that you like the sort of food served there. Sitting in a hipster cafe with a MacBook might enhance your branding (increasingly, sitting in a cafe with a Windows laptop that is not a Surface might mark you out as an overly corporate type). Political events might help iff you are the overly political type (my wife has clients who specify the desired political leadings of potential spouses). Caste groups on Orkut or Facebook might help if that is the sort of thing you like. The axes are endless.

All that matters is that whatever improved base rate you seek to achieve by doing something, the signal you send out needs to be credible. Else you can get downgraded very quickly once you’ve got the target’s attention.