Structures of professions and returns to experience

I’ve written here a few times about the concept of “returns to experience“. Basically, in some fields such as finance, the “returns to experience” is rather high. Irrespective of what you have studied or where, how long you have continuously been in the industry and what you have been doing has a bigger impact on your performance than your way of thinking or education.

In other domains, returns to experience is far less. After a few years in the profession, you would have learnt all you had to, and working longer in the job will not necessarily make you better at it. And so you see that the average 15 years experience people are not that much better than the average 10 years experience people, and so you see salaries stagnating as careers progress.

While I have spoken about returns to experience, till date, I hadn’t bothered to figure out why returns to experience is a thing in some, and only some, professions. And then I came across this tweetstorm that seeks to explain it.

Now, normally I have a policy of not reading tweetstorms longer than six tweets, but here it was well worth it.

It draws upon a concept called “cognitive flexibility theory”.

Basically, there are two kinds of professions – well-structured and ill-structured. To quickly summarise the tweetstorm, well-structured professions have the same problems again and again, and there are clear patterns. And in these professions, first principles are good to reason out most things, and solve most problems. And so the way you learn it is by learning concepts and theories and solving a few problems.

In ill-structured domains (eg. business or medicine), the concepts are largely the same but the way the concepts manifest in different cases are vastly different. As a consequence, just knowing the theories or fundamentals is not sufficient in being able to understand most cases, each of which is idiosyncratic.

Instead, study in these professions comes from “studying cases”. Business and medicine schools are classic examples of this. The idea with solving lots of cases is NOT that you can see the same patterns in a new case that you see, but that having seen lots of cases, you might be able to reason HOW to approach a new case that comes your way (and the way you approach it is very likely novel).

Picking up from the tweetstorm once again:

 

It is not hard to see that when the problems are ill-structured or “wicked”, the more the cases you have seen in your life, the better placed you are to attack the problem. Naturally, assuming you continue to learn from each incremental case you see, the returns to experience in such professions is high.

In securities trading, for example, the market takes very many forms, and irrespective of what chartists will tell you, patterns seldom repeat. The concepts are the same, however. Hence, you treat each new trade as a “case” and try to learn from it. So returns to experience are high. And so when I tried to reenter the industry after 5 years away, I found it incredibly hard.

Chess, on the other hand, is well-structured. Yes, alpha zero might come and go, but a lot of the general principles simply remain.

Having read this tweetstorm, gobbled a large glass of wine and written this blogpost (so far), I’ve been thinking about my own profession – data science. My sense is that data science is an ill-structured profession where most practitioners pretend it is well-structured. And this is possibly because a significant proportion of practitioners come from academia.

I keep telling people about my first brush with what can now be called data science – I was asked to build a model to forecast demand for air cargo (2006-7). The said demand being both intermittent (one order every few days for a particular flight) and lumpy (a single order could fill up a flight, for example), it was an incredibly wicked problem.

Having had a rather unique career path in this “industry” I have, over the years, been exposed to a large number of unique “cases”. In 2012, I’d set about trying to identify patterns so that I could “productise” some of my work, but the ill-structured nature of problems I was taking up meant this simply wasn’t forthcoming. And I realise (after having read the above-linked tweetstorm) that I continue to learn from cases, and that I’m a much better data scientist than I was a year back, and much much better than I was two years back.

On the other hand, because data science attracts a lot of people from pure science and engineering (classically well-structured fields), you see a lot of people trying to apply overly academic or textbook approaches to problems that they see. As they try to divine problem patterns that don’t really exist, they fail to recognise novel “cases”. And so they don’t really learn from their experience.

Maybe this is why I keep saying that “in data science, years of experience and competence are not correlated”. However, fundamentally, that ought NOT to be the case.

This is also perhaps why a lot of data scientists, irrespective of their years of experience, continue to remain “junior” in their thinking.

PS: The last few paragraphs apply equally well to quantitative finance and economics as well. They are ill-structured professions that some practitioners (thanks to well-structured backgrounds) assume are well-structured.

The World After Overbooking

Why do you think you usually have to wait so much to see a doctor, even when you have an appointment? It is because doctors routinely overbook.

You can think of a doctor’s appointment as being a free option. You call up, give your patient number, and are assigned a slot when the doctor sees you. If you choose to see the doctor at that time, you get the doctor’s services, and then pay for the service. If you choose to not turn up, the doctor’s time in that slot is essentially wasted, since there is nobody else to see then. The doctor doesn’t get compensated for this as well.

In order to not waste their time, thus, doctors routinely overbook patients. If the average patient takes fifteen minutes to see, they give appointments once every ten minutes, in the hope of building up a buffer so that their time is not wasted. This way they protect their incomes, and customers pay for this in terms of long waiting hours.

Now, in the aftermath of the covid crisis, this will need to change. People won’t want to spend long hours in a closed waiting room with scores of other sick people. In an ideal world, doctors will want to not let two of their patients even see each other, since that could mean increased disease transmission.

In the inimitable words of Ravishastri, “something’s got to give”.

One way could be for doctors to simply up their fees and give out appointments at intervals that better reflect the time taken per patient. The problem with this is that there are reputation costs to upping fee per patient, and doctors simply aren’t conditioned to unexpected breaks between patients. Moreover, lower number of slots might mean appointments not being available for several days together, and higher cancellations as well, both problems that doctors want to avoid.

As someone with a background in financial derivatives, there is one obvious thing to tackle – the free option being given to patients in terms of the appointment. What if you were to charge people for making appointments?

Now, taking credit card details at the time of booking is not efficient. However, assuming that most patients a doctor sees are “repeat patients”, just keeping track of who didn’t turn up for appointments can be used to charge them extra on the next visit (this needs to have been made clear in advance, at the time of making the appointment).

My take is that even if this appointment booking cost is trivial (say 5% of the session fee), people are bound to take the appointments more seriously. And when people take their appointments more seriously, the amount of buffer built in by doctors in their schedules can be reduced. Which means they can give out appointments at more realistic intervals. Which also means their income overall is protected, while still maintaining social distancing among patients.

I remember modelling this way back when I was working in air cargo pricing. There again, free options abound. I remember building this model that showed that charging a nominal fee for the options could result in a much lower fee for charging the actual cargo. A sort of win-win for customers and airlines alike. Needless to say, I was the only ex-derivatives guy around and it proved to be a really hard sell everywhere.

However, the concept remains. When options that have hitherto been free get monetised, it will lead to a win-win situation and significantly superior experience for all parties involved. The only caveat is that the option pricing should be implemented in a manner with as little friction as possible, else transaction costs can overwhelm the efficiency gains.

Why You Should Not Do An Undergrad in Computer Science at IIT Madras

I did my undergrad in Computer Science and Engineering at IIT Madras. My parents wanted me to study Electrical Engineering, but I had liked programming back in school, and my JEE rank “normally” “implied” Computer Science and Engineering. So I just went with the flow and joined the course. In the short term, I liked some subjects, so I was happy with my decision. Moreover there was a certain aura associated with CS students back in IITM, and I was happy to be a part of it. In the medium term too, the computer science degree did open doors to a few jobs, and I’m happy for that. And I still didn’t regret my decision.

Now, a full seven years after I graduated with my Bachelors, I’m not so sure. I think I should’ve gone for a “lighter” course, but then no one told me. So the thing with a B.Tech. in Computer Science and Engineering at IIT Madras is that it is extremely assignment incentive. Computer Science is that kind of a subject, there is very little you can learn in the classroom. The best way to learn stuff is by actually doing stuff, and “lab” is cheap (all you need is a bunch of computers) so most courses are filled with assignments. Probably from the fourth semester onwards, you spend most of your time doing assignments. Yes, you do end up getting good grades on an average, but you would’ve worked for it. And there’s no choice.

The thing with an Undergrad is that you are clueless. You have no clue what you’re interested in, what kind of a career you want to pursue, what excites you and the stuff. Yes, you have some information from school, from talking to seniors and stuff, but still it’s very difficult to KNOW when you are seventeen as to what you want to do in life. From this perspective, it is important for your to keep your options as open as they can be.

Unfortunately most universities in India don’t allow you to switch streams midway through your undergrad (most colleges are siloed into “arts” or “engineering” or “medicine” and the like). IIT Madras, in fact, is better in that respect since it allows you to choose a “minor” stream of study and courses in pure sciences and the humanities. But still, it is impossible for you to change your stream midway. So how do you signal to the market that you are actually interested in something else?

One way is by doing projects in areas that you think you are really interested in. Projects serve two purposes – first they allow you to do real work in the chosen field, and find out for yourself if it interests you. And if it does interest you, you have an automatic resume bullet point to pursue your career on that axis. Course-related projects are fine but since they’re forced, you have no way out, and they will be especially unpleasant if you happen to not like the course.

So why is CS@IITM a problem? Because it is so hectic, it doesn’t give you the time to pursue your other interests. It doesn’t offer you the kind of time that you need to study and take on projects in other subjects (yeah, it still offers you the 3 + 1 months of vacation per year, when you can do whatever you want, but then in the latter stages you’re so occupied with internships and course projects you’re better off having time during the term). So if you, like me, find out midway through the course that you would rather do something else, there is that much less time for you to explore around, study, and do projects in other subjects.

And there is no downside to joining a less hectic course. How hectic a course inherently is only sets a baseline. If you were to like the course, no one stops you from doing additional projects in the same subject. That way you get to do more of what you like, and get additional bullet points. All for the good, right?

After I graduated, IIT Madras reduced its credit requirement by one-twelfth. I don’t know how effective that has been in reducing the inherent workload of students but it’s a step in the right direction. Nevertheless, if you are going to get into college now, make sure you get into a less hectic course so that the cost of making a mistake in selection is not high.

Alco Haalu

Does anyone know why the colloquial name for liquor in Kannada is “oil” (eNNe) while the corresponding word in Tamil is “water” (thaNNi)?

Is there some kind of a caste/class origin to it, with me being biased given that most Tamilians I know are upper caste/class, and that there is a different colloquial word that is in vogue among other classes? Because “eNNe” has more of a working-class feel to it (the name, that is), and one that has been appropriated by all sections of society.

While on the topic, I learn that the Gult word for alcohol is medicine (mandu)!! Fantastic!

What is the colloquial name for alcohol in your language, and what does it mean? Put it down in the comments here.

PS: and does anyone know why alcohol bottles are sold in black polythene covers? Never seen these things being used elsewhere so if you see a black polythene cover you know there’s a good probability it’ll contain a bottle of alcohol

MBA specializations

During some casual conversation earlier this evening, I realized that I get irritated when people talk about ‘MBA finance’ or ‘MBA marketing’. I realized that I feel like not continuing the conversation when someone asks me my MBA specialization. Later I spoke to Baada about this, and he too agreed about the lack of respect for the counterparty when this topic gets mentioned.

I think it has to do with a lot of people assuming that “MBA” is just a set of courses that one does in order to become a manager. Maybe they assume that one can become a manager in a particular domain by reading a set of books. Maybe they think that an MBA is just like any other course where you get “knowledge” rather than change your way of thinking (ok a lot of people say MBA is useless and suchlike but my MBA certainly changed the way I think).

Or maybe it’s just that people find it easier to classify. Sometimes people overdo it, to the point of stereotyping. I’m reminded of my last company which worked on two kinds of products (let’s call them Product A and Product B – details are, er, classified). I started off doing a bit of A and soon I became “Associate for A”. Soon, I started doing some other stuff, which would easily fall under B. Yet, the CEO kept referring to me as “Associate for A”. It was ridiculous, but somehow he couldn’t get this classification out of his head – even when most of my time was spent doing B.

Anyways, point I’m trying to make is that people are used to classifications in education. For example, in engineering you have electrical, mechanical, etc. – all very easy. Similarly in postgrad for medicine – you can easily classify as ‘eye’, ‘bone’, etc. So isn’t it the duty of “management” also to get duly classified? And it did help the classifiers that there were three or four major areas in which most MBAs sought employment, and this made classification convenient.

Most local MBA colleges use this “specialization” funda to optimize on the number of electives that they need to offer. From a couple of interactions¬† with people from local MBA colleges, I found that they had very few electives – the major choice that they had was in specialization. And once you picked your specialization, your set of courses would get more or less frozen which made it easy for the college to organize.

Some local MBA colleges seem to have taken this specialization thing to ridiculous levels. The other day, one of my cousins had come to me for career gyaan and he said “I’m wondering whether to do an MBA in Aviation or an MBA in media”. I completely lost it at that point and blasted him and asked him to work before thinking of an MBA. Hopefully the current bust will take care of such ridiculousness that exists in the colleges.

Even a large number of good colleges had this “specialization” funda. I’m told that IIMC had this funda of “major” where if you took five electives in a particular area, that would go on your degree certi as a “major”. However, I’ve never heard anyone from IIMC (even from those days when this classification existed) describing themselves as a “MBA in XXX”.

Anyway, the next time you ask me what my specialization was during my MBA, you’ll make sure that I lose all respect for you.