Sometimes we overdo “option value”. We do things that have a small possibility of a big upside, and big possibility of no or very minimal downside, in the belief that “nothing can go wrong in trying”.
My father used to term this “pulling a mountain with a string”, with the reasoning being that if you actually manage to pull, then you have moved a mountain. If not, all that you have lost is a string.
There is one kind of situation, however, where I think we might overindex on option value – these are what I call “one shot events” or “brahmastras”.
Going into a little bit of mythology, there is the story of the Brahmastra in the Mahabharata. Famously, Karna possesses it. It is an incredibly powerful weapon with the feature (or bug, rather) that it can be used only once. Karna would have set it aside to use on Arjuna, but the Pandavas decide to send Ghatotkacha to create havoc during the night fight when Karna is forced to use up his brahmastra on Ghatotkacha – meaning he didn’t have access to it in his battle with Arjuna, where he (Karna) ultimately got killed.
Because the Brahmastra could be used only once, Karna wanted to maximise the impact of the weapon. His initial plan was to use it on what he thought might be a decisive battle with Arjuna. The Pandavas’ counterplan was to force him to use it earlier.
Actually, thinking about it – the Brahmastra can be thought of as another kind of option. The problem here being one of optimal exercise. Actually, there is a very stud paper written by economist Avinash Dixit on this topic – regarding Elaine’s sponges.
Read the whole paper. It is surely worth it. To quickly summarise, Elaine has a limited number of “contraceptive sponges”, and wants to maximise her “utility” of using them. When a guy comes along, she needs to decide whether it is worth expending a sponge on him. Dixit derives a nice equation to determine a function for this.
Basically, Brahmastra occurs when you have only one sponge left, and you need to use it at an “optimal time”. There is another problem in economics called the “secretary problem” (nothing to do with secretary birds) that deals with this.
Recently I’ve been thinking – these kind of Brahmastra / sponge / secretary problems are important to solve when you are thinking of talking to someone.
Let’s say you have what you think is a studmax application of GenAI and want to talk to VCs about it. If you go too early, the VC will only see a half-baked version of your idea, and even if you go to them later once you have fully formed it, the half-baked idea you had showed them will influence them enough to discount your later fully formed idea.
And if you go too late, the idea may not be that studmax any more, and the VC might dismiss it. So it’s a problem of “optimal exercise” (note that this is an issue only with American options, not European).
It is similar with asking someone out (or so I think – I’ve been out of this business for 14 years now). You approach them “too early” (before they know you), they will dismiss you then and then forever. You approach too late and the option would have expired.
In the world of finance, we focus too much on the PRICE of options and (based on my now limited knowledge) too little on optimal expiry of the said options. In the real world, the latter is also important.
Back in 2011-12, when I was about to go freelance, a friend told me about a simple formula on how I should price my services. “Take your expected annual income and divide it by 1000. That will be your hourly rate”, he said. I followed this policy fairly well, with reasonable success (though I think I shortchanged myself in some situations by massively underestimating how long a task would take – but that story is for another day).
The longer term effect of that has been that every time I see someone’s hourly rate, I multiply it by 1000 to guess that person’s approximate annual income (the basis being that as a full time worker, you “bill” for 2000 hours a year. As a freelancer you have “50% utilisation” and so you work 1000 hours).
And one set of people who have fairly transparent hourly rates are doctors – you know the number of appointments they give per hour, and what you paid for that, and you can back calculate their annual income based on that. The interesting thing is, for most doctors I’ve seen, based on this metric, what they earn for their level of eduction and years of experience seems rather low.
“So how do doctors earn?”, I wonder. Why is it still a prized profession while you might have a much better life being an engineer, for example?
Now you should remember that consultations are only one income stream for doctors. Those that practice surgery as well have a more lucrative stream – the hourly rates for surgeries far exceeds hourly rates of consultation. And so surgeons make far more than what I impute from what I’ve paid them for a consultation.
One possible reason for this arbitrage is the way insurance deals are structured – at least in India, out patient care is seldom paid for by insurance. As a consequence, hospitals and doctors cross-subsidise consultations with surgeries. They are able to get away with higher rates for surgeries because insurers are bearing the cost. Consultations, where patients generally pay out of their own pockets, are far more elastic.
This, however, leads to a problem for doctors who don’t do surgeries. Psychiatrists, for example. If they have to make money solely through consultations, their hourly rate must be far higher than that of doctors who also do surgeries. But then, is the market willing to bear this cost?
Now, I’m getting into conspiracy theory mode. If the amount non-surgeon doctors make is limited (thanks to market dynamics), the only way they can make sure they earn a decent living is by limiting supply. Could this be one reason India is under-supplied in a lot of non-surgical doctors? Again this is pure pure speculation, and not based in any fact.
Continuing with conspiracy theories, even for doctors who are surgeons, the only way to make a certain income is to have a threshold on the ratio of surgeries to consultations. And if this ratio (surgeries / consultations) goes too low, the doctors’ income suffers. Again, hippocratic oath aside, do hospitals try to game this metric, based on the current incentives?
On a more serious note, this distortion in the hourly earnings for surgeries versus consultations is one reason that India is also undersupplied with good general practitioners (GPs). Because GPs don’t do surgeries (though the Indian system means they are all licensed to perform surgeries, to the best of my knowledge), their earning potential is naturally capped. So the better doctors don’t want to be GPs.
How can we fix this distortion? How can we make sure we have better GPs? Insurance cover for outpatient care is one thing, but I’m not sure it is the silver bullet I’ve been making it out to be (and it will come with its own set of market distortions).
This entire post is me shooting from my hip. So please feel free to correct me iff I’m wrong.
I remember, about a year or so back, the US weekly non-farm payroll data had shown an uptick in unemployment. Intuitively, a higher unemployment rate indicates lower economic activity, since (among other things) the average purchasing power goes down and fewer things are getting produced (since fewer people are at work). So you would expect the stock market to react to this by going down.
The exact opposite happened. The higher unemployment was greeted with a big rise in the S&P 500. I remember tweeting about it but can’t find it now. But I can find some research someone has done about this:
But here’s the kicker: the S&P500 is inversely related to the unemployment rate, and thus the market actually goes up as a response to a release of a higher than expected unemployment rate. This may seem illogical conceptually, but historical analysis and statistics show that it is true.
In the last 3 years, the unemployment rate in the United States has been surprisingly higher than expected 11 times. The result? The S&P500 went up 80% of those times within a time-frame of 90 minutes (see Fig. 2, click to enlarge the image).
The basic issue (as I see it) is that higher unemployment means lesser likelihood that the US Federal Reserve will raise interest rates. Which means lower rates for the longer foreseeable future, which translates to higher stock prices.
The kicker here is the “discrete action” on part of the Fed. Because their decision (on whether to hike rates or not) is binary, news that decreases their odds of hiking rates, even if it (the news) is bad for the market, leads the market to go up.
You can see this in action elsewhere as well. Let’s say you are the number two at a manufacturing plant, and you are not happy with the way things have been run. However, you know that with the current level of production, the company management will not bother – they only see the numbers and see that the plant is being run well, and they won’t listen to you.
However, if the production drops below a certain level, the management is certain to review the operations, at which point you will be able to make your point to them and be heard, and you will be able to hopefully better influence how the plant is run.
Normally, your incentive is in keeping production as high as possible. But now, with this discrete action (management’s review of your operations) in the picture, your incentives get reversed. It suddenly becomes rational for you to not work so hard to increase production, since lower production means higher chance of a management review.
The problem with a lot of standard economics teaching is that it abstracts away the messiness of real world “step functions” and instead uses a deceptively simple continuously increasing or decreasing demand and supply curves. And so we are conditioned to think that incentives are linear as well.
However, given the step functions inherent in everyday business (which are only made worse (steps become steeper) with discrete actions), the incentives are not linear at all, and there are points in the curve where incentives are actually inverted! And this is everywhere.
I’m writing this on a lazy Sunday morning, having postponed this for over a week, so no enthu da to make pictures and explain my point. However, I guess I’ve explained sufficiently for you to catch my pOint.
Actually – since I have an iPad with a pencil, I did make a simple sketch. Limited by my drawing (and mentally adding curves) skillsBasically normal incentives is like the red line, but the discrete action (modelled here like a negative sigmoid) means that there is a region where the overall payoff is massively downward sloping. Which means your incentives are inverted.
Rob Henderson writes that in general, relative to the value they add to their firms, senior employees are underpaid and junior employees are overpaid. This, he reasons, is because senior employees trade off money for status.
Quoting him in full:
Robert Frank suggests the reason for this is that workers would generally prefer to occupy higher-ranked positions in their work groups than lower-ranked ones. They’re forgoing more earnings to hold a higher-status position in their organization.
But this preference for a higher-status position can be satisfied within any given organization.
After all, 50 percent of the positions in any firm must always be in the bottom half.
So the only way some workers can enjoy the pleasure inherent in positions of high status is if others are willing to bear the dissatisfactions associated with low status.
The solution, then, is to pay the low-status workers a bit more than they are worth to get them to stay. The high-status workers, in contrast, accept lower pay for the benefit of their lofty positions.
I’m not sure I agree. Yes, I do agree that higher productivity employees are underpaid and lower productivity employees are overpaid. However, I don’t think status fully explains it. There are also issues of variance and correlation and liquidity (there – I’m talking like a real quant now).
One the variance front – the higher you are in the organisation and the higher your salary is, the more the variance of your contribution to the organisation. For example, if you are being paid $350,000 (the number Henderson hypothetically uses), the actual value you are bringing to your firm might have a mean of $500,000 and a standard deviation of $200,000 (pulling all these numbers out of thin air, while making some sense checks that broadly risk pricing holds).
On the other hand, if you are being paid $35,000, then it is far more likely that the average value you bring to the firm is $40,000 with a standard deviation of $5,000 (again numbers entirely pulled out of thin air). Notice the drastic difference in the coefficient of variation in the two cases.
Putting it another way, the more productive you are, the harder it is for any organisation to put a precise value on your contribution. Henderson might say “you are worth 500K while you earn 350K” but the former is an average number. It is because of the high variance in your “worth” that you are paid far lower than what you are worth on average.
And why does this variance exist? It’s due to correlation.
More so at higher ranked positions (as an aside – my weird career path means that I’ve NEVER been in middle management) the value you can add to a company is tightly coupled with your interactions with your colleagues and peers. As a junior employee your role can be defined well enough that your contributions are stable irrespective of how you work with the others. At senior levels though a very large part of the value you can add is tied to how you work with others and leverage their work in your contributions.
So one way a company can get you to contribute more is to have a good set of peers you like working with, which increases your average contribution to the firm. Rather paradoxically, because you like your peers (assuming peer liking in senior management is two way), the company can get away with paying you a little less than your average worth and you will continue to stick on. If you don’t like working with your colleagues, there is the double whammy that you will add less to the company and you need to be paid more to stick on. And so if you look at people who are actually successful in their jobs at a senior level, they will all appear to be underpaid relative to their peers.
And finally there is liquidity (can I ever theorise about something without bringing this up?). The more senior you go, the less liquid is the market for your job. The number of potential jobs that you want to do, and which might want you, is very very low. And as I’ve explained in the first chapter of my book, when a market is illiquid, the bid-ask spread can be rather high. This means that even holding the value of your contribution to a company constant, there can be a large variation in what you are actually paid. And that is a gain why, on average, senior employees are underpaid.
So yes, there is an element of status. But there are also considerations of variance, correlation and bid-ask. And selection bias (senior employees who are overpaid relative to the value they add don’t last very long in their jobs). And this is why, on average, you can afford to underpay senior employees.
So I got an award today. It is called “exemplary data scientist”, and was given out by the Analytics India Magazine as part of their MachineCon 2022. I didn’t really do anything to get the award, apart from existing in my current job.
I guess having been out of the corporate world for nearly a decade, I had so far completely missed out on the awards and conferences circuit. I would see old classmates and colleagues put pictures on LinkedIn collecting awards. I wouldn’t know what to make of it when my oldest friend would tell me that whenever he heard “eye of the tiger”, he would mentally prepare to get up and go receive an award (he got so many I think). It was a world alien to me.
Parallelly, I used to crib about how while I’m well networked in India, and especially in Bangalore, my networking within the analytics and data science community is shit. In a way, I was longing for physical events to remedy this, and would lament that the pandemic had killed those.
So I was positively surprised when about a month ago Analytics India Magazine wrote to me saying they wanted to give me this award, and it would be part of this in-person conference. I knew of the magazine, so after asking around a bit on legitimacy of such awards and looking at who had got it the last time round, I happily accepted.
Most of the awardees were people like me – heads of analytics or data science at some company in India. And my hypothesis that my networking in the industry was shit was confirmed when I looked at the list of attendees – of 100 odd people listed on the MachineCon website, I barely knew 5 (of which 2 didn’t turn up at the event today).
Again I might sound like a n00b, but conferences like today are classic two sided markets (read this eminently readable paper on two sided markets and pricing of the same by Jean Tirole of the University of Toulouse). On the one hand are awardees – people like me and 99 others, who are incentivised to attend the event with the carrot of the award. On the other hand are people who want to meet us, who will then pay to attend the event (or sponsor it; the entry fee for paid tickets to the event was a hefty $399).
It is like “ladies’ night” that pubs have, where on a particular days of the week, women who go to the pub get a free drink. This attracts women, which in turn attracts men who seek to court the women. And what the pub spends in subsidising the women it makes back in terms of greater revenue from the men on the night.
And so it was at today’s conference. I got courted by at least 10 people, trying to sell me cloud services, “AI services on the cloud”, business intelligence tools, “AI powered business intelligence tools”, recruitment services and the like. Before the conference, I had received LinkedIn requests from a few people seeking to sell me stuff at the conference. In the middle of the conference, I got a call from an organiser asking me to step out of the hall so that a sponsor could sell to me.
I held a poker face with stock replies like “I’m not the person who makes this purchasing decision” or “I prefer open source tools” or “we’re building this in house”.
With full benefit of hindsight, Radisson Blu in Marathahalli is a pretty good conference venue. An entire wing of the ground floor of the hotel is dedicated for events, and the AIM guys had taken over the place. While I had not attended any such event earlier, it had all the markings of a well-funded and well-organised event.
As I entered the conference hall, the first thing that struck me was the number of people in suits. Most people were in suits (though few wore ties; And as if the conference expected people to turn up in suits, the goodie bag included a tie, a pair of cufflinks and a pocket square). And I’m just not used to that. Half the days I go to office in shorts. When I feel like wearing something more formal, I wear polo T-shirts with chinos.
My colleagues who went to the NSE last month to ring the bell to take us public all turned up company T-shirts and jeans. And that’s precisely what I wore to the conference today, though I had recently procured a “formal uniform” (polo T-shirt with company logo, rather than my “usual uniform” which is a round neck T-shirt). I was pretty much the only person there in “uniform”. Towards the end of the day, I saw one other guy in his company shirt, but he was wearing a blazer over it!
Pretty soon I met an old acquaintance (who I hadn’t known would be at the conference). He introduced me to a friend, and we went for coffee. I was eating a cookie with the coffee, and had an insight – at conferences, you should eat with your left hand. That way, you don’t touch the food with the same hand you use to touch other people’s hands (surprisingly I couldn’t find sanitiser dispensers at the venue).
The talks, as expected, were nothing much to write about. Most were by sponsors selling their wares. The one talk that wasn’t by a sponsor was delivered by a guy who was introduced as “his greatgrandfather did this. His grandfather did that. And now this guy is here to talk about ethics of AI”. Full Challenge Gopalakrishna feels happened (though, unfortunately, the Kannada fellows I’d hung out with earlier that day hadn’t watched the movie).
I was telling some people over lunch (which was pretty good) that talking about ethics in AI at a conference has become like worshipping Ganesha as part of any elaborate pooja. It has become the de riguer thing to do. And so you pay obeisance to the concept and move on.
Ok so i'm really really annoyed by all this focus on "ethical AI". It is just driven by "social desirability bias". Talking about ethical AI seemingly makes you look ethical so people talk about it. fund research in it. etc.
The awards function had three sections. The first section was for “users of AI” (from what I understood). The second (where I was included) was for “exemplary data scientists”. I don’t know what the third was for (my wife is ill today so I came home early as soon as I’d collected my award), except that it would be given by fast bowler and match referee Javagal Srinath. Most of the people I’d hung out with through the day were in the Srinath section of the awards.
Overall it felt good. The drive to Marathahalli took only 45 minutes each way (I drove). A lot of people had travelled from other cities in India to reach the venue. I met a few new people. My networking in data science and analytics is still not great, but far better than it used to be. I hope to go for more such events (though we need to figure out how to do these events without that talks).
PS: Everyone who got the award in my section was made to line up for a group photo. As we posed with our awards, an organiser said “make sure all of you hold the prizes in a way that the Intel (today’s chief sponsor) logo faces the camera”. “I guess they want Intel outside”, I joked. It seemed to be well received by the people standing around me. I didn’t talk to any of them after that, though.
The “intel outside” pic. Courtesy: https://www.linkedin.com/company/analytics-india-magazine/posts/?feedView=all
Yesterday I was reading this article($) about how Liverpool FC is going about (not) retaining its star forwards Sadio Mane and Mo Salah, who have been key parts of the team that has (almost) “cracked it” in the last 5 seasons.
One of the key ideas in the (paywalled) piece is that Liverpool is more careful about spending on its players than other top contemporary clubs. As Oliver Kay writes:
[…] the Spanish club have the financial strength to operate differently — retaining their superstars well into their 30s and paying them accordingly until they are perceived to have served their purpose, at which point either another A-list star or one of the most coveted youngsters in world football (an Eder Militao, an Eduardo Camavinga, a Vinicius Junior, a Rodrygo and perhaps imminently, an Aurelien Tchouameni) will usually emerge to replace them.
In an ideal world, Liverpool would do something similar with Salah and Mane, just as Manchester City did with Vincent Kompany, Fernandinho, Yaya Toure, David Silva and Sergio Aguero — and as they will surely do with De Bruyne.
But the reality is that the Merseyside club are more restricted. Not dramatically so, but restricted enough for Salah, Mane and their agents to know there is more to be earned elsewhere, and that presents a problem not just when it comes to retaining talent but also when it comes to competing for the signings that might fill the footsteps of today’s heroes.
To go back to fundamentals, earnings in sport follow a power law distribution – a small number of elite players make a large portion of the money. And the deal with the power law is that it is self-similar – you can cut off the distribution at any arbitrary amount, and what remains to the right is still a power law.
So income in football follows a power law. Income in elite football also follows the same power law. The English Premier League is at the far right end of this, but wages there again follow a power law. If you look at really elite players in the league, again it is a (sort of – since number of data points would have become small by now) power law.
What this means is that if you can define “marginal returns to additional skill”, at this far right end of the distribution it can be massive. For example, the article talks about how Salah has been offered a 50% hike (to make him the best paid Liverpool player ever), but that is still short of what some other (perceptibly less skilled) footballers earn.
So how do you go about getting value while operating in this kind of a market? One approach, that Liverpool seems to be playing, is to go Moneyball. “The marginal cost of getting a slightly superior player is massive, so we will operate not so far out at the right tail”, seems to be their strategy.
This means not breaking the bank for any particular player. It means ruthlessly assessing each player’s costs and benefits and acting accordingly (though sometimes it comes across as acting without emotion). For example, James Milner has just got an extension in his contract, but at lower wages to reflect his marginally decreased efficiency as he gets older.
Some of the other elite clubs (Real Madrid, PSG, Manchester City, etc.), on the other hand, believe that the premium for marginal quality is worth it, and so splurge on the elite players (including keeping them till fairly late in their careers even if it costs a lot). The rationale here is that the differences (to the “next level”) might be small, but these differences are sufficient to outperform compared to their peers (for example, Manchester City has won the league by one point over Liverpool twice in the last four seasons).
(Liverpool’s moneyball approach, of course, means that they try to get these “marginal advantages” in other (cheaper) ways, like employing a throw in coach or neuroscience consultants).
This approach is not without risk, of course. At the far right end of the tail, the variance in output can be rather high. Because the marginal cost of small increases in competence is so high, even if a player slightly underperforms, the effective monetary value of this underperformance is massive – you have paid for insanely elite players to win you everything, but they win you nothing.
In any case, the story doing the rounds now is that Barcelona want to hire Salah, but given their financial situation, they can’t afford to buy out his contract at Liverpool. So, they are hoping that he will run down his contract and join them on a free transfer next year. Then again, that’s what they had hoped from Gini Wijnaldum two years ago as well. And he’s ended up at PSG, where (to the best of my knowledge) he doesn’t play much.
There are a number of professions where incomes are distributed according to a power law. The most successful people in the professions corner a very large share of the income that people in the profession make, and unless you reach that very high level of success, you might even struggle to make a living wage.
Professions of this nature include the arts (movies, music, drama, standup comedy, painting, sculpture, etc.), sports, writing and entrepreneurship. The thing with such professions is that it needs some degree of “socialism” – if people are left to their own devices, then the 99% confidence payoffs will mean that few people will enter the profession, and when fewer people enter the profession, the overall quality of the profession goes down.
So what is required in this case is some sort of a safety net – people who are reasonably competent at the profession get paid a sort of regular basic income (could either be one-time, periodic or output-based) by “investors” in exchange for a cut of the upside. And this, for a talented but struggling beginner, is usually a good deal – they are assured a basic income to pursue what they love and think they are good at, and anything they have to pay in return is only probabilistic – contingent upon a heavy degree of success.
And in order for this kind of safety net to work, it is important that the investment be of the nature of “equity” rather than “debt” – the extreme power law nature of these professions is that only a small proportion of the people who get the safety net will be able to pay back, and those that are able to pay back will be able to pay disproportionately large amounts.
Entrepreneurship and film acting have sort of done well in terms of providing these safety net. Entrepreneurs get venture capital investment, which allows them to fund their business and take (nominal) salaries, while working on the thing they hope to make it big in. The venture capitalists make money even when a small proportion of their investments don’t fail.
The model in acting is a little different- studios hire actors on long term contracts at negotiated salaries. These salaries give actors the safety net to continue in the profession. And in case the actors become popular, the studios cash out essentially by “encashing the option” of using the actor at the pre-negotiated rate for the duration of the contract.
There are other examples of these safety nets as well – artist studios pay their artists a basic wage, in exchange for a cut on the sale of their paintings. However, the model is not as popular as it seems.
For sportspersons, for example, apart from things like the Ranji Trophy increasing match fees in a big way in the late noughties, this kind of a safety net has been absent. The studio model in acting hasn’t held on. Writers get advances but that doesn’t represent much of a “living wage”.
The good news is that this is changing. Investment in athletes in exchange for a cut of future earnings is gaining traction. And now we have this deal ($):
Taxes will cut into his new 14-year agreement with the Padres, of course. But Tatis also must pay off a previous obligation, a deal he made during the 2017-18 offseason, when he was turning 19 years old and preparing for his first full season at Double A.
It was then that Tatis entered into a contract with Big League Advance (BLA), a company that offers select minor leaguers upfront payments in exchange for a percentage of their future earnings in Major League Baseball. Neither Tatis nor BLA has revealed the exact percentage he owes the company.
The company’s president and CEO, former major-league pitcher Michael Schwimer, told The Athletic in April 2018 that BLA uses a proprietary algorithm to value every player in the minors. Players who receive offers can accept a base-level payout in return for 1 percent of their earnings, with the chance to receive greater incremental payouts and pay back a maximum of 10 percent. If a player never reaches the majors, he keeps the cash advance, with no obligation to pay it back.
This is an awesome thing. For a struggling potential sportsperson, a minor investment (in exchange for equity) can provide a huge boost in their chances of making it – hiring coaches, for example, or eating better food, or living more comfortably.
While the media attention will go to the small proportion of investments that do pay off (like how tech media gives disproportionate coverage, and quite rightly so, to startups that do well), arrangements like this mean that more people will play the sport, and the overall standard in the sport will improve.
We need to see if such arrangements start making a mark in the rest of the arts and writing as well.
Oh, and much has been made of income sharing agreements for professional colleges and “tuition centres”. I’m not sure that is the right model there – the thing is that if you are studying to be a software engineer, your payoffs don’t follow a power law. Yes, if you are successful, you make a few orders of magnitude more money than the less successful ones, but even an average software engineer can expect to make a fairly decent income.
From that perspective, selling equity in your future earnings to get paid to study engineering is not a great idea, and can lead to adverse selection on the part of the candidates (the better ones will prefer to get funding through debt, which their average salaries can help pay off). In that sense I prefer what the likes of MountBlue are doing, where the “training fees” get paid off by simply working for the company for a certain period of time.
In 2002, as part of my undergrad in computer science, I took a course in “Artificial Intelligence”. It was a “restricted elective” – you had to either take that or another course called “Artificial Neural Networks”. That Neural Networks was then considered disjoint from AI will tell you how the field of computer science has changed in the 15 years since I graduated.
In any case, as part of our course on AI, we learnt heuristics. These were approximate algorithms to solve a problem – seldom did well in terms of worst case complexity but in most cases got the job done. Back then, the dominant discourse was that you had to tell a computer how to solve a problem, not just show it a large number of positive and negative examples and allow it to learn by itself (though that was the approach taken by the elective I did not elect for).
One such heuristic was Simulated Annealing. The problem with a classic “hill climbing” algorithm is that you can get caught in local optima. And the deterministic hill climbing algorithm doesn’t let you get off your local optima to search for better optima. Hence there are variants. In Simulated Annealing, in the early part of the algorithm you are allowed to take big steps down (assuming you are trying to find the peak). As the algorithm progresses, it “cools down” (hence simulated annealing) and the extent to which you are allowed to climb down is massively reduced.
It is not just in algorithms, or in the case of AI, do we get stuck in local optima. In a recent post, I had made a passing reference to a paper about the tube strikes of 2014.
It is clearly visible from the two panels that far fewer commuters were able to use their modal station during the strike, which implies that a substantial number of individuals were forced to explore alternative routes. The data also suggest that the strike brought about some lasting changes in behaviour, as the fraction of commuters that made use of their modal station seemingly drops after the strike (in the paper we substantiate this claim econometrically).
Screw the paper if you don’t want to read it. Basically the concept is that the strike of 2014 shook things up. People were forced to explore alternatives. And some alternatives stuck. In other words, a lot of people had got stuck in local maxima. And when an external event (the strike) pushed them off their local pedestals (figuratively speaking), they were able to find better maxima.
And that was only the result of a three-day strike. Now, the pandemic has gone on for 5-6 months now (depending on the part of world you are in). During this time, a lot of behaviour otherwise considered normal have been questioned by people behaving thus. My theory is that a lot of these hitherto “normal behaviours” were essentially local optima. And with the pandemic forcing people to rethink their behaviours, they will find better optima.
I can think of a few examples from my own life.
I wrote about this the other day. I had gotten used to a schedule of heavy weight lifting for my workouts. I had plateaued in all my lifts, and this meant that my upper body had plateaued at a rather suboptimal level. However much I tried to improve my bench press and shoulder press (using only these movements) the bar refused to budge. And my shoulders refused to get bigger. I couldn’t do a (palms facing away) pull up. Thanks to the pandemic, the gym shut, and I was forced to do body weight exercises at home. There was a limit on how much I could load my legs and back, so I focussed more on my upper body, especially doing different progressions of the pushup. And back in the gym today, I discovered I could easily do pullups now.
Similarly, the progression of body weight squats I knew forced me to learn to squat deep (hamstrings touching calves). Today for the first time ever I did deep front squats. This means in a few months I can learn to clean.
I was used to eating Milky Mist set curd (the one that comes in a 1kg box). It was nice and creamy and I loved eating it. It isn’t widely available and there was one supermarket close to home from where I could get it. As soon as the lockdown happened that supermarket shut. Even when it opened it had long lines, and there were physical barricades between my house and that so I couldn’t drive to it.
In the meantime I figured that the guy who delivers milk to my door in the morning could deliver (Nandini) curd as well. And I started buying from him. Well, it’s not as creamy as Milky Mist, but it’s good enough. And I’m not going back.
This was a see-saw. For the first month of the lockdown most bakeries nearby were shut. So I started trying out bread at this supermarket close to home (not where I got Milky Mist from). I loved it. Presently, bakeries reopened and the density of cases in Bangalore meant I became wary of going to supermarkets. So now we’ve shifted back to freshly baked bread from the local bakery
I’d tried intermittent fasting several times in life but had never been able to do it on a consistent basis. In the initial part of the lockdown good bread was hard to come by (since the bakeries shut and I hadn’t discovered the supermarket bread yet). There had been a bird flu scare near Bangalore so we weren’t buying eggs either. What do we do for breakfast? Just skip it. Now i have no problem not having breakfast at all
The list goes on. And I’m sure this applies to you as well. Think of all the behavioural changes that the pandemic has forced on you, and think of which all you will go back on once it has passed. There is likely to be a set of behavioural changes that won’t change back.
Like how one in 20 passengers who changed routes following the 2014 tube strikes never went back to their earlier routes. Except that this time it is a 6-month disruption.
What this means is that even when the pandemic is past us, the economy will not look like the economy that was before the pandemic hit us. There will be winners and losers. And since it will take time and effort for people doing “loser jobs” to retrain themselves (if possible) to do “winner jobs”, the economic downturn will be even longer.
I’m calling it the “tube strike mental model” for behavioural change during the pandemic.
Did you know that trade unions were responsible for apartheid, which devastated the lives of black and coloured people in South Africa for nearly a century?
The logic was simple – black people were willing to work as miners for lower wages than white people. So the white-controlled unions lobbied to not allow black people to work in mines, so that their wages weren’t undercut. And what started as a movement to not let blacks work in diamond mines became an overall anti-black movement that led to apartheid.
This is captured in this beautiful old essay in Econlib. A couple of excerpts:
At first, however, the white capitalist could deal directly only with the few English and Afrikaner managers and foremen who shared his tongue and work habits. But the premium such workers commanded soon became an extravagance. Black workers were becoming capable of performing industrial leadership roles in far greater numbers and at far less cost. Driven by the profit motive, the substitution of black for white in skilled and semiskilled mining jobs rose high on the agenda of the mining companies.
[…]
Nonetheless, the state instituted an array of legal impediments to the promotion of black workers. The notorious Pass Laws sought to sharply limit the supply of nonwhite workers in “white” employment centers. Blacks were not allowed to become lawful citizens, to live permanently near their work, or to travel without government passports. This last restriction created a catch-22. If passports were issued only to those already possessing jobs, how was a nonwhite to get into the job area to procure a job so as to obtain a passport? Nonwhites also were prohibited from bringing their families while working in the mines (reinforcing the transient nature of employment).
[…]
To discourage mine owners from substituting cheaper African labor for more expensive European labor, the trade unions regularly resorted to violence and the strike threat. They also turned to legislation: the Mines and Works Act of 1911 (commonly referred to as the first Colour Bar Act) used the premise of “worker safety” to institute a licensing scheme for labor. A government board was set up to certify individuals for work in “hazardous” occupations. The effect was to decertify non-Europeans, who were deemed “unqualified.”
Read the whole thing. Going by modern American (or British) politics, this kind of a conflict between labour unions and blacks doesn’t make sense. After all, both these “communities” are among the biggest supporters of the Democratic (or Labour) Party, and so based on modern politics, you would imagine that they would be in harmony with each other on most counts.
However, I’m not sure the conflict between mostly-white unions and blacks has completely gone away.
I’ve been thinking of the brutal killing of George Floyd and the subsequent protests all over the USA (and elsewhere) over the last 10 days. The protestors have been protesting against racism, and the many cases of abuse of black people by white policemen in the US.
While the perpetrators of the crime were all racist white men, and the victim was a black man, I don’t know how much of the brutality can be attributed to racism, and how much simply to bad policing. Keep aside the victim’s race for a moment, and think about what happened – a policeman pinned down a suspect, and then knelt upon his neck for eight minutes until he was dead.
Racism has a part to play in that maybe the policemen thought they have a higher chance of getting away with it because the victim was black, but that the cop thought it was okay to brutalise just about anyone the way he did is atrocious.
Having largely been off social media, my reading about this (and related) issues is through the blogs that I follow, and one phrase that repeatedly make an appearance in this context is “police unions”. Policemen, like many other professions, are highly unionised in the United States, and the unions set rules for how the police can be treated, what they can be expected to do, their punishments, etc.
And from the stuff I’ve read (too many to link to everything), the unions give individual cops to behave the way they want to, knowing that punishment is going to be limited.
Today I came across this rather interesting post by Alex Tabarrok about Camden (New Jersey) where policemen marched with the Black Lives Matters protestors. There is a very interesting history to policing in Camden NJ.
In May of 2013, however, the entire police department was disbanded nullifying the union contract and an entirely new county police department was put into place.
And Tabarrok’s post goes on to show that the dissolution and reconstitution of the police force (basically the dissolution of the union) has led to tangible benefits in terms of reduced violent crime.
So it appears that, decades after apartheid was (in letter) abolished, white-controlled unions continue to make life really difficult for blacks.
Why do you think you usually have to wait so much to see a doctor, even when you have an appointment? It is because doctors routinely overbook.
You can think of a doctor’s appointment as being a free option. You call up, give your patient number, and are assigned a slot when the doctor sees you. If you choose to see the doctor at that time, you get the doctor’s services, and then pay for the service. If you choose to not turn up, the doctor’s time in that slot is essentially wasted, since there is nobody else to see then. The doctor doesn’t get compensated for this as well.
In order to not waste their time, thus, doctors routinely overbook patients. If the average patient takes fifteen minutes to see, they give appointments once every ten minutes, in the hope of building up a buffer so that their time is not wasted. This way they protect their incomes, and customers pay for this in terms of long waiting hours.
Now, in the aftermath of the covid crisis, this will need to change. People won’t want to spend long hours in a closed waiting room with scores of other sick people. In an ideal world, doctors will want to not let two of their patients even see each other, since that could mean increased disease transmission.
In the inimitable words of Ravishastri, “something’s got to give”.
One way could be for doctors to simply up their fees and give out appointments at intervals that better reflect the time taken per patient. The problem with this is that there are reputation costs to upping fee per patient, and doctors simply aren’t conditioned to unexpected breaks between patients. Moreover, lower number of slots might mean appointments not being available for several days together, and higher cancellations as well, both problems that doctors want to avoid.
As someone with a background in financial derivatives, there is one obvious thing to tackle – the free option being given to patients in terms of the appointment. What if you were to charge people for making appointments?
Now, taking credit card details at the time of booking is not efficient. However, assuming that most patients a doctor sees are “repeat patients”, just keeping track of who didn’t turn up for appointments can be used to charge them extra on the next visit (this needs to have been made clear in advance, at the time of making the appointment).
My take is that even if this appointment booking cost is trivial (say 5% of the session fee), people are bound to take the appointments more seriously. And when people take their appointments more seriously, the amount of buffer built in by doctors in their schedules can be reduced. Which means they can give out appointments at more realistic intervals. Which also means their income overall is protected, while still maintaining social distancing among patients.
I remember modelling this way back when I was working in air cargo pricing. There again, free options abound. I remember building this model that showed that charging a nominal fee for the options could result in a much lower fee for charging the actual cargo. A sort of win-win for customers and airlines alike. Needless to say, I was the only ex-derivatives guy around and it proved to be a really hard sell everywhere.
However, the concept remains. When options that have hitherto been free get monetised, it will lead to a win-win situation and significantly superior experience for all parties involved. The only caveat is that the option pricing should be implemented in a manner with as little friction as possible, else transaction costs can overwhelm the efficiency gains.