SHAP and WAR

A few months back, at work, a couple of kids in my team taught me this concept called “SHAP“. I won’t go into the technical details here (or maybe I will later on in this post), but it is basically an algo that helps us explain a machine learning model.

It was one of those concepts that I found absolutely mind-blowing, to the extent that after these guys taught this concept to me, it became the proverbial hammer, and I started looking for “nails” all around the company. I’m pretty sure I’ve abused it (SHAP I mean).

Most of the documentation of SHAP is not very good, as you might expect about something that is very deeply technical. So maybe I’ll give a brief intro here. Or maybe not – it’s been a few months since I started using and abusing it, and so I’ve forgotten the maths.

In any case, this is one of those concepts that made me incredibly happy on the day I learnt about it. Basically, to put it “in brief”, what you essentially do is to zero out an explanatory variable, and see what the model predicts with the rest of the variables. The difference between this and the actual model output, approximately speaking, is the contribution of this explanatory variable to this particular prediction.

The beauty of SHAP is that you can calculate the value for hundreds of explanatory variables and millions of observations in fairly quick time. And that’s what’s led me to use and abuse it.

In any case, I was reading something about American sport recently, and I realised that SHAP is almost exactly identical (in concept, though not in maths) to Wins Above Replacement.

WAR works the same way – a player is replaced by a hypothetical “average similar player” (the replacement), and the model calculates how much the team would have won in that case. A player’s WAR is thus the difference between the “actuals” (what the team has actually won) and the hypothetical if this particular player had been replaced by the average replacement.

This, if you think about it, is exactly similar to zeroing out the idiosyncrasies of a particular player. So – let’s say you had a machine learning model where you had to predict wins based on certain sets of features of each player (think of the features they put on those otherwise horrible spider charts when comparing footballers).

You build this model. And then to find out the contribution of a particular player, you get rid of all of this person’s features (or replace it with “average” for all data points). And then look at the prediction and how different it is from the “actual prediction”. Depending on how you look at it, it can either be SHAP or WAR.

In other words, the two concepts are pretty much exactly the same!

War, Terror and Leaderless Protests

A while back, I’d written on this blog that the phrase “war on terror” is incorrect since terrorism is not a war (actually I have written two posts on this topic. Here is the second one). A war is a staged human conflict with the aim being a political victory, and wars inevitably end in a political settlement, which in chess terms can be described as “resignation, rather than check mate”.

The issue with terrorism is that it is usually a distributed method. There is no one leader of terror. You might identify one leader and neutralise him, but that is no guarantee that the protests are going to end, since the rest of the “terrorist organisation” (a bit of an oxymoron) will keep the terror going. With a distributed organisation like a terrorist outfit, political settlements are impossible (who do you really settle with), and so the terrorism continues and there is no “victory”.

It is similar with spontaneous leaderless protests that have become the hallmark of the last decade, from Tunisia and Egypt in 2011 to Occupy Wall Street to the recent anti-CAA protests in India. To take a stark example with two protests based in Delhi, the Anna Hazare protest in 2011 was finished in fairly quick order (it started two days after India won the World Cup, and finished two days before the IPL was about to begin), while the Shaheen Bagh protests against the Citizenship Amendment Act have been going on for nearly three months now.

The difference between these two Delhi protests is that the first one (2011) had a designated leader (Anna Hazare, and maybe even Arvind Kejriwal or Kiran Bedi). And the protestors effectively followed the leader. And so when the government of the day negotiated a settlement with the leader, the protest effectively got “called off” and ended abruptly.

The Shaheen Bagh protests don’t have a designated leader to negotiate with (at least there are no obvious leaders). The government might try to negotiate with or round up or be violent to a handful of people who it thinks are the leaders, but the nature of the protest means that this is unlikely to have much effect since the rest of the “decentralised organisation” will go on.

In that sense, protests by “decentralised groups” are attritional battles where no negotiation is possible, and the only possible end is that the protestors either get bored or decide that the protest is pointless (that’s pretty much what happened with Occupy). Each member of the protesting group takes an independent decision each day (or night) whether to join the protest or not, and the protest will die down over a period of time (how long it will take depends on the size of the universe of people participating in the protest, overall interest level in the protest and how networked the protest is).

From that point of view, a leadered protest (like the Anna Hazare protest) can end suddenly (so everyone can go watch the IPL). A leaderless protest dies slowly and gradually (stronger network effects among the protestors can actually mean that the protest can die a bit faster, but still gradually).

There are claims on social media and WhatsApp groups that the communal violence in Delhi on Monday and Tuesday was designed in part to intimidate the Shaheen Bagh protestors to stop the protests. Even the violence was “successful” in achieving this objective, the leaderless nature of the protest will mean that it will only end “gradually”, more like a “halal process” rather than with a “jhatka”.

The “war” on terror

In light of the terrorist attack in London this morning, when 29 people were hospitalised following an explosion in a peak hour District Line train on a massively crowded route, I nearly re-wrote this old blogpost of mine. I even thought of the very same examples before I figured I should once check.

Recently, Iraqi Prime Minister Haider Al-Abadi declared “victory over Islamic State“, and announced that the organisation had been defeated. In this statement, Al-Abadi conveyed his ignorance of the kind of conflict his government was involved in, for Islamic State is not a “normal country” and the so-called “war” the Iraq and others are fighting it is not a war – since it will never end the way normal wars do.

As I’d mentioned in the other post on wars, wars end in a political decision – a surrender, usually. Sometimes, it takes extreme measures to induce surrender, as it happened with Japan in World War 2. At other times, a slight advantage to one side might lead the other to concede, and strike a treaty. Either ways, in a conventional war, few sides are likely to fight on until last man standing.

The so-called “war on terror” (especially aimed at the Islamic State) is not a war for several reasons. Firstly, Islamic State is not a conventional organisation – it has transcended that to become a concept, to unite radical Islamists worldwide. Irrespective of how many layers of the top management of the Islamic State are eliminated (either by killing or by incarceration), the remainder of the organisation will regroup and continue to thrive. And the organisation continues to grow – with ordinary members constantly seeking to enroll new members.

This feature of the Islamic State not being a conventional organisation also means that there is no central leadership that has the power to concede defeat and declare the war to an end. Even if a nominal leader of the organisation were to take such a decision, the fact that the organisation is an extremist on might imply that this decision might be decried as “selling out” by the more extreme factions of the organisation, who will fight on.

Then, the Islamic State is a distributed organisation – even in terms of geography. The use of the internet for recruitment has meant that they have operatives in most countries, and after some initial training, these operatives operate independently. So even if a nominal “top management” of the organisation were to be eliminated, these independent operatives will continue to thrive. And they need to be taken down – to the last man.

In that sense, the “war” against Islamic State is hardly a war. There is no political objective since the Islamic State lacks a political leadership capable of taking decisions. The organisation is rather distributed and even killing the “main organisation” will not eliminate the branches (reminds me of this demon in Hindu myth who had the property that each drop of his blood that would touch the ground would result in a clone of the demon).

The fight is going to be a long one, and we’ll need measures both conventional and unconventional to defeat the organisation. Declaring victory, like the Iraqi PM did, can only prove counterproductive.

When is a war a war?

War is an inherently political instrument used to achieve a political objective, so a credible political adversary is necessary for war to be war.

As the US Presidential election race hots up (or gets more one-sided, depending upon your interpretation), people continue to refer to former President George W Bush leading the US into two “wars” in Iraq and Afghanistan. Thinking about it, I’m not sure the two can actually be classified as wars.

To use a chess analogy, real wars seldom end in checkmate – they most often end in resignation, or an agreed draw. War is an instrument that is used to achieve a political objective, to get the other party to do what you want them to do.

And so war ends when one side has established such an utter dominance over the other that the counterparty decides that to resign, or “surrender” is superior to continuing fighting the war.

For this to happen, however, the counterparty needs to have a political leadership that is able and willing to take a decision, following which the war actually stops. In the absence of such a political leadership, the war will continue indefinitely until “checkmate”, and assuming that the losing side’s force “decays exponentially”, it can take a really long time for it to actually get over.

So based on this definition that war is a political instrument used to achieve a political objective, I’m not sure what happened in Iraq and Afghanistan can actually be classified as “war”.

The “government” of the day in Afghanistan (Taliban), for example, would have never come to the negotiating table with the US, so short of complete annihilation, there was no other “objective” that the US could achieve there.

Iraq, on the other hand, possessed credible political leadership (Saddam Hussein) when the US invaded, but by actually killing him, the US denied themselves the chance of a “real victory” in terms of a negotiated settlement. A game of chess might end when the king is mated (remember that the king never “dies”, only trapped), but in a situation such as Iraq, the battle will rage until each member of the opposing force is taken out.

And so fighting continues to this day, over a decade since it started, with no hope of it ending in the near future. Real wars never go on indefinitely.

Snow White meets Gandalf

I don’t know how to describe it. “Writing club” or “literature club” makes it sound too serious, and if it were indeed one, then we need not have put “informal” when we put it in our CVs. “Slander club” makes it sound like we were all bitches, which we were not – though I must admit that once in a while we used to bitch a bit. We weren’t even doing campus journalism – we were hardly regular, and never came in print. And we weren’t storytellers, either, since most of what we wrote was based on what actually happened.

Twisted Shout began when Hunger tried to murder War. However, the defining moment for the group happened when Snow White Meets Gandalf (a trilogy in five parts) was released. I must say it was a fairly random story. So random that most of you won’t understand it. If you are not from IIMB, you can give up every hope of figuring it out. If you are from IIMB but not from our batch, you might understand one joke in the entire trilogy. If you are from IIMB and from my batch and not from my section, you might probably understand half the stuff.

SWG had so many characters that I won’t blame you if you would get confused. Most of these characters are based on people in my batch and the batch senior to mine at IIMB. And it’s not a one-to-one correspondence between real and fictional characters. Some people in my batch were so colourful that multiple characters were based on them. On the other hand, the entire commie half of Sumo Yet So Far (my quiz team) had gotten merged into one character called Swaadisht.

We drew inspiration from several sources, with the primary source being our first test in Economics, which had a certain Queen Shilpa taxing coconuts. A number of other characters, and scenes were built based on interactions in class in term 1. There was heavy punning on people’s names, and even seemingly random sentences like “I find Aishwarya Rai so hot that I want her as my wife” found their way into the stories.

If my memory serves me right (it usually does in the long term), the first three parts of the Trilogy were written by Disease, who then proceeded to put NED (this was a full three years before the term NED was coined, btw). Madness, the correspondent from H Base, joined the great institution when he wrote the fourth part. The fifth part, which involved a Great War, based on the Mahabharata, was appropriately penned by War. And he had ensured that he gave due footage to himself, as well as to the Footage Queen.

The beauty of the series was that characterization was not constant. People would change sides more frequently than Disease would change his shorts – which means that they didn’t change sides too often, but did it once in a long time. Characters would disappear from the plot, and occasionally reappear at a strategic time. There would be sudden updates in relationships between characters – to account for similar changes in the real world. Every event of note that took place on campus, and even some insignifcant ones, got due footage. It was a masterpiece of its times.

Looking back at these stories today, I’m feeling nostalgic, and at the same time proud to have been part of such an august institution. We were to come up with a few other masterpieces during our tenure, but SWG would remain our best known work.

Two months back, more than three years after we had first folded up, we thought we should make an attempt to recreate the magic, and thus started the Twisted Shout blog. I admit that we haven’t been too regular in updating it, but each of us has been managing an important transition in our lives, and thus haven’t really had the time to update it. We hope to fix this in the coming months, though I’m not sure how funny we will be since we will be writing for a general audience. In hindsight, it was really easy writing for a restricted audience that knew exactly who each character was based on. Making inside jokes, it seems, is much easier than making generalized jokes.