The Misfit Job Market

Exactly 15 years ago, I was looking for a job. I had graduated from IIMB four months earlier, taken my first ever full time job 3 months earlier, and was already serving notice. Very quickly on, I had figured that I was not a good fit for the job that I had taken up, and so decided to cut my losses and move on.

The only problem was job hunting was hard. Back then, most people I spoke to seemed suspicious of me because I was getting out of my first job so early. For the longest time (years later), people spoke to me as if there was something wrong with me because I had quit my first job within three months. Finally I ended up taking a 20% pay cut to take another job where I seemed a better fit.

Thinking back, I don’t think I’m alone. The sheer randomness of the campus placement process means that a lot of people end up in jobs that they are ill suited for, purely based on a bit of bad judgment here and a lucky interview there. And most smart people figure out quickly enough that in case they are in jobs they are not a good fit for, it’s better to cut losses and move on. If it is their first ever jobs (applies for undergrad jobs, and for MBAs without prior work experience), the desperation to get out of their misfit jobs will be high.

I think this is a highly underserved market. Companies fall head over heels over themselves to access premium slots in the random process called campus placements, without realising that a significant part of the same pool will (theoretically) be available for a proper interview just a few months hence.

5-6 years back, an old friend of mine had started a company which was essentially a clearinghouse targeted at this precise market – to enable companies hire people in their first years of employment. Unfortunately the company didn’t take off, suggesting that the market design problem is not easy to solve.

Anyway, in case you are a just-graduated student who believes you are a misfit in your first job, and instead want to do analytics, get in touch with me. Having been on the other side, I’m more than happy to fish in this pool, and I know that I’ll get some temporarily undervalued talent here.

Just that I don’t know what sort of market or clearinghouse I need to go to to tap this supply, and so I’m putting out a bid here in the form of this blogpost.

PS: In case you’re a recent reader of my blog, I’ve written a book on market design.

Slip fielding meetings

It’s been nearly six months since I returned to corporate life. As you might imagine, I have participated in lots of meetings in this period. Some of them are 1-on-1s. Some are in slightly larger groups. Some meetings have big groups.

Meetings in big groups are of two types – ones where you do a lot of the talking, and what I have come to call as “slip fielder meetings”.

Basically, participating in these meetings is like fielding at slip in a cricket match. For most of the day, you just stand there doing nothing, but occasionally once in a while a ball will come towards you and you are expected to catch it. That means you need to be alert all the time.

These meetings are the same. For most of the discussion you are not necessarily required, but once in a while there might be some matter that comes up where your opinion is required, and you need to be prepared for that.

I can think of at least two occasions in the last six months where I was rudely awoken from my daydreams (no I wasn’t literally napping) with someone saying “Karthik, what do you think we should do about this?”.

And since then I’ve learnt to anticipate. Anticipate when my presence might be required. Figure out from the broad contours of the conversation on when I might be called upon. And remain alert when called upon (though on one occasion early on in the company my internet decided to give way just when I had started talking in a 20 person meeting).

Yesterday, a colleague gave me a good idea on how to stay alert through these “slip fielder meetings”. “Just turn on the automated captions on Google Meet”, he said. “Occasionally it can be super funny. Like one day ‘inbound docks’ was shown as ‘inborn dogs'”.

I think this is a great idea. By continuously looking at the captions, I can remain sufficiently stimulated and entertained, and also know what exactly is happening in the meeting. I’m going to use this today onwards.

I now wonder what real slip fielders do to stay alert. I’m not sure chatting with the wicketkeeper is entertaining enough.

Fifteen years of professional life

I was supposed to begin my first job on the 1st of May 2006. A week before, I got a call from HR stating that my joining date had been shifted to the 2nd. “1st May is Maharashtra Day, and all Mumbai-based employees have a holiday that day. So you start on the second”, she said.

I was thinking about this particular job (where I lasted all of three months) for a totally different reason last night. We will talk about that sometime in another blogpost (once those thoughts are well formed).

The other day I was thinking about how I have changed since the time I was working. I mean there are a lot of cosmetic changes – I’m older now. I can claim to have “experience”. I have a family. I have a better idea now of what I’m good at and all that.

However, if I think about the biggest change from a professional front that has happened to me, it is in (finally, belatedly) coming to realise that the world (especially, “wealth games”) is positive sum, and not zero sum.

The eight years before I started my first job in 2006 were spent in insanely competitive environments. First there was mugging for IIT JEE, where what mattered was the rank, not the absolute number of marks. Then, in IIT, people targeted “branch position” (relative position in class) rather than absolute CGPA. We even had a term for it – “RG” (for relative grading).

And so it went along. More entrance exams. Another round of RG. And then campus interviews where companies came with a fixed number of open positions. I don’t think I realised this then, but all of my late teens and early twenties spent in ultra competitive environments meant that I entered corporate life also thinking that it was a zero sum thing.

I kept comparing myself to everyone around. It didn’t matter if it was the company’s CEO, or my boss, or some junior, or someone completely unconnected in another part of the firm. The only thing that was constant was that I would instinctively compare myself

“Why do people think this person is good? I’m smarter than him”
“Oh, she seems to be much smarter than me. I should be like her”

And that went on for a while. Somewhere along the way I decided to quit corporate altogether and start my own consulting business. Along the way I met a lot of people. Some were people I was trying to sell to. Others I worked with after having sold to some of their colleagues. I saw companies in action. I saw diverse people get together to get work done.

Along the way something flipped. I don’t exactly know what. And I started seeing how things in the real world are not a zero sum game after all. It didn’t matter who was good at what. It didn’t matter if one person “dominated” another (was good at the latter on all counts). People worked together and got things done.

My own sales process also contributed. I spoke to several people. And every sale I achieved was a win-win. Every assignment came about because I was adding value to them, and because they were adding (monetary) value to me. It was all positive sum. There were no favours involved.

And so by the time I got back to corporate life once again at the end of last year, I had changed completely. I had started seeing everything in a “positive sum” sort of way and not “zero sum” like I used to in my first stint in corporate life. That is possibly one reason why I’m enjoying this corporate stint much better.

PS: If you haven’t already done so, listen to this podcast by Naval Ravikant. It is rather profound (I don’t say that easily). Talks about how wealth is a positive sum game while status is a zero sum game. And to summarise this post, I had spent eight years immediately before I started building wealth by competing for status, in zero sum games.

JEE Rank, branch position, getting the “most coveted job” – they were all games of status. It is interesting (and unfortunate) that it took me so long to change my perspective to what was useful in the wealth business.

PPS: I’ve written this blogpost over nearly two hours, while half-watching an old Rajkumar movie. My apologies if it seems a bit rambling or incoherent or repetitive.

 

 

The Office!

For the first time in nearly ten years, I went to an office where I’m employed to work. I’m not going to start going regularly, yet. This was a one off since I had to meet some people who were visiting. On the evidence of today, though, I think i once again sort of enjoy going to an office, and might actually look forward to when I start going regularly again.

Metro

I had initially thought I’d drive to the office, but white topping work on CMH Road means I didn’t fancy driving. Also, the office being literally a stone’s throw away from the Indiranagar Metro Station meant that taking the Metro was an easy enough decision.

The walk to South End Metro station was uneventful, though I must mention that the footpath close to the metro station works after a very long time! However, they’ve changed the gate that’s kept open to enter the station which means that the escalator wasn’t available.

The first order of business upon entering the station was to show my palm to one reader which took my temperature and let me go past. As someone had instructed me on twitter, I put my phone, wallet and watch in my bag as I got it scanned.

Despite not having taken the metro for at least 11 months, the balance on my card remained, and as I swiped it while entering, I heard announcements of a train to Peenya about to enter the station. I bounded up the stairs, only to see that the train was a little distance away.

In 2019, when I had just moved back to Bangalore from London, I had declared that the air conditioning in the Bangalore Metro is the best ever in the city. Unfortunately post-covid protocols mean that the train is kept at a much warmer temperature than usual. So on the way to the office, I kept sweating like a pig.

The train wasn’t too crowded, though. On the green line (till Majestic), everyone was comfortably seated  (despite every alternate seat having been blocked off). I panicked once, though, when a guy seated two seats away from me sneezed. I felt less worried when I saw he was wearing a mask.

The purple line from Majestic was another story. It felt somewhat silly that every alternate seat remained blcoked off when plenty of people were crowding around standing. I must mention, though, that the crowd was nothing like what it normally is. In any case, most of the train emptied out at Vidhana Soudha, and it was a peaceful ride from there on.

40 minute from door to door. Once office starts regularly, I plan to take the metro every day.

The Office

While the office was thinly populated, it felt good being back there. I was meeting several of my colleagues for the first time ever, and it was good to see them in person. We sat together for lunch (ordered from Thai House), and spoke about random things while eating. There was an office boy who, from time to time, ensured that my water glass and bottle were always filled up.

In the evening, one colleague and I went for coffee to the darshini next door. That the coffee was provided in paper cups meant we could safely socially distance from the little crowd at that restaurant. The coffee at this place is actually good – which again bodes well for my office.

And then some usual office-y things happened. I was in a meeting room doing a call with my team when someone else knocked asking if he could use the room. I got into a constant cycle of “watering and dewatering”, something I always do when I’m in an office. The combination of the thin attendance and the office boy, though, meant that there was no need to crowd around the water cooler.

I guess this is what 2020 has done to us. Normally, going to office to work should be the “most normal and boring thing ever”. However, 2020 means that it is now an event worth blogging about. Then again, I don’t need much persuasion to write about anything, do I?

Should this have been my SOP?

I was chatting with a friend yesterday about analytics and “data science” and machine learning and data engineering and all that, and he commented that in his opinion a lot of the work mostly involves gathering and cleaning the data, and that any “analytics” is mostly around averaging and the sort.

This reminded me of an old newsletter I’d written way back in January 2018, soon after I’d read Raphael Honigstein‘s Das Reboot. A short discussion ensued. I sent him the link to that newsletter. And having read the bit about Das Reboot (I was talking about how SAP had helped the German national team win the 2014 FIFA World Cup) and the subsequent section of the newsletter, my friend remarked that I could have used that newsletter edition as a “statement of purpose for my job hunt”.

Now that my job hunt is done, and I’m no more in the job market, I don’t need an SOP. However, for the purpose that I don’t forget this, and keep in mind the next time I’m applying for a job, I’m reproducing a part of that newsletter here. Even if you subscribed to that newsletter, I recommend that you read it again. It’s been a long time, and this is still relevant.

Das Reboot

This is not normally the kind of book you’d see being recommended in a Data Science newsletter, but I found enough in Raphael Honigstein’s book on the German football renaissance in the last 10 years for it to merit a mention here.

So the story goes that prior to the 2014 edition of the Indian Premier League (cricket), Kolkata Knight Riders had announced a partnership with tech giant SAP, and claimed that they would use “big data insights” from SAP’s HANA system to power their analytics. Back then, I’d scoffed, since I wasn’t sure if the amount of data that’s generated in all cricket matches till then wasn’t big enough to merit “big data analytics”.

As it happens, the Knight Riders duly won that edition of the IPL. Perhaps coincidentally, SAP entered into a partnership with another champion team that year – the German national men’s football team, and Honigstein dedicates a chapter of his book to this, and other, partnerships, and the role of analytics in helping the team’s victory in that year’s World Cup.

If you look past all the marketing spiel (“HANA”, “big data”, etc.) what SAP did was to group data, generate insights and present it to the players in an easily consumable format. So in the football case, they developed an app for players where they could see videos of specific opponents doing things. It made it easy for players to review certain kinds of their own mistakes. And so on. Nothing particularly fancy; simply simple data put together in a nice easy-to-consume format.

A couple of money quotes from the book. One on what makes for good analytics systems:

‘It’s not particularly clever,’ says McCormick, ‘but its ease of use made it an effective tool. We didn’t want to bombard coaches or players with numbers. We wanted them to be able to see, literally, whether the data supported their gut feelings and intuition. It was designed to add value for a coach or athlete who isn’t that interested in analytics otherwise. Big data needed to be turned into KPIs that made sense to non-analysts.’

And this one on how good analytics can sometimes invert hierarchies, and empower the people on the front to make their own good decisions rather than always depend on direction from the top:

In its user-friendliness, the technology reversed the traditional top-down flow of tactical information in a football team. Players would pass on their findings to Flick and Löw. Lahm and Mertesacker were also allowed to have some input into Siegenthaler’s and Clemens’ official pre-match briefing, bringing the players’ perspective – and a sense of what was truly relevant on the pitch – to the table.

A lot of business analytics is just about this – presenting the existing data in an easily consumable format. There might be some statistics or machine learning involved somewhere, but ultimately it’s about empowering the analysts and managers with the right kind of data and tools. And what SAP’s experience tells us is that it may not be that bad a thing to tack on some nice marketing on top!

Hiring data scientists

I normally don’t click through on articles in my LinkedIn feed, but this article about the churn in senior data scientists caught my eye enough for me to click through and read the whole thing. I must admit to some degree of confirmation bias – the article reflected my thoughts a fair bit.

Given this confirmation bias, I’ll spare you my commentary and simply put in a few quotes:

Many large companies have fallen into the trap that you need a PhD to do data science, you don’t.

Not to mention, I have yet to see a data science program I would personally endorse. It’s run by people who have never done the job of data science outside of a lab. That’s not what you want for your company.

Doing data science and managing data science are not the same. Just like being an engineer and a product manager are not the same. There is a lot of overlap but overlap does not equal sameness.

Most data scientists are just not ready to lead the teams. This is why the failure rate of data science teams is over 90% right now. Often companies put a strong technical person in charge when they really need a strong business person in charge. I call it a data strategist.

I have worked with companies that demand agile and scrum for data science and then see half their team walk in less than a year. You can’t tell a team they will solve a problem in two sprints. If they don’t’ have the data or tools it won’t happen.

I’ll end this blog post with what my friend had to say (yesterday) about what I’d written about how SAP helped the German National team. “This is what everyone needs to do first. (All that digital transformation everyone is working on should be this kind of work)”.

I agree with him on this.

Proper Job

For the first time in over nine years, I’m taking up one of these.

If someone, sometime, were to do a compendium of stories of people whose careers changed because of covid-19, then I might feature in it. To be very honest, my present career change had been in the works for a while now. However, a bunch of things that covid-19 forced upon me this year made it that much easier to take the plunge.

As the more perceptive of you might have observed by now, I quit full time employment to embark on a “portfolio life” in late 2011. Apart from getting control over my own time, this change allowed me to do a lot of interesting things apart from my “core work”, which I took on such that most of the work I did was things I was good at or interested in.

So over the last nine years, apart from doing a lot of very interesting consulting work around data and analytics and AI and ML and “data science” and all that, I did a lot of interesting stuff otherwise as well. I wrote a book. I wrote a column for Mint. I taught at IIMB. I did public policy work for Takshashila.

I met lots of people and had loads of interesting discussions. There were times, yes, when I went into every meeting or catchup with a “sales mindset”, trying to sell something to someone. Thankfully these times were infrequent, and short. At all other times, I enjoyed all these random catchups, without any expectation  that anything come out of it.

My network expanded like crazy during these years. For the first time in my life, I came to be known for something apart from entrance exams. I spent time living in other places. I “followed my wife” when she first went to Barcelona, and then to London. It was all smooth.

In any case, you might be wondering how the pandemic resulted in my transition to employment being easier. The main way in which it has eased this transition is by ruining my carefully constructed lifestyle of the last nine years.

I’ve loved going around and meeting people. On an average, I would meet two to three people a week, for things completely unrelated to work. That has come down to nearly zero in the last nine months.

I had grown used to having massive control of my time and schedule. The prolonged school shutdown has completely sent it for a toss, with shared childcare responsibilities. “If I don’t have control over my time any ways, I might as well take up a job”, went one line of my reasoning.

I sometimes think I have a fear of open offices (I’ve felt this even during my consulting times when some clients have asked me to do “face time” in their offices). I hate having other people looking at my screen when I’m working. Maybe it has to do with some bad bosses / colleagues I’ve had over the years. The pandemic means I start working from the comfort of my home. And by the time I go to an office I will have hopefully settled down in this job.

And speaking of offices, the pandemic has normalised remote or hybrid working to an extent that I applied to jobs without having the constraint that they necessarily need to have an office in Central Bangalore. The company I’m joining – I’m not sure I would have thought of them in a “normal job search”. As it happens, while they’re not primarily based here, they do have a small office not far from Central Bangalore, and I’ll be going there once it reopens.

Then, thanks to the pandemic, I have successfully concluded my jobhunt without stepping out of home. All interviews, with a big range of companies, happened through video conferencing. In terms of my personal experience, Zoom >> Teams >> Meet.

But yeah, the biggest impact of the pandemic has  been on my lifestyle. So many things that I craved, and took as given, have been taken away from my life, that changing lifestyle seems to have become far easier than I had imagined. It’s like the tube strike model. I got shaken out of my earlier local optimum, and that has enabled me to convince myself that this new lifestyle will work.

In any case, I hope this works out. Just before joining, I feel positive, and excited in a good way.

Oh, and I guess I need to add here, and maybe at the beginning of every subsequent post.

All opinions expressed here on this blog are mine, and only mine. They don’t reflect the thoughts or opinions or positions of any organisation(s) that I might be associated with. Also, none of what I write on this blog is to be taken as investment advice. 

 

Join a boss or join a company?

“You don’t quit your job. You quit your boss”.

Versions of this keep popping up on my LinkedIn with amazing regularity. People have told me this in a non-ironic way in personal conversations as well, so I assume that it is true.

And now that I’m back in the job market, I’ve been thinking of a corollary to this – basically, if you apply “backward induction” to the above statement, then it essentially means that you “join a boss” rather than “join a company”?

I mean – if the boss is the reason why you quit a particular job, then shouldn’t you be thinking about this at the time when you’re joining as well? And so, while you’re interviewing and having these conversations, shouldn’t you be on the lookout for potential bad bosses as well?

In that sense, as I go through my hunt, I’ve been evaluating companies not just on the basis of what they do and what they might expect me to do, but also on the basis of what I feel about the people I talk to. In some places, I have an idea on who I could potentially report to, and in some I don’t. However, I treat pretty much everyone I talk to as people I have to potentially report to or work with at some point of time or the other, and evaluate the company based on these conversations.

Sometimes I think this might be too conservative, but at other times I think that this conservatism now is worth any potential trouble later.

What do you think about this approach?

Record of my publicly available work

A few people who I’ve spoken to as part of my job hunt have asked to see some “detailed descriptions” of work that I’ve done. The other day, I put together an email with some of these descriptions. I thought it might make sense to “document” it in one place (and for me, the “obvious one place” is this blog). So here it is. As you might notice, this takes the form of an email.


I’m putting together links to some of the publicly available work that i’ve done.
1. Cricket
I have a model to evaluate and “tell the story of a cricket match”. This works for all limited overs games, and is based on a dynamic programming algorithm similar to the WASP. The basic idea is to estimate the odds of each team winning at the end of each ball, and then chart that out to come up with a “match story”.
And through some simple rules-based intelligence, the key periods in the game are marked out.
The model can also be used to evaluate the contributions of individual batsmen and bowlers towards their teams’ cause, and when aggregated across games and seasons, can be used to evaluate players’ overall contributions.
Here is a video where I explain the model and how to interpret it:
The algorithm runs live during a game. You can evaluate the latest T20 game here:
Here is a more interactive version , including a larger selection of matches going back in time.
Related to this is a cricket analytics newsletter I actively wrote during the World Cup last year. Most Indians might find this post from the newsletter interesting:
2. Covid-19
At the beginning of the pandemic (when we had just gone under a national lockdown), I had built a few agent based models to evaluate the risk associated with different kinds of commercial activities. They are described here.
Every morning, a script that I have written parses the day’s data from covid19india.org and puts out some graphs to my twitter account  This is a daily fully automated feature.
Here is another agent based model that I had built to model the impact of social distancing on covid-19.
tweetstorm based on Bayes Theorem that I wrote during the pandemic went viral enough that I got invited to a prime time news show (I didn’t go).
3. Visualisations
I used to collect bad visualisations.
I also briefly wrote a newsletter analysing “good and bad visualisations”.
4. I have an “app” to predict which single malts you might like based on your existing likes. This blogpost explains the process behind (a predecessor of ) this model.
5. I had some fun with machine learning, using different techniques to see how they perform in terms of predicting different kinds of simple patterns.
6. I used to write a newsletter on “the art of data science”.
In addition to this, you can find my articles for Mint here. Also, this page on my website  as links to some anonymised case studies.

I guess that’s a lot? In any case, now I’m wondering if I did the right thing by choosing “skthewimp” as my Github username.

Core quants and desk quants on main street

The more perceptive of you might have realised that I’m in the job market.

Over the last one month, my search has mostly be “breadth first” (lots of exploratory conversations with lots of companies), and I’m only now starting to “go deep” into some of them. As part of this process, I need to send out a pitch to a company I’ve been in conversation with regarding what I can do for them.

So I’ve been thinking of how to craft my mandate while keeping in mind that they have an existing data science team. And while I was thinking about this problem, I realised that I can model it like how investment banks (at least one that I worked for) do – in terms of “core quants” and “desk quants”.

I have written about this on my blog before – most “data scientists” in industry are equivalent to what investment banks call “core quants”. They are usually highly technically accomplished people; in many cases they are people who were on an academic path that they left to turn to industry. They do very well in “researchy” environments.

They’re great at running long-gestation-period assignments, working on well defined technical problems and expressing their ideas in code. In general, though (I know I’m massively generalising), they are not particularly close to the business and struggle to deal with the ambiguities that business throws at them from time to time.

What I had mentioned in my earlier post is that “main street” (the American word for “general industry”) lacks “desk quants”. In investment banks, desk quants are attached to trading desks and work significantly closer to the business. They may work less on firmwide or long term strategic projects, but their strength is in blending the models and the markets, and building and making simple tweaks to models so that they remain relevant to the business.

And this is the sort of role in which I’m planning to pitch myself – to all potential employers. That while I’m rather comfortable technically, and all sorts of different modelling techniques, I’m not “deep into tech” and like to work close to the markets. I realise that this analogy will be lost on most people, so I need to figure out a better way of marketing myself. Any ideas will be appreciated.

Over the last month or so I’ve been fairly liberal and using my network to get introductions and references. The one thing I’ve struggled with there is how they describe me as. Most people end up describing me as a “data scientist”, and I’m not sure that’s an accurate description of what I do. Then again, it’s my responsibility to help them figure out how best to describe me. And that’s another thing I’m struggling in. “Desk quant” doesn’t translate well.

Meetings from home

For the last eight years, I’ve worked from home with occasional travel to clients’ offices. How occasional this travel has been has mostly depended on how far away the client is, and how insistent they are on seeing my face. Nevertheless, I’ve always made it a point to visit them for any important meetings, and do them in person.

Now, with the Covid-19 crisis, this hybrid model has broken down. Like most other people in the world, I work entirely from home nowadays, even for important meetings.

At the face of this, this seems like a good thing – for example, nowadays, however important a meeting is, the transaction cost is low. An hour long meeting means spending an hour for it (the time taken for prep is separate and hasn’t changed), and there’s no elaborate song-and-dance about it with travel and dressing up and all that.

While this seems far more efficient use of my time, I’m not sure I’m so happy about it. Essentially, I miss the sense of occasion. Now, an important meeting feels no different from an internal meeting with partners, or some trivial update.

Travel to and from an important meeting was a good time to mentally prepare for it, and then take stock of how it was gone. Now, until ten minutes before a meeting, I’m living my life as usual, and the natural boundaries that used to help me prep are also gone.

The other problem with remotely being there in large but important meetings is that it’s really easy to switch off. If you’re not the one who is doing a majority of the talking (or even the listening), it becomes incredibly hard to focus, and incredibly easy to get distracted elsewhere in the computer (it helps if your camera is switched off).

In a “real” physical meeting, however, large the gathering is, it is naturally easy for you to focus (and naturally more difficult to be distracted), and also easier to get involved in the meeting. An online meeting sometimes feels a bit too much like a group discussion, and without visual cues involved, it becomes really hard to butt in and make a point.

So once we are allowed to travel, and to meet, I’m pretty certain that I’ll start travelling a bit for work again. I’ll start with meetings in Bangalore (inter-city travel is likely to be painful for a very long time).

It might involve transaction cost, but a lot of the transaction cost gets recovered in terms of collateral benefits.