Average skill and peak skill

One way to describe how complex a job is is to measure the “average level of skill” and “peak level of skill” required to do the job. The more complex the job is, the larger this difference is. And sometimes, the frequency at which the peak level of skill is required can determine the quality of people you can expect to attract to the job.

Let us start with one extreme – the classic case of someone  turning screws in a Ford factory. The design has been done so perfectly and the assembly line so optimised that the level of skill required by this worker each day is identical. All he/she (much more likely a he) has to do is to show up at the job, stand in the assembly line, and turn the specific screw in every single car (or part thereof) that passes his way.

The delta between the complexity of the average day and the “toughest day” is likely to be very low in this kind of job, given the amount of optimisation already put in place by the engineers at the factory.

Consider a maintenance engineer (let’s say at an oil pipeline) on the other hand. On most days, the complexity required of the job is very close to zero, for there is nothing much to do. The engineer just needs to show up and potter around and make a usual round of checks and all izz well.

On a day when there is an issue however, things are completely different – the engineer now needs to identify the source of the issue, figure out how to fix it and then actually put in the fix. Each of this is an insanely complex process requiring insane skill. This maintenance engineer needs to be prepared for this kind of occasional complexity, and despite the banality of most of his days on the job, maintain the requisite skill to do the job on these peak days.

In fact, if you think of it, a lot of “knowledge” jobs, which are supposed to be quite complex, actually don’t require a very high level of skill on most days. Yet, most of these jobs tend to employ people at a far higher skill level than what is required on most days, and this is because of the level of skill required on “peak days” (however you define “peak”).

The challenge in these cases, though, is to keep these high skilled people excited and motivated enough when the job on most days requires pretty low skill. Some industries, such as oil and gas, resolve this issue by paying well and giving good “benefits” – so even an engineer who might get bored by the lack of work on most days stays on to be able to contribute in times when there is a problem.

The other way to do this is in terms of the frequency of high skill days – if you can somehow engineer your organisation such that the high skilled people have a reasonable frequency of days when high skills are required, then they might find more motivation. For example, you might create an “internal consulting” team of some kind – they are tasked with performing a high skill task across different teams in the org. Each time this particular high skill task is required, the internal consulting team is called for. This way, this team can be kept motivated and (more importantly, perhaps) other teams can be staffed at a lower average skill level (since they can get help on high peak days).

I’m reminded of my first ever real taste of professional life – an internship in an investment bank in London in 2005. That was the classic “high variance in skills” job. Having been tested on fairly extreme maths and logic before I got hired, I found that most of my days were spent just keying in numbers in to an Excel sheet to call a macro someone else had written to price swaps (interest rate derivatives).

And being fairly young and immature, I decided this job is not worth it for me, and did not take up the full time offer they made me. And off I went on a rather futile “tour” to figure out what kind of job has sufficient high skill work to keep me interested. And then left it all to start my own consultancy (where others would ONLY call me when there was work of my specialty; else I could chill).

With the benefit of hindsight (and having worked in a somewhat similar job later in life), though, I had completely missed the “skill gap” (delta between peak and average skill days) in my internship, and thus not appreciated why I had been hired for it. Also, that I spent barely two months in the internship meant I didn’t have sufficient data to know the frequency of “interesting days”.

And this is why – most of your time might be spent in writing some fairly ordinary code, but you will still be required to know how to reverse a red-black tree.

Most of your time might be spent in writing SQL queries or pulling some averages, but on the odd day you might need to know that a chi square test is the best way to test your current hypothesis.

Most of your time might be spent in managing people and making sure the metrics are alright, but on the odd day you might have to redesign the process at the facility that you are in charge of.

In most complex jobs, the average day is NOT similar to the most complex day by any means. And thus the average day is NOT representative of the job. The next time someone I’m interviewing asks me what my “average day looks like”, I’ll maybe point that person to this post!

Computer science and psychology

This morning, when I got back from the gym, my wife and daughter were playing 20 questions, with my wife having just taught my daughter the game.

Given that this was the first time they were playing, they started with guessing “2 digit numbers”. And when I came in, they were asking questions such as “is this number divisible by 6” etc.

To me this was obviously inefficient. “Binary search is O(log n)“, I realised in my head, and decided this is a good time to teach my daughter binary search.

So for the next game, I volunteered to guess, and started with “is the number \ge 55“? And went on to “is the number \ge 77“, and got to the number in my wife’s mind (74) in exactly  7 guesses (and you might guess that \lceil log_2 90 \rceil (90 is the number of 2 digit numbers) is 7).

And so we moved on. Next, I “kept” 41, and my wife went through a rather random series of guesses (including “is it divisible by 4” fairly early on) to get in 8 tries. By this time I had been feeling massively proud, of putting to good use my computer science knowledge in real life.

“See, you keep saying that I’m not a good engineer. See how I’m using skills that I learnt in my engineering to do well in this game”, I exclaimed. My wife didn’t react.

It was finally my daughter’s turn to keep a number in mind, and my turn to guess.

“Is the number \ge 55?”
“Yes”

“Is the number \ge 77?”
“Yes”

“Is the number \ge 88?”
“Yes”

My wife started grinning. I ignored it and continued with my “process”, and I got to the right answer (99) in 6 tries. “You are stupid and know nothing”, said my wife. “As soon as she said it’s greater than 88, I knew it is 99. You might be good at computer science but I’m good at psychology”.

She had a point. And then I started thinking – basically the binary search method works under the assumption that the numbers are all uniformly distributed. Clearly, my wife had some superior information to me, which made 99 far more probable than any number between 89 and 98. And s0 when the answer to “Is the number \ge 88?”turned out to by “yes”, she made an educated guess that it’s 99.

And since I’m used to writing algorithms, and  teaching dumb computers to solve problems, I used a process that didn’t make use of any educated guesses! And thus took far many more steps to get to the answer.

When the numbers don’t follow a uniform distribution, binary search works differently. You don’t start with the middle number – instead, you start with the weighted median of all the numbers! And then go on to the weighted median of whichever half you end up in. And so on and so forth until you find the number in the counterparty’s mind. That is the most optimal algo.

Then again, how do you figure out what the prior distribution of numbers is? For that, I guess knowing some psychology helps.

 

Books, Music, Disruption and Distribution

Having watched this short film by The Economist on disruption in the music business, I find the parallels between the books and the music businesses uncanny.

https://youtu.be/eAGPvnF7q7U

Both industries have been traditionally controlled by the middlemen – labels in the case of music, and publishers in the case of books. Both sets of middlemen are oligopolies – there are three big music labels and four (?) major publishers. This is primarily a result of production costs – traditionally, professional recording equipment has been both expensive and hard to get. Similarly, typesetting and printing a book was expensive business.

However, both industries have been massively disrupted in the last couple of decades, primarily thanks to new distribution models – streaming in the case of music, and online vendors and e-books in the case of books. Simultaneously, the cost of production have also plummeted – I can get studio quality recording and mixing software on my Macbook Pro, and I already have a version of my book that looks good on the Kindle.

Yet, in both industries, the incumbents strongly believe that they continue to add value despite the disruption, and staunchly defend the value of the marketing and distribution they bring. In the above video, for example, a record studio executive talks about how established artistes may do well going “indie”, but new artistes require support in production, marketing and distribution.

If you see blogs and news articles on publishing and self-publishing, on the other hand, most of the talk is about how little value publishers themselves bring into the marketing and distribution process. While publishers continue to have a broad monopoly on the traditional distribution chain (bookstores, primarily), they have no particular competitive advantage in the new channels.

One of the successful indie artistes interviewed in the above video talks about how he was successful thanks to the brand and following he built up on social media, which ensured that his album had several takers as soon as it was released. It is again similar to advice that authors who want to self-publish get!

As someone who has completed a book manuscript and is looking for production and distribution options, I find the developments in the indie space (across products) rather interesting. Going by all this, maybe I should just give up on the “stamp of approval” I’m looking for from a traditional publisher, and go indie myself!

I leave you with a few lines from one of my favourite poems, which I believe is a commentary about the music record label industry!

Now the frog puffed up with rage.
“Brainless bird – you’re on the stage –
Use your wits and follow fashion.
Puff your lungs out with your passion.”
Trembling, terrified to fail,
Blind with tears, the nightingale
Heard him out in silence, tried,
Puffed up, burst a vein, and died.