Tag: hiring

Chat GPT and Information Content

For the last week or so I’ve been fooling around with Chat GPT. It’s been a lot of fun, if you can endure the verbosity of its answers. Yesterday, at work, I used it for tech support when some people I was talking to were using too much jargon.

Before I came home from work yesterday, my last conversation was with a couple of recruiters, about a role that I’m hiring for. So when I got home, on a whim, I wondered if I could check out of Chat GPT could write job descriptions for me.

I asked:

write a job description for a senior data scientist at India’s largest logistics company. 4-7 years exp. Needs to have experience in ML, OR and statistics, and be able to work close to business and on ambiguous problems

It answered (pasting in full since the screenshot above is limited):

Job Title: Senior Data Scientist

Company: India’s Largest Logistics Company

Location: India

Experience: 4-7 years

We are seeking an experienced and highly skilled Senior Data Scientist to join our team at India’s largest logistics company. In this role, you will be responsible for using machine learning, operations research, and statistics to drive business insights and decision making.

As a Senior Data Scientist, you will have a deep understanding of data science techniques and technologies, and will be able to apply them to complex and ambiguous problems. You will be able to work closely with business teams to understand their needs and to develop solutions that drive value for the company.

To be successful in this role, you should have at least 4-7 years of experience in data science, and should have a strong background in machine learning, operations research, and statistics. You should be able to work independently and be comfortable working on complex and ambiguous problems.

Responsibilities:

  • Use machine learning, operations research, and statistics to drive business insights and decision making
  • Work closely with business teams to understand their needs and develop solutions
  • Develop and maintain data pipelines and systems
  • Collaborate with other data scientists and cross-functional teams to develop and implement solutions

Qualifications:

  • Bachelor’s or Master’s degree in a quantitative field, such as computer science, statistics, or engineering
  • At least 4-7 years of experience in data science
  • Strong background in machine learning, operations research, and statistics
  • Ability to work independently and on complex and ambiguous problems
  • Excellent communication and collaboration skills
  • Experience with Python, R, or other programming languages is a plus

We offer a competitive salary and benefits package, as well as opportunities for growth and advancement. If you are an experienced data scientist with a passion for driving business insights and decision making, we encourage you to apply.

In the past when I’ve been hiring, I’ve been a bit annoyed by people asking for job descriptions. I’ve largely found them to be fluff and boilerplate, and I’ve tried to get away with a sentence or two. However, due to the insistence of recruiters and candidates alike, I’ve ended up writing elaborate job descriptions. I don’t need to any more.

This post, however, is not about hiring (but yeah – I am hiring, and the above job description is reasonably accurate). It’s about information content in general.

Traditionally, information content has been measured in bits. A bit is the amount of information contained in an equi-probable binary event.

Sometimes when we find that someone is too verbose and using too many words when fewer would suffice, we say that their bit rate is low. We also use “low bit rate” to describe people such as former Prime Minister Atal Behari Vajpayee, who would speak incredibly slowly.

However, beyond the bit, which is a fairly technical concept, it has been difficult to quantify information content. Sometimes you read an article or a story and find that there is nothing much to it. But given the natural language, and the context of various words, it is impossible to quantify the information content.

Now, with Chat GPT, maybe it becomes a bit easier (though one would need a “reverse chat GPT algo”, to find the set of prompts required for Chat GPT to churn out a particular essay). Above, for example, I’ve shown how much fluff there generally is to the average job description – a fairly short prompt generated this longish description that is fairly accurate.

So you can define the information content of a piece or essay in terms of the number of words in the minimum set of prompts required for Chat GPT (or something like it) to come up with it. If you are a boring stereotypical writer, the set of prompts required will be lower. If you are highly idiosyncratic, then you will need to give a larger number of prompts for Chat GPT to write like you. You know where I’m going.

This evening, in office, a colleague commented that now it will be rather easy to generate marketing material. “Even blogs might become dead, since with a few prompts you can get that content”, he said (it can be a legit service to build off the Chat GPT API to take a tweet and convert it into an essay).

I didn’t tell him then but I have decided to take it up as a challenge. I consider myself to be a fairly idiosyncratic writer, which means I THINK there is a fair bit of information content in what I write, and so this blog will stay relevant. Let’s see how it goes.

PS: I still want to train a GAN on my blog (well over a million words, at last count) and see how it goes. If you know of any tools I can use for this, let me know!

 

The Misfit Job Market

Exactly 15 years ago, I was looking for a job. I had graduated from IIMB four months earlier, taken my first ever full time job 3 months earlier, and was already serving notice. Very quickly on, I had figured that I was not a good fit for the job that I had taken up, and so decided to cut my losses and move on.

The only problem was job hunting was hard. Back then, most people I spoke to seemed suspicious of me because I was getting out of my first job so early. For the longest time (years later), people spoke to me as if there was something wrong with me because I had quit my first job within three months. Finally I ended up taking a 20% pay cut to take another job where I seemed a better fit.

Thinking back, I don’t think I’m alone. The sheer randomness of the campus placement process means that a lot of people end up in jobs that they are ill suited for, purely based on a bit of bad judgment here and a lucky interview there. And most smart people figure out quickly enough that in case they are in jobs they are not a good fit for, it’s better to cut losses and move on. If it is their first ever jobs (applies for undergrad jobs, and for MBAs without prior work experience), the desperation to get out of their misfit jobs will be high.

I think this is a highly underserved market. Companies fall head over heels over themselves to access premium slots in the random process called campus placements, without realising that a significant part of the same pool will (theoretically) be available for a proper interview just a few months hence.

5-6 years back, an old friend of mine had started a company which was essentially a clearinghouse targeted at this precise market – to enable companies hire people in their first years of employment. Unfortunately the company didn’t take off, suggesting that the market design problem is not easy to solve.

Anyway, in case you are a just-graduated student who believes you are a misfit in your first job, and instead want to do analytics, get in touch with me. Having been on the other side, I’m more than happy to fish in this pool, and I know that I’ll get some temporarily undervalued talent here.

Just that I don’t know what sort of market or clearinghouse I need to go to to tap this supply, and so I’m putting out a bid here in the form of this blogpost.

PS: In case you’re a recent reader of my blog, I’ve written a book on market design.