board exams – Pertinent Observations

RG@ICSE,ISC

I did my higher education at two “institutes of national importance”. Both institutions followed what is called “relative grading”. It didn’t matter on an absolute scale how well or how badly you did. Your grade for the course would depend on how everyone else who took the course did. So for example, there was this one course at IIT Madras where I got 80/100, and got an S grade (the highest grade possible). The general performance of the class had not been great, so in that course 80 merited an S. In another course, however, 80 fetched me only a B (the third highest grade) – the general performance of the class had been much better.

While IITs and IIMs and some other autonomous institutions practice relative grading, it is not the “done thing” in most of the rest of India. Most of our board and university exams follow what is known as “absolute grading” – your grade for the course depends solely on your performance, without taking into account the performance of others. So it is theoretically possible to have a case where practically everyone in the class scores “90%”. Given that this is the prevailing system of grading in most of India, we assume that the board exams follow this principle, too.

Two or three days back, Debarghya Das, a student at Cornell set a cat among the pigeons by scraping the marks of every single student who took the ICSE or ISC exams (10th and 12th board respectively administered by the CICSE). What he noticed was that certain marks had gone missing – for example nobody scored 81, 82, 84, 85, 87, 89, 91 or 93 in any of the courses. This is just a sample of marks that have gone missing. There are several other numbers which are effectively “unattainable” in any of the courses. Das, on his account, has alleged some kind of “fraud”.

What is the first thing that comes to your mind when you see this rather jagged distribution? I wouldn’t blame you if you saw a hedgehog. But can you think of a graph that looks like that?

Three years back I bought myself a DSLR camera, after which I pretend to be an expert photographer. I even use Photoshop/Gimp to manipulate some of the images I click. And a decidedly much better photographer friend has told me that the first thing you do while editing a photo is to adjust “levels”. See this to know what you can do with levels. Basically, the concept is that some parts of the colour spectrum are unrepresented in an image, and by adjusting levels you make sure the full spectrum is used, thus improving the contrast of the image.

There is something known as the image histogram. I took a picture that I had shot and adjusted the “levels”. On the left you see what the histogram looks like after the levels. On the right, you see the histogram as it was before you adjusted the levels.

Image histogram after (left) and before (right) adjusting the levels of an image. From a random photograph I had shot

Doesn’t the histogram on the left remind you of the distribution of ICSE/ISC marks? And how did we get that histogram? By taking the histogram on the right (which is smoothed but all bunched up in one part of the distribution) and stretching it so that it falls across the entire distribution. And what happened when we did that? We got gaps, as you can see in the histogram on the left or the distribution of ICSE/ISC marks.

There is an article in The Hindu today that again explores this issue of missing marks in ICSE/ISC. In that the ICSE council, which administers these exams is quoted saying:

“In keeping with the practice followed by examination conducting bodies, a process of standardisation is applied to the results, so as to take into account the variations in difficulty level of questions over the years (which may occur despite applying various norms and yardsticks), as well as the marginal variations in evaluation of answer scripts by hundreds of examiners (inter-examiner variability), for each subject.”

Another money quote from the same article:

“The word tampering is wrong. There is moderation that happens across education boards,” explained a teacher, who has worked with ICSE schools in Hyderabad and Chennai. “After the first round of corrections, raw data is given to officials and head examiners who analyse how students have performed. They try to ensure the bell curve of the results does not look awkward. If it does, the implication is that the checking has been either too liberal or very strict.”

So there you go. The ICSE Council effectively follows relative grading. There is a certain distribution of marks that they desire, and they adjust the “levels” of the overall distribution of marks so that the desired distribution is achieved. The desired distribution of marks is something like “X% students get between 95 and 100, Y% get between 90 and 95”, and so on. Now, two students who had got the same number of marks as per the initial marking have to get the same number of marks after recalibration. So what the missing marks indicates is that there was clustering – a large number of students had ended up scoring in the same narrow range, and so after normalization, this range got expanded because of which you have gaps. Now, when certain sections of the range in the middle are expanded, some at the end have to get contracted (for example, if someone who originally got 70 is given 90, a person who originally got 90 deserves so much more). Which is why you see that at one end – 94-100 all possible marks are represented.

This still doesn’t explain one thing though – why is it that the same marks have gone missing in all subjects? It is impossible that the initial distribution of marks was identical across subjects. I have only one explanation for this – there was one overall mapping algorithm that was used across subjects, that converted marks obtained to the relative marks. This is also seen in the fact that the shape of the distribution across subjects varies widely (again refer to Das’s post).

So that explains the weird distribution of marks in the ICSE / ISC exams. But what explains the title of the post? In IITian English, “RG” is a term derived from “relative grading”. It is a rather derogatory term used to describe people who prefer to pull down others in their quest to get ahead (note that this is a consequence of relative grading). Taking some more liberties and using IITian English, you can say that the ICSE/ISC board has “RGed” students!

Common kids versus coaching factories

Given that I can consider it as my “specialist subject” I continue to comment on entrance exams. Glanced through an article about changes in medical entrance exams today in the newspaper, which talked about “eliminating negative marking”, and also talked about having a common entrance exams for all colleges (didn’t read in enough detail to figure out the details).

Now, when you have a broad-based entrance exam which is supposed to cater to people of varying backgrounds, there is a need to keep it simple. There is a need to announce a “syllabus”, and stick to it. And to pre-announce a format which will allow students to prepare adequately for the exam. The problem with this, however, is that it plays right into the hands of coaching factories, whose influence the examiners want to try and reduce. Given a syllabus and a format, it becomes easier to cram for an exam without understanding fundamentals, and this is what coaching schools want.

When the format of the examination is unknown, it becomes harder to “prepare for the exam”, and all one can do is to “prepare the concepts”. In theory, a random examination format allows the examiner to examine concepts better, and doesn’t give unfair advantage to people who go to coaching factories.

That makes me wonder if the attempt to make heavily-coached entrance exams “easier” (this applies to IIT admissions also) can be explained with a baptists-and-bootleggers argument. The baptists in this case are the inclusionists, who want to keep entrance exam papers simple and reasonably deterministic so that “common kids” are not disadvantaged. The bootleggers are the coaching factories, since a deterministic exam will make it easier to coach and thus increase their demand.

For the same reasons, the move to using board exam scores for IIT admissions is daft. Board exams are inherently designed to make people pass, which means they have a defined syllabus and a deterministic format. Use of that for something as competitive as IIT admissions is only going to play into the hands of coaching factories.

College Admissions

Why does the government require colleges in India to have “objective criteria” for admissions? I understand that such criteria are necessary for government-owned or run or aided colleges where there’s scope for rent seeking. But why is it that “private” colleges are also forced to adopt “objective criteria” such as board exam marks or entrance test scores for admission?

Abroad, and here, too for MBA admissions, admission is more “subjective”. While of course this has the scope to introduce bias, and is a fairly random process (though I’d argue that the JEE is also a fairly random process), won’t it reduce pressure on the overall student population, and bring in more diversity into colleges?

As a natural experiment I want to see a few state governments deregulating the admissions process for private colleges, making it possible for the colleges to choose their students based on whatever criterion. So what would happen? Of course, some seats would be “reserved” for those with big moneybags. Some more would be reserved for people who are well connected with the college management. But would it be rational for the college boards to “reserve” all the seats this way?

Maybe some colleges would take a short-term view and try to thus “cash out”. The cleverer ones will realize that they need to build up a reputation. So while some seats will be thus “reserved”, others will be used to attract what the college thinks are “good students”. Some might define “good students” to be those that got high marks in board exams. Others might pick students based on how far they can throw a cricket ball. The colleges have a wide variety of ways in which to make a name, and they’ll pick students accordingly.

The problem with such a measure is that there is a transient cost. A few batches of students might get screwed, since they wouldn’t have figured out the reputations of colleges (or maybe not – assuming colleges don’t change drastically from the way they are right now). But in a few years’ time, reputations of various sorts would have been built. Colleges would have figured out various business models. The willingness to pay of the collective population would ensure that reasonably priced “seats” are available.

And remember that I mentioned that a few states should implement this, with the others sticking to the current system of regulating admissions and fees and all such. In due course of time it’ll be known what works better. Rather, it’ll be known what the students prefer.

It’s crazy that colleges now require students to get “cent per cent” in their board exams as a prerequisite to admission. It’s crazy that hundreds of thousands of students all over India, every year, spend two years of their prime youth just preparing to get into a good college (nowadays the madness is spreading. A cousin-in-law is in 9th standard, and he’s already joined JEE coaching). On reflection, it’s crazy that I wasted all of my 12th standard simply mugging, for an exam that would admit me to a college that I knew little about. Madness, sheer madness.

Anecdotes from school: Copying In exams

A couple of not-so-hilarious incidents from our pre-board exams in 10th standards. It being election year (1998) we had 2 rounds of pre-boards instead of the usual one. The formation in the classroom was interesting – we sat normally two to a desk, and there were two sets of question papers. Since these were pre-boards and not boards, many of us didn’t really take them seriously. I must say that the entire set of exams was a riot. After all, it was the last thing that we did in that wonderful school (the school didn’t have 11th and 12th, so all of us had to shift out).

The biology section of the science exam contained a question on habit-forming substances. Something on the lines of “what are habit forming substances and why are they bad”. A certain mahaanubhaavva thought he didn’t know the answer. Or maybe he didn’t understand the question properly. So using a set of excellently-planned cheat codes, he managed to communicate to the guy in the next row (note that he couldn’t ask the person next to him since she had a different question paper) about this question.

The guy in the next row wasn’t such a stud in dumb-charades, and decided to use standard gestures rather than excellently-worked-out codes. He wanted to show booze in as intuitive way as possible. Putting his fist near his mouth, and with a clever movement of his thumb, he indicated drink. Sitting behind him, I thought this was excellent for someone not well-versed in Dumb Charades. Unfortunately, people well-versed in Dumb Charades tend to think too much. In went the answer paper “the primary habitat forming substance is water. It is bad because people and animals can fall and drown in it”. He must count himself lucky he got the hall ticket.

This incident has had far-reaching consequences. The mahaanubhaava who didn’t know the answer was so traumatised by the incident that he is yet to taste alcohol. He is afraid of drowning in it – that dreaded habitat forming substance.

****************

One day later was the social sciences paper. Unfortunately I wasn’t part of the Dumb Charades study group, so I hadn’t been introduced to the art of communicating the question number across the class. I realized that with my skills I was unable to even communicate across the aisle. I wasn’t even as good as the guy in front of me who put his fist to his mouth. So it had to be the cute girl next to me who had to help me out with the question that I didn’t have a clue of. If I remember right, she was partially trained in Dumb Charades.

What I didn’t realize was that you are not supposed to copy if you are seated in the first row – it is too easy to get caught. Moreover, if you are in one of the middle columns (like I was) you are in the direct line of vision of the invigilator. So it is never a good idea to copy. But then, I’d never copied in my life, and I knew this was the last opportunity for me to make amends. So what if I didn’t know the codes? So what if I was seated on the first row? So what if the cute girl next to me had a different set of questions? This was my last chance to profitably copy, and I had to take it.

I usually pride myself on being good at eye contact. I pride myself on the fact that I can communicate anything to someone of the opposite gender by just looking deeply into her eyes. I know that if I were to copy from a girl who was seated in FRONT of me, I could have done it with just eye contact. Unfortunately, the only person seated across me and looking towards me was the invigilator. Obviously I couldn’t ask her the answer?

The rules of copying state that it is always the dumber person who copies from the smarter person. The class topper never copies. If he were a copycat, his topping could never have been this sustainable. By knownig the two names, you can easily know who is the copier and who is the copied. Things always go by the rules. So if you try to invert these rules, it is usually easy to fool the invigilator. And so forth.

So unlike the mahaanubhaava who hadn’t understood the question, I didn’t get caught for the attempt to copy. No one threatened to not give me my hall ticket – that honour went to the cute girl who had been sitting next to me. I didn’t do well in my social science pre-boards – I hadn’t been able to get the answer from the benchmate – she had got caught for copying from me before that. Despite now knowing the codes, and having zero experience in this department, I had played my cards well. I never repeated this experiment. Even if I wnated to, I think I’d’ve never found a counterparty.