Tag: bayesian

Bayesian Reasoning and Indian Philosophy

I’m currently reading a book called How the World Thinks: A global history of philosophy by Julian Baggini. I must admit I bought this by mistake – I was at a bookshop where I saw this book and went to the Amazon website to check reviews. And by mistake I ended up hitting buy. And before I got around to returning it, I started reading and liking it, so I decided to keep it.

In any case, this book is a nice comparative history of world philosophies, with considerable focus on Indian, Chinese, Japanese and Islamic philosophies. The author himself is trained in European/Western philosophy, but he keeps an open mind and so far it’s been an engaging read.

Rather than approaching the topic in chronological order, like some historians might have been tempted to do, this book approaches it by concept, comparing how different philosophies treat the same concept. And the description of Indian philosophy in the “Logic” chapter caught my eye, in the sense that it reminded me of Bayesian logic, and a piece I’d written a few years back.

Talking about Hindu philosophy and logic, Baggins writes:

For instance, the Veda affirms that when the appropriate sacrifice for the sake of a son is performed, a son will be produced. But it is often observed that a son is not produced, even though the sacrifice has been performed. This would seem to be pretty conclusive proof that the sacrifices don’t work and so the Veda is flawed. Not, however, if you start from the assumption that the Veda cannot be flawed.

In other words, Hindu Philosophy starts with the Bayesian prior that the Veda cannot be flawed. Consequently, irrespective of how strong the empirical evidence that the Vedas are flawed, the belief in the Vedas can never change! On the other hand, if the prior probability that the Vedas were flawed were positive but even infinitesimal, then the amount of evidences such as the above (where sacrifices that are supposed to have produced sons but fail to do so) would over time result in the probability of the Vedas being flawed increasing, and soon tending to 1.

In 2015, I had written in Mint about how Bayesian logic can be used to explain online flame wars. There again, I had written about how when people start with extreme opinions (probabilities equal to 0 or 1), even the strongest contrary evidence is futile to get them to change their opinions. And hence in online flame wars you have people simply talking past each other because neither is willing to update their opinions in the face of evidence.

Coming back to Hindu philosophy, this prior belief that the Vedas cannot be flawed reminds me of the numerous futile arguments with some of my relatives who are of a rather religious persuasion. In each case I presented to them what seemed like strong proof that some of their assumptions of religion are flawed. In each case, irrespective of the strength of my evidence, they refused to heed my argument. Now, looking at the prior of a religious Hindu – that the likelihood of the Vedas being flawed is 0 (not infinitesimal, but 0), it is clear why my arguments fell on deaf ears.

In any case, Baggini goes on to say:

By this logic, if ‘a son is sure to be produced as a result of performing the sacrifice’ but a son is not produced, it can only follow that the sacrifice was not performed correctly, however much it seems that it was performed properly. By such argument, the Ny?ya S?tra can safely conclude, ‘Therefore there is no untruth in the Veda.’

Sudden death and the discount rate

It’s six years today since my mother passed away. She died in the early hours of Friday, 23rd October 2009 following a rather brief illness. The official death summary that the hospital issued reported the cause of her death as “sepsis”. She only officially died on the 23rd. As far as I’m concerned, I’d lost her two Mondays earlier, on 12th October 2009, when she complained of extreme breathing difficulty and was put on ventilator in the ICU.

Looking back (this year’s calendar is identical to that of 2009, so memories of that year have been coming back rather strongly this year), I realise that the suddenness with which it all happened have left me with a deep sense of paranoia, which can be described in financial terms as a “high discount rate”.

Having moved back from Gurgaon in June of that year, my mother and I had settled down in a rented house in Tata Silk Farm (she didn’t want to go back to our own house in Kathriguppe where we’d lived until 2008). She had settled well, and living not far from her sisters, had developed a nice routine. There were certain temples she would visit on certain days of the week, for example.

And then suddenly one day in September she complained of breathing trouble (she took thirty minutes to walk from our then house to my aunt’s house, which is only a ten minute walk away). Initial medical tests revealed nothing. More tests were prescribed, as her breathing got worse. There was no diagnosis yet.

She started seeing specialists – a pulmonologist and her cardiovascular surgeon (she had had trouble with some veins for a few years). More tests. Things getting worse. And before we knew it, she was in hospital – for a “routine three day admission” for an invasive test. The test got postponed, and the surgery finally done a week later. She got out of the ICU and remained there for hardly two days before she complained of insane breathing trouble and had to be put on ventilator – the only purpose the 12 days she spent on that served was to help me prepare for her impending death.

In all, it took less than a month end to end – from initially complaining of breathlessness to going on ventilator. What seemed to be a harmless problem leading to death.

I realise it’s caused insane paranoia in me which I’m yet to come out of. Every time I, or a relative or a friend, show minor signs of sickness, I start fearing the worst. I stop thinking about the symptoms in a Bayesian fashion – by looking at prior probabilities of the various illnesses that could be causing them – and overweight the more morbid causes of the symptoms. And that adds paranoia and anxiety to what I’m already suffering from.

Like two weeks back I had a little trouble breathing, but no apparent cold. It wasn’t something that happens to me normally. A quick Bayesian analysis would have revealed that the most probable cause is a sinus (which it was), but I spent half a day wondering what had become of me before I applied Vicks and quickly recovered. When my wife told me a week after she reached the US that she had got a high fever, I got paranoid again before realising that the most probable cause was a flu caused due to a change of seasons (which it was!).

Another consequence of my mother’s rather sudden death in 2009 (and my father’s death in 2007, though that was by no means sudden, as he had been diagnosed with cancer two years earlier) was that I suddenly stopped being able to make plans. I started overestimating the odds of something drastic happening, and planning didn’t make sense in such scenarios, I reasoned. As a consequence I became extremely short-term in my thinking, and couldn’t see beyond a few days away.

There have been several occasions where I’ve left a decision (such as booking tickets for something, for example) until it has been too late. There have been times when I’ve optimised for too short a term in some of my decisions, effectively jacking up my “discount rate”.

I’d written a while earlier about how in case of rare events, the probabilities we observe can be much higher than actual probabilities, and how that can lead to impaired decision-making. Thinking about it now, I’ve seen that playing out in my life over the last six years.  And it will take a considerable amount of effort to become more rational (i.e. use the “true” rather than “observed” probabilities) in these things.

Review: The Theory That Would Not Die

I was introduced to Bayes’ Theorem of Conditional Probabilities in a rather innocuous manner back when I was in Standard 12. KVP Raghavan, our math teacher, talked about pulling black and white balls out of three different boxes. “If you select a box at random, draw two balls and find that both are black, what is the probability you selected box one?” , he asked and explained to us the concept of Bayes’ Theorem. It was intuitive, and I accepted it as truth.

I wouldn’t come across the theorem, however, for another four years or so, until in a course on Communication, I came across a concept called “Hidden Markov Models”. If you were to observe a signal, and it could have come out of four different transmitters, what are the odds that it was generated by transmitter one? Once again, it was rather intuitive. And once again, I wouldn’t come across or use this theorem for a few years.

A couple of years back, I started following the blog of Columbia Statistics and Social Sciences Professor Andrew Gelman. Here, I came across the terms “Bayesian” and “non-Bayesian”. For a long time, the terms baffled me to no end. I just couldn’t get what the big deal about Bayes’ Theorem was – as far as I was concerned it was intuitive and “truth” and saw no reason to disbelieve it. However, Gelman frequently allured to this topic, and started using the term “frequentists” for non-Bayesians. It was puzzling as to why people refused to accept such an intuitive rule.

The Theory That Would Not Die is Shannon Bertsch McGrayne’s attempt to tell the history of the Bayes’ Theorem. The theorem, according to McGrayne,

survived five near-fatal blows: Bayes had shelved it; Price published it but was ignored; Laplace discovered his own version but later favored his frequency theory; frequentists virstually banned it; and the military kept it secret.

The book is about the development of the theorem and associated methods over the last two hundred and fifty years, ever since Rev. Thomas Bayes first came up with it. It talks about the controversies associated with the theorem, about people who supported, revived or opposed it; about key applications of the theorem, and about how it was frequently and for long periods virtually ostracized.

While the book is ostensibly about Bayes’s Theorem, it is also a story of how science develops, and comes to be. Bayes proposed his theorem but didn’t publish it. His friend Price put things together and published it but without any impact. Laplace independently discovered it, but later in his life moved away from it, using frequency-based methods instead. The French army revived it and used it to determine the most optimal way to fire artillery shells. But then academic statisticians shunned it and “Bayes” became a swearword in academic circles. Once again, it saw a revival at the Second World War, helping break codes and test weapons, but all this work was classified. And then it found supporters in unlikely places – biology departments, Harvard Business School and military labs, but statistics departments continued to oppose.

The above story is pretty representative of how a theory develops – initially it finds few takers. Then popularity grows, but the establishment doesn’t like it. It then finds support from unusual places. Soon, this support comes from enough places to build momentum. The establishment continues to oppose but is then bypassed. Soon everyone accepts it, but some doubters remain..

Coming back to Bayes’ Theorem – why is it controversial and why was it ostracized for long periods of time? Fundamentally it has to do with the definition of probability. According to “frequentists”, who should more correctly be called “objectivists”, probability is objective, and based on counting. Objectivists believe that probability is based on observation and data alone, and not from subjective beliefs. If you ask an objectivist, for example, the probability of rain in Bangalore tomorrow, he will be unable to give you an answer – “rain in Bangalore tomorrow” is not a repeatable event, and cannot be observed multiple times in order to build a model.

Bayesians, who should be more correctly be called “subjectivists”, on the other hand believe that probability can also come from subjective beliefs. So it is possible to infer the probability of rain in Bangalore tomorrow based on other factors – like the cloud cover in Bangalore today or today’s maximum temperature. According to subjectivists (which is the current prevailing thought), probability for one-time events is also defined, and can be inferred from other subjective factors.

Essentially, the the battle between Bayesians and frequentists is more to do with the definition of probability than with whether it makes sense to define inverse probabilities as in Bayes’ Theorem. The theorem is controversial only because the prevailing statistical establishment did not agree with the “subjectivist” definition of probability.

There are some books that I call as ‘blog-books’. These usually contain ideas that could be easily explained in a blog post, but is expanded into book length – possibly because it is easier to monetize a book-length manuscript than a blog-length one. When I first downloaded a sample of this book to my Kindle I was apprehensive that this book might also fall under that category – after all, how much can you talk about a theorem without getting too technical? However, McGrayne avoids falling into that trap. She peppers the book with interesting stories of the application of Bayes’ Theorem through the years, and also short biographical tidbits of some of the people who helped shape the theorem. Sometimes (especially towards the end) some of these examples (of applications) seem a bit laboured, but overall, the books sustains adequate interest from the reader through its length.

If I had one quibble with the book, it would be that even after the descriptions of the story of the theorem, the book talks about “Bayesian” and ‘non-Bayesian” camps, and talk about certain scientists “not doing enough to further the Bayesian cause”. For someone who is primarily interested in getting information out of data, and doesn’t care about the methods involved, it was a bit grating that scientists be graded on their “contribution to the Bayesian cause” rather than their “contribution to science”. Given the polarizing history of the theorem, however, it is perhaps not that surprising.

The Theory That Would Not Die: How Bayes’ Rule Cracked the Enigma Code, Hunted Down Russian Submarines, and Emerged Triumphant from Two Centuries of Controversy
by Sharon Bertsch McGrayne
USD 12.27 (Kindle edition)
360 pages (including appendices and notes)