Tag: ml

Stable Diffusion and Chat GPT and Logistic Regression

For a long time I have had this shibboleth on whether someone is a “statistics person or a machine learning person”. It is based on what they call regressions where the dependent variable is binary. Statisticians simply call it “logit” (there is also a “probit“).

Now, in terms of implementation as well, there is one big difference between the way “logit” is modelled versus “logistic regression”. For a logit model (if you are using python, you need to use the “statsmodels” package for this, not scikit learn), the number of observations needs to far exceed the number of independent variables.

Else, a matrix that needs to be inverted as part of the solution will turn out to be singular, and there will be no solution. I guess I betrayed my greater background in statistics than in Machine Learning when, in 2018, I wrote this blogpost on machine learning being a “process to tie down coefficients in maths models“.

For “logistic regression” (as opposed to “logit”) puts no such constraint – on the regression matrix being invertible. Instead of actually inverting the matrix, machine learning approaches simply focus on learning the terms of the inverted matrix using gradient descent (basically the opposite of hill climbing), so mathematical inconveniences such as matrices that cannot be inverted are moot there.

And so you have logistic regression models with thousands of variables, often calibrated with a fewer number of data points. To be honest, I can’t understand this fully – without sufficient information (data points) to calibrate the coefficients, there will always be a sense of randomness in the output. The model has too many degrees of freedom, and so there is additional information the model is supplying (apart from what was supplied in the training data!).

Of late I have been playing a fair bit with generative AI (primarily ChatGPT and Stable Diffusion). The other day, my daughter and I were alone in my in-laws’ house, and I told her “look I’ve brought my personal laptop along, if you want we can play with it”. And she had demanded that she “play with stable diffusion”. This is the image she got for “tiger chasing deer”.

I have written earlier here about how the likes of ChatGPT and Stable Diffusion in a way redefine “information content“.

 

And if you think about it, almost by definition, “generative AI” creates information (and hallucinates, like in the above pic). Traditionally speaking, a “picture is worth a thousand words”, but if you can generate a picture with just a few words of prompt, the information content in it is far less than a thousand words.

In some sense, this reminds me of “logistic regression” once again. By definition (because it is generative), there is insufficient “tying down of coefficients”, because of which the AI inevitably ends up “adding value of its own”, which by definition is random.

So, you will end up getting arbitrary results. ChatGPT often gives you wrong answers to questions. Dall-E and Midjourney and Stable Diffusion will return nonsense images such as the above. Because a “generative AI” needs to create information, by definition, all the coefficients of the model cannot be well calibrated. 

And the consequence of this is that however good these AIs get, however much data is used to train them, there will always be an element of randomness to them. There will always be test cases where they give funny results.

No, AGI is not here yet.

Red wine and mirchi

is such an awesome combo. As we just discovered, here in Monastiraki square in Athens. It was this restaurant called Savas. Specializing in one “Sauvlaki”.

So over the last week or so of vacation, the girlfriend has been complaining of not eating spicy enough food. So as we settle down today, and get our can of wine (yeah, you get half a litre of wine in an aluminium can here. Awesome it is), I see this “spicy hot peppers” on the menu.

And given that the girlfriend has been deprived of spicy food, and I like peppers it doesn’t take long for me to order it. And boy was it hot.

I gave up after a couple of bites of the pepper. No amounts of pita bread and Tzatziki (the Greek version of raita – with cucumber and garlic blended into curd) could cure the hotness on my tongue. With there being no water on the table, I went straight for the wine.

I’ve always suspected it when the girlfriend has claimed to have Gult roots. Of course, I’ve seen a lot of Gult being spoken in her family, and had half my pre-wedding dance party inundated with Gult songs, but still find it hard to accept she’s Gult. And did she prove it! She ate four whole peppers, as I struggled to finish half..

A couple of minutes back, we staggered back to the hotel. Absolutely drunk. We’d had 250 ml of red wine each, “house wine” according to the restaurant. And mirchi. Whatta combo. Surprised the “shady bars” of Bangalore haven’t exploited it yet. Maybe no one drinks wine there.

Coffee in America

I could have finished this post in one word  – “horrible”. But for the sake of blogging and detailed description, let me sacrifice brevity, like I usually do. I’m writing this after having drunk a cup of absolutely atrocious self-made coffee. Yes, it is proper traditional filter coffee made using Coffee Day Ultra Rich powder, but somewhere I seem to have messed it up. And the quality of this coffee, the first time I’ve made the brew after returning from America, reflects the general quality of coffee they make in America!

I think I can count on the fingers of one hand the number of times I had good coffee. One was at a friend’s place in New Jersey, where I had traditional South Indian filter coffee out of a steel tumbler. Another was the Turkish coffee at The Hummus Place in Greenwich Village which I’ve talked about in my previous post. That is it! Maybe the odd capuccino somewhere but I can’t remember anything else specific.

The funny thing about America is the size of the coffee lOTas. The average coffee cup in my office was some 400 ml, and each time I would put it under the machine and ask for capuccino it would get filled up! It was extremely disorienting for someone like me who is used to several small doses of coffee during the day. There was another dispenser which dispensed decoction but that was horrible, too. But later on I started drinking from that since I could then control the volume of each dose!

I think I have mentioned this in some other post but another problem in America is they give you hot black coffee and COLD milk. Again extremely disorienting for someone who is used to coffee made with boiling milk. I’m told that the typical American puts such little milk in his coffee that the temperature of milk doesn’t matter. Just that I found it hard to digest (not literally).

Then there was this coffee maker in my apartment. I had to google to figure out how it worked and then realized that an essential part of making it work was to buy filter paper (the first time I’d come across this thing since high school chemistry lab). Since I didn’t have enthu to buy the said paper, I just made do with the two complimentary sheets that had been kindly provided in my apartment. Needless to say the coffee came out to be horrible and I didn’t use the machine again.

One of my regrets of my America trip is that I didn’t order coffee post my several Italian meals. Maybe the Italian restaurants would have made coffee much better than what was available in the rest of the country. And one of the amusing things i remember from the trip is the length of the queues at the Starbucks outlets! That made me realize that people actually go to Starbucks for coffee unlike us here who use Cafe Coffee Day as a convenient hangout location!

Yesterday I did my bit to make up for all the horrible coffee that I’d endured during my America trip. Had two awesome cups of filter coffee at a friend’s place, and then three doses of “sugarless strong” at three diffferent darshini-level places. Unfortunately this morning’s mess-up (now I realize I put 2 spoons of powder into the filter instead of the usual 4) has taken me back to square one, of American quality coffee.

Interior Design

Recently it has been reported that former ML MD John Thain spent some 1.2 million dollars  in decorating his office. And people say that this is very conservative by normal CEO standards. Normal people (like me) might wonder why one needs to spend so much on one’s office. Even if you were to list out what needs to go into an office, and then go on to buy the best possible item in each category, this kind of money seems obscene.

So if you are still wondering why people end up spending so much on their offices, you will need to get in touch with someone from the profession called interior design. It’s quite fascinating. The way these people think is extremely instructive, and actually it would make sense for an investment bank CEO to learn this from the designer and then use such ideas to trade. They way these people imagine stuff, they comparisons that they make, the associations that they draw, are incredible. Actually, I think interior designers might be good people to partner on a quiz team.

So it cannot be any random painting that needs to go on the walls. The painting needs to have a theme, and this theme needs to fit in with the general theme of the company. And interior designers being interior designers, will develop their own idea of the company’s theme. And then use this to design the office. So coming back, the painting needs to conform to the ideals of the company. Next, the painter who painted this painting needs to conform to the ideals of the company. Put these two together and the painting will cost a bomb. Doesn’t matter, they need to get everything right. And perfect. And in order

Interior designers also seem to be proficient in stuff such as vaastu, feng shui, numerology, and all such. So each and every desk in the office needs to be oriented in the right way. It’s ok if the employee doesn’t have space to stretch his legs. Doesn’t matter if the position of one particular desk means they can’t play gulli cricket in office. It has to be that way.

It is excellent that interior designers do their jobs so diligently. The way they think, their attention to detail, the way they see the big picture, is all extremely good. In fact, interior design is probably one profession where, to succeed, you need to be both a stud and a fighter. So kudos to the entire community. However, there is a small issue.

The biggest problem with interior design is that it’s all so subtle. Ok, the colour of this wall matches the theme colour of the company. But who would notice? Ok, the painter who painted this exquisite painting just outside the CEO’s door might belong to the same moon nakshatra as the CEO. Excellent attention to detail. But does anyone notice it? it is quite a pity. These designers spend so much time and clients’ money in bringing out the perfect design, but most of their excellent thinking, and work, goes unnoticed.

There is this story about Michelangelo painting the Sistine Chapel. He was painting an extremely dark corner, which was out of eyesight of most visitors, or maybe all visitors to the Chapel. Someone goes up to him and asks why he is taking so much trouble in painting this particular nook when no one will notice it. He replies that he is doing it because God is watching. Extremely commendable. And I suppose interior designers also work on the same principle. However, I’m not sure if Michelangelo billed any additional amount to the Chapel for painting this unseen corner.

The other day, I was talking to my uncle about the design of his drawing room in his Gurgaon house. He mentioned to me that soon after he bought the place, he had called an interior designer to help him design the drawing room. The lady broadly told him about her plans for the house, which my uncle seemed to appreciate, and they sat down to discuss fees. The deal was that the interior designer would instruct my uncle about where he needs to get each and every piece of his furniture from. She would determine the design, the designer and the shops. And my uncle would have to do exactly as she said. And here is the clincher: the interior designer’s fee would be 2% of my uncle’s total expenses on his drawing room.

I don’t think incentives can be more misaligned than this. You get paid to help your client spend his/her money, and the more money you make your client spend, the more money you make. So it is always in your best interest to make sure that the client spends as much as possible. The only limitation might be the client’s budget, but your incentives make sure that you will stretch it to its limit. In case of professional CEO’s, they don’t really have limits, and it is their shareholders that pay. Which is why you get situations like Thain’s expense of $1.2mm on his office room being considered low by industry standards.

It intrigues me as to how interior design fee structures have settled down this way. And the only thing I can think of is that most people are spending someone else’s money. Their shareholders’ money, in most cases. If I were to engage an interior designer some day, I would try and structure her fees differently. I would tell her (numbers here are indicative only) “I’m willing to spend Rs. 10 lakh, and I will pay you a minimum of 20,000 rupees. For every lakh less than 10 lakh that I end up spending, I will give you Rs. 10,000 more.”  Or maybe not. I may just negotiate a fixed fee.