AI is wrong 75.3% of the time.

Ok, maybe not. Probably not

But I just did something AI largely doesn’t do. I admitted uncertainty: AI acts like a first-year associate at an elite consulting firm, not allowing for the possibility that it is wrong.

“Hallucination” is the sanitized term. In plain English: “I made it up and hoped you wouldn’t notice.”

So why all the excitement?

Three big reasons:
– People are talking their book. Companies have already invested hundreds of billions in AI. They need it to work. (A big part of my portfolio is in AI-related companies, so I really want it to work, too!)

– A lot of the excitement around AI is coding tools. “Look at this app I built with a few sentences.” AI is a technology that is over-fitted for coding. There is so much high quality training data out there from sources like Stack Overflow, language documentation, open-source community, university lectures among other things. In product management, this is called “the happy path” – what happens when everything goes right. Clean specs, deterministic outputs, tons of labeled data. Real life isn’t like Stack Overflow.

– It feels like magic. No pop-ups, no autoplay videos, no sites inhaling 6 GB of RAM. Just a clean answer box. Take that, EU cookie banners. But “feels like magic” isn’t the same as being right.

Going outside that domain, things get a lot dicier:
– I did some medical queries and if I’d listened to ChatGPT’s advice, I would be having *literal* hallucinations. It confused a benign drug with a powerful psychotropic.

– Earlier this year, it was still thinking Biden was president. It didn’t even know that Biden had dropped out and Kamala ran for president.

– When I was researching Boeing, ChatGPT’s performance was 80 points off. Boeing’s 5‑year total return was +~30%, ChatGPT said –50%.

Across thousands of queries where I’m a SME, I saw a lot of cracks. Where I’m not, everything just “sounds right.” That’s dangerous.

There are a lot of technical reasons for this that I’ll get into in another post.

I’m still very excited about where AI will go. With tech, we way overestimate what it will do in the next year, but way underestimate what it will do in ten years. (See: cell phones, iPhones, apps, social, Amazon.)

We will get there. Until then, take the responses with a brick of salt.

(I reviewed this post with ChatGPT. It had some good thoughts, including adding the “feels like magic” bullet. It tightened my wording in a few places.)