Chat has become the default AI interface across B2B SaaS, and for most use cases it's the wrong one. A CPO at a B2B listings management platform told me something recently: buyers ask "Do you have AI?" when what they mean is "Do you have a chat window?" Teams build the widget to answer that question, and most of the time the same LLM embedded in the actual workflow would serve users better.
The same story keeps playing out across B2B SaaS teams. A team spends three months iterating on prompts for their AI assistant. They rewrite the system instructions a dozen times, tune the retrieval pipeline, test two different models. Eval scores look strong and the CEO loves the demo, but then someone checks the analytics: barely any users have tried it more than once. So the team goes back and rewrites the prompt again. Nobody thinks to check where in the product the feature actually lives, and when someone finally does, it takes four clicks to find it.
The numbers back this up. MIT researchers studying 300 enterprise AI deployments found that 95% of generative AI pilots fail to deliver measurable impact. The culprit was what they called a "learning gap" between the tool and the organization around it. Teams assume low adoption means the prompt isn't good enough, or the retrieval needs work, or maybe they should switch models, so they keep iterating on the technical side, which was never the actual problem.