Mantras for building actually good generative ai products

Some mantras for building actually good generative ai products

From building in the space in the last few years, I’ve tried to summarize some key learnings. I’m open to being wrong and corrected on any of these concepts. The caveat here is nobody really knows what the future holds for gen ai products, and if they claim to they are probably trying to sell you something. This is a particularly timely post given Apple’s announcements at WWDC this week, which will I think lead to many consumers’ first productive experience with gen ai.

1. Your user shouldn’t know what a prompt is

Back in the 80’s, before GUI’s existed you had to interact with a computer only via the command line. This meant that only users willing to have a back-and-forth input/output experience could get any value from the computer. Sound familiar?

This is basically the same as prompting for ChatGPT and similar systems today. Think of how primitive the old DOS systems are today compared to our Graphic User Interfaces? The personal computer is a completely different (and many times better) product with a GUI. This is how we’ll view prompting vs. the end user experiences that are to come with generative ai products.

2. Chatbots are terrible products for most use cases

Remove the concept of gen ai. Is the ideal user experience still a chatbot? If the answer is no, your gen ai product should also ideally not be a chatbot.

Most end users do not want a chatbot experience. They want a product that delivers value with as little effort as possible.

A lot of people have mistaken chatbots as the goal user experience instead of what they are - a necessary evil to test out a proof-of-concept. It is a lot more difficult to create a product that doesn’t have a chatbot user experience, and everyone is figuring this out in real time, but ultimately the products that get over this hurdle while maintaining end product quality for the user will win.

I think for most cases to get around this, you need to get the clarifying data off the user in some automated fashion and have some logic that fills in the blanks in an intelligent manner. This is probably problem-specific. For example, for marketing use cases like Jasper or Coso.ai you can read in brand guidelines and historic marketing material instead of having the user provide this in a chat experience.

3. Generating drafts and letting a user choose is a limitation of your tech, not a design choice

The core concepts of the previous two points should illustrate this point well enough I think.

Is providing the user with drafts the ideal user experience to solve the problem? If the answer is no, your final product probably shouldn’t do it either.

4. You have to solve the “last mile” problem in a way bespoke to your problem (for now)

Large language models are really good at getting you 90% of the way there. They are currently really bad at the last 10%. This last 10% is often the hardest aspect of building an LLM product, and why so many people settle for chatbot or experiences that generate multiple options.

I don’t think there is a one size fits all solution here. The two prominent solutions seem to be having a human in the loop, or leveraging an LLM Ops solution - some solutions that are being built by friends at the moment include Outerop, Inspeq and Sentify.

Some processes have a natural expected lag, due to historically being a service-based workflow. For example, many businesses are used to marketing cycles running at a weekly/monthly cadence - this creates opportunities for gen ai marketing products to have a human in the loop for now, without the end-user experience suffering.

If anyone is solving this last mile problem with a production product that people are paying for today, without a human being in the loop, I’d absolutely love to hear how you are doing it - james [a-t] coso dot ai.

Thanks a million for reading, please let me know what you agree or disagree with.

James Flynn
CEO @ Coso.ai - Your Social Media Co-Pilot

Subscribe to the newsletter for more