Chat Limitations, Rise of LLM’s, Generative UIs
Chat Limitations
As AI becomes more prevalent in our daily lives, and with the rise in popularity of LLMs, search and chat are becoming growing mediums of user interaction. However, it is unlikely that chat will replace all other modes of interaction or become the main mode of interaction. Chat is a limited interaction mode, regardless of the quality of the model or bot underneath. Take this example from OpenAI’s Greg Brockman where he uses ChatGPT to trim the first 5 seconds of a video using natural language.
Although this is an impressive showcase of the capabilities of the underneath technology this is a task that's much better suited for a different type of UI that allows direct manipulation (I don’t see myself going back and forth with a chatbot trying to figure out the best exact time to trim a video file)
It is important to consider the nature of the task and the mode of interaction that would be most effective.
Another example where chat may not always be the optimal mode of interaction would be e-commerce. E-commerce involves a lot of discovery and browsing which is much more suitable for a full-blown UI unless you really know what you want and are looking only for a very specific thing.
One possible solution is to integrate chat with some browsing elements. For example, a chat assistant could suggest items to browse based on previous purchases or interests. It could also provide a curated list of items that the user might be interested in and redirect you to a complete UI experience with chat acting as a starting point.
Rise of LLMs
The improvement of LLMs could lead to unexpected changes in software usage. One potential outcome is the democratization of AI-generated software. There are already impressive examples of GPT-4 generated code or even UI’s:
https://twitter.com/mlejva/status/1641151421830529042?s=20
These capabilities and their improvements as these models get more powerful and faster could lead to a world where UI’s are generated on demand based on the user intent, optimized for a specific need or action
https://twitter.com/sgrove/status/1640417065650778113?s=20
Also in a world where code is just a natural language command away extending systems could become a mandatory feature, especially in the world of SAAS where you could much more easily adapt the software to your needs
https://twitter.com/mitchellh/status/1638967450510458882?s=20
Generative UIs
“As one’s reach is amplified, so to one’s vision” Opening the hood of a word processor - Alan Kay
This led me to think about the concept of UIs that can be generated on the fly, not just within a chat interface but within a full-blown application, that can include ai powered search and chat.
For example, if you have a design system and provide the AI with information about each component and its purpose, it could self-assemble a UI for a user based on a request to complete a specific task or a search intent. The user doesn’t need to start from a blank chat, but also may not need any navigation and just use natural language to browse and discover.
The experience could then be refined further by either chatting or using direct manipulation filters, depending on what's more efficient for the user and the task at hand.
This opens possibilities for applications that adapt to the user's task and self-assemble based on a natural language request.
Applications in this world will be able to showcase and interact with everything an API provides within a design system representation and LLM-defined boundaries.