Introduction

Since the emergence of Foundation Models, various organizations have started attempting to harness their power. They do so using Prompt Engineering, Fine Tuning, LoRA (Low Rank Adaptation), RLHF (Reinforcement Learning from Human Feedback), or a combination of these techniques.

From a UX perspective, it’s clear that the introduction of ChatGPT will change the interfaces we need to create. ChatGPT has demonstrated a level of AI interaction never seen before. While conversational interfaces already existed, ChatGPT’s impact makes it the killer application that will be remembered as the start of a revolution.

Another crucial aspect of this revolution is OpenAI’s provision of APIs for programmatic access to Foundation Models, which is comparable to Apple and Google allowing developers to build applications on their mobile operating systems. This API exposure represents an engineering feat perhaps as significant as the models themselves.

In this article, we will explore four emerging UX interaction patterns that leverage these new technologies.

Chat to Explore Contents

Problem: 

Searching for something in particular and answering questions about large amounts of information (documents, PDFs, articles, etc) is not an easy task.

There is a growing demand for a ChatGPT-like interface that can be tailored to specific knowledge domains, whether they involve private or public information. However, it is crucial to develop this interface in a manner that ensures privacy and allows for control over the language model’s output, specifically addressing any potential issues with hallucinations in its responses.

Solution: 

This pattern was quickly adopted by search giants like Google (Bard) and Microsoft (Bing). The Chat to Explore pattern involves conducting an initial search and then refining it using a chat model.

This approach narrows the scope of interaction with the chat model by providing context from the initial search. Both Google and Bing use their search engines to contextualize the models, thus attempting to “tame the beast” behind them.

At Globant we are applying this pattern in several projects already. It’s incredible how this pattern matches the mental model on how to interact with information for almost any user.

Combining Jakob Nielsen’s UX heuristics with the Chat to Explore pattern:

1. Visibility of system status: The chat interface can provide users with real-time feedback as they refine their search query, ensuring they are aware of the system’s status and progress. The system status is shown to the user directly in the thread of the conversation.
2. User control and freedom: Users can switch between search and chat interactions at will, allowing them to explore information and navigate the system in a more intuitive way.
   In this pattern the user has the freedom to chat but at the same time feel in control because the interface is going to present to the user the sources for the responses. At the same time the user can retry different questions to refine an initial response.
3. Consistency and standards: By leveraging a consistent design language for both search and chat components, users will be able to understand and predict the system’s behavior more effectively. With millions of users using Bard, Bing, ChatGPT like interfaces we can ensure some consistency on how to interact with enterprise by following similar interfaces.
4. Flexibility and efficiency: The Chat to Explore pattern enables users to quickly access the desired information, whether through the search functionality or by engaging in a conversation with the AI model.

Chat to Analyze

Problem: 

Users are viewing an entity, document, or spreadsheet on a page or application and have questions about what they see.

Solution:

A key heuristic of interaction is matching the system’s display with the user’s mental model. For example, listing the technical specifications of a product is not enough to help users understand how it fits their needs. With more powerful models, we can engage in conversations using terms that the user understands.

This pattern can be implemented in productivity interfaces, like Office 365, which utilize tool windows to present an assistant for context-sensitive inquiries.This pattern will be used in ERPs and other traditional web interfaces, like the one created in K2B ERP.

Incorporating Nielsen’s heuristics in the Chat to Analyze pattern:

1. Match between system and real world: The chat interface allows users to interact with the system using natural language, bridging the gap between the user’s mental model and the system’s representation of information.
2. Aesthetic and minimalist design: By integrating the chat interface as a complementary tool, the primary interface can remain clutter-free, focusing on essential content and reducing cognitive load.
3. Help users recognize, diagnose, and recover from errors: The chat interface can assist users in understanding errors, providing suggestions, and guiding them through the recovery process.
4. Help and documentation: The Chat to Analyze pattern allows for context-sensitive help, providing users with the information they need without leaving the current interface.

Intent-Based UI Navigation

Problem: 

Users often enter an enterprise system intending to perform a specific task or getting a specific information, but these systems tend to grow in functionalities over time, leading to complex navigation, resulting in decreased user efficiency.

Solution:

LLMs will be used for progressive discovery based on user intentions, combining the productivity of direct manipulation interfaces with conversational ones. Navigation will become hybrid in certain contexts, allowing users to discover new parts of the system using natural language and guiding them to the most productive areas.

Enhancing Nielsen’s heuristics with the Intent-Based Navigation pattern:

1. Recognition rather than recall: By using natural language processing to understand user intentions, this pattern reduces the need for users to remember specific navigation paths or commands.
2. Error prevention: By understanding the user’s intent, the system can guide them to the desired destination, reducing the chance of making navigation errors.
3. Flexibility and efficiency: Intent-Based Navigation offers a more personalized and efficient way to navigate complex systems, as it caters to individual users’ needs and preferences.
4. Consistency and standards: By incorporating the same language model across the system, users can expect consistent responses and guidance, regardless of where they are in the application.

Magic Auto-Complete

Problem: 

Human-generated content often requires summarizing, editing, and formatting, which can be time-consuming.

Solution: 

AI models are pushing the boundaries of completing, summarizing, formatting, and comparing content across various interfaces. This goes beyond simple autocomplete features found in programming environments.

While designing interfaces, we must ask ourselves if it is necessary to fill or write something. If so, we must consider the Magic Auto-Complete interface that will best serve the purpose.

For example in StarMeUp there are several places where we are using this pattern, for generating images content, reviews, etc. In developer tools there are several tools applying this pattern for completing code, summarizing documentation, etc. Productivity tools like Google Apps and Office 365 are applying  this pattern everywhere.  

Integrating Nielsen’s heuristics into the Auto-Magic Complete pattern:

1. Error prevention: Auto-Magic Complete helps users avoid errors by providing context-aware suggestions and reducing the likelihood of input mistakes.
2. Flexibility and efficiency: This pattern caters to both novice and expert users, as it offers intelligent suggestions to save time without compromising the user’s control over content creation.
3. Aesthetic and minimalist design: By seamlessly integrating Magic Auto-Complete into the existing interface, it does not add any unnecessary elements, maintaining a clean and focused design.
4. Recognition rather than recall: Magic Auto-Complete aids users in recalling relevant information by providing suggestions and completions based on their input and the system’s knowledge, reducing the need for users to remember specific details.

Conclusion

In conclusion, we believe that the convergence of the LLMs and the emergence of API for programmatic access to these models, represent an inflection point paralleled to the release of the iPhone (that brings mainstream touch interfaces) and later the developer tools for creating apps for that platform. The UX experience of software changed overnight, and this will also happen in the current AI scenario.

The emergence of LLMs has led to the development of various techniques like Prompt Engineering, Fine Tuning, LoRA, and RLHF. OpenAI’s APIs for programmatic access to these models represent an engineering feat as significant as the models themselves.

We believe that these emerging UX interaction patterns (Chat to Explore, Chat to Analyze, Intent-Based Navigation, and Magic Auto-Complete) cater to users’ needs, providing personalized and efficient ways of navigating complex systems, completing, and enhancing content, and new ways to finding information and answers. By incorporating Jakob Nielsen’s heuristics, designers can ensure that these patterns are usable, efficient, and easy to learn. And they’re only the tip of the iceberg!

As foundation models continue to evolve and become more powerful, and we continue to learn how to apply it better, it’s exciting to imagine what new UX interaction patterns they will enable. 

We’ll be working hard creating this new future, and so should you!