Augmenting human-centered design research with AI.
Screenshot of the proof-of-concept web application.
I've been curious for a while how AI can help with synthesizing user research, and help design teams interact with the knowledge they've already gained from customer touchpoints. I’ve experienced the huge time commitment that it takes to interview research participants, and review what was said to find key insights that repeat across participants that can be acted upon to inform design strategy.
It’s common practice for transcripts to be produced from recordings of user research. From experience, it would be a great help to design teams and researchers to have a tool to parse these transcripts to identify key information and insights about users, without having to repeatedly listen to the recordings or depend on notes. Not only is doing so labor-intensive, but parsing research this way is subject to the biases of the person doing so. If the most pertinent information to glean from each session is understood, it follows that this could be used as a template to prompt AI to do the parsing for us.
Furthermore, if we are able to distill the important details from each user, we could use them in combination to simulate the “user”, using the available information to experimentally extrapolate their reactions to different concepts without having to conduct additional research.
To that end, I've developed a proof-of-concept web application I’m calling “Chatty Insights” for synthesizing research transcripts into summaries and using those summaries to launch a "conversation" with the "user". The app takes the large token count, raw transcripts and digests them into a succinct meta-summary and interactive chat using prompt engineering, off-model storage, and GPT-4 through the OpenAI API.
The application uses this prompting strategy to to effectively focus the information from transcripts:
It takes in transcript files and summarizes each using a user-entered summarization prompt.
It merges the summaries into a meta-summary with the same structure.
It begins a conversation with the application user, using the meta-summary as the basis for its character.
The idea behind this strategy is:
The UX Researcher will be able to change the summarization as needed by changing the summarization prompt, given the focus of the design research project.
The AI model will not get "lost in the sauce" from the very large context window that would be necessary for processing all transcripts at once (this may not be as much of an issue in future models, such as Gemini 1.5 Pro, but is currently a limitation of GPT-4).
Processing time for the conversational interface will be reduced from one that attempts to use all transcript data simultaneously.
Creating a more abstracted version of a set of users to converse with may make it easier for UX Researchers to avoid cognitive biases, such as gravitating to the mindset of specific memorable interviewees. This should help them better understand overall thought patterns in the user set.
I implemented this proof-of-concept application with client-side JavaScript managing the details of synthesizing the transcripts and the chatting interface. The server-side is a Node.js/Express application hosted on AWS to handle interop with the OpenAI API, as to not expose or require an API key (Update August 2024: I’ve taken down the AWS hosted version due to low utilization).
I’m planning to keep working on this project. Some improvements I’d like to make are:
Better communicate the processing activity of the application and estimated processing time.
Break the individual transcript and meta summaries out of the chat window interface and into individual cards that can be opened and closed for viewing.
Develop a preprocessor to scrub personal details from transcripts.
Optimize the tradeoff between having more summary tasks in one prompt to run on transcripts, potentially missing accurate gleaning of some details, or re-running transcripts with different requested focuses, incurring more processing time and cost.