The rapid advancement of Generative AI (GenAI) has democratized visual content production, allowing users to derive high-fidelity outputs from a simple prompt. However, the transactional nature of current tools fundamentally compromises human agency, especially in open-ended tasks that require exploration and iterative refinement. In this talk, I will present my research works that address this challenge by redesigning user interfaces to support programmable visual media. I advocate for a new interaction paradigm of “intent first, nuances on demand” that bridges the gap between abstract prompting and precise direct manipulation. I will demonstrate how this approach restores agency by allowing users to steer the semantic logic, visual structures, and dynamic behaviors of the generated content. The talk concludes by mapping the landscape of future AI tools, proposing a shift from static generation to dynamic, user-led co-evolution.
DataSway: Vivifying Metaphoric Visualization with Animation Clip Generation and Coordination
Animating metaphoric visualizations brings data to life, enhancing the comprehension of abstract data encodings and fostering deeper engagement. However, creators face significant challenges in designing these animations, such as crafting motions that align semantically with the metaphors, maintaining faithful data representation during animation, and seamlessly integrating interactivity. We propose a human-AI co-creation workflow that facilitates creating animations for SVG-based metaphoric visualizations. Users can initially derive animation clips for data elements from vision-language models (VLMs) and subsequently coordinate their timelines based on entity order, attribute values, spatial layout, or randomness. Our design decisions were informed by a formative study with experienced designers (N=8). We further developed a prototype, DataSway, and conducted a user study (N=14) to evaluate its creativity support and usability. A gallery with seven cases demonstrates its capabilities and applications in web-based hypermedia. We conclude with implications for future research on bespoke data visualization animation.
DataWink: Reusing and Adapting SVG-based Visualization Examples with Large Multimodal Models
Creating aesthetically pleasing data visualizations remains challenging for users without design expertise or familiarity with visualization tools. To address this gap, we present DataWink, a system that enables users to create custom visualizations by adapting high-quality examples. Our approach combines large multimodal models (LMMs) to extract data encoding from existing SVG-based visualization examples, featuring an intermediate representation of visualizations that bridges primitive SVG and visualization programs. Users may express adaptation goals to a conversational agent and control the visual appearance through widgets generated on demand. With an interactive interface, users can modify both data mappings and visual design elements while maintaining the original visualization’s aesthetic quality. To evaluate DataWink, we conduct a user study (N=12) with replication and free-form exploration tasks. As a result, DataWink is recognized for its learnability and effectiveness in personalized authoring tasks. Our results demonstrate the potential of example-driven approaches for democratizing visualization creation.
Wait, GPT: How to Build Interactive Scaffolds in Visualization Generation?
Expressive visualizations play a crucial role in effectively conveying information about the data to diverse audiences. Authoring these visualizations often requires a blend of technical skill and creativity to design engaging and memorable representations. This talk explores the intersection of creativity support and the authoring process for communicative visualizations. In combination of our explorations in animation design, we will review representative works in this area, highlighting their key contributions to enabling and enhancing creative processes in visualization design. Through these examples, we will identify common themes and challenges when developing tools and techniques to support creativity in this domain. The talk will conclude by discussing potential avenues for future research and development of creative visualizations in digital storytelling, especially with the recent advance of generative AI.
Wait, GPT! How Can We Support Human-AI Communication in Personalized Visualization Tasks?
—
Societal AI Group, Microsoft Research Asia,Beijing, China, hosted by Haotian Li
—
Hello World Forum, College of Software, Beihang University,Beijing, China, hosted by Zipeng Liu
—
Beijing Institute of Technology,Beijing, China, hosted by Guozheng Li
The rapid advancement of AI has democratized information consumption and communication, enabling a broader audience to create visualizations for diverse personalized tasks. However, fostering effective human-AI communication remains a significant challenge. This challenge lies in helping users understand AI behavior while also giving them control over the content generation process, particularly when data constraints limit workflows. In this talk, I will present three projects that address these challenges through innovative user interface design. By examining design studies in data analysis and content creation tasks, I will highlight how dynamic, user-centered abstractions can bridge the gap between general users and AI. The talk will conclude by discussing broader implications for designing future systems augmenting data-driven workflows.
WaitGPT: Monitoring and Steering Conversational LLM Agent in Data Analysis with On-the-Fly Code Visualization
LLMs support data analysis through conversational user interfaces, as exemplified in OpenAI’s ChatGPT (formally known as Advanced Data Analysis or Code Interpreter). Essentially, LLMs produce code for accomplishing diverse analysis tasks. However, presenting raw code can obscure the logic and hinder user verification. To empower users with enhanced comprehension and augmented control over analysis conducted by LLMs, we propose a novel approach to transform LLM-generated code into an interactive visual representation. In the approach, users are provided with a clear, step-by-step visualization of the LLM-generated code in real time, allowing them to understand, verify, and modify individual data operations in the analysis. Our design decisions are informed by a formative study (N=8) probing into user practice and challenges. We further developed a prototype named WaitGPT and conducted a user study (N=12) to evaluate its usability and effectiveness. The findings from the user study reveal that WaitGPT facilitates monitoring and steering of data analysis performed by LLMs, enabling participants to enhance error detection and increase their overall confidence in the results.
Creativity Support in Visualization Authoring
—
HKUST MSBD 5005 (Guest Lecture),Hong Kong SAR, China
Communicative visualizations play a crucial role in effectively conveying information about the data to diverse audiences. Authoring these visualizations often requires a blend of technical skill and creativity to design engaging and memorable representations. This talk explores the intersection of creativity support and the authoring process for communicative visualizations. We will review representative works in this area, highlighting their key contributions to enabling and enhancing creative processes in visualization design. Through these examples, we will identify common themes and challenges when developing tools and techniques to support creativity in this domain. The talk will conclude by discussing potential avenues for future research and development, especially with the recent advance of generative AI.
Wakey-Wakey: Animate Text by Mimicking Characters in a GIF
With appealing visual effects, kinetic typography (animated text) has prevailed in movies, advertisements, and social media. However, it remains challenging and time-consuming to craft its animation scheme. We propose an automatic framework to transfer the animation scheme of a rigid body on a given meme GIF to text in vector format. First, the trajectories of key points on the GIF anchor are extracted and mapped to the text’s control points based on local affine transformation. Then the temporal positions of the control points are optimized to maintain the text topology. We also develop an authoring tool that allows intuitive human control in the generation process. A questionnaire study provides evidence that the output results are aesthetically pleasing and well preserve the animation patterns in the original GIF, where participants were impressed by a similar emotional semantics of the original GIF. In addition, we evaluate the utility and effectiveness of our approach through a workshop with general users and designers.
We propose emordle, a conceptual design that animates wordles (compact word clouds) to deliver their emotional context to the audiences. To inform the design, we first reviewed online examples of animated texts and animated wordles, and summarized strategies for injecting emotion into the animations. We introduced a composite approach that extends an existing animation scheme for one word to multiple words in a wordle with two global factors: the randomness of text animation (entropy) and the animation speed (speed). To create an emordle, general users can choose one predefined animated scheme that matches the intended emotion class and fine-tune the emotion intensity with the two parameters. We designed proof-of-concept emordle examples for four basic emotion classes, namely happiness, sadness, anger, and fear. We conducted two controlled crowdsourcing studies to evaluate our approach. The first study confirmed that people generally agreed on the conveyed emotions from well-crafted animations, and the second one demonstrated that our identified factors helped fine-tune the delivered emotion extent. We also invited general users to create emordles on their own based on our proposed framework. Through this user study, we confirmed the effectiveness of the approach. We concluded with implications for future research opportunities of supporting emotion expression in visualizations.
Interative Time-Series of Measures for Exploring Dynamic Networks
We present MeasureFlow, an interface to visually and interactively explore dynamic networks through time-series of network measures such as link number, graph density, or node activation. When networks contain many time steps, become large and more dense, or contain high frequencies of change, traditional visualizations that focus on network topology, such as animations or small multiples, fail to provide adequate overviews and thus fail to guide the analyst towards interesting time points and periods. MeasureFlow presents a complementary approach that relies on visualizing time-series of common network measures to provide a detailed yet comprehensive overview of when changes are happening and which network measures they involve. As dynamic networks undergo changes of varying rates and characteristics, network measures provide important hints on the pace and nature of their evolution and can guide an analysts in their exploration; based on a set of interactive and signal-processing methods, MeasureFlow allows an analyst to select and navigate periods of interest in the network. We demonstrate MeasureFlow through case studies with real-world data.