LAVE: LLM-Powered Agent Assistance and Language Augmentation for Video Editing
Authors:
- Bryan WangBWBryan Wang
- Yuliang LiYLYuliang Li
- Zhaoyang LvZLZhaoyang Lv
- Haijun XiaHXHaijun Xia
- Yan XuYXYan Xu
- Raj SodhiRSRaj Sodhi
Preprint
Abstract
Video creation has become increasingly popular, yet the expertise and effort required for editing often pose barriers to beginners. In this paper, we explore the integration of large language models (LLMs) into the video editing workflow to reduce these barriers. Our design vision is embodied in LAVE, a novel system that provides LLM-powered agent assistance and language-augmented editing features. LAVE automatically generates language descriptions for the user's footage, serving as the foundation for enabling the LLM to process videos and assist in editing tasks. When the user provides editing objectives, the agent plans and executes relevant actions to fulfill them. Moreover, LAVE allows users to edit videos through either the agent or direct UI manipulation, providing flexibility and enabling manual refinement of agent actions. Our user study, which included eight participants ranging from novices to proficient editors, demonstrated LAVE's effectiveness. The results also shed light on user perceptions of the proposed LLM-assisted editing paradigm and its impact on users' creativity and sense of co-creation. Based on these findings, we propose design implications to inform the future development of agent-assisted content editing.Keywords
All Related Versions
- Version 1, 2024-02-15, ArXiv
- Published version: , 699.
This publication has 38 references indexed in Scilit:
- Improved Baselines with Visual Instruction Tuning2024
- Humans as Creativity Gatekeepers: Are We Biased Against AI Creativity?Journal of Business and Psychology, 2023
- AI agency vs. human agency: understanding human–AI interactions on TikTok and their implications for user engagementJournal of Computer-Mediated Communication, 2022
- Ownership, liability, patentability, and creativity issues in artificial intelligenceInformation Security Journal: A Global Perspective, 2022
- In AI We Trust? Effects of Agency Locus and Transparency on Uncertainty Reduction in Human–AI InteractionJournal of Computer-Mediated Communication, 2021
- Human ownership of artificial creativityNature Machine Intelligence, 2020
- Text-based editing of talking-head videoACM Transactions on Graphics, 2019
- How to Design Voice Based Navigation for How-To Videos2019
- Explainable AI for Designers: A Human-Centered Perspective on Mixed-Initiative Co-Creation2018
- Quantifying the Creativity Support of Digital Tools through the Creativity Support IndexACM Transactions on Computer-Human Interaction, 2014