Stephanie M. Lukin
Current Organizations / Institutes
The organizations/institutes from the scholar's latest article
Stephanie M. Lukin has a total of 8 co-authors. Here are the top co-authors by number of shared publications:
- Brett A. Halperin
Human Centered Design & Engineering, University of Washington, Seattle, Washington, USA
3 shared publications - Claire Bonial
DEVCOM Army Research Laboratory
1 shared publication - Clare Voss
Army Research Laboratory, Adelphi, MD, USA
1 shared publication - Cynthia Matuszek
Computer Science & Electrical Engineering, University of Maryland, Baltimore County, Baltimore, USA
1 shared publication Jaci South
Southern Illinois University
1 shared publication- Matthew Marge
CCDC Army Research Laboratory 2800 Powder Mill Rd. 20783 Adelphi MD
1 shared publication
Latest Publications
- Open Access
- 30 June 2025
- Published by Association for Computing Machinery (ACM)
- p. 32-37
Towards Integrated Multimodal Interaction: Merging Immersive 3D Worlds with Language Based Retrieval for 3D Scene Understanding
- 30 June 2025
- Published by Association for Computing Machinery (ACM)
- p. 32-37
Open AccessTowards Integrated Multimodal Interaction: Merging Immersive 3D Worlds with Language Based Retrieval for 3D Scene Understanding
- Open Access
- 25 April 2025
- Published by Association for Computing Machinery (ACM)
- p. 1-18
From Camera-Eye to AI: Exploring the Interplay of Cinematography and Computational Visual Storytelling
- 25 April 2025
- Published by Association for Computing Machinery (ACM)
- p. 1-18
Open AccessFrom Camera-Eye to AI: Exploring the Interplay of Cinematography and Computational Visual Storytelling
- Top Cited PapersOpen Access
- 1 July 2024
- Published by Association for Computing Machinery (ACM)
- p. 619-637
Artificial Dreams: Surreal Visual Storytelling as Inquiry Into AI 'Hallucination'
What does it mean for stochastic artificial intelligence (AI) to “hallucinate” when performing a literary task as open-ended as creative visual storytelling? In this paper, we investigate AI “hallucination” by stress-testing a visual storytelling algorithm with different visual and textual inputs designed to probe dream logic inspired by cinematic surrealism. Following a close reading of 100 visual stories that we deem artificial dreams, we describe how AI “hallucination” in computational visual storytelling is the opposite of groundedness: literary expression that is ungrounded in the visual or textual inputs. We find that this lack of grounding can be a source of either creativity or harm entangled with bias and illusion. In turn, we disentangle these obscurities and discuss steps toward addressing the perils while harnessing the potentials for innocuous cases of AI “hallucination” to enhance the creativity of visual storytelling.
- 1 July 2024
- Published by Association for Computing Machinery (ACM)
- p. 619-637
Top Cited PapersOpen AccessArtificial Dreams: Surreal Visual Storytelling as Inquiry Into AI 'Hallucination'
What does it mean for stochastic artificial intelligence (AI) to “hallucinate” when performing a literary task as open-ended as creative visual storytelling? In this paper, we investigate AI “hallucination” by stress-testing a visual storytelling algorithm with different visual and textual inputs designed to probe dream logic inspired by cinematic surrealism. Following a close reading of 100 visual stories that we deem artificial dreams, we describe how AI “hallucination” in computational visual storytelling is the opposite of groundedness: literary expression that is ungrounded in the visual or textual inputs. We find that this lack of grounding can be a source of either creativity or harm entangled with bias and illusion. In turn, we disentangle these obscurities and discuss steps toward addressing the perils while harnessing the potentials for innocuous cases of AI “hallucination” to enhance the creativity of visual storytelling.
- 22 April 2024
- Published by Defense Technical Information Center (DTIC)
CHRIS-Bot: A Robot for Dialogue and Scene Understanding of Anomalous Environments in Virtual Reality
Robots can play a critical role in supporting human teammates; however, there are many challenges to ensuring effective collaboration under unknown or anomalous conditions. Natural language is a useful method for allowing humans to issue instructions at a high level. However, we further enhance the human-robot dialogue paradigm by increasing the robots ability to provide common ground for the conversation by performing scene understanding and reporting back on its findings. We offer the following contributions in this report: 1) a human-robot vignette centered around the train derailment in East Palestine, Ohio, USA, in February 2023 that we modeled in a simulated platform; 2) a robot implementation to autonomously navigate this 3-Dspace as dictated by natural language instructions using sentence embeddings and cosine similarity for the robots dialogue management; and 3) scene understanding using Vision-Language Models to analyze a visual snapshot of the environment and generate a textual analysis for the human teammate. We conclude with a list of planned tasks to evaluate the models at the algorithmic level as well as for their efficacy in assisting a human in information gathering and disaster relief tasks.
- 22 April 2024
- Published by Defense Technical Information Center (DTIC)
CHRIS-Bot: A Robot for Dialogue and Scene Understanding of Anomalous Environments in Virtual Reality
Robots can play a critical role in supporting human teammates; however, there are many challenges to ensuring effective collaboration under unknown or anomalous conditions. Natural language is a useful method for allowing humans to issue instructions at a high level. However, we further enhance the human-robot dialogue paradigm by increasing the robots ability to provide common ground for the conversation by performing scene understanding and reporting back on its findings. We offer the following contributions in this report: 1) a human-robot vignette centered around the train derailment in East Palestine, Ohio, USA, in February 2023 that we modeled in a simulated platform; 2) a robot implementation to autonomously navigate this 3-Dspace as dictated by natural language instructions using sentence embeddings and cosine similarity for the robots dialogue management; and 3) scene understanding using Vision-Language Models to analyze a visual snapshot of the environment and generate a textual analysis for the human teammate. We conclude with a list of planned tasks to evaluate the models at the algorithmic level as well as for their efficacy in assisting a human in information gathering and disaster relief tasks.
- Open Access
- 19 April 2023
- Published by Association for Computing Machinery (ACM)
- p. 1-21
Envisioning Narrative Intelligence: A Creative Visual Storytelling Anthology
In this paper, we collect an anthology of 100 visual stories from authors who participated in our systematic creative process of improvised story-building based on image sequences. Following close reading and thematic analysis of our anthology, we present five themes that characterize the variations found in this creative visual storytelling process: (1) Narrating What is in Vision vs. Envisioning; (2) Dynamically Characterizing Entities/Objects; (3) Sensing Experiential Information About the Scenery; (4) Modulating the Mood; (5) Encoding Narrative Biases. In understanding the varied ways that people derive stories from images, we offer considerations for collecting story-driven training data to inform automatic story generation. In correspondence with each theme, we envision narrative intelligence criteria for computational visual storytelling as: creative, reliable, expressive, grounded, and responsible. From these criteria, we discuss how to foreground creative expression, account for biases, and operate in the bounds of visual storyworlds.
- 19 April 2023
- Published by Association for Computing Machinery (ACM)
- p. 1-21
Open AccessEnvisioning Narrative Intelligence: A Creative Visual Storytelling Anthology
In this paper, we collect an anthology of 100 visual stories from authors who participated in our systematic creative process of improvised story-building based on image sequences. Following close reading and thematic analysis of our anthology, we present five themes that characterize the variations found in this creative visual storytelling process: (1) Narrating What is in Vision vs. Envisioning; (2) Dynamically Characterizing Entities/Objects; (3) Sensing Experiential Information About the Scenery; (4) Modulating the Mood; (5) Encoding Narrative Biases. In understanding the varied ways that people derive stories from images, we offer considerations for collecting story-driven training data to inform automatic story generation. In correspondence with each theme, we envision narrative intelligence criteria for computational visual storytelling as: creative, reliable, expressive, grounded, and responsible. From these criteria, we discuss how to foreground creative expression, account for biases, and operate in the bounds of visual storyworlds.