Sejune Joo
Current Organizations / Institutes
The organizations/institutes from the scholar's latest article
Research Subjects
Sejune Joo has a total of 3 co-authors. Here are the top co-authors by number of shared publications:
- Beomjoon Kim
Graduate School of AI, Korea Advanced Institute of Science and Technology, Seoul, Korea
1 shared publication - Dongryung Lee
Graduate School of AI, Korea Advanced Institute of Science and Technology, Seoul, Korea
1 shared publication Kimin Lee
Graduate School of AI, Korea Advanced Institute of Science and Technology, Seoul, Korea
1 shared publication
Latest Publications
- 6 June 2025
- Published by SAGE Publications in The International Journal of Robotics Research
Prime the search: Using large language models for guiding geometric task and motion planning by warm-starting tree search
The problem of relocating a set of objects to designated areas amidst movable obstacles can be framed as a Geometric Task and Motion Planning ( g-tamp ), a subclass of task and motion planning problem (TAMP). Traditional approaches to g-tamp have relied either on domain-independent heuristics or on learning from planning experience to guide the search, both of which typically demand significant computational resources or data. In contrast, humans often use common sense to intuitively decide which objects to manipulate in g-tamp problems. Inspired by this, we propose leveraging Large Language Models (LLMs), which have common sense knowledge acquired from internet-scale data, to guide task planning in g-tamp problems. To enable LLMs to perform geometric reasoning, we design a predicate-based prompt that encodes geometric information derived from a motion planning algorithm. We then query the LLM to generate a task plan, which is then used to search for a feasible set of continuous parameters. Since LLM is prone to mistakes, instead of committing to LLM’s outputs we extend Monte Carlo Tree Search (MCTS) to a hybrid action space and use the LLM to guide the search. Unlike the previous approach that calls an LLM at every node and incurs high computational costs, we use it to warm-start the MCTS with the nodes explored in completing the LLM’s task plan. On six different g-tamp problems, we show our method outperforms previous LLM planners and pure search algorithms. Code can be found at https://github.com/iMSquared/prime-the-search .
- 6 June 2025
- Published by SAGE Publications in The International Journal of Robotics Research
Prime the search: Using large language models for guiding geometric task and motion planning by warm-starting tree search
The problem of relocating a set of objects to designated areas amidst movable obstacles can be framed as a Geometric Task and Motion Planning ( g-tamp ), a subclass of task and motion planning problem (TAMP). Traditional approaches to g-tamp have relied either on domain-independent heuristics or on learning from planning experience to guide the search, both of which typically demand significant computational resources or data. In contrast, humans often use common sense to intuitively decide which objects to manipulate in g-tamp problems. Inspired by this, we propose leveraging Large Language Models (LLMs), which have common sense knowledge acquired from internet-scale data, to guide task planning in g-tamp problems. To enable LLMs to perform geometric reasoning, we design a predicate-based prompt that encodes geometric information derived from a motion planning algorithm. We then query the LLM to generate a task plan, which is then used to search for a feasible set of continuous parameters. Since LLM is prone to mistakes, instead of committing to LLM’s outputs we extend Monte Carlo Tree Search (MCTS) to a hybrid action space and use the LLM to guide the search. Unlike the previous approach that calls an LLM at every node and incurs high computational costs, we use it to warm-start the MCTS with the nodes explored in completing the LLM’s task plan. On six different g-tamp problems, we show our method outperforms previous LLM planners and pure search algorithms. Code can be found at https://github.com/iMSquared/prime-the-search .