Darkhan Zholtayev
Current Organizations / Institutes
The organizations/institutes from the scholar's latest article
Research Subjects
Darkhan Zholtayev has a total of 5 co-authors. Here are the top co-authors by number of shared publications:
- Aigul Meirmanova
School of Artificial Intelligence and Data Science, Astana IT University, Mangilik Yel Ave., Astana 020000, Kazakhstan
1 shared publication - Akhmetbek Yernar
Computer Science Department, SDU University, Almatinskaya St. 1/1, Almaty Region, Kaskelen 040900, Kazakhstan
1 shared publication - Ayaulym Parmash
Computer Science Department, SDU University, Almatinskaya St. 1/1, Almaty Region, Kaskelen 040900, Kazakhstan
1 shared publication - Azamat Yeshmukhametov
Institute of Smart Systems and Artificial Intelligence (ISSAI), Nazarbayev University, Qabanbay Batyr Ave., Block 1, Astana 010000, Kazakhstan
1 shared publication - Temirlan Meiramkhanov
Astana IT University, Kazakhstan
1 shared publication
Latest Publications
- Open Access
A Hierarchical VLM-to-TD3 Framework with Novel Object Coordinate Estimation and Persistent Spatial Memory for Semantically Guided Indoor Navigation
Autonomous semantic indoor navigation requires robust low-level control and high-level understanding of objects and spatial context in cluttered and partially occluded environments. While deep reinforcement learning (DRL) methods such as twin delayed deep deterministic policy gradient (TD3) enable reactive obstacle avoidance, they typically struggle with long-horizon semantic navigation, where object-location memory and language-level reasoning are required. We present a lightweight hierarchical two-stage framework that, to the best of our knowledge, is introduced for the first time to integrate a locally deployed vision–language model (VLM), semantic object coordinate memory, and a TD3-based DRL controller for language-conditioned indoor navigation. In Stage 1, the robot performs semantic exploration using odometry, 2D LiDAR, and VLM-based object recognition to build a geometric map and store detected object categories with their estimated world coordinates in a structured javaScript object notation (JSON) semantic memory. In Stage 2, a natural language query is used to retrieve the target object coordinates from memory and pass them to a TD3 target point navigation policy, which performs mapless navigation using odometry and RealSense RGB-D perception. The proposed framework combines open-vocabulary VLM-based object coordinate estimation, LiDAR mapping, RGB-D perception, and language grounding within a unified semantic memory representation. Experiments in a ROS-integrated realistic simulation demonstrate consistent goal-reaching performance and improved navigation efficiency compared with an Artificial Potential Field baseline using the same VLM and a DRL + GPT-4o mini configuration. We also compare the proposed VLM-based recognition module with YOLO-World v2.6 and Grounding DINO, showing that the VLM-based approach provides more reliable semantic grounding and target-coordinate estimation in the tested indoor navigation scenarios, particularly for flexible natural language object queries.
Open AccessA Hierarchical VLM-to-TD3 Framework with Novel Object Coordinate Estimation and Persistent Spatial Memory for Semantically Guided Indoor Navigation
Autonomous semantic indoor navigation requires robust low-level control and high-level understanding of objects and spatial context in cluttered and partially occluded environments. While deep reinforcement learning (DRL) methods such as twin delayed deep deterministic policy gradient (TD3) enable reactive obstacle avoidance, they typically struggle with long-horizon semantic navigation, where object-location memory and language-level reasoning are required. We present a lightweight hierarchical two-stage framework that, to the best of our knowledge, is introduced for the first time to integrate a locally deployed vision–language model (VLM), semantic object coordinate memory, and a TD3-based DRL controller for language-conditioned indoor navigation. In Stage 1, the robot performs semantic exploration using odometry, 2D LiDAR, and VLM-based object recognition to build a geometric map and store detected object categories with their estimated world coordinates in a structured javaScript object notation (JSON) semantic memory. In Stage 2, a natural language query is used to retrieve the target object coordinates from memory and pass them to a TD3 target point navigation policy, which performs mapless navigation using odometry and RealSense RGB-D perception. The proposed framework combines open-vocabulary VLM-based object coordinate estimation, LiDAR mapping, RGB-D perception, and language grounding within a unified semantic memory representation. Experiments in a ROS-integrated realistic simulation demonstrate consistent goal-reaching performance and improved navigation efficiency compared with an Artificial Potential Field baseline using the same VLM and a DRL + GPT-4o mini configuration. We also compare the proposed VLM-based recognition module with YOLO-World v2.6 and Grounding DINO, showing that the VLM-based approach provides more reliable semantic grounding and target-coordinate estimation in the tested indoor navigation scenarios, particularly for flexible natural language object queries.