JZJingtao Zhou

Jingtao Zhou

*This is an algorithmically generated author record based on publicly available information.

Current Organizations / Institutes

The organizations/institutes from the scholar's latest article


Jingtao Zhou has a total of 25 co-authors. Here are the top co-authors by number of shared publications:


Latest Publications

  • Open Access

    InstructPipe: Generating Visual Blocks Pipelines with Human Instructions and LLMs

    by Zhongyi Zhou, Jing Jin, Vrushank Phadnis, Xiuxiu Yuan, Jun Jiang, Xun Qian, Kristen Wright, Mark Sherwood, Jason Mayes, Jingtao Zhou, et al.


  • Open Access

    Experiencing InstructPipe: Building Multi-modal AI Pipelines via Prompting LLMs and Visual Programming

    by Zhongyi Zhou, Jing Jin, Vrushank Phadnis, Xiuxiu Yuan, Jun Jiang, Xun Qian, Jingtao Zhou, Yiyi Huang, Zheng Xu, Yinda Zhang, et al.

    Foundational multi-modal models have democratized AI access, yet the construction of complex, customizable machine learning pipelines by novice users remains a grand challenge. This paper demonstrates a visual programming system that allows novices to rapidly prototype multimodal AI pipelines. We first conducted a formative study with 58 contributors and collected 236 proposals of multimodal AI pipelines that served various practical needs. We then distilled our findings into a design matrix of primitive nodes for prototyping multimodal AI visual programming pipelines, and implemented a system with 65 nodes. To support users’ rapid prototyping experience, we built InstructPipe, an AI assistant based on large language models (LLMs) that allows users to generate a pipeline by writing text-based instructions. We believe InstructPipe enhances novice users onboarding experience of visual programming and the controllability of LLMs by offering non-experts a platform to easily update the generation.


  • Open Access

    Experiencing Visual Blocks for ML: Visual Prototyping of AI Pipelines

    by Ruofei Du, Na Li, Jing Jin, Michelle Carney, Xiuxiu Yuan, Kristen Wright, Mark Sherwood, Jason Mayes, Lin Chen, Jun Jiang, et al.

    We demonstrate Visual Blocks for ML, a visual programming platform that facilitates rapid prototyping of ML-based multimedia applications. As the public version of Rapsai [3], we further integrated large language models and custom APIs into the platform. In this demonstration, we will showcase how to build interactive AI pipelines in a few drag-and-drops, how to perform interactive data augmentation, and how to integrate pipelines into Colabs. In addition, we demonstrate a wide range of community-contributed pipelines in Visual Blocks for ML, covering various aspects including interactive graphics, chains of large language models, computer vision, and multi-modal applications. Finally, we encourage students, designers, and ML practitioners to contribute ML pipelines through https://github.com/google/visualblocks/tree/main/pipelines to inspire creative use cases. Visual Blocks for ML is available at http://visualblocks.withgoogle.com.


  • Solving Tracking Challenges: Lightweight Algorithm for Single Pedestrian Tracking on Nano-UAVs

    by Jingtao Zhou, Lei Feng, Haolin Chen, Bing Liu

    With its pocket size and flexible flight characteristics, the nano-UAV can achieve more stealthy flight. If the full on-board deployment of the single pedestrian tracking algorithm can be realized on the nano-UAV, It will completely unmanned tasks such as personnel monitoring and security inspections, and has a broad application prospects. However, it is very challenging to design single pedestrian tracking algorithms for nano-UAV platforms. On the one hand, nano-UAVs are not able to deploy the algorithms fully on-board due to their strictly limited computing and storage resources; on the other hand, there are various tracking challenges (multi-scale changing target tracking, small-scale target tracking) for single pedestrian tracking tasks. In this paper, we design a lightweight single pedestrian tracking algorithm that ensures its resource footprint meets the limitations of nano-UAVs. The algorithm proposed in this paper consists of a multi-task learning front-end and a fused ReID tracking back-end, and we design a complete algorithm deployment process, including dataset construction, model training quantification and model deployment, which can effectively solve the challenges of current tracking tasks. A high efficiency (computational efficiency reaches 4.8 MACs/Cycle, and energy efficiency reaches 4.7 mJ/frame) has been achieved on the nano-UAV, achieving an accuracy score of 0.988 and a success score of 0.578 with good robustness in flight. The full-board deployment of nano-UAVs can achieve real-time tracking efficiency (28fps), which has application value and greatly promotes the research work of nano-UAVs.