JSJinwoo Shin

Jinwoo Shin

*This is an algorithmically generated author record based on publicly available information.

Current Organizations / Institutes

The organizations/institutes from the scholar's latest article


Jinwoo Shin has a total of 176 co-authors. Here are the top co-authors by number of shared publications:


Latest Publications

  • SpatialBoost: Enhancing Visual Representation Through Language-Guided Reasoning

    by Byungwoo Jeon, Dongyoung Kim, Huiwon Jang, Insoo Kim, Jinwoo Shin

  • Open Access

    Extremely Low-resolution RFID Vision for Real-time and Visually-anonymized Action Recognition

    by Hyeonho Shin, Seungwoo Shim, Myeongkyun Cho, Youngki Lee, Jinwoo Shin, Song Min Kim

    Despite the potential of vision-based personal monitoring (e.g., healthcare), private data leakage concerns hinder its wide deployment in personal spaces (e.g., bedrooms). A body of data anonymization designs was proposed throughout image processing and federated learning. They commonly store high-quality images and videos locally, which are anonymized via post-processing before cloud upload. However, the recent IoT camera hacking and local data leakage call for anonymized data at the sensing stage. Also, continuous and pervasive monitoring without blind spots in complicated indoor spaces requires a scalable and economic system. This article presents Mosaic , a vision-based end-to-end action recognition framework that (i) intrinsically achieves data anonymity from the sensing stage and (ii) battery-free operation for blind spot-free continuous monitoring. Mosaic leverages an extremely low resolution (eLR) Near-Infrared (NIR) image sensor with 6 \(\times\) 10 pixels for video anonymity and an RFID-compliant fully-passive tag with four solar cells for real-time eLR video streaming under as low as 30 lux (e.g., deep in the shelf without direct light). This is accompanied by a lightweight action recognition neural network for real-time inference (18.4 ms on Intel(R) Core i7-8700). Mosaic achieves an average of 98% accuracy on 10 action classes, hitting the balance between data anonymity and high-precision action recognition. Taking advantage of the NIR (non-visible) frequency, Mosaic also works in the dark without disturbing sleep. Lastly, wildfire detection reaching 20 m was demonstrated, showcasing the potential for outdoor monitoring.


  • Open Access

    RoboCurate: Harnessing Diversity with Action-Verified Neural Trajectory for Robot Learning

    by Seungku Kim, Suhyeok Jang, Byungjun Yoon, Dongyoung Kim, John Won, Jinwoo Shin

    Synthetic data generated by video generative models has shown promise for robot learning as a scalable pipeline, but it often suffers from inconsistent action quality due to imperfectly generated videos. Recently, vision-language models (VLMs) have been leveraged to validate video quality, but they have limitations in distinguishing physically accurate videos and, even then, cannot directly evaluate the generated actions themselves. To tackle this issue, we introduce RoboCurate, a novel synthetic robot data generation framework that evaluates and filters the quality of annotated actions by comparing them with simulation replay. Specifically, RoboCurate replays the predicted actions in a simulator and assesses action quality by measuring the consistency of motion between the simulator rollout and the generated video. In addition, we unlock observation diversity beyond the available dataset via image-to-image editing and apply action-preserving video-to-video transfer to further augment appearance. We observe RoboCurate's generated data yield substantial relative improvements in success rates compared to using real data only, achieving +70.1% on GR-1 Tabletop (300 demos), +16.1% on DexMimicGen in the pre-training setup, and +179.9% in the challenging real-world ALLEX humanoid dexterous manipulation setting.

  • Open Access

    Vision-aligned Latent Reasoning for Multi-modal Large Language Model

    by Byungwoo Jeon, Yoonwoo Jeong, Hyunseok Lee, Minsu Cho, Jinwoo Shin

    Despite recent advancements in Multi-modal Large Language Models (MLLMs) on diverse understanding tasks, these models struggle to solve problems which require extensive multi-step reasoning. This is primarily due to the progressive dilution of visual information during long-context generation, which hinders their ability to fully exploit test-time scaling. To address this issue, we introduce Vision-aligned Latent Reasoning (VaLR), a simple, yet effective reasoning framework that dynamically generates vision-aligned latent tokens before each Chain of Thought reasoning step, guiding the model to reason based on perceptual cues in the latent space. Specifically, VaLR is trained to preserve visual knowledge during reasoning by aligning intermediate embeddings of MLLM with those from vision encoders. Empirical results demonstrate that VaLR consistently outperforms existing approaches across a wide range of benchmarks requiring long-context understanding or precise visual perception, while exhibiting test-time scaling behavior not observed in prior MLLMs. In particular, VaLR improves the performance significantly from 33.0% to 52.9% on VSI-Bench, achieving a 19.9%p gain over Qwen2.5-VL.

  • Open Access

    Scalable and Robust LLM Unlearning by Correcting Responses with Retrieved Exclusions

    by Junbeom Kim, Kyuyoung Kim, Jihoon Tack, Dongha Lim, Jinwoo Shin

    Language models trained on web-scale corpora risk memorizing and exposing sensitive information, prompting the need for effective machine unlearning. Prior methods mainly focus on input queries to suppress sensitive outputs, yet this often fails to eliminate the underlying knowledge and limits scalability. To address this, we propose Corrective Unlearning with Retrieved Exclusions (CURE), a novel unlearning framework that verifies model outputs for leakage and revises them into safe responses. Specifically, CURE employs a lightweight corrector that is applied to the original model to verify whether outputs contain target knowledge and to rewrite them if any leakage is detected. To efficiently handle large-scale unlearning requests, CURE retrieves unlearning targets that are relevant to the initial response and provides them as in-context references to the corrector for detection and conditional revision. By leveraging this retrieval augmentation, the corrector can adapt to new unlearning requests without additional training. Extensive evaluations demonstrate that CURE substantially reduces information leakage, even from indirect queries where prior works fall short, while maintaining response quality and general utility. Moreover, it demonstrates robustness under continual unlearning scenarios, making it practical for real-world applications.

  • Open Access

    StarFT: Robust Fine-tuning of Zero-shot Models via Spuriosity Alignment

    by Younghyun Kim, Jongheon Jeong, Sangkyung Kwak, Kyungmin Lee, Juho Lee, Jinwoo Shin

    Learning robust representations from data often requires scale, which has led to the success of recent zero-shot models such as CLIP. However, the obtained robustness can easily be deteriorated when these models are fine-tuned on other downstream tasks (e.g., of smaller scales). Previous works often interpret this phenomenon in the context of domain shift, developing fine-tuning methods that aim to preserve the original domain as much as possible. However, in a different context, fine-tuned models with limited data are also prone to learning features that are spurious to humans, such as background or texture. In this paper, we propose StarFT (Spurious Textual Alignment Regularization), a novel framework for fine-tuning zero-shot models to enhance robustness by preventing them from learning spuriosity. We introduce a regularization that aligns the output distribution for spuriosity-injected labels with the original zero-shot model, ensuring that the model is not induced to extract irrelevant features further from these descriptions. We leverage recent language models to get such spuriosity-injected labels by generating alternative textual descriptions that highlight potentially confounding features. Extensive experiments validate the robust generalization of StarFT and its emerging properties: zero-shot group robustness and improved zero-shot classification. Notably, StarFT boosts both worst-group and average accuracy by 14.30% and 3.02%, respectively, in the Waterbirds group shift scenario, where other robust fine-tuning baselines show even degraded performance.

  • Open Access

    Sparsified State-Space Models are Efficient Highway Networks

    by Woomin Song, Jihoon Tack, Sangwoo Mo, Seunghyuk Oh, Jinwoo Shin

    State-space models (SSMs) offer a promising architecture for sequence modeling, providing an alternative to Transformers by replacing expensive self-attention with linear recurrences. In this paper, we propose a simple yet effective trick to enhance SSMs within given computational budgets by sparsifying them. Our intuition is that tokens in SSMs are highly redundant due to gradual recurrent updates, and dense recurrence operations block the delivery of past information. In particular, we observe that upper layers of SSMs tend to be more redundant as they encode global information, while lower layers encode local information. Motivated by this, we introduce Simba, a hierarchical sparsification method for SSMs based on token pruning. Simba sparsifies upper layers more than lower layers, encouraging the upper layers to behave like highways. To achieve this, we propose a novel token pruning criterion for SSMs, measuring the global impact of tokens on the final output by accumulating local recurrences. We demonstrate that Simba outperforms the baseline model, Mamba, with the same FLOPS in various natural language tasks. Moreover, we illustrate the effect of highways, showing that Simba not only enhances efficiency but also improves the information flow across long sequences. Code is available at https://github.com/woominsong/Simba.

  • NeFL: Nested Model Scaling for Federated Learning With System Heterogeneous Clients

    by Honggu Kang, Seohyeon Cha, Jinwoo Shin, Jongmyeong Lee, Joonhyuk Kang

    Federated learning (FL) enables distributed training while preserving data privacy, but stragglers—slow or incapable clients can significantly slow down the total training time and degrade performance. To mitigate the impact of stragglers, system heterogeneity, including heterogeneous computing and network bandwidth, has been addressed. While previous studies have addressed system heterogeneity by splitting models into submodels, they offer limited flexibility in model architecture design, without considering potential inconsistencies arising from training multiple submodel architectures. We propose nested federated learning (NeFL), a generalized framework that efficiently divides deep neural networks into submodels using both depthwise and widthwise scaling. To address the inconsistency arising from training multiple submodel architectures, NeFL decouples a subset of parameters from those being trained for each submodel. An averaging method is proposed to handle these decoupled parameters during aggregation. NeFL enables resource-constrained devices to effectively participate in the FL pipeline, facilitating larger datasets for model training. Experiments demonstrate that NeFL achieves performance gain, especially for the worst-case submodel compared to baseline approaches (7.63% improvement on CIFAR-100). Furthermore, NeFL aligns with recent advances in FL, such as leveraging pre-trained models and accounting for statistical heterogeneity.


  • Open Access

    Training Text-to-Molecule Models with Context-Aware Tokenization

    by Seojin Kim, Hyeontae Song, Jaehyun Nam, Jinwoo Shin

    Seojin Kim, Hyeontae Song, Jaehyun Nam, Jinwoo Shin. Findings of the Association for Computational Linguistics: EMNLP 2025. 2025.

  • Open Access

    Personalized Language Models via Privacy-Preserving Evolutionary Model Merging

    by Kyuyoung Kim, Jinwoo Shin, Jaehyung Kim

    Kyuyoung Kim, Jinwoo Shin, Jaehyung Kim. Proceedings of the 2025 Conference on Empirical Methods in Natural Language Processing. 2025.