Current Organizations / Institutes

The organizations/institutes from the scholar's latest article


Yen-Che Hsiao has a total of 5 co-authors. Here are the top co-authors by number of shared publications:


Latest Publications

  • Shaping robust dynamic inversion control of neural cell dynamics

    by Rongting Yue, Yen-Che Hsiao, Abhishek Dutta

    Objective. In this work, we aim to enforce the spiking of the membrane potential of a single neuron or a neuronal network, described by dynamical models, by controlling the current injection in the presence of model uncertainty and synaptic noise. Approach. In this study, we propose Shaping Robust Dynamic Inversion (SRDI) as a robust nonlinear control technique, which uses dynamic inversion of neuronal dynamical systems and shapes the error surface to derive a current control signal that enforces the spiking of membrane potential under model uncertainty. Main results. We apply SRDI to Hodgkin-Huxley model, integrate-and-fire model, and FitzHugh-Nagumo model to achieve controlled neuron spiking. Comparative studies show that SRDI outperforms classical dynamic inversion in robustness and linear model predictive control in computational time. Significance. SRDI enables precise and efficient neural control by shaping error dynamics, handling nonlinearities, and maintaining robustness to noise and model uncertainty, achieving controlled timing for single spikes, spike trains, and small neuronal networks.

  • Open Access

    System Theoretic Methods in Drug Discovery and Vaccine Formulation: Review and Perspectives

    by Ankita Sharma, Yen-Che Hsiao, Abhishek Dutta

    The methods utilized in the drug discovery pipeline routinely combine machine learning and deep learning algorithms to enhance the outputs. The generation of a drug target, through virtual screening and computational analysis of databases used for target discovery, has increased the reliability of the machine learning and deep learning incorporated techniques. Recent technological advances in human immunology have provided improved tools that allow a better understanding of the biological and molecular mechanisms leading to the protective human immune response to pathogens, inspiring new strategies for vaccine design. Immunoinformatics approaches are more beneficial, and thus there is a demand for modern technologies such as reverse vaccinology, structural vaccinology, and system approaches in developing potential vaccine candidates. System theory, defined as a set of machine learning, control theory, and optimization-based methods applied to networked systems, provides a unifying framework for modeling and analyzing biological complexity. In this review, we explore the application of such computational methods at every stage of the therapeutic pipeline, including lead discovery, optimization, and dosing, as well as vaccine target prediction and immunogen design. Here, we summarize the system theoretic methods which provide insights into developed approaches and their applications in rational drug discovery and vaccine formulations. The approaches ranged in the review yield accurate predictions and insights. This review is intended to serve as a resource for researchers seeking to understand, adopt, or build upon system theoretic techniques in drug and vaccine development, offering both conceptual foundations and practical directions.


  • Derivation of Back-Propagation for Graph Convolutional Networks Using Matrix Calculus and Its Application to Explainable Artificial Intelligence

    by Yen-Che Hsiao, Rongting Yue, Abhishek Dutta

    This paper provides a comprehensive and detailed derivation of the backpropagation algorithm for graph convolutional neural networks using matrix calculus. The derivation is extended to include arbitrary element-wise activation functions and an arbitrary number of layers. The study addresses two fundamental problems, namely node classification and link prediction. To validate our method, we compare it with reverse-mode automatic differentiation. The experimental results demonstrate that the median sum of squared errors of the updated weight matrices, when comparing our method to the approach using reverse-mode automatic differentiation, falls within the range of 10-18 to 10-14. These outcomes are obtained from conducting experiments on a five-layer graph convolutional network, applied to a node classification problem on Zachary’s karate club social network and a link prediction problem on a drug-drug interaction network. Furthermore, we derive an element-wise solution for backpropagation in node classification using graph convolutional networks, demonstrating that it can achieve faster computation on large-scale graphs compared to reverse-mode automatic differentiation. Finally, we show how the derived closed-form solution can facilitate the development of explainable AI and sensitivity analysis.


  • Open Access

    Unveiling Reasoning Thresholds in Language Models: Scaling, Fine-Tuning, and Interpretability through Attention Maps

    by Yen-Che Hsiao, Abhishek Dutta

    This study investigates the in-context learning capabilities of various decoder-only transformer-based language models with different model sizes and training data, including GPT2, SmolLM2, OpenELM, TinyLlama, Stable LM, and Gemma 2. We identify a critical parameter threshold (~1.6 billion), beyond which reasoning performance improves significantly in tasks such as commonsense reasoning in multiple-choice question answering and deductive reasoning. Specifically, models above this threshold achieve better success rates in chain-of-thought (CoT) prompting for deductive reasoning tasks, especially those requiring longer reasoning chains, such as proof by contradiction and disjunction elimination. To address limitations in sub-threshold models, we demonstrate that fine-tuning with task-specific exemplars substantially enhances reasoning performance, enabling accurate CoT generation even without additional exemplars in the prompt for tasks with shorter reasoning chains. Finally, our analysis of attention maps reveals that models capable of generating correct CoTs exhibit higher token-level attention scores on subsequent correct tokens and the correct parts of speech, providing interpretability insights into reasoning processes. These findings collectively advance understanding of reasoning capabilities in decoder-only transformer-based models. The code is available at: https://github.com/AnnonymousForPapers/CoT_Reasoning_Test.

  • Open Access

    Design and Analysis of few Million Parameter Transformer-based Language Models trained over a few Million Tokens Dataset

    by Yen-Che Hsiao, Abhishek Dutta

    Yen-Che Hsiao, Abhishek Dutta. Proceedings of the First BabyLM Workshop. 2025.

  • Open Access

    Adaptive Reasoning and Acting in Medical Language Agents

    by Abhishek Dutta, Yen-Che Hsiao

    This paper presents an innovative large language model (LLM) agent framework for enhancing diagnostic accuracy in simulated clinical environments using the AgentClinic benchmark. The proposed automatic correction enables doctor agents to iteratively refine their reasoning and actions following incorrect diagnoses, fostering improved decision-making over time. Experiments show that the implementation of the adaptive LLM-based doctor agents achieve correct diagnoses through dynamic interactions with simulated patients. The evaluations highlight the capacity of autonomous agents to adapt and improve in complex medical scenarios. Future enhancements will focus on refining the algorithm and expanding its applicability across a wider range of tasks and different large language models.

  • Open Access

    Efficient transformer with reinforced position embedding for language models

    by Yen-Che Hsiao, Abhishek Dutta

    In this paper, we propose an efficient transformer architecture that uses reinforced positional embedding to obtain superior performance with half the number of encoder decoder layers. We demonstrate that concatenating positional encoding with trainable token embeddings, normalizing columns in the token embedding matrix, and using the normalized token embedding matrix as the value of the attention layer improve the training and validation loss and the training time in an encoder-decoder Transformer model for a Portuguese-English translation task with 10 epochs or 12 hours of training across 10 trials. Our method, with roughly a threefold parameter reduction compared to the baseline model, yields a mean training loss of 1.21, a mean validation loss of 1.51, and an average training time of 1352.27 seconds per epoch, surpassing the baseline model with the same embedding dimension that employs addition of positional encoding and token embeddings, which achieves a mean training loss of 1.96, a validation loss of 2.18, and an average training time of 4297.79 seconds per epoch. Additionally, we evaluated our proposed architecture and the baseline across 14 diverse translation datasets from TensorFlow. The results indicate that our method consistently achieves lower or comparable training and validation losses, suggesting enhanced learning efficiency.

  • Open Access

    Derivation of Back-propagation for Graph Convolutional Networks using Matrix Calculus and its Application to Explainable Artificial Intelligence

    by Yen-Che Hsiao, Rongting Yue, Abhishek Dutta

    This paper provides a comprehensive and detailed derivation of the backpropagation algorithm for graph convolutional neural networks using matrix calculus. The derivation is extended to include arbitrary element-wise activation functions and an arbitrary number of layers. The study addresses two fundamental problems, namely node classification and link prediction. To validate our method, we compare it with reverse-mode automatic differentiation. The experimental results demonstrate that the median sum of squared errors of the updated weight matrices, when comparing our method to the approach using reverse-mode automatic differentiation, falls within the range of $10^{-18}$ to $10^{-14}$. These outcomes are obtained from conducting experiments on a five-layer graph convolutional network, applied to a node classification problem on Zachary's karate club social network and a link prediction problem on a drug-drug interaction network. Finally, we show how the derived closed-form solution can facilitate the development of explainable AI and sensitivity analysis.

  • Network Modeling and Control of Dynamic Disease Pathways, Review and Perspectives

    by Yen-Che Hsiao, Abhishek Dutta

    Dynamic disease pathways are a combination of complex dynamical processes among bio-molecules in a cell that leads to diseases. Network modeling of disease pathways considers disease-related bio-molecules (e.g. DNA, RNA, transcription factors, enzymes, proteins, and metabolites) and their interaction (e.g. DNA methylation, histone modification, alternative splicing, and protein modification) to study disease progression and predict therapeutic responses. These bio-molecules and their interactions are the basic elements in the study of the misregulation in the disease-related gene expression that lead to abnormal cellular responses. Gene regulatory networks, cell signaling networks, and metabolic networks are the three major types of intracellular networks for the study of the cellular responses elicited from extracellular signals. The disease-related cellular responses can be prevented or regulated by designing control strategies to manipulate these extracellular or other intracellular signals. The paper reviews the regulatory mechanisms, the dynamic models, and the control strategies for each intracellular network. The applications, limitations and the prospective for modeling and control are also discussed.