SSSanjay Kumar Singh

Sanjay Kumar Singh

*This is an algorithmically generated author record based on publicly available information.

Sanjay Kumar Singh has a total of 6 co-authors. Here are the top co-authors by number of shared publications:


Latest Publications

  • Open Access

    MemGuard-Alpha: Detecting and Filtering Memorization-Contaminated Signals in LLM-Based Financial Forecasting via Membership Inference and Cross-Model Disagreement

    by Dip Roy, Rajiv Misra, Sanjay Kumar Singh, Anisha Roy

    Large language models (LLMs) are increasingly used to generate financial forecasts and alpha signals, but recent evidence suggests they memorize historical data from their training corpora, producing spurious in-sample accuracy that collapses out-of-sample. Existing remedies require costly retraining or input anonymization, with no practical method for filtering contamination at the signal level. We introduce MemGuard-Alpha, comprising two novel algorithms: (i) the MemGuard Composite Score (MCS), which combines five membership inference attack (MIA) methods with a temporal proximity feature via logistic regression, achieving Cohen's d = 18.57 in separating in-sample from out-of-sample prompts; and (ii) Cross-Model Memorization Disagreement (CMMD), which exploits natural variation in training cutoff dates across multiple LLMs and admits an interpretation as a fuzzy regression discontinuity design. Evaluated across seven LLMs (124M to 7B parameters), 50 S&P 100 constituents, 42,800 prompts, and 5.5 years of daily data (January 2019 to June 2024), CMMD achieves a Sharpe ratio of 4.11 versus 2.76 for unfiltered LLM signals, a 49% improvement after transaction costs. Signal accuracy reveals a crossover: in-sample accuracy rises with contamination (40.8% to 52.5% across quintiles) while out-of-sample accuracy falls (47% to 42%), providing direct empirical evidence that memorization inflates apparent accuracy at the cost of generalization.

  • Open Access

    Superposition and Polysemanticity in Mamba’s Selective State Space: A Mechanistic Interpretability Study via Sparse Autoencoders

    by Dip Roy, Rajiv Misra, Sanjay Kumar Singh, Anisha Roy

    State space models (SSMs), particularly the Mamba architecture, have emerged as competitive alternatives to Transformers for sequence modeling, offering linear-time complexity during both training and inference. Despite their growing adoption, the internal representations of SSMs remain poorly understood from an interpretability standpoint. This paper presents the first systematic mechanistic interpretability study of superposition and polysemanticity in Mamba’s selective state space, addressing whether SSM hidden-state channels exhibit representational phenomena analogous to those documented in Transformers. We train 114 TopK sparse autoencoders (SAEs) in a primary experimental campaign across two Mamba models (130M and 370M parameters), two GPT-2 baselines (124M and 355M parameters), multiple network depths, three expansion factors, and three random seeds, using both WikiText-103 (out-of-distribution) and The Pile (in-distribution) corpora. An additional 13 validation SAEs bring the total to 127 trained models. In our experiments we discovered three primary findings. First, Mamba and Transformers use fundamentally different geometric representations of what the activation of each unit represents. In this regard, Mamba's activation is distributed over an approximately five to 179 times larger dimensional manifold than GPT-2's (based upon participation ratio, which is 5.9–179 for Mamba, 1.1–13.4 for GPT-2) with a significantly higher SAE reconstruction error (FVU is 0.181 for Mamba, 0.064 for GPT-2) and consistently less polysemous representation for the same expansion factors (mean Polysemanticity Index 0.473 for Mamba, 0.505 for GPT-2, Cohen's d = 0.54) and therefore, the SSM's state accumulation process disperses representations throughout a number of hidden dimensions as opposed to condensing these into a relatively small subspace through superposition. Second, the selective differential sparsification analysis demonstrated that the delta gate projection (dt_proj) is a significant selectivity bottleneck; when only 10% of the elements in dt_proj were pruned, there was a complete collapse of perplexity (+290 PPL), whereas pruning at the same rate the x_proj and in_proj elements had little impact on perplexity (ΔPPL < 1), a sensitivity differential that does not exist within the Transformer attention projections. Finally, by employing an activation patching approach, we demonstrated that there are substantial differences in causal structure between Mamba and GPT-2. Specifically, patching Mamba's mixer output resulted in near zero restoration (range -0.122 to +0.018), while the same type of patching applied to GPT-2's residual streams resulted in perfect restoration (1.0 at every layer) and demonstrated that the recurrent states in Mamba carry information that is independent of the residual stream. The findings presented above have been validated through cross-dataset consistency (PI r = 0.934, p = 0.006; SI r = 0.988, p < 0.001), sequence length robustness (seq128 vs. seq512 ΔPI < 0.01), and TopK sensitivity analysis (k = 16/32/64).

  • Open Access

    Opening the Selective Gate: Mechanistic Interpretability of Mamba’s Latent State Dynamics for Financial Market Regime Discovery

    by Dip Roy, Rajiv Misra, Sanjay Kumar Singh, Anisha Roy

    Selectively constrained state-space models (SSM) including specifically the Mamba model, are becoming increasingly well-recognized as computationally cost-effective options relative to the Transformer model for modeling financial time-series. However, the way in which selectively constrained SSMs internally represent knowledge regarding market regimes – those transition periods among bull, bear, or sideways markets when optimal trading strategy decisions should be made -- remain opaque and largely unexamined. This paper represents an exploratory application of Mechanistic Interpretability (MI) to understand how a Mamba model encodes information in a financial context. In doing so, it is important to note that the focus of this paper is on developing an understanding of how financial selectively constrained SSMs represent information internally, rather than applying such an understanding to develop new trading strategies. A six-layer selectively constrained SSM was trained using approximately twenty-one years of daily equity price data on fifty components of the S&P 500 index, in addition to incorporating several other types of information (i.e., macroeconomic indicators, Fama-French style risk factors, and cross-sectoral features), and the hidden states within each layer were subsequently analyzed via Top-K Sparse Autoencoder decomposition. Five key insights were developed. Firstly, a layer-specific hierarchical relationship between selectively constrained gates (Δt) was observed where Layer 1 serves as a "crisis detection" layer and exhibits the largest discretization step during periods corresponding to bear markets. Conversely, Layers 3-4 served as "noise filtering" layers with the maximum value of the gate during sideways markets. Secondly, intervention experiments provided evidence that the SAE features produced by selectively constrained SSMs are linked to regime-specific processing. Specifically, the intervention effect of bear market specific SAE features on processing regimes exhibited the largest effect size (effect size = 0.873). Thirdly, a polysemanticity gradient emerges across layers with earlier layers exhibiting extreme super-position (251 polysemantic features at Layer 1) whereas later layers (Layer 5) resolve into monosemantic representations (66 monosemantic features). Fourthly, SAE features are capable of achieving nearly ceiling level performance in terms of regime classification (F1=1.00 under matched feature conditions; refer to Section 5.5) with geometrically distinct regime clusterings within the latent space. Finally, a comparative analysis revealed that Mamba distributed regime information throughout significantly more selectively constrained features than its comparable Transformer (283 selectively constrained features vs. 170 selectively constrained features). The results presented above demonstrate the viability of MI as a methodological framework for understanding financial SSMs and provide insight into potential regulatory compliance issues under the European Union's AI Act.

  • Machine Learning for Predicting Clinical Outcomes of Metformin Dosage on Patient Health

    by Niraj Dineshkumar Bhagchandani, Sanjay Kumar Singh, Rajiv Misra

    An ability to predict clinical outcomes in different doses of the drug is important in enhancing clinical trials and drug development. The paper is a hybrid machine learning model predicting clinical outcomes of Metformin (ChEMBL1703) based on bioactivity data, molecular structures, and assay conditions of the CHEMBL 28 database. The model combines the approaches of Random Forest Regression (RF), Graph Neural Networks (GNNs), and Natural Language Processing (NLP) to forecast the impact of Metformin on such health markers as blood pressure, cholesterol, and glucose. Preprocess is done in harmonization of bioactivity data (IC50, EC50, ChEMBL values) through unit standardization and logtransformation. Cross-validation on 5 folds would yield an $\mathbf{R}^{\mathbf{2}}$ of 0.89 and a Mean Absolute Error (MAE) of 0.35, which is 20 % lower than the bioactivity-only model. The statistical analysis (ttest and ANOVA) shows that there is significant agreement between the results of the simulated and reference results (0.05). The process of validation was done on synthetic and publicly available datasets. The future work will use real patient-level data to generalize more to the clinical trials. The proposed model shows a high potential in the simulated drug trials and the bettering of clinical decision-making through the prediction of outcomes at various levels of Metformin dosing.

  • Maturity grading of jujube for industrial applications harnessing deep learning

    by Atif Mahmood, Amod Kumar Tiwari, Sanjay Kumar Singh

    Purpose: To develop and examine an efficient and reliable jujube grading model with reduced computational time, which could be utilized in the food processing and packaging industries to perform quick grading and pricing of jujube as well as for the other similar types of fruits.Design/methodology/approach: The whole process begins with manual analysis and collection of four jujube grades from the jujube tree, in addition to this jujube image acquisition was performed utilizing MVS which is further followed by image pre-processing and augmentation tasks. Eventually, classification models (i.e. proposed model, from scratch and pre-trained VGG16 and AlexNet) were trained and validated over the original and augmented datasets to discriminate the jujube into maturity grades.Findings: The highest success rates reported over the original and augmented datasets were 97.53% (i.e. error of 2.47%) and 99.44% (i.e. error of 0.56%) respectively using Adam optimizer and a learning rate of 0.003.Research limitations/implications: The investigation relies upon a single view of the jujube image and the outer appearance of the jujube. In the future, multi-view image capturing system could be employed for the model training/validation.Practical implications: Due to the vast functional derivatives of jujube, the identification of maturity grades of jujube is paramount in the fruit industry, functional food production industries and pharmaceutical industry. Therefore, the proposed model which is practically feasible and easy to implement could be utilized in such industries.Originality/value: This research examines the performance of proposed CNN models for selected optimizer and learning rates for the grading of jujube maturity into four classes and compares them with the classical models to depict the sublime model in terms of accuracy, the number of parameters, epochs and computational time. After a thorough investigation of the models, it was discovered that the proposed model transcends both classical models in all aspects for both the original and augmented datasets utilizing Adam optimizer with learning rate of 0.003.