Dip Roy
Current Organizations / Institutes
The organizations/institutes from the scholar's latest article
Research Subjects
Dip Roy has a total of 7 co-authors. Here are the top co-authors by number of shared publications:
- Anisha Roy
Department of Electronics and Communication Engineering, Jaypee Institute of Information Technology, Noida, India
6 shared publications Rajiv Misra
4 shared publications- Rajiv Misra
Indian Institute of Technology Patna
4 shared publications Sanjay Kumar Singh
4 shared publications- Sanjay Kumar Singh
Rajarshi School of Management and Technology,Computer Science,Varanasi,India
3 shared publications Anisha Roy
1 shared publication
Latest Publications
- 27 August 2026
- Published by Elsevier in Pattern Recognition
- Vol. 182
Mechanistic interpretability of Mamba’s selective gate in financial markets: A leakage-controlled study
- 27 August 2026
- Published by Elsevier in Pattern Recognition
- Vol. 182
Mechanistic interpretability of Mamba’s selective gate in financial markets: A leakage-controlled study
- 1 August 2026
- Published by Springer Nature in Neural Computing and Applications
- Vol. 38 (15) , 610
Posterior-calibrated causal circuits in variational autoencoders: why image-domain interpretability fails on tabular data
- 1 August 2026
- Published by Springer Nature in Neural Computing and Applications
- Vol. 38 (15) , 610
Posterior-calibrated causal circuits in variational autoencoders: why image-domain interpretability fails on tabular data
- Open Access
- 5 June 2026
- Published by Elsevier in SSRN Electronic Journal
MemGuard-Alpha: Detecting and Filtering Memorization-Contaminated Signals in LLM-Based Financial Forecasting via Membership Inference and Cross-Model Disagreement
Large language models (LLMs) are increasingly used to generate financial forecasts and alpha signals, but recent evidence suggests they memorize historical data from their training corpora, producing spurious in-sample accuracy that collapses out-of-sample. Existing remedies require costly retraining or input anonymization, with no practical method for filtering contamination at the signal level. We introduce MemGuard-Alpha, comprising two novel algorithms: (i) the MemGuard Composite Score (MCS), which combines five membership inference attack (MIA) methods with a temporal proximity feature via logistic regression, achieving Cohen's d = 18.57 in separating in-sample from out-of-sample prompts; and (ii) Cross-Model Memorization Disagreement (CMMD), which exploits natural variation in training cutoff dates across multiple LLMs and admits an interpretation as a fuzzy regression discontinuity design. Evaluated across seven LLMs (124M to 7B parameters), 50 S&P 100 constituents, 42,800 prompts, and 5.5 years of daily data (January 2019 to June 2024), CMMD achieves a Sharpe ratio of 4.11 versus 2.76 for unfiltered LLM signals, a 49% improvement after transaction costs. Signal accuracy reveals a crossover: in-sample accuracy rises with contamination (40.8% to 52.5% across quintiles) while out-of-sample accuracy falls (47% to 42%), providing direct empirical evidence that memorization inflates apparent accuracy at the cost of generalization.
- 5 June 2026
- Published by Elsevier in SSRN Electronic Journal
Open AccessMemGuard-Alpha: Detecting and Filtering Memorization-Contaminated Signals in LLM-Based Financial Forecasting via Membership Inference and Cross-Model Disagreement
Large language models (LLMs) are increasingly used to generate financial forecasts and alpha signals, but recent evidence suggests they memorize historical data from their training corpora, producing spurious in-sample accuracy that collapses out-of-sample. Existing remedies require costly retraining or input anonymization, with no practical method for filtering contamination at the signal level. We introduce MemGuard-Alpha, comprising two novel algorithms: (i) the MemGuard Composite Score (MCS), which combines five membership inference attack (MIA) methods with a temporal proximity feature via logistic regression, achieving Cohen's d = 18.57 in separating in-sample from out-of-sample prompts; and (ii) Cross-Model Memorization Disagreement (CMMD), which exploits natural variation in training cutoff dates across multiple LLMs and admits an interpretation as a fuzzy regression discontinuity design. Evaluated across seven LLMs (124M to 7B parameters), 50 S&P 100 constituents, 42,800 prompts, and 5.5 years of daily data (January 2019 to June 2024), CMMD achieves a Sharpe ratio of 4.11 versus 2.76 for unfiltered LLM signals, a 49% improvement after transaction costs. Signal accuracy reveals a crossover: in-sample accuracy rises with contamination (40.8% to 52.5% across quintiles) while out-of-sample accuracy falls (47% to 42%), providing direct empirical evidence that memorization inflates apparent accuracy at the cost of generalization.
- Open Access
- 11 May 2026
- Published by Elsevier in SSRN Electronic Journal
Superposition and Polysemanticity in Mamba’s Selective State Space: A Mechanistic Interpretability Study via Sparse Autoencoders
State space models (SSMs), particularly the Mamba architecture, have emerged as competitive alternatives to Transformers for sequence modeling, offering linear-time complexity during both training and inference. Despite their growing adoption, the internal representations of SSMs remain poorly understood from an interpretability standpoint. This paper presents the first systematic mechanistic interpretability study of superposition and polysemanticity in Mamba’s selective state space, addressing whether SSM hidden-state channels exhibit representational phenomena analogous to those documented in Transformers. We train 114 TopK sparse autoencoders (SAEs) in a primary experimental campaign across two Mamba models (130M and 370M parameters), two GPT-2 baselines (124M and 355M parameters), multiple network depths, three expansion factors, and three random seeds, using both WikiText-103 (out-of-distribution) and The Pile (in-distribution) corpora. An additional 13 validation SAEs bring the total to 127 trained models. In our experiments we discovered three primary findings. First, Mamba and Transformers use fundamentally different geometric representations of what the activation of each unit represents. In this regard, Mamba's activation is distributed over an approximately five to 179 times larger dimensional manifold than GPT-2's (based upon participation ratio, which is 5.9–179 for Mamba, 1.1–13.4 for GPT-2) with a significantly higher SAE reconstruction error (FVU is 0.181 for Mamba, 0.064 for GPT-2) and consistently less polysemous representation for the same expansion factors (mean Polysemanticity Index 0.473 for Mamba, 0.505 for GPT-2, Cohen's d = 0.54) and therefore, the SSM's state accumulation process disperses representations throughout a number of hidden dimensions as opposed to condensing these into a relatively small subspace through superposition. Second, the selective differential sparsification analysis demonstrated that the delta gate projection (dt_proj) is a significant selectivity bottleneck; when only 10% of the elements in dt_proj were pruned, there was a complete collapse of perplexity (+290 PPL), whereas pruning at the same rate the x_proj and in_proj elements had little impact on perplexity (ΔPPL < 1), a sensitivity differential that does not exist within the Transformer attention projections. Finally, by employing an activation patching approach, we demonstrated that there are substantial differences in causal structure between Mamba and GPT-2. Specifically, patching Mamba's mixer output resulted in near zero restoration (range -0.122 to +0.018), while the same type of patching applied to GPT-2's residual streams resulted in perfect restoration (1.0 at every layer) and demonstrated that the recurrent states in Mamba carry information that is independent of the residual stream. The findings presented above have been validated through cross-dataset consistency (PI r = 0.934, p = 0.006; SI r = 0.988, p < 0.001), sequence length robustness (seq128 vs. seq512 ΔPI < 0.01), and TopK sensitivity analysis (k = 16/32/64).
- 11 May 2026
- Published by Elsevier in SSRN Electronic Journal
Open AccessSuperposition and Polysemanticity in Mamba’s Selective State Space: A Mechanistic Interpretability Study via Sparse Autoencoders
State space models (SSMs), particularly the Mamba architecture, have emerged as competitive alternatives to Transformers for sequence modeling, offering linear-time complexity during both training and inference. Despite their growing adoption, the internal representations of SSMs remain poorly understood from an interpretability standpoint. This paper presents the first systematic mechanistic interpretability study of superposition and polysemanticity in Mamba’s selective state space, addressing whether SSM hidden-state channels exhibit representational phenomena analogous to those documented in Transformers. We train 114 TopK sparse autoencoders (SAEs) in a primary experimental campaign across two Mamba models (130M and 370M parameters), two GPT-2 baselines (124M and 355M parameters), multiple network depths, three expansion factors, and three random seeds, using both WikiText-103 (out-of-distribution) and The Pile (in-distribution) corpora. An additional 13 validation SAEs bring the total to 127 trained models. In our experiments we discovered three primary findings. First, Mamba and Transformers use fundamentally different geometric representations of what the activation of each unit represents. In this regard, Mamba's activation is distributed over an approximately five to 179 times larger dimensional manifold than GPT-2's (based upon participation ratio, which is 5.9–179 for Mamba, 1.1–13.4 for GPT-2) with a significantly higher SAE reconstruction error (FVU is 0.181 for Mamba, 0.064 for GPT-2) and consistently less polysemous representation for the same expansion factors (mean Polysemanticity Index 0.473 for Mamba, 0.505 for GPT-2, Cohen's d = 0.54) and therefore, the SSM's state accumulation process disperses representations throughout a number of hidden dimensions as opposed to condensing these into a relatively small subspace through superposition. Second, the selective differential sparsification analysis demonstrated that the delta gate projection (dt_proj) is a significant selectivity bottleneck; when only 10% of the elements in dt_proj were pruned, there was a complete collapse of perplexity (+290 PPL), whereas pruning at the same rate the x_proj and in_proj elements had little impact on perplexity (ΔPPL < 1), a sensitivity differential that does not exist within the Transformer attention projections. Finally, by employing an activation patching approach, we demonstrated that there are substantial differences in causal structure between Mamba and GPT-2. Specifically, patching Mamba's mixer output resulted in near zero restoration (range -0.122 to +0.018), while the same type of patching applied to GPT-2's residual streams resulted in perfect restoration (1.0 at every layer) and demonstrated that the recurrent states in Mamba carry information that is independent of the residual stream. The findings presented above have been validated through cross-dataset consistency (PI r = 0.934, p = 0.006; SI r = 0.988, p < 0.001), sequence length robustness (seq128 vs. seq512 ΔPI < 0.01), and TopK sensitivity analysis (k = 16/32/64).
- Open Access
- 29 April 2026
- Published by Elsevier in SSRN Electronic Journal
Opening the Selective Gate: Mechanistic Interpretability of Mamba’s Latent State Dynamics for Financial Market Regime Discovery
Selectively constrained state-space models (SSM) including specifically the Mamba model, are becoming increasingly well-recognized as computationally cost-effective options relative to the Transformer model for modeling financial time-series. However, the way in which selectively constrained SSMs internally represent knowledge regarding market regimes – those transition periods among bull, bear, or sideways markets when optimal trading strategy decisions should be made -- remain opaque and largely unexamined. This paper represents an exploratory application of Mechanistic Interpretability (MI) to understand how a Mamba model encodes information in a financial context. In doing so, it is important to note that the focus of this paper is on developing an understanding of how financial selectively constrained SSMs represent information internally, rather than applying such an understanding to develop new trading strategies. A six-layer selectively constrained SSM was trained using approximately twenty-one years of daily equity price data on fifty components of the S&P 500 index, in addition to incorporating several other types of information (i.e., macroeconomic indicators, Fama-French style risk factors, and cross-sectoral features), and the hidden states within each layer were subsequently analyzed via Top-K Sparse Autoencoder decomposition. Five key insights were developed. Firstly, a layer-specific hierarchical relationship between selectively constrained gates (Δt) was observed where Layer 1 serves as a "crisis detection" layer and exhibits the largest discretization step during periods corresponding to bear markets. Conversely, Layers 3-4 served as "noise filtering" layers with the maximum value of the gate during sideways markets. Secondly, intervention experiments provided evidence that the SAE features produced by selectively constrained SSMs are linked to regime-specific processing. Specifically, the intervention effect of bear market specific SAE features on processing regimes exhibited the largest effect size (effect size = 0.873). Thirdly, a polysemanticity gradient emerges across layers with earlier layers exhibiting extreme super-position (251 polysemantic features at Layer 1) whereas later layers (Layer 5) resolve into monosemantic representations (66 monosemantic features). Fourthly, SAE features are capable of achieving nearly ceiling level performance in terms of regime classification (F1=1.00 under matched feature conditions; refer to Section 5.5) with geometrically distinct regime clusterings within the latent space. Finally, a comparative analysis revealed that Mamba distributed regime information throughout significantly more selectively constrained features than its comparable Transformer (283 selectively constrained features vs. 170 selectively constrained features). The results presented above demonstrate the viability of MI as a methodological framework for understanding financial SSMs and provide insight into potential regulatory compliance issues under the European Union's AI Act.
- 29 April 2026
- Published by Elsevier in SSRN Electronic Journal
Open AccessOpening the Selective Gate: Mechanistic Interpretability of Mamba’s Latent State Dynamics for Financial Market Regime Discovery
Selectively constrained state-space models (SSM) including specifically the Mamba model, are becoming increasingly well-recognized as computationally cost-effective options relative to the Transformer model for modeling financial time-series. However, the way in which selectively constrained SSMs internally represent knowledge regarding market regimes – those transition periods among bull, bear, or sideways markets when optimal trading strategy decisions should be made -- remain opaque and largely unexamined. This paper represents an exploratory application of Mechanistic Interpretability (MI) to understand how a Mamba model encodes information in a financial context. In doing so, it is important to note that the focus of this paper is on developing an understanding of how financial selectively constrained SSMs represent information internally, rather than applying such an understanding to develop new trading strategies. A six-layer selectively constrained SSM was trained using approximately twenty-one years of daily equity price data on fifty components of the S&P 500 index, in addition to incorporating several other types of information (i.e., macroeconomic indicators, Fama-French style risk factors, and cross-sectoral features), and the hidden states within each layer were subsequently analyzed via Top-K Sparse Autoencoder decomposition. Five key insights were developed. Firstly, a layer-specific hierarchical relationship between selectively constrained gates (Δt) was observed where Layer 1 serves as a "crisis detection" layer and exhibits the largest discretization step during periods corresponding to bear markets. Conversely, Layers 3-4 served as "noise filtering" layers with the maximum value of the gate during sideways markets. Secondly, intervention experiments provided evidence that the SAE features produced by selectively constrained SSMs are linked to regime-specific processing. Specifically, the intervention effect of bear market specific SAE features on processing regimes exhibited the largest effect size (effect size = 0.873). Thirdly, a polysemanticity gradient emerges across layers with earlier layers exhibiting extreme super-position (251 polysemantic features at Layer 1) whereas later layers (Layer 5) resolve into monosemantic representations (66 monosemantic features). Fourthly, SAE features are capable of achieving nearly ceiling level performance in terms of regime classification (F1=1.00 under matched feature conditions; refer to Section 5.5) with geometrically distinct regime clusterings within the latent space. Finally, a comparative analysis revealed that Mamba distributed regime information throughout significantly more selectively constrained features than its comparable Transformer (283 selectively constrained features vs. 170 selectively constrained features). The results presented above demonstrate the viability of MI as a methodological framework for understanding financial SSMs and provide insight into potential regulatory compliance issues under the European Union's AI Act.
- Open Access
- 27 April 2026
- Published by Elsevier in SSRN Electronic Journal
Superposition and Polysemanticity in Mamba's Selective State Space: A Mechanistic Interpretability Study via Sparse Autoencoders
State space models (SSMs), particularly the Mamba architecture, have emerged as competitive alternatives to Transformers for sequence modeling, offering linear-time complexity during both training and inference. Despite their growing adoption, the internal representations of SSMs remain poorly understood from an interpretability standpoint. This paper presents the first systematic mechanistic interpretability study of superposition and polysemanticity in Mamba's selective state space, addressing whether SSM hiddenstate channels exhibit representational phenomena analogous to those documented in Transformers. We train 114 TopK sparse autoencoders (SAEs) in a primary experimental campaign across two Mamba models (130M and 370M parameters), two GPT-2 baselines (124M and 355M parameters), multiple network depths, three expansion factors, and three random seeds, using both WikiText-103 (out-of-distribution) and The Pile (in-distribution) corpora. An additional 13 validation SAEs bring the total to 127 trained models. In our experiments we discovered three primary findings. First, Mamba and Transformers use fundamentally different geometric representations of what the activation of each unit represents. In this regard, Mamba's activation is distributed over an approximately five to 179 times larger dimensional manifold than GPT-2's (based upon participation ratio, which is 5.9-179 for Mamba, 1.1-13.4 for GPT-2) with a significantly higher SAE reconstruction error (FVU is 0.181 for Mamba, 0.064 for GPT-2) and consistently less polysemous representation for the same expansion factors (mean Polysemanticity Index 0.473 for Mamba, 0.505 for GPT-2, Cohen's d = 0.54) and therefore, the SSM's state accumulation process disperses representations throughout a number of hidden dimensions as opposed to condensing these into a relatively small subspace through superposition. Second, the selective differential sparsification analysis demonstrated that the delta gate projection (dt_proj) is a significant selectivity bottleneck; when only 10% of the elements in dt_proj were pruned, there was a complete collapse of perplexity (+290 PPL), whereas pruning at the same rate the x_proj and in_proj elements had little impact on perplexity (ΔPPL < 1), a sensitivity differential that does not exist within the Transformer attention projections. Finally, by employing an activation patching approach, we demonstrated that there are substantial differences in causal structure between Mamba and GPT-2. Specifically, patching Mamba's mixer output resulted in near zero restoration (range-0.122 to +0.018), while the same type of patching applied to GPT-2's residual streams resulted in perfect restoration (1.0 at every layer) and demonstrated that the recurrent states in Mamba carry information that is independent of the residual stream. The findings presented above have been validated through cross-dataset consistency (PI r = 0.934, p = 0.006; SI r = 0.988, p < 0.001), sequence length robustness (seq128 vs. seq512 ΔPI < 0.01), and TopK sensitivity analysis (k = 16/32/64).
- 27 April 2026
- Published by Elsevier in SSRN Electronic Journal
Open AccessSuperposition and Polysemanticity in Mamba's Selective State Space: A Mechanistic Interpretability Study via Sparse Autoencoders
State space models (SSMs), particularly the Mamba architecture, have emerged as competitive alternatives to Transformers for sequence modeling, offering linear-time complexity during both training and inference. Despite their growing adoption, the internal representations of SSMs remain poorly understood from an interpretability standpoint. This paper presents the first systematic mechanistic interpretability study of superposition and polysemanticity in Mamba's selective state space, addressing whether SSM hiddenstate channels exhibit representational phenomena analogous to those documented in Transformers. We train 114 TopK sparse autoencoders (SAEs) in a primary experimental campaign across two Mamba models (130M and 370M parameters), two GPT-2 baselines (124M and 355M parameters), multiple network depths, three expansion factors, and three random seeds, using both WikiText-103 (out-of-distribution) and The Pile (in-distribution) corpora. An additional 13 validation SAEs bring the total to 127 trained models. In our experiments we discovered three primary findings. First, Mamba and Transformers use fundamentally different geometric representations of what the activation of each unit represents. In this regard, Mamba's activation is distributed over an approximately five to 179 times larger dimensional manifold than GPT-2's (based upon participation ratio, which is 5.9-179 for Mamba, 1.1-13.4 for GPT-2) with a significantly higher SAE reconstruction error (FVU is 0.181 for Mamba, 0.064 for GPT-2) and consistently less polysemous representation for the same expansion factors (mean Polysemanticity Index 0.473 for Mamba, 0.505 for GPT-2, Cohen's d = 0.54) and therefore, the SSM's state accumulation process disperses representations throughout a number of hidden dimensions as opposed to condensing these into a relatively small subspace through superposition. Second, the selective differential sparsification analysis demonstrated that the delta gate projection (dt_proj) is a significant selectivity bottleneck; when only 10% of the elements in dt_proj were pruned, there was a complete collapse of perplexity (+290 PPL), whereas pruning at the same rate the x_proj and in_proj elements had little impact on perplexity (ΔPPL < 1), a sensitivity differential that does not exist within the Transformer attention projections. Finally, by employing an activation patching approach, we demonstrated that there are substantial differences in causal structure between Mamba and GPT-2. Specifically, patching Mamba's mixer output resulted in near zero restoration (range-0.122 to +0.018), while the same type of patching applied to GPT-2's residual streams resulted in perfect restoration (1.0 at every layer) and demonstrated that the recurrent states in Mamba carry information that is independent of the residual stream. The findings presented above have been validated through cross-dataset consistency (PI r = 0.934, p = 0.006; SI r = 0.988, p < 0.001), sequence length robustness (seq128 vs. seq512 ΔPI < 0.01), and TopK sensitivity analysis (k = 16/32/64).
- Open Access
- 15 April 2026
- Published by Elsevier in SSRN Electronic Journal
Before the First Token: Scale-dependent Emergence of Hallucination Signals in Autoregressive Language Models
When do large language models make decisions to generate fake information? Doing so in fields such as health care, law, science research, or making financial decisions has serious consequences; but still there are few formal answers to this question. Recent studies have shown that there are differences in how autoregressive language models represent internally whether they are providing factual versus fictional responses, demonstrating that these models have some form of internal understanding about their reliability that does not appear in their probability distributions of output; however, the dynamic of when these internal representations are most active — specifically, how quickly this activity peaks as a function of the size of the model — remains poorly understood. We investigate this gap by conducting a comprehensive study of the time course of when internal representations are most reliably indicative of factual versus fictional responses across 7 autoregressive transformers of varying sizes, ranging from 117 million to 7 billion parameters total, using three fact-based datasets (TriviaQA, Simple Facts, Biography) containing a total of 552 labeled examples. We find that a "scale dependent" transition occurs in the representation of hallucinations: we find that models less than 400M parameters produce chance-level probe accuracy at every position during generation (AUC = 0.48 – 0.67); therefore, we conclude that there is little-to-no reliable internal signal distinguishing between factual and fictional responses at this scale. At a scale above ~1B parameters, we see a qualitatively different regime emerge: the maximum of the detectability signal occurs at position zero (i.e., before any output tokens are produced), and then decreases as generation progresses. Importantly, this pre-generation signal is statistically significant in both Pythia-1.4B (p = 0.012) and Qwen2.5-7B (p = 0.038), two models based on entirely separate architectures that were trained on distinctly different corpuses. One of the most interesting results is at the largest scale of 7B: Pythia-6.9B (a base-only model trained on The Pile) produces an almost perfectly flat temporal profile (Δ = +0.001, p = 0.989), whereas Qwen2.5-7B (which was instruction-tuned on a very diverse corpus) demonstrates a clearly dominant pre-generation effect. This difference between models indicates that raw scale is not sufficient — knowledge organization through instruction tuning or equivalent post-training is apparently required for pre-commitment encoding at the 7B level. We also note that probing along probe-derived directions via activation steering fails to correct hallucinations for all models, which confirms that the signal is correlational rather than causally preceding generation. Together, our findings provide scale-calibrated detection protocols, and lay out a concrete hypothesis regarding the role of instruction tuning in developing the knowledge circuits that support factual knowledge.
- 15 April 2026
- Published by Elsevier in SSRN Electronic Journal
Open AccessBefore the First Token: Scale-dependent Emergence of Hallucination Signals in Autoregressive Language Models
When do large language models make decisions to generate fake information? Doing so in fields such as health care, law, science research, or making financial decisions has serious consequences; but still there are few formal answers to this question. Recent studies have shown that there are differences in how autoregressive language models represent internally whether they are providing factual versus fictional responses, demonstrating that these models have some form of internal understanding about their reliability that does not appear in their probability distributions of output; however, the dynamic of when these internal representations are most active — specifically, how quickly this activity peaks as a function of the size of the model — remains poorly understood. We investigate this gap by conducting a comprehensive study of the time course of when internal representations are most reliably indicative of factual versus fictional responses across 7 autoregressive transformers of varying sizes, ranging from 117 million to 7 billion parameters total, using three fact-based datasets (TriviaQA, Simple Facts, Biography) containing a total of 552 labeled examples. We find that a "scale dependent" transition occurs in the representation of hallucinations: we find that models less than 400M parameters produce chance-level probe accuracy at every position during generation (AUC = 0.48 – 0.67); therefore, we conclude that there is little-to-no reliable internal signal distinguishing between factual and fictional responses at this scale. At a scale above ~1B parameters, we see a qualitatively different regime emerge: the maximum of the detectability signal occurs at position zero (i.e., before any output tokens are produced), and then decreases as generation progresses. Importantly, this pre-generation signal is statistically significant in both Pythia-1.4B (p = 0.012) and Qwen2.5-7B (p = 0.038), two models based on entirely separate architectures that were trained on distinctly different corpuses. One of the most interesting results is at the largest scale of 7B: Pythia-6.9B (a base-only model trained on The Pile) produces an almost perfectly flat temporal profile (Δ = +0.001, p = 0.989), whereas Qwen2.5-7B (which was instruction-tuned on a very diverse corpus) demonstrates a clearly dominant pre-generation effect. This difference between models indicates that raw scale is not sufficient — knowledge organization through instruction tuning or equivalent post-training is apparently required for pre-commitment encoding at the 7B level. We also note that probing along probe-derived directions via activation steering fails to correct hallucinations for all models, which confirms that the signal is correlational rather than causally preceding generation. Together, our findings provide scale-calibrated detection protocols, and lay out a concrete hypothesis regarding the role of instruction tuning in developing the knowledge circuits that support factual knowledge.
- 27 March 2026
- Published by Elsevier in Neurocomputing
- Vol. 682
Fundamental limits of neural network sparsification: Evidence from catastrophic interpretability collapse
- 27 March 2026
- Published by Elsevier in Neurocomputing
- Vol. 682
Fundamental limits of neural network sparsification: Evidence from catastrophic interpretability collapse
- Open Access
Bayesian Autoencoder for Medical Anomaly Detection: Uncertainty-Aware Approach for Brain 2 MRI Analysis
In medical imaging, anomaly detection is a vital element of healthcare diagnostics, especially for neurological conditions which can be life-threatening. Conventional deterministic methods often fall short when it comes to capturing the inherent uncertainty of anomaly detection tasks. This paper introduces a Bayesian Variational Autoencoder (VAE) equipped with multi-head attention mechanisms for detecting anomalies in brain magnetic resonance imaging (MRI). For the purpose of improving anomaly detection performance, we incorporate both epistemic and aleatoric uncertainty estimation through Bayesian inference. The model was tested on the BraTS2020 dataset, and the findings were a 0.83 ROC AUC and a 0.83 PR AUC. The data in our paper suggests that modeling uncertainty is an essential component of anomaly detection, enhancing both performance and interpretability and providing confidence estimates, as well as anomaly predictions, for clinicians to leverage in making medical decisions.
Open AccessBayesian Autoencoder for Medical Anomaly Detection: Uncertainty-Aware Approach for Brain 2 MRI Analysis
In medical imaging, anomaly detection is a vital element of healthcare diagnostics, especially for neurological conditions which can be life-threatening. Conventional deterministic methods often fall short when it comes to capturing the inherent uncertainty of anomaly detection tasks. This paper introduces a Bayesian Variational Autoencoder (VAE) equipped with multi-head attention mechanisms for detecting anomalies in brain magnetic resonance imaging (MRI). For the purpose of improving anomaly detection performance, we incorporate both epistemic and aleatoric uncertainty estimation through Bayesian inference. The model was tested on the BraTS2020 dataset, and the findings were a 0.83 ROC AUC and a 0.83 PR AUC. The data in our paper suggests that modeling uncertainty is an essential component of anomaly detection, enhancing both performance and interpretability and providing confidence estimates, as well as anomaly predictions, for clinicians to leverage in making medical decisions.
- Open Access
- 22 April 2025
- Published by Springer Nature in Research Square
Bayesian Autoencoder for Medical Anomaly Detection: Uncertainty-Aware Approach for Brain MRI Analysis
In medical imaging, anomaly detection is a vital element of healthcare diagnostics, especially for neurological conditions which can be life-threatening. Conventional deterministic methods often fall short when it comes to capturing the inherent uncertainty of anomaly detection tasks. This paper introduces a Bayesian Variational Autoencoder (VAE) equipped with multi-head attention mechanisms for detecting anomalies in brain magnetic resonance imaging (MRI). For the purpose of improving anomaly detection performance, we incorporate both epistemic and aleatoric uncertainty estimation through Bayesian inference. The model was tested on the BraTS2020 dataset, and the findings were a 0.83 ROC AUC and a 0.83 PR AUC. The data in our paper suggests that modeling uncertainty is an essential component of anomaly detection, enhancing both performance and interpretability and providing confidence estimates, as well as anomaly predictions, for clinicians to leverage in making medical decisions.
- 22 April 2025
- Published by Springer Nature in Research Square
Open AccessBayesian Autoencoder for Medical Anomaly Detection: Uncertainty-Aware Approach for Brain MRI Analysis
In medical imaging, anomaly detection is a vital element of healthcare diagnostics, especially for neurological conditions which can be life-threatening. Conventional deterministic methods often fall short when it comes to capturing the inherent uncertainty of anomaly detection tasks. This paper introduces a Bayesian Variational Autoencoder (VAE) equipped with multi-head attention mechanisms for detecting anomalies in brain magnetic resonance imaging (MRI). For the purpose of improving anomaly detection performance, we incorporate both epistemic and aleatoric uncertainty estimation through Bayesian inference. The model was tested on the BraTS2020 dataset, and the findings were a 0.83 ROC AUC and a 0.83 PR AUC. The data in our paper suggests that modeling uncertainty is an essential component of anomaly detection, enhancing both performance and interpretability and providing confidence estimates, as well as anomaly predictions, for clinicians to leverage in making medical decisions.