Amos Storkey
Current Organizations / Institutes
The organizations/institutes from the scholar's latest article
Previous Organizations / Institutes
The organizations / institutes from the scholar's other articles in the past ten years
Research Subjects
Amos Storkey has a total of 131 co-authors. Here are the top co-authors by number of shared publications:
- Elliot J. Crowley
School of Informatics University of Edinburgh
7 shared publications - Jack Turner
School of Informatics University of Edinburgh
3 shared publications - Joanna Marguerite Wardlaw
Centre for Clinical Brain Sciences, UK Dementia Research Institute, University of Edinburgh, Edinburgh, UK
3 shared publications - Massimiliano Patacchiola
School of Informatics, University of Edinburgh, Edinburgh, U.K.
3 shared publications - Michael F. P. O'Boyle
University of Edinburgh, Edinburgh, United Kingdom
3 shared publications - Paul Micaelli
University of Edinburgh University of Edinburgh
3 shared publications
Latest Publications
- Open Access
- 3 September 2026
- Published by MDPI AG in Preprints.org
Theory of Agent: The Science of Internalization and Externalization for LLM-Based Agents
A large language model (LLM) agent is a model coupled to a harness: parametric weights that hold knowledge and reasoning ability, wrapped in an external apparatus of memory, tools, and orchestration through which the model perceives and acts. Progress over the past years has come from pushing on both, with stronger models and richer harnesses, but the design decision that quietly governs an agent’s behavior is where competence should live: which knowledge, skills, and control to internalize into the model’s parameters, and which to externalize onto the surrounding harness. These decisions increasingly determine whether an agent is reliable, efficient, and able to keep improving. We investigate this core problem through the lens of the Theory of Agent (ToA), which views an LLM agent not as a task executor but as an epistemic decision-maker in terms of reasoning or acting at every step. The two means it has for that epistemic uncertainty reduction, namely reasoning over what it already holds and acting on the world to obtain what it does not, are the run-time faces of internalization and externalization: reasoning draws on competence that has been absorbed into the model, while acting exploits competence that remains in the harness. We review this internalization and externalization across five chapters. Foundations sets up the basic picture: an agent that, at each step, decides whether to answer from what it already knows or reach outside for what it does not, and what separates a good version of that decision from a wasteful one. Learning examines how training places competence on either side of the boundary, spanning internalization of memory, reasoning, tool-use policy, and world models, and externalization of memory, skills, and orchestration. Aligning treats calibration as the connecting thread across an agent’s cognitive boundaries, efficiency constraints, human preferences, environment constraints, and other agents. Evolving studies how agents revise both sides of the boundary over time, from model-side and harness-side updates to agent–environment co-evolution. Evaluation reinterprets metrics and benchmarks by the epistemic effort they expose and argues for measuring the boundary decision rather than task completion alone. We conclude with the open problems that follow once internalization and externalization are treated as the governing axis of agent design.
- 3 September 2026
- Published by MDPI AG in Preprints.org
Open AccessTheory of Agent: The Science of Internalization and Externalization for LLM-Based Agents
A large language model (LLM) agent is a model coupled to a harness: parametric weights that hold knowledge and reasoning ability, wrapped in an external apparatus of memory, tools, and orchestration through which the model perceives and acts. Progress over the past years has come from pushing on both, with stronger models and richer harnesses, but the design decision that quietly governs an agent’s behavior is where competence should live: which knowledge, skills, and control to internalize into the model’s parameters, and which to externalize onto the surrounding harness. These decisions increasingly determine whether an agent is reliable, efficient, and able to keep improving. We investigate this core problem through the lens of the Theory of Agent (ToA), which views an LLM agent not as a task executor but as an epistemic decision-maker in terms of reasoning or acting at every step. The two means it has for that epistemic uncertainty reduction, namely reasoning over what it already holds and acting on the world to obtain what it does not, are the run-time faces of internalization and externalization: reasoning draws on competence that has been absorbed into the model, while acting exploits competence that remains in the harness. We review this internalization and externalization across five chapters. Foundations sets up the basic picture: an agent that, at each step, decides whether to answer from what it already knows or reach outside for what it does not, and what separates a good version of that decision from a wasteful one. Learning examines how training places competence on either side of the boundary, spanning internalization of memory, reasoning, tool-use policy, and world models, and externalization of memory, skills, and orchestration. Aligning treats calibration as the connecting thread across an agent’s cognitive boundaries, efficiency constraints, human preferences, environment constraints, and other agents. Evolving studies how agents revise both sides of the boundary over time, from model-side and harness-side updates to agent–environment co-evolution. Evaluation reinterprets metrics and benchmarks by the epistemic effort they expose and argues for measuring the boundary decision rather than task completion alone. We conclude with the open problems that follow once internalization and externalization are treated as the governing axis of agent design.
- Open Access
- 27 April 2026
- Published by Association for Computing Machinery (ACM)
- p. 407-416
On Harnessing Idle Compute at the Edge for Foundation Model Training
- 27 April 2026
- Published by Association for Computing Machinery (ACM)
- p. 407-416
Open AccessOn Harnessing Idle Compute at the Edge for Foundation Model Training
- Open Access
Probing Dec-POMDP Reasoning in Cooperative MARL
Cooperative multi-agent reinforcement learning (MARL) is typically framed as a decentralised partially observable Markov decision process (Dec-POMDP), a setting whose hardness stems from two key challenges: partial observability and decentralised coordination. Genuinely solving such tasks requires Dec-POMDP reasoning, where agents use history to infer hidden states and coordinate based on local information. Yet it remains unclear whether popular benchmarks actually demand this reasoning or permit success via simpler strategies. We introduce a diagnostic suite combining statistically grounded performance comparisons and information-theoretic probes to audit the behavioural complexity of baseline policies (IPPO and MAPPO) across 37 scenarios spanning MPE, SMAX, Overcooked, Hanabi, and MaBrax. Our diagnostics reveal that success on these benchmarks rarely requires genuine Dec-POMDP reasoning. Reactive policies match the performance of memory-based agents in over half the scenarios, and emergent coordination frequently relies on brittle, synchronous action coupling rather than robust temporal influence. These findings suggest that some widely used benchmarks may not adequately test core Dec-POMDP assumptions under current training paradigms, potentially leading to over-optimistic assessments of progress. We release our diagnostic tooling to support more rigorous environment design and evaluation in cooperative MARL.
Open AccessProbing Dec-POMDP Reasoning in Cooperative MARL
Cooperative multi-agent reinforcement learning (MARL) is typically framed as a decentralised partially observable Markov decision process (Dec-POMDP), a setting whose hardness stems from two key challenges: partial observability and decentralised coordination. Genuinely solving such tasks requires Dec-POMDP reasoning, where agents use history to infer hidden states and coordinate based on local information. Yet it remains unclear whether popular benchmarks actually demand this reasoning or permit success via simpler strategies. We introduce a diagnostic suite combining statistically grounded performance comparisons and information-theoretic probes to audit the behavioural complexity of baseline policies (IPPO and MAPPO) across 37 scenarios spanning MPE, SMAX, Overcooked, Hanabi, and MaBrax. Our diagnostics reveal that success on these benchmarks rarely requires genuine Dec-POMDP reasoning. Reactive policies match the performance of memory-based agents in over half the scenarios, and emergent coordination frequently relies on brittle, synchronous action coupling rather than robust temporal influence. These findings suggest that some widely used benchmarks may not adequately test core Dec-POMDP assumptions under current training paradigms, potentially leading to over-optimistic assessments of progress. We release our diagnostic tooling to support more rigorous environment design and evaluation in cooperative MARL.
- Open Access
- 3 November 2025
- Published by Association for Computing Machinery (ACM)
- p. 1329-1331
Poster: On Harnessing Idle Compute at the Edge for Foundation Model Training
- 3 November 2025
- Published by Association for Computing Machinery (ACM)
- p. 1329-1331
Open AccessPoster: On Harnessing Idle Compute at the Edge for Foundation Model Training
- Open Access
- 9 October 2025
- Published by Springer Nature
- p. C1
Correction to: Self-consistent Deep Approximation of Retinal Traits for Robust and Highly Efficient Vascular Phenotyping of Retinal Colour Fundus Images
- 9 October 2025
- Published by Springer Nature
- p. C1
Open AccessCorrection to: Self-consistent Deep Approximation of Retinal Traits for Robust and Highly Efficient Vascular Phenotyping of Retinal Colour Fundus Images
- Open Access
Remembering the Markov Property in Cooperative MARL
Cooperative multi-agent reinforcement learning (MARL) is typically formalised as a Decentralised Partially Observable Markov Decision Process (Dec-POMDP), where agents must reason about the environment and other agents' behaviour. In practice, current model-free MARL algorithms use simple recurrent function approximators to address the challenge of reasoning about others using partial information. In this position paper, we argue that the empirical success of these methods is not due to effective Markov signal recovery, but rather to learning simple conventions that bypass environment observations and memory. Through a targeted case study, we show that co-adapting agents can learn brittle conventions, which then fail when partnered with non-adaptive agents. Crucially, the same models can learn grounded policies when the task design necessitates it, revealing that the issue is not a fundamental limitation of the learning models but a failure of the benchmark design. Our analysis also suggests that modern MARL environments may not adequately test the core assumptions of Dec-POMDPs. We therefore advocate for new cooperative environments built upon two core principles: (1) behaviours grounded in observations and (2) memory-based reasoning about other agents, ensuring success requires genuine skill rather than fragile, co-adapted agreements.
Open AccessRemembering the Markov Property in Cooperative MARL
Cooperative multi-agent reinforcement learning (MARL) is typically formalised as a Decentralised Partially Observable Markov Decision Process (Dec-POMDP), where agents must reason about the environment and other agents' behaviour. In practice, current model-free MARL algorithms use simple recurrent function approximators to address the challenge of reasoning about others using partial information. In this position paper, we argue that the empirical success of these methods is not due to effective Markov signal recovery, but rather to learning simple conventions that bypass environment observations and memory. Through a targeted case study, we show that co-adapting agents can learn brittle conventions, which then fail when partnered with non-adaptive agents. Crucially, the same models can learn grounded policies when the task design necessitates it, revealing that the issue is not a fundamental limitation of the learning models but a failure of the benchmark design. Our analysis also suggests that modern MARL environments may not adequately test the core assumptions of Dec-POMDPs. We therefore advocate for new cooperative environments built upon two core principles: (1) behaviours grounded in observations and (2) memory-based reasoning about other agents, ensuring success requires genuine skill rather than fragile, co-adapted agreements.
- Open Access
- 21 March 2025
- Published by Association for Computing Machinery (ACM) in ACM Transactions on Architecture and Code Optimization
- Vol. 22 (1) , 1-28
DLAS: A Conceptual Model for Across-Stack Deep Learning Acceleration
Deep Neural Networks (DNNs) are very computationally demanding, which presents a significant barrier to their deployment, especially on resource-constrained devices. Significant work from both the machine learning and computing systems communities has attempted to accelerate DNNs. However, the number of techniques available and the required domain knowledge for their exploration continue to grow, making design space exploration (DSE) increasingly difficult. To unify the perspectives from these two communities, this article introduces the Deep Learning Acceleration Stack (DLAS), a conceptual model for DNN deployment and acceleration. We adopt a six-layer representation that organizes and illustrates the key areas for DNN acceleration, from machine learning to software and computer architecture. We argue that the DLAS model balances simplicity and expressiveness, assisting practitioners from various domains in tackling co-design acceleration challenges. We demonstrate the interdependence of the DLAS layers, and thus the need for co-design, through an across-stack perturbation study, using a modified tensor compiler to generate experiments for combinations of a few parameters across the DLAS layers. Our perturbation study assesses the impact on inference time and accuracy when varying DLAS parameters across two datasets, seven popular DNN architectures, four compression techniques, three algorithmic primitives (with sparse and dense variants), untuned and auto-scheduled code generation, and four hardware platforms. The study observes significant changes in the relative performance of design choices with the introduction of new DLAS parameters (e.g., the fastest algorithmic primitive varies with the level of quantization). Given the strong evidence for the need for co-design, and the high costs of DSE, DLAS offers a valuable conceptual model for better exploring advanced co-designed accelerated deep learning solutions.
- 21 March 2025
- Published by Association for Computing Machinery (ACM) in ACM Transactions on Architecture and Code Optimization
- Vol. 22 (1) , 1-28
Open AccessDLAS: A Conceptual Model for Across-Stack Deep Learning Acceleration
Deep Neural Networks (DNNs) are very computationally demanding, which presents a significant barrier to their deployment, especially on resource-constrained devices. Significant work from both the machine learning and computing systems communities has attempted to accelerate DNNs. However, the number of techniques available and the required domain knowledge for their exploration continue to grow, making design space exploration (DSE) increasingly difficult. To unify the perspectives from these two communities, this article introduces the Deep Learning Acceleration Stack (DLAS), a conceptual model for DNN deployment and acceleration. We adopt a six-layer representation that organizes and illustrates the key areas for DNN acceleration, from machine learning to software and computer architecture. We argue that the DLAS model balances simplicity and expressiveness, assisting practitioners from various domains in tackling co-design acceleration challenges. We demonstrate the interdependence of the DLAS layers, and thus the need for co-design, through an across-stack perturbation study, using a modified tensor compiler to generate experiments for combinations of a few parameters across the DLAS layers. Our perturbation study assesses the impact on inference time and accuracy when varying DLAS parameters across two datasets, seven popular DNN architectures, four compression techniques, three algorithmic primitives (with sparse and dense variants), untuned and auto-scheduled code generation, and four hardware platforms. The study observes significant changes in the relative performance of design choices with the introduction of new DLAS parameters (e.g., the fastest algorithmic primitive varies with the level of quantization). Given the strong evidence for the need for co-design, and the high costs of DSE, DLAS offers a valuable conceptual model for better exploring advanced co-designed accelerated deep learning solutions.
- 9 February 2025
- Published by Springer Nature
- p. 213-223
Self-consistent Deep Approximation of Retinal Traits for Robust and Highly Efficient Vascular Phenotyping of Retinal Colour Fundus Images
- 9 February 2025
- Published by Springer Nature
- p. 213-223
Self-consistent Deep Approximation of Retinal Traits for Robust and Highly Efficient Vascular Phenotyping of Retinal Colour Fundus Images
- 13 September 2024
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Transactions on Medical Imaging
- Vol. 44 (9) , 3574-3585
Diffusion Models for Counterfactual Generation and Anomaly Detection in Brain Images
Segmentation masks of pathological areas are useful in many medical applications, such as brain tumour and stroke management. Moreover, healthy counterfactuals of diseased images can be used to enhance radiologists’ training files and to improve the interpretability of segmentation models. In this work, we present a weakly supervised method to generate a healthy version of a diseased image and then use it to obtain a pixel-wise anomaly map. To do so, we start by considering a saliency map that approximately covers the pathological areas, obtained with ACAT. Then, we propose a technique that allows to perform targeted modifications to these regions, while preserving the rest of the image. In particular, we employ a diffusion model trained on healthy samples and combine Denoising Diffusion Probabilistic Model (DDPM) and Denoising Diffusion Implicit Model (DDIM) at each step of the sampling process. DDPM is used to modify the areas affected by a lesion within the saliency map, while DDIM guarantees reconstruction of the normal anatomy outside of it. The two parts are also fused at each timestep, to guarantee the generation of a sample with a coherent appearance and a seamless transition between edited and unedited parts. We verify that when our method is applied to healthy samples, the input images are reconstructed without significant modifications. We compare our approach with alternative weakly supervised methods on the task of brain lesion segmentation, achieving the highest mean Dice and IoU scores among the models considered.
- 13 September 2024
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Transactions on Medical Imaging
- Vol. 44 (9) , 3574-3585
Diffusion Models for Counterfactual Generation and Anomaly Detection in Brain Images
Segmentation masks of pathological areas are useful in many medical applications, such as brain tumour and stroke management. Moreover, healthy counterfactuals of diseased images can be used to enhance radiologists’ training files and to improve the interpretability of segmentation models. In this work, we present a weakly supervised method to generate a healthy version of a diseased image and then use it to obtain a pixel-wise anomaly map. To do so, we start by considering a saliency map that approximately covers the pathological areas, obtained with ACAT. Then, we propose a technique that allows to perform targeted modifications to these regions, while preserving the rest of the image. In particular, we employ a diffusion model trained on healthy samples and combine Denoising Diffusion Probabilistic Model (DDPM) and Denoising Diffusion Implicit Model (DDIM) at each step of the sampling process. DDPM is used to modify the areas affected by a lesion within the saliency map, while DDIM guarantees reconstruction of the normal anatomy outside of it. The two parts are also fused at each timestep, to guarantee the generation of a sample with a coherent appearance and a seamless transition between edited and unedited parts. We verify that when our method is applied to healthy samples, the input images are reconstructed without significant modifications. We compare our approach with alternative weakly supervised methods on the task of brain lesion segmentation, achieving the highest mean Dice and IoU scores among the models considered.
- Open Access
Noisy Early Stopping for Noisy Labels
Training neural network classifiers on datasets contaminated with noisy labels significantly increases the risk of overfitting. Thus, effectively implementing Early Stopping in noisy label environments is crucial. Under ideal circumstances, Early Stopping utilises a validation set uncorrupted by label noise to effectively monitor generalisation during training. However, obtaining a noise-free validation dataset can be costly and challenging to obtain. This study establishes that, in many typical learning environments, a noise-free validation set is not necessary for effective Early Stopping. Instead, near-optimal results can be achieved by monitoring accuracy on a noisy dataset - drawn from the same distribution as the noisy training set. Referred to as `Noisy Early Stopping' (NES), this method simplifies and reduces the cost of implementing Early Stopping. We provide theoretical insights into the conditions under which this method is effective and empirically demonstrate its robust performance across standard benchmarks using common loss functions.
Open AccessNoisy Early Stopping for Noisy Labels
Training neural network classifiers on datasets contaminated with noisy labels significantly increases the risk of overfitting. Thus, effectively implementing Early Stopping in noisy label environments is crucial. Under ideal circumstances, Early Stopping utilises a validation set uncorrupted by label noise to effectively monitor generalisation during training. However, obtaining a noise-free validation dataset can be costly and challenging to obtain. This study establishes that, in many typical learning environments, a noise-free validation set is not necessary for effective Early Stopping. Instead, near-optimal results can be achieved by monitoring accuracy on a noisy dataset - drawn from the same distribution as the noisy training set. Referred to as `Noisy Early Stopping' (NES), this method simplifies and reduces the cost of implementing Early Stopping. We provide theoretical insights into the conditions under which this method is effective and empirically demonstrate its robust performance across standard benchmarks using common loss functions.