Laura Wynter
Research Subjects
Laura Wynter has a total of 35 co-authors. Here are the top co-authors by number of shared publications:
- Sebastien Blandin
IBM Research Singapore
5 shared publications - Shiau Hong Lim
IBM Research
4 shared publications - Achintya Kundu
IBM Research IBM Research IBM Research IBM Research
3 shared publications - Baoyang Song
Ecole Polytechnique Palaiseau France
2 shared publications - Hasan Poonawala
IBM Research Singapore Ecole Polytechnique France
2 shared publications Pengqian Yu
IBM Research
2 shared publications
Latest Publications
- Open Access
Efficiently Distilling LLMs for Edge Applications
Supernet training of LLMs is of great interest in industrial applications as it confers the ability to produce a palette of smaller models at constant cost, regardless of the number of models (of different size / latency) produced. We propose a new method called Multistage Low-rank Fine-tuning of Super-transformers (MLFS) for parameter-efficient supernet training. We show that it is possible to obtain high-quality encoder models that are suitable for commercial edge applications, and that while decoder-only models are resistant to a comparable degree of compression, decoders can be effectively sliced for a significant reduction in training time.
Open AccessEfficiently Distilling LLMs for Edge Applications
Supernet training of LLMs is of great interest in industrial applications as it confers the ability to produce a palette of smaller models at constant cost, regardless of the number of models (of different size / latency) produced. We propose a new method called Multistage Low-rank Fine-tuning of Super-transformers (MLFS) for parameter-efficient supernet training. We show that it is possible to obtain high-quality encoder models that are suitable for commercial edge applications, and that while decoder-only models are resistant to a comparable degree of compression, decoders can be effectively sliced for a significant reduction in training time.
- 30 October 2023
- Published by Institute of Electrical and Electronics Engineers (IEEE)
- p. 39-44
M3: Towards Efficient Mixed Machine Learning Model Co-Location on Constrained Edge Devices
This paper explores the effects of machine learning (ML) model co-location on resource-constrained edge devices, and proposes M3, a mixed machine learning model co-location framework that leverages techniques such as network architecture search to generate resource-aware models, as well as quantization to further reduce resource requirements, alongside a runtime that dynamically switches between resource-aware models to serve specific tasks.
- 30 October 2023
- Published by Institute of Electrical and Electronics Engineers (IEEE)
- p. 39-44
M3: Towards Efficient Mixed Machine Learning Model Co-Location on Constrained Edge Devices
This paper explores the effects of machine learning (ML) model co-location on resource-constrained edge devices, and proposes M3, a mixed machine learning model co-location framework that leverages techniques such as network architecture search to generate resource-aware models, as well as quantization to further reduce resource requirements, alongside a runtime that dynamically switches between resource-aware models to serve specific tasks.
- Open Access
Transfer-Once-For-All: AI Model Optimization for Edge
Weight-sharing neural architecture search aims to optimize a configurable neural network model (supernet) for a variety of deployment scenarios across many devices with different resource constraints. Existing approaches use evolutionary search to extract models of different sizes from a supernet trained on a very large data set, and then fine-tune the extracted models on the typically small, real-world data set of interest. The computational cost of training thus grows linearly with the number of different model deployment scenarios. Hence, we propose Transfer-Once-For-All (TOFA) for supernet-style training on small data sets with constant computational training cost over any number of edge deployment scenarios. Given a task, TOFA obtains custom neural networks, both the topology and the weights, optimized for any number of edge deployment scenarios. To overcome the challenges arising from small data, TOFA utilizes a unified semi-supervised training loss to simultaneously train all subnets within the supernet, coupled with on-the-fly architecture selection at deployment time.
Open AccessTransfer-Once-For-All: AI Model Optimization for Edge
Weight-sharing neural architecture search aims to optimize a configurable neural network model (supernet) for a variety of deployment scenarios across many devices with different resource constraints. Existing approaches use evolutionary search to extract models of different sizes from a supernet trained on a very large data set, and then fine-tune the extracted models on the typically small, real-world data set of interest. The computational cost of training thus grows linearly with the number of different model deployment scenarios. Hence, we propose Transfer-Once-For-All (TOFA) for supernet-style training on small data sets with constant computational training cost over any number of edge deployment scenarios. Given a task, TOFA obtains custom neural networks, both the topology and the weights, optimized for any number of edge deployment scenarios. To overcome the challenges arising from small data, TOFA utilizes a unified semi-supervised training loss to simultaneously train all subnets within the supernet, coupled with on-the-fly architecture selection at deployment time.
- Open Access
Robustness and Personalization in Federated Learning: A Unified Approach via Regularization
We present a class of methods for robust, personalized federated learning, called Fed+, that unifies many federated learning algorithms. The principal advantage of this class of methods is to better accommodate the real-world characteristics found in federated training, such as the lack of IID data across parties, the need for robustness to outliers or stragglers, and the requirement to perform well on party-specific datasets. We achieve this through a problem formulation that allows the central server to employ robust ways of aggregating the local models while keeping the structure of local computation intact. Without making any statistical assumption on the degree of heterogeneity of local data across parties, we provide convergence guarantees for Fed+ for convex and non-convex loss functions under different (robust) aggregation methods. The Fed+ theory is also equipped to handle heterogeneous computing environments including stragglers without additional assumptions; specifically, the convergence results cover the general setting where the number of local update steps across parties can vary. We demonstrate the benefits of Fed+ through extensive experiments across standard benchmark datasets.
Open AccessRobustness and Personalization in Federated Learning: A Unified Approach via Regularization
We present a class of methods for robust, personalized federated learning, called Fed+, that unifies many federated learning algorithms. The principal advantage of this class of methods is to better accommodate the real-world characteristics found in federated training, such as the lack of IID data across parties, the need for robustness to outliers or stragglers, and the requirement to perform well on party-specific datasets. We achieve this through a problem formulation that allows the central server to employ robust ways of aggregating the local models while keeping the structure of local computation intact. Without making any statistical assumption on the degree of heterogeneity of local data across parties, we provide convergence guarantees for Fed+ for convex and non-convex loss functions under different (robust) aggregation methods. The Fed+ theory is also equipped to handle heterogeneous computing environments including stragglers without additional assumptions; specifically, the convergence results cover the general setting where the number of local update steps across parties can vary. We demonstrate the benefits of Fed+ through extensive experiments across standard benchmark datasets.
- 8 February 2022
- Published by Springer Nature
- p. 467-482
Federated Reinforcement Learning for Portfolio Management
- 8 February 2022
- Published by Springer Nature
- p. 467-482
Federated Reinforcement Learning for Portfolio Management
- 8 February 2022
- Published by Springer Nature
- p. 99-123
Personalized, Robust Federated Learning with Fed+
- 8 February 2022
- Published by Springer Nature
- p. 99-123
Personalized, Robust Federated Learning with Fed+
- Open Access
A Deep Ensemble Multi-Agent Reinforcement Learning Approach for Air Traffic Control
Air traffic control is an example of a highly challenging operational problem that is readily amenable to human expertise augmentation via decision support technologies. In this paper, we propose a new intelligent decision making framework that leverages multi-agent reinforcement learning (MARL) to dynamically suggest adjustments of aircraft speeds in real-time. The goal of the system is to enhance the ability of an air traffic controller to provide effective guidance to aircraft to avoid air traffic congestion, near-miss situations, and to improve arrival timeliness. We develop a novel deep ensemble MARL method that can concisely capture the complexity of the air traffic control problem by learning to efficiently arbitrate between the decisions of a local kernel-based RL model and a wider-reaching deep MARL model. The proposed method is trained and evaluated on an open-source air traffic management simulator developed by Eurocontrol. Extensive empirical results on a real-world dataset including thousands of aircraft demonstrate the feasibility of using multi-agent RL for the problem of en-route air traffic control and show that our proposed deep ensemble MARL method significantly outperforms three state-of-the-art benchmark approaches.
Open AccessA Deep Ensemble Multi-Agent Reinforcement Learning Approach for Air Traffic Control
Air traffic control is an example of a highly challenging operational problem that is readily amenable to human expertise augmentation via decision support technologies. In this paper, we propose a new intelligent decision making framework that leverages multi-agent reinforcement learning (MARL) to dynamically suggest adjustments of aircraft speeds in real-time. The goal of the system is to enhance the ability of an air traffic controller to provide effective guidance to aircraft to avoid air traffic congestion, near-miss situations, and to improve arrival timeliness. We develop a novel deep ensemble MARL method that can concisely capture the complexity of the air traffic control problem by learning to efficiently arbitrate between the decisions of a local kernel-based RL model and a wider-reaching deep MARL model. The proposed method is trained and evaluated on an open-source air traffic management simulator developed by Eurocontrol. Extensive empirical results on a real-world dataset including thousands of aircraft demonstrate the feasibility of using multi-agent RL for the problem of en-route air traffic control and show that our proposed deep ensemble MARL method significantly outperforms three state-of-the-art benchmark approaches.
- 2 November 2019
- Published by Springer Nature
- p. 337-368
The Internet of Things and Machine Learning, Solutions for Urban Infrastructure Management
- 2 November 2019
- Published by Springer Nature
- p. 337-368
The Internet of Things and Machine Learning, Solutions for Urban Infrastructure Management
- 1 July 2019
- Published by Institute for Operations Research and the Management Sciences (INFORMS) in INFORMS Journal on Applied Analytics
- Vol. 49 (4) , 269-280
MPA-IBM Project SAFER: Sense-Making Analytics for Maritime Event Recognition
The authors describe Project SAFER, which they established to conceptualize, develop, and test new analytics-based technologies to enhance port operations and cater to the increasing growth in vessel traffic in Singapore.
- 1 July 2019
- Published by Institute for Operations Research and the Management Sciences (INFORMS) in INFORMS Journal on Applied Analytics
- Vol. 49 (4) , 269-280
MPA-IBM Project SAFER: Sense-Making Analytics for Maritime Event Recognition
The authors describe Project SAFER, which they established to conceptualize, develop, and test new analytics-based technologies to enhance port operations and cater to the increasing growth in vessel traffic in Singapore.
- Open Access
FASTER: Fusion AnalyticS for public Transport Event Response
Increasing urban concentration raises operational challenges that can benefit from integrated monitoring and decision support. Such complex systems need to leverage the full stack of analytical methods, from state estimation using multi-sensor fusion for situational awareness, to prediction and computation of optimal responses. The FASTER platform that we describe in this work, deployed at nation scale and handling 1.5 billion public transport trips a year, offers such a full stack of techniques for this large-scale, real-time problem. FASTER provides fine-grained situational awareness and real-time decision support with the objective of improving the public transport commuter experience. The methods employed range from statistical machine learning to agent-based simulation and mixed-integer optimization. In this work we present an overview of the challenges and methods involved, with details of the commuter movement prediction module, as well as a discussion of open problems.
Open AccessFASTER: Fusion AnalyticS for public Transport Event Response
Increasing urban concentration raises operational challenges that can benefit from integrated monitoring and decision support. Such complex systems need to leverage the full stack of analytical methods, from state estimation using multi-sensor fusion for situational awareness, to prediction and computation of optimal responses. The FASTER platform that we describe in this work, deployed at nation scale and handling 1.5 billion public transport trips a year, offers such a full stack of techniques for this large-scale, real-time problem. FASTER provides fine-grained situational awareness and real-time decision support with the objective of improving the public transport commuter experience. The methods employed range from statistical machine learning to agent-based simulation and mixed-integer optimization. In this work we present an overview of the challenges and methods involved, with details of the commuter movement prediction module, as well as a discussion of open problems.