Daniel Bienstock
Current Organizations / Institutes
The organizations/institutes from the scholar's latest article
Previous Organizations / Institutes
The organizations / institutes from the scholar's other articles in the past ten years
Daniel Bienstock has a total of 63 co-authors. Here are the top co-authors by number of shared publications:
- Mauro Escobar
LIX CNRS, Ecole Polytechnique, Institut Polytechnique de Paris, Palaiseau, France
4 shared publications - Leo Liberti
LIX CNRS, Ecole Polytechnique, Institut Polytechnique de Paris, Palaiseau, France
3 shared publications - Michael Chertkov
Program in Applied Mathematics, Department of Mathematics, University of Arizona, Tucson, AZ 85721, USA
3 shared publications - Russell Bent
Los Alamos National Laboratory Los Alamos NM USA
3 shared publications - Yury Dvorkin
Johns Hopkins University, Baltimore, USA
3 shared publications - Alberto Del Pia
Department of Industrial and Systems Engineering & Wisconsin Institute for Discovery, University of Wisconsin-Madison, Madison, USA
2 shared publications
Latest Publications
- Open Access
Advanced Cutting-Plane Algorithms for ACOPF
We propose a disciplined, numerically stable, and scalable approach to SDP relaxations of the ACOPF problem based on linear cutting-planes. Our method can be warm-started and, owing to its linear nature, enables the computation of tight and accurate bounds for large-scale multi-period relaxations -- well beyond what nonlinear convex solvers can achieve. Preliminary experiments show promising results when benchmarked against state-of-the-art bounds on PGLIB instances.
Open AccessAdvanced Cutting-Plane Algorithms for ACOPF
We propose a disciplined, numerically stable, and scalable approach to SDP relaxations of the ACOPF problem based on linear cutting-planes. Our method can be warm-started and, owing to its linear nature, enables the computation of tight and accurate bounds for large-scale multi-period relaxations -- well beyond what nonlinear convex solvers can achieve. Preliminary experiments show promising results when benchmarked against state-of-the-art bounds on PGLIB instances.
- 7 August 2025
- Published by Springer Nature in Mathematical Programming Computation
- Vol. 18 (1) , 79-133
Accurate linear cutting-plane relaxations for ACOPF
- 7 August 2025
- Published by Springer Nature in Mathematical Programming Computation
- Vol. 18 (1) , 79-133
Accurate linear cutting-plane relaxations for ACOPF
- 18 September 2024
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Transactions on Energy Markets, Policy and Regulation
- Vol. 2 (4) , 536-551
Risk-Aware Security-Constrained Unit Commitment: Taming the Curse of Real-Time Volatility and Consumer Exposure
We propose an enhancement to wholesale electricity markets to contain the exposure of consumers to increasingly large and volatile consumer payments arising as a byproduct of volatile real-time net loads – i.e., loads minus renewable outputs – and prices, both compared to day-ahead cleared values. We incorporate a trade-off, motivated by portfolio optimization methods, between standard day-ahead payments and a robust estimate of such excess payments into the day-ahead computation and specifically seek to account for volatility in real-time net loads and renewable generation. Our model features a data-driven uncertainty set based on principal component analysis, which accommodates both load and wind production volatility and captures locational correlation of uncertain data. To solve the model more efficiently, we develop a decomposition algorithm that can handle nonconvex subproblems. Our extensive experiments on a realistic NYISO data set show that the risk-aware model protects the consumers from potential high costs caused by adverse circumstances.
- 18 September 2024
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Transactions on Energy Markets, Policy and Regulation
- Vol. 2 (4) , 536-551
Risk-Aware Security-Constrained Unit Commitment: Taming the Curse of Real-Time Volatility and Consumer Exposure
We propose an enhancement to wholesale electricity markets to contain the exposure of consumers to increasingly large and volatile consumer payments arising as a byproduct of volatile real-time net loads – i.e., loads minus renewable outputs – and prices, both compared to day-ahead cleared values. We incorporate a trade-off, motivated by portfolio optimization methods, between standard day-ahead payments and a robust estimate of such excess payments into the day-ahead computation and specifically seek to account for volatility in real-time net loads and renewable generation. Our model features a data-driven uncertainty set based on principal component analysis, which accommodates both load and wind production volatility and captures locational correlation of uncertain data. To solve the model more efficiently, we develop a decomposition algorithm that can handle nonconvex subproblems. Our extensive experiments on a realistic NYISO data set show that the risk-aware model protects the consumers from potential high costs caused by adverse circumstances.
- 1 May 2024
- Published by Society for Industrial & Applied Mathematics (SIAM) in SIAM Review
- Vol. 66 (3) , 403-477
Cardinality Minimization, Constraints, and Regularization: A Survey
We survey optimization problems that involve the cardinality of variable vectors in constraints or the objective function. We provide a unified viewpoint on the general problem classes and models, and we give concrete examples from diverse application fields such as signal and image processing, portfolio selection, and machine learning. The paper discusses general-purpose modeling techniques and broadly applicable as well as problem-specific exact and heuristic solution approaches. While our perspective is that of mathematical optimization, a main goal of this work is to reach out to and build bridges between the different communities in which cardinality optimization problems are frequently encountered. In particular, we highlight that modern mixed-integer programming, which is often regarded as impractical due to the commonly unsatisfactory behavior of black-box solvers applied to generic problem formulations, can in fact produce provably high-quality or even optimal solutions for cardinality optimization problems, even in large-scale real-world settings. Achieving such performance typically involves drawing on the merits of problem-specific knowledge that may stem from different fields of application and, e.g., can shed light on structural properties of a model or its solutions, or can lead to the development of efficient heuristics. We also provide some illustrative examples.
- 1 May 2024
- Published by Society for Industrial & Applied Mathematics (SIAM) in SIAM Review
- Vol. 66 (3) , 403-477
Cardinality Minimization, Constraints, and Regularization: A Survey
We survey optimization problems that involve the cardinality of variable vectors in constraints or the objective function. We provide a unified viewpoint on the general problem classes and models, and we give concrete examples from diverse application fields such as signal and image processing, portfolio selection, and machine learning. The paper discusses general-purpose modeling techniques and broadly applicable as well as problem-specific exact and heuristic solution approaches. While our perspective is that of mathematical optimization, a main goal of this work is to reach out to and build bridges between the different communities in which cardinality optimization problems are frequently encountered. In particular, we highlight that modern mixed-integer programming, which is often regarded as impractical due to the commonly unsatisfactory behavior of black-box solvers applied to generic problem formulations, can in fact produce provably high-quality or even optimal solutions for cardinality optimization problems, even in large-scale real-world settings. Achieving such performance typically involves drawing on the merits of problem-specific knowledge that may stem from different fields of application and, e.g., can shed light on structural properties of a model or its solutions, or can lead to the development of efficient heuristics. We also provide some illustrative examples.
- 9 March 2024
- Published by Springer Nature in Optimization and Engineering
- Vol. 25 (4) , 2047-2072
Optimizing ventilation in medium- and short-term mine planning
- 9 March 2024
- Published by Springer Nature in Optimization and Engineering
- Vol. 25 (4) , 2047-2072
Optimizing ventilation in medium- and short-term mine planning
- Open Access
Incentivizing Investment and Reliability: A Study on Electricity Capacity Markets
The capacity market, a marketplace to exchange available generation capacity for electricity production, provides a major revenue stream for generators and is adopted in several U.S. regions. A subject of ongoing debate, the capacity market is viewed by its proponents as a crucial mechanism to ensure system reliability, while critics highlight its drawbacks such as market distortion. Under a novel analytical framework, we rigorously evaluate the impact of the capacity market on generators' revenue and system reliability. More specifically, based on market designs at New York Independent System Operator (NYISO), we propose market equilibrium-based models to capture salient aspects of the capacity market and its interaction with the energy market. We also develop a leader-follower model to study market power. We show that the capacity market incentivizes the investment of generators with lower net cost of new entry. It also facilitates reliability by preventing significant physical withholding when the demand is relatively high. Nevertheless, the capacity market may not provide enough revenue for peaking plants. Moreover, it is susceptible to market power, which necessitates tailored market power mitigation measures depending on market dynamics. We provide further insights via large-scale experiments on data from NYISO markets.
Open AccessIncentivizing Investment and Reliability: A Study on Electricity Capacity Markets
The capacity market, a marketplace to exchange available generation capacity for electricity production, provides a major revenue stream for generators and is adopted in several U.S. regions. A subject of ongoing debate, the capacity market is viewed by its proponents as a crucial mechanism to ensure system reliability, while critics highlight its drawbacks such as market distortion. Under a novel analytical framework, we rigorously evaluate the impact of the capacity market on generators' revenue and system reliability. More specifically, based on market designs at New York Independent System Operator (NYISO), we propose market equilibrium-based models to capture salient aspects of the capacity market and its interaction with the energy market. We also develop a leader-follower model to study market power. We show that the capacity market incentivizes the investment of generators with lower net cost of new entry. It also facilitates reliability by preventing significant physical withholding when the demand is relatively high. Nevertheless, the capacity market may not provide enough revenue for peaking plants. Moreover, it is susceptible to market power, which necessitates tailored market power mitigation measures depending on market dynamics. We provide further insights via large-scale experiments on data from NYISO markets.
- Open Access
- 22 September 2023
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Transactions on Energy Markets, Policy and Regulation
- Vol. 2 (1) , 40-51
Physics-Informed Machine Learning for Electricity Markets: A NYISO Case Study
This paper addresses the challenge of efficiently recovering exact solutions to the optimal power flow problem in real-time electricity markets. The proposed solution, named Physics-Informed Market-Aware Active Set learning OPF (PIMA-AS-OPF), leverages physical constraints and market properties to ensure physical and economic feasibility of market-clearing outcomes. Specifically, PIMA-AS-OPF employs the active set learning technique and expands its capabilities to account for curtailment in load or renewable power generation, which is a common challenge in real-world power systems. The core of PIMA-AS-OPF is a fully-connected neural network that takes the net load and the system topology as input. The outputs of this neural network include active constraints such as saturated generators and transmission lines, as well as non-zero load shedding and wind curtailments. These outputs allow for reducing the original market-clearing optimization to a system of linear equations, which can be solved efficiently and yield both the dispatch decisions and the locational marginal prices (LMPs). The dispatch decisions and LMPs are then tested for their feasibility with respect to the requirements for efficient market- clearing results. The accuracy and scalability of the proposed method is tested on a realistic 1814-bus NYISO system with current and future renewable energy penetration levels.
- 22 September 2023
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Transactions on Energy Markets, Policy and Regulation
- Vol. 2 (1) , 40-51
Open AccessPhysics-Informed Machine Learning for Electricity Markets: A NYISO Case Study
This paper addresses the challenge of efficiently recovering exact solutions to the optimal power flow problem in real-time electricity markets. The proposed solution, named Physics-Informed Market-Aware Active Set learning OPF (PIMA-AS-OPF), leverages physical constraints and market properties to ensure physical and economic feasibility of market-clearing outcomes. Specifically, PIMA-AS-OPF employs the active set learning technique and expands its capabilities to account for curtailment in load or renewable power generation, which is a common challenge in real-world power systems. The core of PIMA-AS-OPF is a fully-connected neural network that takes the net load and the system topology as input. The outputs of this neural network include active constraints such as saturated generators and transmission lines, as well as non-zero load shedding and wind curtailments. These outputs allow for reducing the original market-clearing optimization to a system of linear equations, which can be solved efficiently and yield both the dispatch decisions and the locational marginal prices (LMPs). The dispatch decisions and LMPs are then tested for their feasibility with respect to the requirements for efficient market- clearing results. The accuracy and scalability of the proposed method is tested on a realistic 1814-bus NYISO system with current and future renewable energy penetration levels.
- Open Access
- 28 May 2023
- Published by Springer Nature in Journal of Optimization Theory and Applications
- Vol. 198 (1) , 428-447
On a Computationally Ill-Behaved Bilevel Problem with a Continuous and Nonconvex Lower Level
It is well known that bilevel optimization problems are hard to solve both in theory and practice. In this paper, we highlight a further computational difficulty when it comes to solving bilevel problems with continuous but nonconvex lower levels. Even if the lower-level problem is solved to $$\varepsilon $$ -feasibility regarding its nonlinear constraints for an arbitrarily small but positive $$\varepsilon $$ , the obtained bilevel solution as well as its objective value may be arbitrarily far away from the actual bilevel solution and its actual objective value. This result even holds for bilevel problems for which the nonconvex lower level is uniquely solvable, for which the strict complementarity condition holds, for which the feasible set is convex, and for which Slater’s constraint qualification is satisfied for all feasible upper-level decisions. Since the consideration of $$\varepsilon $$ -feasibility cannot be avoided when solving nonconvex problems to global optimality, our result shows that computational bilevel optimization with continuous and nonconvex lower levels needs to be done with great care. Finally, we illustrate that the nonlinearities in the lower level are the key reason for the observed bad behavior by showing that linear bilevel problems behave much better at least on the level of feasible solutions.- 28 May 2023
- Published by Springer Nature in Journal of Optimization Theory and Applications
- Vol. 198 (1) , 428-447
Open AccessOn a Computationally Ill-Behaved Bilevel Problem with a Continuous and Nonconvex Lower Level
It is well known that bilevel optimization problems are hard to solve both in theory and practice. In this paper, we highlight a further computational difficulty when it comes to solving bilevel problems with continuous but nonconvex lower levels. Even if the lower-level problem is solved to $$\varepsilon $$ -feasibility regarding its nonlinear constraints for an arbitrarily small but positive $$\varepsilon $$ , the obtained bilevel solution as well as its objective value may be arbitrarily far away from the actual bilevel solution and its actual objective value. This result even holds for bilevel problems for which the nonconvex lower level is uniquely solvable, for which the strict complementarity condition holds, for which the feasible set is convex, and for which Slater’s constraint qualification is satisfied for all feasible upper-level decisions. Since the consideration of $$\varepsilon $$ -feasibility cannot be avoided when solving nonconvex problems to global optimality, our result shows that computational bilevel optimization with continuous and nonconvex lower levels needs to be done with great care. Finally, we illustrate that the nonlinearities in the lower level are the key reason for the observed bad behavior by showing that linear bilevel problems behave much better at least on the level of feasible solutions. - Open Access
On a Computationally Ill-Behaved Bilevel Problem with a Continuous and Nonconvex Lower Level
It is well known that bilevel optimization problems are hard to solve both in theory and practice. In this paper, we highlight a further computational difficulty when it comes to solving bilevel problems with continuous but nonconvex lower levels. Even if the lower-level problem is solved to $\varepsilon$-feasibility regarding its nonlinear constraints for an arbitrarily small but positive $\varepsilon$, the obtained bilevel solution as well as its objective value may be arbitrarily far away from the actual bilevel solution and its actual objective value. This result even holds for bilevel problems for which the nonconvex lower level is uniquely solvable, for which the strict complementarity condition holds, for which the feasible set is convex, and for which Slater's constraint qualification is satisfied for all feasible upper-level decisions. Since the consideration of $\varepsilon$-feasibility cannot be avoided when solving nonconvex problems to global optimality, our result shows that computational bilevel optimization with continuous and nonconvex lower levels needs to be done with great care. Finally, we illustrate that the nonlinearities in the lower level are the key reason for the observed bad behavior by showing that linear bilevel problems behave much better at least on the level of feasible solutions.Open AccessOn a Computationally Ill-Behaved Bilevel Problem with a Continuous and Nonconvex Lower Level
It is well known that bilevel optimization problems are hard to solve both in theory and practice. In this paper, we highlight a further computational difficulty when it comes to solving bilevel problems with continuous but nonconvex lower levels. Even if the lower-level problem is solved to $\varepsilon$-feasibility regarding its nonlinear constraints for an arbitrarily small but positive $\varepsilon$, the obtained bilevel solution as well as its objective value may be arbitrarily far away from the actual bilevel solution and its actual objective value. This result even holds for bilevel problems for which the nonconvex lower level is uniquely solvable, for which the strict complementarity condition holds, for which the feasible set is convex, and for which Slater's constraint qualification is satisfied for all feasible upper-level decisions. Since the consideration of $\varepsilon$-feasibility cannot be avoided when solving nonconvex problems to global optimality, our result shows that computational bilevel optimization with continuous and nonconvex lower levels needs to be done with great care. Finally, we illustrate that the nonlinearities in the lower level are the key reason for the observed bad behavior by showing that linear bilevel problems behave much better at least on the level of feasible solutions. - Open Access
Cardinality Minimization, Constraints, and Regularization: A Survey
We survey optimization problems that involve the cardinality of variable vectors in constraints or the objective function. We provide a unified viewpoint on the general problem classes and models, and give concrete examples from diverse application fields such as signal and image processing, portfolio selection, or machine learning. The paper discusses general-purpose modeling techniques and broadly applicable as well as problem-specific exact and heuristic solution approaches. While our perspective is that of mathematical optimization, a main goal of this work is to reach out to and build bridges between the different communities in which cardinality optimization problems are frequently encountered. In particular, we highlight that modern mixed-integer programming, which is often regarded as impractical due to commonly unsatisfactory behavior of black-box solvers applied to generic problem formulations, can in fact produce provably high-quality or even optimal solutions for cardinality optimization problems, even in large-scale real-world settings. Achieving such performance typically draws on the merits of problem-specific knowledge that may stem from different fields of application and, e.g., shed light on structural properties of a model or its solutions, or lead to the development of efficient heuristics; we also provide some illustrative examples.
Open AccessCardinality Minimization, Constraints, and Regularization: A Survey
We survey optimization problems that involve the cardinality of variable vectors in constraints or the objective function. We provide a unified viewpoint on the general problem classes and models, and give concrete examples from diverse application fields such as signal and image processing, portfolio selection, or machine learning. The paper discusses general-purpose modeling techniques and broadly applicable as well as problem-specific exact and heuristic solution approaches. While our perspective is that of mathematical optimization, a main goal of this work is to reach out to and build bridges between the different communities in which cardinality optimization problems are frequently encountered. In particular, we highlight that modern mixed-integer programming, which is often regarded as impractical due to commonly unsatisfactory behavior of black-box solvers applied to generic problem formulations, can in fact produce provably high-quality or even optimal solutions for cardinality optimization problems, even in large-scale real-world settings. Achieving such performance typically draws on the merits of problem-specific knowledge that may stem from different fields of application and, e.g., shed light on structural properties of a model or its solutions, or lead to the development of efficient heuristics; we also provide some illustrative examples.