Bin Hu
Research Subjects
Research Interests
Bin Hu has a total of 9 co-authors. Here are the top co-authors by number of shared publications:
- Bo Cao
Taobao & Tmall Group, Hangzhou, China
1 shared publication - Chen Yang
Department of Industrial Engineering and Decision Analytics, Hong Kong University of Science and Technology, Clear Water Bay, Hong Kong, China
1 shared publication - Daniel Zhuoyu Long
Department of Systems Engineering and Engineering Management, Chinese University of Hong Kong, Shatin, Hong Kong
1 shared publication - Feng Wang
Ant Group, Hangzhou, China
1 shared publication - Jin Qi
The Hong Kong University of Science and Technology
1 shared publication - Ruohan Zhan
University College London, London, United Kingdom
1 shared publication
Latest Publications
- Open Access
- 4 August 2023
- Published by Association for Computing Machinery (ACM)
- p. 2918-2929
Fragility Index: A New Approach for Binary Classification
In binary classification problems, many performance metrics evaluate the probability that some error exceeds a threshold. Nevertheless, they focus more on the probability and fail to capture the magnitude of the error, which evaluates how large this error exceeds the threshold. Capturing the magnitude of error is desired in many applications. For example, in detecting disease and predicting credit default, the magnitude of error illustrates the confidence in making the wrong prediction. We propose a novel metric, the Fragility Index (FI), to evaluate the performance of binary classifiers by capturing the magnitude of the error. FI alleviates the risk of misclassification by penalizing the large error greatly, which is seldom considered by standard metrics. Moreover, to strengthen the generalization ability and handle unseen samples, we adopt the framework of distributionally robust optimization and robust satisficing, which allows us to derive and control the maximum degree of fragility of the classifier when the distribution of samples shifts. We show that FI can be easily calculated and optimized for common probabilistic distance measures. Experiments with real datasets demonstrate the new insights brought by FI and the advantages of classifiers selected under FI, which always improve the robustness and reduce the risk of large errors as compared to classifiers selected by alternative metrics.
- 4 August 2023
- Published by Association for Computing Machinery (ACM)
- p. 2918-2929
Open AccessFragility Index: A New Approach for Binary Classification
In binary classification problems, many performance metrics evaluate the probability that some error exceeds a threshold. Nevertheless, they focus more on the probability and fail to capture the magnitude of the error, which evaluates how large this error exceeds the threshold. Capturing the magnitude of error is desired in many applications. For example, in detecting disease and predicting credit default, the magnitude of error illustrates the confidence in making the wrong prediction. We propose a novel metric, the Fragility Index (FI), to evaluate the performance of binary classifiers by capturing the magnitude of the error. FI alleviates the risk of misclassification by penalizing the large error greatly, which is seldom considered by standard metrics. Moreover, to strengthen the generalization ability and handle unseen samples, we adopt the framework of distributionally robust optimization and robust satisficing, which allows us to derive and control the maximum degree of fragility of the classifier when the distribution of samples shifts. We show that FI can be easily calculated and optimized for common probabilistic distance measures. Experiments with real datasets demonstrate the new insights brought by FI and the advantages of classifiers selected under FI, which always improve the robustness and reduce the risk of large errors as compared to classifiers selected by alternative metrics.