Ehsan S. Soofi
Current Organizations / Institutes
The organizations/institutes from the scholar's latest article
Previous Organizations / Institutes
The organizations / institutes from the scholar's other articles in the past ten years
Research Subjects
Ehsan S. Soofi has a total of 33 co-authors. Here are the top co-authors by number of shared publications:
- Majid Asadi
Department of Statistics, Faculty of Mathematics and Statistics, University of Isfahan, Isfahan, Iran
9 shared publications - Nader Ebrahimi
Division of Statistics, Northern Illinois University, Dekalb, IL, USA
8 shared publications Nader Ebrahimi
Department of Statistics, Northern Illinois University, Dekalb, Illinois, USA
5 shared publicationsMajid Asadi
Department of Statistics, University of Isfahan, Isfahan, IRAN
4 shared publicationsG. G. Hamedani
Marquette University
3 shared publications- Nader Ebrahimi
Division of Statistics, Northern Illinois University, Dekalb, IL, USA
3 shared publications
Latest Publications
- 1 January 2025
- Published by Wiley in Applied Stochastic Models in Business and Industry
- Vol. 41 (1)
Information About the Moments or the Likelihood Model Parameters? A Chicken and Egg Problem
This article compares the information content of a sample for two competing Bayesian approaches. One approach follows Dennis Lindley's Bayesian standpoint, where one begins by formulating a prior for a parameter related to the problem in question and incorporates a likelihood to transition to a posterior. This contrasts with the usual Bayesian approach, where one starts with a likelihood model, formulates a prior distribution for its parameters, and derives the corresponding posterior. In both cases, the sample information content is measured using the difference between the prior and posterior entropies. We investigate this contrast in the context of learning about the moments of a variable. The maximum entropy principle is used to construct the likelihood model consistent with the given moment parameters. This likelihood model is then combined with the prior information on the parameters to derive the posterior. The model parameters are the Lagrange multipliers for the moment constraints. A prior for the moments induces a prior for the model parameters; however, the data provides differing amounts of information about them. The results obtained for several problems show that the information content using the two formulations can differ significantly. Additional information measures are derived to assess the effects of operating environments on the lifetimes of system components.
- 1 January 2025
- Published by Wiley in Applied Stochastic Models in Business and Industry
- Vol. 41 (1)
Information About the Moments or the Likelihood Model Parameters? A Chicken and Egg Problem
This article compares the information content of a sample for two competing Bayesian approaches. One approach follows Dennis Lindley's Bayesian standpoint, where one begins by formulating a prior for a parameter related to the problem in question and incorporates a likelihood to transition to a posterior. This contrasts with the usual Bayesian approach, where one starts with a likelihood model, formulates a prior distribution for its parameters, and derives the corresponding posterior. In both cases, the sample information content is measured using the difference between the prior and posterior entropies. We investigate this contrast in the context of learning about the moments of a variable. The maximum entropy principle is used to construct the likelihood model consistent with the given moment parameters. This likelihood model is then combined with the prior information on the parameters to derive the posterior. The model parameters are the Lagrange multipliers for the moment constraints. A prior for the moments induces a prior for the model parameters; however, the data provides differing amounts of information about them. The results obtained for several problems show that the information content using the two formulations can differ significantly. Additional information measures are derived to assess the effects of operating environments on the lifetimes of system components.
- Open Access
Information Architecture for Data Disclosure
Preserving confidentiality of individuals in data disclosure is a prime concern for public and private organizations. The main challenge in the data disclosure problem is to release data such that misuse by intruders is avoided while providing useful information to legitimate users for analysis. We propose an information theoretic architecture for the data disclosure problem. The proposed framework consists of developing a maximum entropy (ME) model based on statistical information of the actual data, testing the adequacy of the ME model, producing disclosure data from the ME model and quantifying the discrepancy between the actual and the disclosure data. The architecture can be used both for univariate and multivariate data disclosure. We illustrate the implementation of our approach using financial data.
Open AccessInformation Architecture for Data Disclosure
Preserving confidentiality of individuals in data disclosure is a prime concern for public and private organizations. The main challenge in the data disclosure problem is to release data such that misuse by intruders is avoided while providing useful information to legitimate users for analysis. We propose an information theoretic architecture for the data disclosure problem. The proposed framework consists of developing a maximum entropy (ME) model based on statistical information of the actual data, testing the adequacy of the ME model, producing disclosure data from the ME model and quantifying the discrepancy between the actual and the disclosure data. The architecture can be used both for univariate and multivariate data disclosure. We illustrate the implementation of our approach using financial data.
- 20 April 2022
- Published by Wiley in International Statistical Review
- Vol. 90 (3) , 499-524
Elaboration Models with Symmetric Information Divergence
Summary: Various statistical methodologies embed a probability distribution in a more flexible family of distributions. The latter is calledelaboration model, which is constructed by choice or a formal procedure and evaluated by asymmetric measures such as the likelihood ratio and Kullback–Leibler information. The use of asymmetric measures can be problematic for this purpose. This paper introduces two formal procedures, referred to as link functions, that embed any baseline distribution with a continuous density on the real line into model elaborations. Conditions are given for the link functions to render symmetric Kullback–Leibler divergence, Rényi divergence and phi‐divergence family. The first link function elaborates quantiles of the baseline probability distribution. This approach produces continuous counterparts of the binary probability models. Examples include the Cauchy, probit, logit, Laplace and Student's links. The second link function elaborates the baseline survival function. Examples include the proportional odds and change point links. The logistic distribution is characterised as the one that satisfies the conditions for both links. An application demonstrates advantages of symmetric divergence measures for assessing the efficacy of covariates.
- 20 April 2022
- Published by Wiley in International Statistical Review
- Vol. 90 (3) , 499-524
Elaboration Models with Symmetric Information Divergence
Summary: Various statistical methodologies embed a probability distribution in a more flexible family of distributions. The latter is calledelaboration model, which is constructed by choice or a formal procedure and evaluated by asymmetric measures such as the likelihood ratio and Kullback–Leibler information. The use of asymmetric measures can be problematic for this purpose. This paper introduces two formal procedures, referred to as link functions, that embed any baseline distribution with a continuous density on the real line into model elaborations. Conditions are given for the link functions to render symmetric Kullback–Leibler divergence, Rényi divergence and phi‐divergence family. The first link function elaborates quantiles of the baseline probability distribution. This approach produces continuous counterparts of the binary probability models. Examples include the Cauchy, probit, logit, Laplace and Student's links. The second link function elaborates the baseline survival function. Examples include the proportional odds and change point links. The logistic distribution is characterised as the one that satisfies the conditions for both links. An application demonstrates advantages of symmetric divergence measures for assessing the efficacy of covariates.
- Open Access
- 1 June 2021
- Published by CMV Verlag in Journal of the Iranian Statistical Society
- Vol. 20 (1) , 27-59
Variants of Mixtures: Information Properties and Applications
In recent years, we have studied information properties of various types of mixtures of probability distributions and introduced a new type, which includes previously known mixtures as special cases. These studies are disseminated in different fields: reliability engineering, econometrics, operations research, probability, the information theory, and data mining. This paper presents ...
- 1 June 2021
- Published by CMV Verlag in Journal of the Iranian Statistical Society
- Vol. 20 (1) , 27-59
Open AccessVariants of Mixtures: Information Properties and Applications
In recent years, we have studied information properties of various types of mixtures of probability distributions and introduced a new type, which includes previously known mixtures as special cases. These studies are disseminated in different fields: reliability engineering, econometrics, operations research, probability, the information theory, and data mining. This paper presents ...
- 9 June 2020
- Published by Wiley in Statistical Analysis and Data Mining: An ASA Data Science Journal
- Vol. 13 (4) , 405-418
MR plot: A big data tool for distinguishing distributions
Big data enables reliable estimation of continuous probability density, cumulative distribution, survival, hazard rate, and mean residual functions (MRFs). We illustrate that plot of the MRF provides the best resolution for distinguishing between distributions. At each point, the MRF gives the mean excess of the data beyond the threshold. Graph of the empirical MRF, called here the MR plot, provides an effective visualization tool. A variety of theoretical and data driven examples illustrate that MR plots of big data preserve the shape of the MRF and complex models require bigger data. The MRF is an optimal predictor of the excess of the random variable. With a suitable prior, the expected MRF gives the Bayes risk in the form of the entropy functional of the survival function, called here the survival entropy. We show that the survival entropy is dominated by the standard deviation (SD) and the equality between the two measures characterizes the exponential distribution. The empirical survival entropy provides a data concentration statistic which is strongly consistent, easy to compute, and less sensitive than the SD to heavy tailed data. An application uses the New York City Taxi database with millions of trip times to illustrate the MR plot as a powerful tool for distinguishing distributions.
- 9 June 2020
- Published by Wiley in Statistical Analysis and Data Mining: An ASA Data Science Journal
- Vol. 13 (4) , 405-418
MR plot: A big data tool for distinguishing distributions
Big data enables reliable estimation of continuous probability density, cumulative distribution, survival, hazard rate, and mean residual functions (MRFs). We illustrate that plot of the MRF provides the best resolution for distinguishing between distributions. At each point, the MRF gives the mean excess of the data beyond the threshold. Graph of the empirical MRF, called here the MR plot, provides an effective visualization tool. A variety of theoretical and data driven examples illustrate that MR plots of big data preserve the shape of the MRF and complex models require bigger data. The MRF is an optimal predictor of the excess of the random variable. With a suitable prior, the expected MRF gives the Bayes risk in the form of the entropy functional of the survival function, called here the survival entropy. We show that the survival entropy is dominated by the standard deviation (SD) and the equality between the two measures characterizes the exponential distribution. The empirical survival entropy provides a data concentration statistic which is strongly consistent, easy to compute, and less sensitive than the SD to heavy tailed data. An application uses the New York City Taxi database with millions of trip times to illustrate the MR plot as a powerful tool for distinguishing distributions.
- 1 September 2019
- Published by Cambridge University Press (CUP) in Journal of Applied Probability
- Vol. 56 (4) , 1151-1167
The alpha-mixture of survival functions
This paper presents a flexible family which we call the $\alpha$ -mixture of survival functions. This family includes the survival mixture, failure rate mixture, models that are stochastically closer to each of these conventional mixtures, and many other models. The $\alpha$ -mixture is endowed by the stochastic order and uniquely possesses a mathematical property known in economics as the constant elasticity of substitution, which provides an interpretation for $\alpha$ . We study failure rate properties of this family and establish closures under monotone failure rates of the mixture’s components. Examples include potential applications for comparing systems.- 1 September 2019
- Published by Cambridge University Press (CUP) in Journal of Applied Probability
- Vol. 56 (4) , 1151-1167
The alpha-mixture of survival functions
This paper presents a flexible family which we call the $\alpha$ -mixture of survival functions. This family includes the survival mixture, failure rate mixture, models that are stochastically closer to each of these conventional mixtures, and many other models. The $\alpha$ -mixture is endowed by the stochastic order and uniquely possesses a mathematical property known in economics as the constant elasticity of substitution, which provides an interpretation for $\alpha$ . We study failure rate properties of this family and establish closures under monotone failure rates of the mixture’s components. Examples include potential applications for comparing systems.- 23 October 2018
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Transactions on Information Theory
- Vol. 65 (4) , 2316-2321
Mixture Models, Bayes Fisher Information, and Divergence Measures
This paper presents the Bayes Fisher information measures, defined by the expected Fisher information under a distribution for the parameter, for the arithmetic, geometric, and generalized mixtures of two probability density functions. The Fisher information of the arithmetic mixture about the mixing parameter is related to chi-square divergence, Shannon entropy, and the Jensen-Shannon divergence. The Bayes Fisher measures of the three mixture models are related to the Kullback-Leibler, Jeffreys, Jensen-Shannon, Rényi, and Tsallis divergences. These measures indicate that the farther away are the components from each other, the more informative are data about the mixing parameter. We also unify three different relative entropy derivations of the geometric mixture scattered in statistics and physics literatures. Extensions of two of the formulations to the minimization of Tsallis divergence give the generalized mixture as the solution.
- 23 October 2018
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Transactions on Information Theory
- Vol. 65 (4) , 2316-2321
Mixture Models, Bayes Fisher Information, and Divergence Measures
This paper presents the Bayes Fisher information measures, defined by the expected Fisher information under a distribution for the parameter, for the arithmetic, geometric, and generalized mixtures of two probability density functions. The Fisher information of the arithmetic mixture about the mixing parameter is related to chi-square divergence, Shannon entropy, and the Jensen-Shannon divergence. The Bayes Fisher measures of the three mixture models are related to the Kullback-Leibler, Jeffreys, Jensen-Shannon, Rényi, and Tsallis divergences. These measures indicate that the farther away are the components from each other, the more informative are data about the mixing parameter. We also unify three different relative entropy derivations of the geometric mixture scattered in statistics and physics literatures. Extensions of two of the formulations to the minimization of Tsallis divergence give the generalized mixture as the solution.
- 1 October 2018
- Published by Elsevier in European Journal of Operational Research
- Vol. 270 (2) , 723-733
Optimal hazard models based on partial information
- 1 October 2018
- Published by Elsevier in European Journal of Operational Research
- Vol. 270 (2) , 723-733
Optimal hazard models based on partial information
- 25 April 2018
- Published by Wiley in International Statistical Review
- Vol. 86 (3) , 442-468
Ranking Forecasts by Stochastic Error Distance, Information and Reliability Measures
Summary: The stochastic error distance (SED) introduced by Diebold and Shin () ranks forecast models by divergence between distributions of the errors of the actual and perfect forecast models. The basic SED is defined by the variation distance and provides a representation of the mean absolute error, but by basing ranking on the entire error distribution and divergence, the SED moves beyond the traditional forecast evaluations. First, we establish connections between ranking forecast models by the SED, error entropy and some partial orderings of distributions. Then, we introduce the notion of excess error for forecast errors of magnitudes larger than a tolerance threshold and give the SED representation of the mean excess error (MEE). As a function of the threshold, the MEE is a local risk measure. With the distribution of the absolute error as a prior for the threshold, its Bayes risk is the entropy functional of the survival function, which is a known measure in the information theory and reliability. Notions and results are illustrated using various distributions for the error. The empirical versions of SED, MEE and its Bayes risk are compared with the mean squared error in ranking regression and autoregressive integrated moving average models for forecasting bond risk premia.
- 25 April 2018
- Published by Wiley in International Statistical Review
- Vol. 86 (3) , 442-468
Ranking Forecasts by Stochastic Error Distance, Information and Reliability Measures
Summary: The stochastic error distance (SED) introduced by Diebold and Shin () ranks forecast models by divergence between distributions of the errors of the actual and perfect forecast models. The basic SED is defined by the variation distance and provides a representation of the mean absolute error, but by basing ranking on the entire error distribution and divergence, the SED moves beyond the traditional forecast evaluations. First, we establish connections between ranking forecast models by the SED, error entropy and some partial orderings of distributions. Then, we introduce the notion of excess error for forecast errors of magnitudes larger than a tolerance threshold and give the SED representation of the mean excess error (MEE). As a function of the threshold, the MEE is a local risk measure. With the distribution of the absolute error as a prior for the threshold, its Bayes risk is the entropy functional of the survival function, which is a known measure in the information theory and reliability. Notions and results are illustrated using various distributions for the error. The empirical versions of SED, MEE and its Bayes risk are compared with the mean squared error in ranking regression and autoregressive integrated moving average models for forecasting bond risk premia.
- 30 November 2017
- Published by Cambridge University Press (CUP) in Journal of Applied Probability
- Vol. 54 (4) , 1027-1050
Connections of Gini, Fisher, and Shannon by Bayes risk under proportional hazards
The proportional hazards (PH) model and its associated distributions provide suitable media for exploring connections between the Gini coefficient, Fisher information, and Shannon entropy. The connecting threads are Bayes risks of the mean excess of a random variable with the PH distribution and Bayes risks of the Fisher information of the equilibrium distribution of the PH model. Under various priors, these Bayes risks are generalized entropy functionals of the survival functions of the baseline and PH models and the expected asymptotic age of the renewal process with the PH renewal time distribution. Bounds for a Bayes risk of the mean excess and the Gini's coefficient are given. The Shannon entropy integral of the equilibrium distribution of the PH model is represented in derivative forms. Several examples illustrate implementation of the results and provide insights for potential applications.
- 30 November 2017
- Published by Cambridge University Press (CUP) in Journal of Applied Probability
- Vol. 54 (4) , 1027-1050
Connections of Gini, Fisher, and Shannon by Bayes risk under proportional hazards
The proportional hazards (PH) model and its associated distributions provide suitable media for exploring connections between the Gini coefficient, Fisher information, and Shannon entropy. The connecting threads are Bayes risks of the mean excess of a random variable with the PH distribution and Bayes risks of the Fisher information of the equilibrium distribution of the PH model. Under various priors, these Bayes risks are generalized entropy functionals of the survival functions of the baseline and PH models and the expected asymptotic age of the renewal process with the PH renewal time distribution. Bounds for a Bayes risk of the mean excess and the Gini's coefficient are given. The Shannon entropy integral of the equilibrium distribution of the PH model is represented in derivative forms. Several examples illustrate implementation of the results and provide insights for potential applications.