Measurement invariance is an important topic in the psychometric literature of group comparison in confirmatory factor analysis. It is frequently stated in the literature that measurement invariance is a prerequisite for the comparison of group means or standard deviations of the latent variable in a factor model. This article argues that measurement invariance is not necessary for meaningful and valid comparisons across groups. Moreover, we argue that there is unavoidable arbitrariness in how researchers can define group comparisons in the violation of invariance. In particular, there is no support for the belief that the partial invariance approach should be preferred over competing approaches such as invariance alignment, robust linking, or Bayesian approximate invariance. Furthermore, we also argue why an intentionally misspecified multiple-group factor model with diagonally weighted least squares estimation with invariant item parameters can be justified even if the measurement invariance assumption is violated.