Auditory perception studies using the probe‐signal method [Greenberg and Larkin (1968)] have demonstrated greater detection sensitivity for tones occurring at “expected” frequencies relative to tones occurring at “unexpected” frequencies. An important, but largely overlooked, methodological issue in studies using the probe‐signal and related methods is that measured sensitivity (d́) can underestimate true sensitivity for conditions with relatively few observations, implying that enhanced sensitivity observed for “expected” conditions with more observations compared to “unexpected” conditions with fewer observations can potentially be due to a measurement artifact. To address this possibility, the current research combines results of two behavioral studies with a set of simulations and finds that heightened sensitivity to an expected signal relative to an unexpected signal can be observed when there are no differences in true sensitivity across conditions. Specifically, enhanced detection sensitivity for more frequent signals attributable to a measurement artifact is more likely when true detection sensitivity is high, the number of stimulus presentations is relatively low, and a correction for response proportions of 0 and 1 is required to calculate d’. Implications are discussed for psychoacoustics research using the probe‐signal method and more broadly for psychological studies examining effects of listener expectations on perception.