ESPaDOnS is a high-resolution, cross-dispersed, fiber fed, echelle spectrograph in use at the Canada-France-Hawaii Telescope (CFHT). The light from the telescope passes through a polarimeter stage before being injected into the fibers that feed the spectrograph, so the instrument is capable of determining the polarization of the stellar spectra from 370 - 1000 nm in wavelength. One limit to the accuracy of the polarimetry is the inevitable polarization crosstalk added by all optics prior to polarization analysis. The main source of this crosstalk is stress birefringence in the glass of the optics; either residual from the annealing process or induced by the mounting of the optics. The process by which the crosstalk in ESPaDOnS has been reduced from 5% or more to less than 1% is discussed in this paper.