Accommodating the variation present in second language (L2) accented speech can pose a challenge for listeners. Extensive research has revealed that exposure to acoustically-atypical phonemes presented in a lexical context can prompt rapid recalibration of phonemic category representations. Accumulating evidence suggests that, in addition to these boundary shifts, more nuanced changes in category “flexibility” may also occur. We investigate these two phenomena and their effects on identification of sound categories and recognition of lexical items. First language (L1) Spanish listeners were exposed to an L1 French speaker producing accented L2 Spanish. We randomly assigned participants to one of three conditions, manipulating the primary acoustic cue that supports recognition of the /a/ vowel: (1) a control group (i.e., the on-target primary cue to /a/ as “central”), (2) a “fronted” group (more /e/-like than normal), and (3) a “backed” group (more /o/-like). Using pupillometry, a measure of cognitive processing load, a benefit of prior exposure to an atypical /a/ vowel was detected for recognition of words containing the same vowel manipulation. Critically, this benefit was non-directional: Listeners exposed to a backed /a/ during training, for example, showed reductions in cognitive load for recognition of both backed and fronted /a/ exemplars. In combination with behavioral responses from a vowel identification task, the results suggest that two separate mechanisms may support adaptation to accent in parallel: (1) a recalibration mechanism that adjusts phonemic category representations (i.e., boundary shifts), and (2) a criterion relaxation mechanism that adjusts how atypical phonemes facilitate lexical activation.