Feshbach Insulator from Atom-Molecule Coherence of Bosons in Optical Lattices

Feshbach resonances - namely resonances between an unbound two-body state (atomic state) and a bound (molecular) state, differing in magnetic moment - are a unique tool to tune the interaction properties of ultracold atoms. Here we show that the spin-changing interactions, coherently coupling the atomic and molecular state, can lead to a novel insulating phase - the $\textit{Feshbach insulator}$ - for bosons in an optical lattice close to a $\textit{narrow}$ Feshbach resonance. Making use of quantum Monte Carlo simulations and mean-field theory, we show that the Feshbach insulator appears around the resonance, preventing the system from collapsing when the effective atomic scattering length becomes negative. On the atomic side of the resonance, the transition from condensate to Feshbach insulator has a characteristic $\textit{first-order}$ nature, due to the simultaneous loss of coherence in the atomic and molecular components. These features appear clearly in the ground-state phase diagram of $\textit{e.g.}$ $^{87}$Rb around the 414 G resonance, and they are therefore directly amenable to experimental observation.