Low temperature penetration depth experiments in highly underdoped (UD) cuprates have been interpreted in terms of proximity to a superconductor-insulator quantum critical point (QCP). We discuss two ways by which such a proximate QCP could arise in the UD cuprates. One possibility is that the UD regime is near a superconductor (SC) to Mott-insulator transition at half-filling. We obtain such a transition using slave rotor mean field theory. The second scenario is that the UD cuprates are granular, and described by a Josephson junction array model which undergoes a SC-insulator transition. We show that the critical theory of bosons at such a QCP naturally leads to Fermi arcs in the normal state whose length scales linearly with temperature as seen in photoemission experiments.