As market demand for fast-charging grows to alleviate range anxiety, prognosing battery degradation under high-rate charging protocols becomes crucial for reliable operation. However, diverse fast-charging protocols induce varying internal polarization, causing data distribution shifts that fail standard prognosis models. While domain adaptation (DA) mitigates this issue, transmitting massive standard datasets incurs high communication and storage costs. To overcome these limitations, this work formulates degradation prognosis as a source-free DA problem and propose a physics-guided semantic source-free adaptation (PS2FA) algorithm. PS2FA adapts the network using only a pretrained source model (i.e, the standard prognosis model) and unlabeled target data (i.e., fast-charging data), thus eliminating the need for the transmission of source data. Specifically, a rate-conditioned autoencoder projects target features into the source latent space to bridge distribution gaps. Furthermore, a physical and statistical alignment mechanism further confines predictions within valid physical boundaries. Validated on MIT-Stanford and 15-minute fast charging datasets, PS2FA reduces training data number by 80% and decreases prediction RMSE% by up to 52% under complex protocols, demonstrating highly efficient and reliable battery health management.