BACKGROUND Despite a growing awareness among healthcare professionals regarding the benefits of exercise and physical activity, the integration of exercise recommendations into healthcare practices remains sub-optimal. The complexity of the recommendations, which necessitate interdisciplinary knowledge, poses challenges in formulating the structured exercise plans. The rapid development of Large Language Models (LLMs) and Chatbots, offers a potential solution by leveraging AI's capabilities to enhance the precision and personalization of fitness and health recommendations. OBJECTIVE This paper provides a systematic summary of existing research in the landscape of LLMs and Chatbot applications for exercise recommendations and physical activity, encompassing important topics, theoretical frameworks, methodologies, and potential avenues for further exploration. METHODS A comprehensive literature search was conducted across databases, including Web of Science, PubMed, IEEE, and arXiv. The focus was on studies that utilized LLMs or Chatbots in recommendations for exercise, physical activity or fitness. The search was limited to English-language articles, and the selection process involved two reviews and arbitration in case of disagreement among reviewers. RESULTS The review identified 598 articles initially, narrowing down to 11 reports after applying inclusion and exclusion criteria. The current research domain is remarkably active, playing a pivotal role in the field of exercise health, and studies predominantly focus on integrating LLMs and Chatbots with Just-In-Time Adaptive Interventions (JITAIs) to design personalized exercise programs and evaluate the quality of AI-generated exercise recommendations. The primary LLMs utilized in this field are predominantly from the ChatGPT series developed by OpenAI. CONCLUSIONS Notably, the review underscores the need for further advancements to match the expertise of exercise professionals and address individualized, evidence-based physical activity recommendations. While LLMs and Chatbots demonstrate promising utility in exercise recommendations and physical activity recommendations, there is a critical need for further development to enhance their accuracy, adaptability, and alignment with professional healthcare standards. The future direction emphasizes the fine-tuning of models, enhanced integration of wearable technology with AI and dynamic feedback loops in optimizing personalized health interventions.