Kai Xin

The future of artificial intelligence specialist doctors depends on foundational large language models (LLMs). However, there is currently a lack of evaluation studies on their Q&A capabilities in female malignancy. This cross-sectional study created a benchmark dataset consisting of 205 female malignancy Q&A to evaluate and compare the performance of 7 popular LLMs, with 3 human oncologists serving as a comparison. Only OpenAI o1, DeepSeek-R1, and Llama-3.1-405B achieved both accuracy and consistency rates exceeding 80%. Among these, Llama-3.1-405B demonstrated significantly higher accuracy than OpenAI o1 ( P = .002) and potentially higher than DeepSeek-R1 ( P = .059), with significantly lower latency ( P < .001). After removing token limits, both OpenAI o1 and DeepSeek-R1 exhibited strong reasoning abilities, achieving accuracy rates of 98.5% and 95.6%, respectively, with no significant difference from Llama-3.1-405B ( P > .05). The agreement between the models was fair, with an agreement rate of 0.318. Error analysis revealed that the highest error rates were associated with questions involving flexible medical scenarios and complex medical terminology. In female malignancy Q&A tasks, Llama-3.1-405B serves as a strong baseline model, while OpenAI o1 and DeepSeek-R1 show promise in inference, especially when token limitations are removed. However, their reasoning processes lack full consistency, likely due to differences in architectures and training methods. LLMs continue to face challenges in real-world medical scenarios and with complex medical terminology, highlighting the need for ongoing updates and the integration of clinical data to refine these models and improve their ability to address specialized medical needs.

Introduction: Malignant tumors represent a significant public health threat, and the integration of artificial intelligence in health care is increasingly becoming a priority. Many oncology institutions are already considering the use of DeepSeek-R1 to assist doctors in making complex medical decisions. However, there remains a lack of sufficient evidence regarding the accuracy, consistency, and cost-efficiency of DeepSeek-R1 and its distilled models in oncology decision-making. This study aims to fill this gap by evaluating the performance and cost-effectiveness of DeepSeek-R1 and its distilled models in oncology, providing critical insights into their potential for clinical integration. Objectives: This study aimed to systematically evaluate the performance, consistency, and cost-efficiency of the open-source large language model (LLM) DeepSeek-R1 and its distilled variants in the context of oncology decision-making, using a benchmark derived from the MedQA dataset. Methods: A custom oncology question set containing 1,206 multiple choice questions was curated from MedQA. Seven models, including DeepSeek-R1 and six distilled versions, were evaluated using an automated testing framework. Accuracy, consistency, latency, and token consumption were compared across models. Statistical tests, including McNemar and Wilcoxon signed-rank, were used to assess differences in performance. Questions were also categorized into clinical task types (diagnosis, treatment, triage, and follow-up) for subgroup analysis. Results: DeepSeek-R1 achieved the highest performance (accuracy: 91.38%; consistency: 90.47%), whereas DeepSeek-R1-Distill-Qwen-32B was the only distilled model to exceed both metrics at the 0.8 threshold (accuracy: 88.72%; consistency: 81.44%). DeepSeek-R1 demonstrated significantly higher accuracy than its distilled counterpart (p<0.05), particularly in diagnosis- and treatment-related tasks (p<0.05). However, it also exhibited significantly greater latency and token consumption. A Cohen’s kappa value of 0.575 indicated moderate agreement between the two models. Conclusion: DeepSeek-R1 is more suitable for high-stakes oncology tasks requiring high accuracy and consistency, whereas DeepSeek-R1-Distill-Qwen-32B offers a cost-effective alternative for use in outpatient or resource-limited settings. These findings support a task- and resource-adaptive deployment strategy for LLMs in clinical oncology.

Background: To evaluate the clinical efficacy and safety of immune checkpoint inhibitors in patients with advanced gastric cancer in the real world. Methods: The retrospective analysis was conducted on the clinical records of 402 patients with advanced gastric cancer who were admitted to the Nanjing Drum Tower Hospital between December 2017 and April 2022 and who had received immunotherapy. Observation target: drug use, treatment, adverse reaction type and grade, objective response rate (ORR), disease control rate (DCR), progression free survival (PFS), and overall survival (OS). Results: By retrospectively analyzing the data of patients with advanced gastric cancer treated with ICIs previously admitted to our medical center, we found some clinical characteristic factors associated with the occurrence of irAEs as well as the efficacy and prognosis: the presence or absence of hypertension, whether or not to receive targeted therapies can predict the occurrence of immune-related adverse events (irAEs), and the more the presence of irAEs, the better the prognosis. These can help clinicians in clinical drug selection. Conclusions: The results of this paper show that the occurrence of irAEs is associated with patients’ OS. irAEs occurrence can prolong patients’ OS. irAEs occurrence may serve as a surrogate marker for ICIs.

To establish a satisfactory delivery system for the delivery of salinomycin (Sal), a novel, selective cancer stem cell inhibitor with prominent toxicity, gelatinase-responsive core-shell nanoparticles (NPs), were prepared by nanoprecipitation method (NR-NPs) and single emulsion method (SE-NPs). The gelatinase-responsive copolymer was prepared by carboxylation and double amination method. We studied the stability of NPs prepared by nanoprecipitation method with different proportions of F68 in aqueous phase to determine the best proportion used in our study. Then, the NPs were prepared by nanoprecipitation method with the best proportion of F68 and single emulsion method, and their physiochemical traits including morphology, particle size, zeta potential, drug loading content, stability, and in vitro release profiles were studied. The SE-NPs showed significant differences in particle size, drug loading content, stability, and in vitro release profiles compared to NR-NPs. The SE-NPs presented higher drug entrapment efficiency and superior stability than the NR-NPs. The drug release rate of SE-NPs was more sustainable than that of the NR-NPs, and in vivo experiment indicated that NPs could prominently reduce the toxicity of Sal. Our study demonstrates that the SE-NPs could be a satisfactory method for the preparation of gelatinase-responsive NPs for intelligent delivery of Sal.