Kai Ishida
Current Organizations / Institutes
The organizations/institutes from the scholar's latest article
Previous Organizations / Institutes
The organizations / institutes from the scholar's other articles in the past ten years
Kai Ishida has a total of 43 co-authors. Here are the top co-authors by number of shared publications:
Minoru Hirose
17 shared publicationsNoriaki Ikeda
Graduate School of Medical Science, Kitasato University, Sagamihara, Kanagawa, Japan Department of Medical Informatics, School of Allied Health Sciences, Kitasato University, Sagamihara, Kanagawa, Japan
10 shared publications- Eisuke Hanada
Faculty of Science and Engineering, Saga University, Saga 8408502, Japan
8 shared publications Sazu Arie
Applied Electromagnetic Research Institute, National Institute of Information and Communications Technology (NICT), Tokyo, 184-8795, Japan
8 shared publicationsAkihiro Takeuchi
7 shared publicationsHarukazu Tsuruta
Graduate School of Medical Science, Kitasato University, Sagamihara, Kanagawa, Japan Department of Medical Informatics, School of Allied Health Sciences, Kitasato University, Sagamihara, Kanagawa, Japan
7 shared publications
Latest Publications
- 10 October 2026
- Published by Springer Nature
- p. 255-264
Fundamental Study of a Monitoring System for Clinical Settings Using LoRaWAN
- 10 October 2026
- Published by Springer Nature
- p. 255-264
Fundamental Study of a Monitoring System for Clinical Settings Using LoRaWAN
- 2 August 2026
- Published by Springer Nature
- p. 782-788
Recognition of Auditory Alarms of Medical Devices Using Machine Learning
- 2 August 2026
- Published by Springer Nature
- p. 782-788
Recognition of Auditory Alarms of Medical Devices Using Machine Learning
- Open Access
- 21 May 2026
- Published by IOS Press
- Vol. 336, 1070-1071
Evaluation of the Construction and Accuracy of a Local LLM Based on Clinical Engineering
This study aimed to develop a local large language model (LLM) specialized in clinical engineering and to evaluate its performance on the Japanese National Clinical Engineer Licensing Examination. Three local LLMs (Cogito-32b-think, GPT-4o-mini, and Qwen3-14B) and Gemini-2.5 Pro were tested. Among the local models, Qwen3-14B achieved the highest accuracy and was further improved through supervised fine-tuning (SFT) using reasoning data generated by Gemini-2.5 Pro. After SFT, the overall accuracy of Qwen3-14B increased from 77% to 82%, with improvements in image (39% to 52%) and calculation (58% to 68%) questions. Although Gemini-2.5 Pro outperformed Qwen3-14B with 94% accuracy, the results demonstrate that SFT effectively enhances local LLMs, offering a promising, secure, and cost-efficient approach for specialized clinical engineering applications.
- 21 May 2026
- Published by IOS Press
- Vol. 336, 1070-1071
Open AccessEvaluation of the Construction and Accuracy of a Local LLM Based on Clinical Engineering
This study aimed to develop a local large language model (LLM) specialized in clinical engineering and to evaluate its performance on the Japanese National Clinical Engineer Licensing Examination. Three local LLMs (Cogito-32b-think, GPT-4o-mini, and Qwen3-14B) and Gemini-2.5 Pro were tested. Among the local models, Qwen3-14B achieved the highest accuracy and was further improved through supervised fine-tuning (SFT) using reasoning data generated by Gemini-2.5 Pro. After SFT, the overall accuracy of Qwen3-14B increased from 77% to 82%, with improvements in image (39% to 52%) and calculation (58% to 68%) questions. Although Gemini-2.5 Pro outperformed Qwen3-14B with 94% accuracy, the results demonstrate that SFT effectively enhances local LLMs, offering a promising, secure, and cost-efficient approach for specialized clinical engineering applications.
- Open Access
- 10 May 2026
- Published by JMIR Publications Inc. in JMIR Preprints
Evaluation of Local Vision Language Models on the Japanese National Clinical Engineer Licensing Examination Using Image-Only Inputs: A Benchmarking Study (Preprint)
UNSTRUCTURED This study systematically evaluated the performance of locally deployable vision language models (VLMs) on the Japanese National Clinical Engineer Licensing Examination (JNCELE) in a fully image-based input setting. Unlike prior studies that primarily relied on text inputs, all questions including problem statements and answer choices were provided exclusively as images to facilitate an integrated assessment of the VLMs’ optical character recognition (OCR), visual understanding, and reasoning capabilities. A benchmark dataset was constructed based on the 35th–39th JNCELEs, and multiple VLMs with varying architectures and parameter sizes were evaluated. Among the local VLMs, Qwen3.5-27B achieved the highest overall accuracy of 79% and an accuracy exceeding 90% in several domains. However, smaller models showed substantial degradation in both accuracy and output stability, and they often exhibited OCR errors, hallucinations, and repetitive responses. The performance was particularly degraded for questions requiring domain-specific knowledge such as Japanese Industrial Standards and for questions involving reading charts and diagrams and performing calculations. In contrast, the cloud-based Gemini-3-Pro achieved an overall accuracy of 93%, which highlights the current performance gap between local and cloud-based VLMs. These findings demonstrate that, while local VLMs are approaching practical utility in selected domains, their performance is strongly constrained by OCR accuracy, reasoning stability, and domain-specific knowledge. The developed benchmark dataset provides a standardized framework for evaluating the multimodal reasoning capabilities of VLMs and is expected to support the future development of reliable and locally deployable artificial intelligence systems in the clinical engineering field.
- 10 May 2026
- Published by JMIR Publications Inc. in JMIR Preprints
Open AccessEvaluation of Local Vision Language Models on the Japanese National Clinical Engineer Licensing Examination Using Image-Only Inputs: A Benchmarking Study (Preprint)
UNSTRUCTURED This study systematically evaluated the performance of locally deployable vision language models (VLMs) on the Japanese National Clinical Engineer Licensing Examination (JNCELE) in a fully image-based input setting. Unlike prior studies that primarily relied on text inputs, all questions including problem statements and answer choices were provided exclusively as images to facilitate an integrated assessment of the VLMs’ optical character recognition (OCR), visual understanding, and reasoning capabilities. A benchmark dataset was constructed based on the 35th–39th JNCELEs, and multiple VLMs with varying architectures and parameter sizes were evaluated. Among the local VLMs, Qwen3.5-27B achieved the highest overall accuracy of 79% and an accuracy exceeding 90% in several domains. However, smaller models showed substantial degradation in both accuracy and output stability, and they often exhibited OCR errors, hallucinations, and repetitive responses. The performance was particularly degraded for questions requiring domain-specific knowledge such as Japanese Industrial Standards and for questions involving reading charts and diagrams and performing calculations. In contrast, the cloud-based Gemini-3-Pro achieved an overall accuracy of 93%, which highlights the current performance gap between local and cloud-based VLMs. These findings demonstrate that, while local VLMs are approaching practical utility in selected domains, their performance is strongly constrained by OCR accuracy, reasoning stability, and domain-specific knowledge. The developed benchmark dataset provides a standardized framework for evaluating the multimodal reasoning capabilities of VLMs and is expected to support the future development of reliable and locally deployable artificial intelligence systems in the clinical engineering field.
- Open Access
- 7 August 2025
- Published by IOS Press
- Vol. 329, 1788-1789
Accuracy of ChatGPT Responses to Examinations for Senior Medical Information Technicians in Japan
This study recruited senior medical information technicians in Japan to evaluate the performance of ChatGPT. The average correct response rates for multiple-choice and written questions were 82.1% and 76.3%, respectively. Although ChatGPT-generated plausible sentences, it failed to describe the appropriate role of medical information technicians in short essay exams.
- 7 August 2025
- Published by IOS Press
- Vol. 329, 1788-1789
Open AccessAccuracy of ChatGPT Responses to Examinations for Senior Medical Information Technicians in Japan
This study recruited senior medical information technicians in Japan to evaluate the performance of ChatGPT. The average correct response rates for multiple-choice and written questions were 82.1% and 76.3%, respectively. Although ChatGPT-generated plausible sentences, it failed to describe the appropriate role of medical information technicians in short essay exams.
- Open Access
- 15 May 2025
- Published by IOS Press
- Vol. 327, 900-901
Response Analysis of Health Care Information Technologists in Japan Using Chat Generative Pretrained Transformer
This study used the default model of ChatGPT on health care information technologist exams in Japan to evaluate the reliability of this model for medical information knowledge in non-English languages. The average correct response rates for all questions in the health care, information technology, and health information systems fields were 86%, 93% and 81%, respectively. ChatGPT lacked knowledge of standards and laws; therefore, the accuracy rates for related questions were low. Thus, the study concludes that careful attention is required when using ChatGPT, because several of the explanations it provides lack correct descriptions.
- 15 May 2025
- Published by IOS Press
- Vol. 327, 900-901
Open AccessResponse Analysis of Health Care Information Technologists in Japan Using Chat Generative Pretrained Transformer
This study used the default model of ChatGPT on health care information technologist exams in Japan to evaluate the reliability of this model for medical information knowledge in non-English languages. The average correct response rates for all questions in the health care, information technology, and health information systems fields were 86%, 93% and 81%, respectively. ChatGPT lacked knowledge of standards and laws; therefore, the accuracy rates for related questions were low. Thus, the study concludes that careful attention is required when using ChatGPT, because several of the explanations it provides lack correct descriptions.
- Open Access
- 18 April 2025
- Published by Springer Nature in Cureus
- Vol. 17 (4)
Evaluation of the Characteristics and Machine Learning-Based Identification Accuracy of the International Electrotechnical Commission (IEC) 60601-1-8 Medical Equipment Auditory Alarms
Introduction Auditory alarms in clinical settings signal sudden changes in a patient’s condition and failures in medical equipment. However, distinguishing between simultaneously sounding alarms, particularly when superimposed with various ambient sounds, remains challenging. This study aimed to develop a machine learning (ML) model for identifying auditory alarms issued by medical equipment. Methods We targeted old and new auditory alarms for medical equipment as specified in the International Electrotechnical Commission (IEC) 60601-1-8 standard. First, we evaluated the characteristics of both normal and degraded auditory alarms using cosine similarity among old and new alarms. Next, we evaluated the accuracy of ML-based identification of deteriorated alarm sound sources in both the old and new alarm groups. Results The cosine similarity among old alarms was over 0.99, while new alarms ranged from 0.886 to 0.985, and exhibited more distinct characteristics. When noise was superimposed, the similarity among old alarms increased further, making differentiation more difficult. In contrast, for most new alarms, cosine similarity values exceeded 0.99 but retained slight acoustic differences even after noise-induced degradation, demonstrating improved distinguishability. The accuracy for identifying a single degraded alarm sound was 71.9% for the support vector machine. The models exhibited a high number of misclassifications when identifying the old alarms. Conversely, the models achieved higher accuracy when classifying new alarms, with recall exceeding 80%, precision above 70%, and F-measure greater than 80% for all new alarms. The identification accuracies for two simultaneous alarms were under 20% and approximately 50% for old and new alarms, respectively. The accuracy declined when estimating two simultaneous new alarms; however, when at least one of the two alarms was correctly classified, the accuracy exceeded 90%. Conclusions This study evaluated the characteristics of old and new auditory alarms issued by medical equipment as specified in IEC 60601-1-8 and constructed ML models for identifying the type of alarms.
- 18 April 2025
- Published by Springer Nature in Cureus
- Vol. 17 (4)
Open AccessEvaluation of the Characteristics and Machine Learning-Based Identification Accuracy of the International Electrotechnical Commission (IEC) 60601-1-8 Medical Equipment Auditory Alarms
Introduction Auditory alarms in clinical settings signal sudden changes in a patient’s condition and failures in medical equipment. However, distinguishing between simultaneously sounding alarms, particularly when superimposed with various ambient sounds, remains challenging. This study aimed to develop a machine learning (ML) model for identifying auditory alarms issued by medical equipment. Methods We targeted old and new auditory alarms for medical equipment as specified in the International Electrotechnical Commission (IEC) 60601-1-8 standard. First, we evaluated the characteristics of both normal and degraded auditory alarms using cosine similarity among old and new alarms. Next, we evaluated the accuracy of ML-based identification of deteriorated alarm sound sources in both the old and new alarm groups. Results The cosine similarity among old alarms was over 0.99, while new alarms ranged from 0.886 to 0.985, and exhibited more distinct characteristics. When noise was superimposed, the similarity among old alarms increased further, making differentiation more difficult. In contrast, for most new alarms, cosine similarity values exceeded 0.99 but retained slight acoustic differences even after noise-induced degradation, demonstrating improved distinguishability. The accuracy for identifying a single degraded alarm sound was 71.9% for the support vector machine. The models exhibited a high number of misclassifications when identifying the old alarms. Conversely, the models achieved higher accuracy when classifying new alarms, with recall exceeding 80%, precision above 70%, and F-measure greater than 80% for all new alarms. The identification accuracies for two simultaneous alarms were under 20% and approximately 50% for old and new alarms, respectively. The accuracy declined when estimating two simultaneous new alarms; however, when at least one of the two alarms was correctly classified, the accuracy exceeded 90%. Conclusions This study evaluated the characteristics of old and new auditory alarms issued by medical equipment as specified in IEC 60601-1-8 and constructed ML models for identifying the type of alarms.
- Open Access
- 23 March 2025
- Published by Springer Nature in Cureus
- Vol. 17 (3) , e81029
Evaluating Chat Generative Pretrained Transformer (GPT-4o) Problem-Solving Performance in the Japan Certificate Examination for Biomedical Engineering Class 1
Introduction Chat generative pretrained transformer (ChatGPT; OpenAI, San Francisco, CA) has developed rapidly and is used in various fields, including medical engineering. Japan’s Certificate Examination for Biomedical Engineering class 1 (CEBM1) is responsible for the assessment of comprehensive specialized knowledge and skills centered on the maintenance and safety management of medical devices, systems, and related equipment. This study evaluated the performance of ChatGPT (GPT-4o) on CEBM1 for comparison to human-level expectations. Methods We targeted 171 questions including testing for knowledge with fundamental, applied, and problem-solving abilities from the 26th to 28th CEBM1s. We inputted the Japanese version of questions to ChatGPT (GPT-4o), and evaluated performance based on question difficulty. No prompt optimizations were used. We compared the responses provided by ChatGPT with the correct answers. Results The number of correct answers was 39 (68.4±10.5%) for questions testing fundamental knowledge, 33 (57.9±5.3%) for questions testing applied knowledge, and 38 (59.6±8.0%) for questions testing problem-solving ability. There was no statistically significant difference among the three groups. The passing criteria of 60% or higher was achieved only for the 28th examination. However, over 80% of the questions answered incorrectly were due to a lack of knowledge or incorrect knowledge. When asked questions about the background causes and specific countermeasures for problems related to medical devices, the questions were misunderstood, and in certain cases, answers were generated as hallucinations. Conclusions Currently, ChatGPT possesses a certain level of knowledge in medical engineering; however, it cannot be considered universally accurate in solving all possible problems.
- 23 March 2025
- Published by Springer Nature in Cureus
- Vol. 17 (3) , e81029
Open AccessEvaluating Chat Generative Pretrained Transformer (GPT-4o) Problem-Solving Performance in the Japan Certificate Examination for Biomedical Engineering Class 1
Introduction Chat generative pretrained transformer (ChatGPT; OpenAI, San Francisco, CA) has developed rapidly and is used in various fields, including medical engineering. Japan’s Certificate Examination for Biomedical Engineering class 1 (CEBM1) is responsible for the assessment of comprehensive specialized knowledge and skills centered on the maintenance and safety management of medical devices, systems, and related equipment. This study evaluated the performance of ChatGPT (GPT-4o) on CEBM1 for comparison to human-level expectations. Methods We targeted 171 questions including testing for knowledge with fundamental, applied, and problem-solving abilities from the 26th to 28th CEBM1s. We inputted the Japanese version of questions to ChatGPT (GPT-4o), and evaluated performance based on question difficulty. No prompt optimizations were used. We compared the responses provided by ChatGPT with the correct answers. Results The number of correct answers was 39 (68.4±10.5%) for questions testing fundamental knowledge, 33 (57.9±5.3%) for questions testing applied knowledge, and 38 (59.6±8.0%) for questions testing problem-solving ability. There was no statistically significant difference among the three groups. The passing criteria of 60% or higher was achieved only for the 28th examination. However, over 80% of the questions answered incorrectly were due to a lack of knowledge or incorrect knowledge. When asked questions about the background causes and specific countermeasures for problems related to medical devices, the questions were misunderstood, and in certain cases, answers were generated as hallucinations. Conclusions Currently, ChatGPT possesses a certain level of knowledge in medical engineering; however, it cannot be considered universally accurate in solving all possible problems.
- Open Access
- 1 January 2025
- Published by Springer Nature in Cureus
- Vol. 17 (1) , e76775
ChatGPT (GPT-4V) Performance on the Healthcare Information Technologist Examination in Japan
Introduction The Chat Generative Pretrained Transformer (ChatGPT) has developed rapidly and is used in many fields, including healthcare informatics. This study evaluated ChatGPT (GPT-4V)'s performance on the Healthcare Information Technologist (HCIT) certification exam in Japan, which assesses certified professionals who work with electronic health records to improve patient care. Methodology Four hundred seventy-six questions from the HCIT exam were targeted over three years. ChatGPT (GPT-4V) was tested on its ability to answer questions from an HCIT exam to determine if it could perform as well as or better than aspirants taking the exam. Moreover, its performance was evaluated for each academic category, format, presence or absence of images, and calculations. Results The mean correct answer rate for all questions was 84%. ChatGPT achieved the exam passing criteria. The correct answer rate for simple-choice (A-type) questions was higher than that for multiple-choice (X2-type) questions (P < 0.05). The success rate for questions with images was lower than for text-only questions (P < 0.01), and the success rate for questions requiring calculations was lower than for those without calculations (P < 0.05). Conclusions ChatGPT (GPT-4V) met the passing criteria for the 19th to 21st HCIT exams, suggesting that its performance is effective in passing the HCIT exam. ChatGPT may possess the minimum required knowledge, understanding, and application skills for the HCIT certification.
- 1 January 2025
- Published by Springer Nature in Cureus
- Vol. 17 (1) , e76775
Open AccessChatGPT (GPT-4V) Performance on the Healthcare Information Technologist Examination in Japan
Introduction The Chat Generative Pretrained Transformer (ChatGPT) has developed rapidly and is used in many fields, including healthcare informatics. This study evaluated ChatGPT (GPT-4V)'s performance on the Healthcare Information Technologist (HCIT) certification exam in Japan, which assesses certified professionals who work with electronic health records to improve patient care. Methodology Four hundred seventy-six questions from the HCIT exam were targeted over three years. ChatGPT (GPT-4V) was tested on its ability to answer questions from an HCIT exam to determine if it could perform as well as or better than aspirants taking the exam. Moreover, its performance was evaluated for each academic category, format, presence or absence of images, and calculations. Results The mean correct answer rate for all questions was 84%. ChatGPT achieved the exam passing criteria. The correct answer rate for simple-choice (A-type) questions was higher than that for multiple-choice (X2-type) questions (P < 0.05). The success rate for questions with images was lower than for text-only questions (P < 0.01), and the success rate for questions requiring calculations was lower than for those without calculations (P < 0.05). Conclusions ChatGPT (GPT-4V) met the passing criteria for the 19th to 21st HCIT exams, suggesting that its performance is effective in passing the HCIT exam. ChatGPT may possess the minimum required knowledge, understanding, and application skills for the HCIT certification.
- 11 September 2024
- Published by Springer Nature in Journal of Medical Systems
- Vol. 48 (1) , 1-9
Analysis of Responses of GPT-4 V to the Japanese National Clinical Engineer Licensing Examination
Chat Generative Pretrained Transformer (ChatGPT; OpenAI) is a state-of-the-art large language model that can simulate human-like conversations based on user input. We evaluated the performance of GPT-4 V in the Japanese National Clinical Engineer Licensing Examination using 2,155 questions from 2012 to 2023. The average correct answer rate for all questions was 86.0%. In particular, clinical medicine, basic medicine, medical materials, biological properties, and mechanical engineering achieved a correct response rate of ≥ 90%. Conversely, medical device safety management, electrical and electronic engineering, and extracorporeal circulation obtained low correct answer rates ranging from 64.8% to 76.5%. The correct answer rates for questions that included figures/tables, required numerical calculation, figure/table ∩ calculation, and knowledge of Japanese Industrial Standards were 55.2%, 85.8%, 64.2% and 31.0%, respectively. The reason for the low correct answer rates is that ChatGPT lacked recognition of the images and knowledge of standards and laws. This study concludes that careful attention is required when using ChatGPT because several of its explanations lack the correct description.
- 11 September 2024
- Published by Springer Nature in Journal of Medical Systems
- Vol. 48 (1) , 1-9
Analysis of Responses of GPT-4 V to the Japanese National Clinical Engineer Licensing Examination
Chat Generative Pretrained Transformer (ChatGPT; OpenAI) is a state-of-the-art large language model that can simulate human-like conversations based on user input. We evaluated the performance of GPT-4 V in the Japanese National Clinical Engineer Licensing Examination using 2,155 questions from 2012 to 2023. The average correct answer rate for all questions was 86.0%. In particular, clinical medicine, basic medicine, medical materials, biological properties, and mechanical engineering achieved a correct response rate of ≥ 90%. Conversely, medical device safety management, electrical and electronic engineering, and extracorporeal circulation obtained low correct answer rates ranging from 64.8% to 76.5%. The correct answer rates for questions that included figures/tables, required numerical calculation, figure/table ∩ calculation, and knowledge of Japanese Industrial Standards were 55.2%, 85.8%, 64.2% and 31.0%, respectively. The reason for the low correct answer rates is that ChatGPT lacked recognition of the images and knowledge of standards and laws. This study concludes that careful attention is required when using ChatGPT because several of its explanations lack the correct description.