Current Organizations / Institutes

The organizations/institutes from the scholar's latest article


Kai Ishida has a total of 43 co-authors. Here are the top co-authors by number of shared publications:

  • Minoru Hirose

    17 shared publications
  • Noriaki Ikeda

    Graduate School of Medical Science, Kitasato University, Sagamihara, Kanagawa, Japan Department of Medical Informatics, School of Allied Health Sciences, Kitasato University, Sagamihara, Kanagawa, Japan

    10 shared publications
  • Eisuke Hanada

    Faculty of Science and Engineering, Saga University, Saga 8408502, Japan

    8 shared publications
  • Sazu Arie

    Applied Electromagnetic Research Institute, National Institute of Information and Communications Technology (NICT), Tokyo, 184-8795, Japan

    8 shared publications
  • Akihiro Takeuchi

    7 shared publications
  • Harukazu Tsuruta

    Graduate School of Medical Science, Kitasato University, Sagamihara, Kanagawa, Japan Department of Medical Informatics, School of Allied Health Sciences, Kitasato University, Sagamihara, Kanagawa, Japan

    7 shared publications

Latest Publications

  • Open Access

    Evaluation of the Construction and Accuracy of a Local LLM Based on Clinical Engineering

    by Kai Ishida

    This study aimed to develop a local large language model (LLM) specialized in clinical engineering and to evaluate its performance on the Japanese National Clinical Engineer Licensing Examination. Three local LLMs (Cogito-32b-think, GPT-4o-mini, and Qwen3-14B) and Gemini-2.5 Pro were tested. Among the local models, Qwen3-14B achieved the highest accuracy and was further improved through supervised fine-tuning (SFT) using reasoning data generated by Gemini-2.5 Pro. After SFT, the overall accuracy of Qwen3-14B increased from 77% to 82%, with improvements in image (39% to 52%) and calculation (58% to 68%) questions. Although Gemini-2.5 Pro outperformed Qwen3-14B with 94% accuracy, the results demonstrate that SFT effectively enhances local LLMs, offering a promising, secure, and cost-efficient approach for specialized clinical engineering applications.

  • Open Access

    Evaluation of Local Vision Language Models on the Japanese National Clinical Engineer Licensing Examination Using Image-Only Inputs: A Benchmarking Study (Preprint)

    by Kai Ishida

    UNSTRUCTURED This study systematically evaluated the performance of locally deployable vision language models (VLMs) on the Japanese National Clinical Engineer Licensing Examination (JNCELE) in a fully image-based input setting. Unlike prior studies that primarily relied on text inputs, all questions including problem statements and answer choices were provided exclusively as images to facilitate an integrated assessment of the VLMs’ optical character recognition (OCR), visual understanding, and reasoning capabilities. A benchmark dataset was constructed based on the 35th–39th JNCELEs, and multiple VLMs with varying architectures and parameter sizes were evaluated. Among the local VLMs, Qwen3.5-27B achieved the highest overall accuracy of 79% and an accuracy exceeding 90% in several domains. However, smaller models showed substantial degradation in both accuracy and output stability, and they often exhibited OCR errors, hallucinations, and repetitive responses. The performance was particularly degraded for questions requiring domain-specific knowledge such as Japanese Industrial Standards and for questions involving reading charts and diagrams and performing calculations. In contrast, the cloud-based Gemini-3-Pro achieved an overall accuracy of 93%, which highlights the current performance gap between local and cloud-based VLMs. These findings demonstrate that, while local VLMs are approaching practical utility in selected domains, their performance is strongly constrained by OCR accuracy, reasoning stability, and domain-specific knowledge. The developed benchmark dataset provides a standardized framework for evaluating the multimodal reasoning capabilities of VLMs and is expected to support the future development of reliable and locally deployable artificial intelligence systems in the clinical engineering field.

  • Open Access

    Accuracy of ChatGPT Responses to Examinations for Senior Medical Information Technicians in Japan

    by Kai Ishida, Ryoma Seto, Eisuke Hanada

    This study recruited senior medical information technicians in Japan to evaluate the performance of ChatGPT. The average correct response rates for multiple-choice and written questions were 82.1% and 76.3%, respectively. Although ChatGPT-generated plausible sentences, it failed to describe the appropriate role of medical information technicians in short essay exams.

  • Open Access

    Response Analysis of Health Care Information Technologists in Japan Using Chat Generative Pretrained Transformer

    by Kai Ishida, Eisuke Hanada

    This study used the default model of ChatGPT on health care information technologist exams in Japan to evaluate the reliability of this model for medical information knowledge in non-English languages. The average correct response rates for all questions in the health care, information technology, and health information systems fields were 86%, 93% and 81%, respectively. ChatGPT lacked knowledge of standards and laws; therefore, the accuracy rates for related questions were low. Thus, the study concludes that careful attention is required when using ChatGPT, because several of the explanations it provides lack correct descriptions.


  • Open Access

    Evaluation of the Characteristics and Machine Learning-Based Identification Accuracy of the International Electrotechnical Commission (IEC) 60601-1-8 Medical Equipment Auditory Alarms

    by Kai Ishida, Kiyotaka Fujii

    Introduction Auditory alarms in clinical settings signal sudden changes in a patient’s condition and failures in medical equipment. However, distinguishing between simultaneously sounding alarms, particularly when superimposed with various ambient sounds, remains challenging. This study aimed to develop a machine learning (ML) model for identifying auditory alarms issued by medical equipment. Methods We targeted old and new auditory alarms for medical equipment as specified in the International Electrotechnical Commission (IEC) 60601-1-8 standard. First, we evaluated the characteristics of both normal and degraded auditory alarms using cosine similarity among old and new alarms. Next, we evaluated the accuracy of ML-based identification of deteriorated alarm sound sources in both the old and new alarm groups. Results The cosine similarity among old alarms was over 0.99, while new alarms ranged from 0.886 to 0.985, and exhibited more distinct characteristics. When noise was superimposed, the similarity among old alarms increased further, making differentiation more difficult. In contrast, for most new alarms, cosine similarity values exceeded 0.99 but retained slight acoustic differences even after noise-induced degradation, demonstrating improved distinguishability. The accuracy for identifying a single degraded alarm sound was 71.9% for the support vector machine. The models exhibited a high number of misclassifications when identifying the old alarms. Conversely, the models achieved higher accuracy when classifying new alarms, with recall exceeding 80%, precision above 70%, and F-measure greater than 80% for all new alarms. The identification accuracies for two simultaneous alarms were under 20% and approximately 50% for old and new alarms, respectively. The accuracy declined when estimating two simultaneous new alarms; however, when at least one of the two alarms was correctly classified, the accuracy exceeded 90%. Conclusions This study evaluated the characteristics of old and new auditory alarms issued by medical equipment as specified in IEC 60601-1-8 and constructed ML models for identifying the type of alarms.

  • Open Access

    Evaluating Chat Generative Pretrained Transformer (GPT-4o) Problem-Solving Performance in the Japan Certificate Examination for Biomedical Engineering Class 1

    by Kai Ishida

    Introduction Chat generative pretrained transformer (ChatGPT; OpenAI, San Francisco, CA) has developed rapidly and is used in various fields, including medical engineering. Japan’s Certificate Examination for Biomedical Engineering class 1 (CEBM1) is responsible for the assessment of comprehensive specialized knowledge and skills centered on the maintenance and safety management of medical devices, systems, and related equipment. This study evaluated the performance of ChatGPT (GPT-4o) on CEBM1 for comparison to human-level expectations. Methods We targeted 171 questions including testing for knowledge with fundamental, applied, and problem-solving abilities from the 26th to 28th CEBM1s. We inputted the Japanese version of questions to ChatGPT (GPT-4o), and evaluated performance based on question difficulty. No prompt optimizations were used. We compared the responses provided by ChatGPT with the correct answers. Results The number of correct answers was 39 (68.4±10.5%) for questions testing fundamental knowledge, 33 (57.9±5.3%) for questions testing applied knowledge, and 38 (59.6±8.0%) for questions testing problem-solving ability. There was no statistically significant difference among the three groups. The passing criteria of 60% or higher was achieved only for the 28th examination. However, over 80% of the questions answered incorrectly were due to a lack of knowledge or incorrect knowledge. When asked questions about the background causes and specific countermeasures for problems related to medical devices, the questions were misunderstood, and in certain cases, answers were generated as hallucinations. Conclusions Currently, ChatGPT possesses a certain level of knowledge in medical engineering; however, it cannot be considered universally accurate in solving all possible problems.


  • Open Access

    ChatGPT (GPT-4V) Performance on the Healthcare Information Technologist Examination in Japan

    by Kai Ishida, Eisuke Hanada

    Introduction The Chat Generative Pretrained Transformer (ChatGPT) has developed rapidly and is used in many fields, including healthcare informatics. This study evaluated ChatGPT (GPT-4V)'s performance on the Healthcare Information Technologist (HCIT) certification exam in Japan, which assesses certified professionals who work with electronic health records to improve patient care. Methodology Four hundred seventy-six questions from the HCIT exam were targeted over three years. ChatGPT (GPT-4V) was tested on its ability to answer questions from an HCIT exam to determine if it could perform as well as or better than aspirants taking the exam. Moreover, its performance was evaluated for each academic category, format, presence or absence of images, and calculations. Results The mean correct answer rate for all questions was 84%. ChatGPT achieved the exam passing criteria. The correct answer rate for simple-choice (A-type) questions was higher than that for multiple-choice (X2-type) questions (P < 0.05). The success rate for questions with images was lower than for text-only questions (P < 0.01), and the success rate for questions requiring calculations was lower than for those without calculations (P < 0.05). Conclusions ChatGPT (GPT-4V) met the passing criteria for the 19th to 21st HCIT exams, suggesting that its performance is effective in passing the HCIT exam. ChatGPT may possess the minimum required knowledge, understanding, and application skills for the HCIT certification.


  • Analysis of Responses of GPT-4 V to the Japanese National Clinical Engineer Licensing Examination

    by Kai Ishida, Naoya Arisaka, Kiyotaka Fujii

    Chat Generative Pretrained Transformer (ChatGPT; OpenAI) is a state-of-the-art large language model that can simulate human-like conversations based on user input. We evaluated the performance of GPT-4 V in the Japanese National Clinical Engineer Licensing Examination using 2,155 questions from 2012 to 2023. The average correct answer rate for all questions was 86.0%. In particular, clinical medicine, basic medicine, medical materials, biological properties, and mechanical engineering achieved a correct response rate of ≥ 90%. Conversely, medical device safety management, electrical and electronic engineering, and extracorporeal circulation obtained low correct answer rates ranging from 64.8% to 76.5%. The correct answer rates for questions that included figures/tables, required numerical calculation, figure/table ∩ calculation, and knowledge of Japanese Industrial Standards were 55.2%, 85.8%, 64.2% and 31.0%, respectively. The reason for the low correct answer rates is that ChatGPT lacked recognition of the images and knowledge of standards and laws. This study concludes that careful attention is required when using ChatGPT because several of its explanations lack the correct description.