Shima Salehi
Current Organizations / Institutes
The organizations/institutes from the scholar's latest article
Research Subjects
Shima Salehi has a total of 34 co-authors. Here are the top co-authors by number of shared publications:
- Carl Wieman
Physics Department and Graduate School of Education, Stanford University, Stanford, CA 94305, USA
3 shared publications Cissy J. Ballen
Graduate School of Eduction, Stanford University, Stanford, CA, United States
3 shared publications- Karen D. Wang
Stanford University, USA
3 shared publications Carl Wieman
Graduate School of Eduction, Stanford University, Stanford, CA, United States
2 shared publications- Eric Burkholder
Auburn University
2 shared publications - Javier Toro-Pérez
Universidad de Chile, Santiago, Chile
2 shared publications
Latest Publications
- Top Cited PapersOpen Access
- 29 April 2024
- Published by JMIR Publications Inc. in JMIR Medical Education
- Vol. 10, e55048
Exploring the Performance of ChatGPT Versions 3.5, 4, and 4 With Vision in the Chilean Medical Licensing Examination: Observational Study
Background: The deployment of OpenAI’s ChatGPT-3.5 and its subsequent versions, ChatGPT-4 and ChatGPT-4 With Vision (4V; also known as “GPT-4 Turbo With Vision”), has notably influenced the medical field. Having demonstrated remarkable performance in medical examinations globally, these models show potential for educational applications. However, their effectiveness in non-English contexts, particularly in Chile’s medical licensing examinations—a critical step for medical practitioners in Chile—is less explored. This gap highlights the need to evaluate ChatGPT’s adaptability to diverse linguistic and cultural contexts. Objective: This study aims to evaluate the performance of ChatGPT versions 3.5, 4, and 4V in the EUNACOM (Examen Único Nacional de Conocimientos de Medicina), a major medical examination in Chile. Methods: Three official practice drills (540 questions) from the University of Chile, mirroring the EUNACOM’s structure and difficulty, were used to test ChatGPT versions 3.5, 4, and 4V. The 3 ChatGPT versions were provided 3 attempts for each drill. Responses to questions during each attempt were systematically categorized and analyzed to assess their accuracy rate. Results: All versions of ChatGPT passed the EUNACOM drills. Specifically, versions 4 and 4V outperformed version 3.5, achieving average accuracy rates of 79.32% and 78.83%, respectively, compared to 57.53% for version 3.5 (P<.001). Version 4V, however, did not outperform version 4 (P=.73), despite the additional visual capabilities. We also evaluated ChatGPT’s performance in different medical areas of the EUNACOM and found that versions 4 and 4V consistently outperformed version 3.5. Across the different medical areas, version 3.5 displayed the highest accuracy in psychiatry (69.84%), while versions 4 and 4V achieved the highest accuracy in surgery (90.00% and 86.11%, respectively). Versions 3.5 and 4 had the lowest performance in internal medicine (52.74% and 75.62%, respectively), while version 4V had the lowest performance in public health (74.07%). Conclusions: This study reveals ChatGPT’s ability to pass the EUNACOM, with distinct proficiencies across versions 3.5, 4, and 4V. Notably, advancements in artificial intelligence (AI) have not significantly led to enhancements in performance on image-based questions. The variations in proficiency across medical fields suggest the need for more nuanced AI training. Additionally, the study underscores the importance of exploring innovative approaches to using AI to augment human cognition and enhance the learning process. Such advancements have the potential to significantly influence medical education, fostering not only knowledge acquisition but also the development of critical thinking and problem-solving skills among health care professionals.
- 29 April 2024
- Published by JMIR Publications Inc. in JMIR Medical Education
- Vol. 10, e55048
Top Cited PapersOpen AccessExploring the Performance of ChatGPT Versions 3.5, 4, and 4 With Vision in the Chilean Medical Licensing Examination: Observational Study
Background: The deployment of OpenAI’s ChatGPT-3.5 and its subsequent versions, ChatGPT-4 and ChatGPT-4 With Vision (4V; also known as “GPT-4 Turbo With Vision”), has notably influenced the medical field. Having demonstrated remarkable performance in medical examinations globally, these models show potential for educational applications. However, their effectiveness in non-English contexts, particularly in Chile’s medical licensing examinations—a critical step for medical practitioners in Chile—is less explored. This gap highlights the need to evaluate ChatGPT’s adaptability to diverse linguistic and cultural contexts. Objective: This study aims to evaluate the performance of ChatGPT versions 3.5, 4, and 4V in the EUNACOM (Examen Único Nacional de Conocimientos de Medicina), a major medical examination in Chile. Methods: Three official practice drills (540 questions) from the University of Chile, mirroring the EUNACOM’s structure and difficulty, were used to test ChatGPT versions 3.5, 4, and 4V. The 3 ChatGPT versions were provided 3 attempts for each drill. Responses to questions during each attempt were systematically categorized and analyzed to assess their accuracy rate. Results: All versions of ChatGPT passed the EUNACOM drills. Specifically, versions 4 and 4V outperformed version 3.5, achieving average accuracy rates of 79.32% and 78.83%, respectively, compared to 57.53% for version 3.5 (P<.001). Version 4V, however, did not outperform version 4 (P=.73), despite the additional visual capabilities. We also evaluated ChatGPT’s performance in different medical areas of the EUNACOM and found that versions 4 and 4V consistently outperformed version 3.5. Across the different medical areas, version 3.5 displayed the highest accuracy in psychiatry (69.84%), while versions 4 and 4V achieved the highest accuracy in surgery (90.00% and 86.11%, respectively). Versions 3.5 and 4 had the lowest performance in internal medicine (52.74% and 75.62%, respectively), while version 4V had the lowest performance in public health (74.07%). Conclusions: This study reveals ChatGPT’s ability to pass the EUNACOM, with distinct proficiencies across versions 3.5, 4, and 4V. Notably, advancements in artificial intelligence (AI) have not significantly led to enhancements in performance on image-based questions. The variations in proficiency across medical fields suggest the need for more nuanced AI training. Additionally, the study underscores the importance of exploring innovative approaches to using AI to augment human cognition and enhance the learning process. Such advancements have the potential to significantly influence medical education, fostering not only knowledge acquisition but also the development of critical thinking and problem-solving skills among health care professionals.
- Open Access
- 18 March 2024
- Published by Association for Computing Machinery (ACM)
- p. 498-506
Discovering Players’ Problem-Solving Behavioral Characteristics in a Puzzle Game through Sequence Mining
Digital games offer promising platforms for assessing student higher-order competencies such as problem-solving. However, processing and analyzing the large volume of interaction log data generated in these platforms to uncover meaningful behavioral patterns remain a complex research challenge. In this study, we employ sequence mining and clustering techniques to examine students’ log data in an interactive puzzle game that requires player to change rules to win the game. Our goal is to identify behavioral characteristics associated with the problem-solving practices adopted by individual students. The findings indicate that the most effective problem solvers made fewer rule changes and took longer time to make those changes across both an introductory and a more advanced level of the game. Conversely, rapid rule change actions were linked to ineffective problem-solving. This research underscores the potential of sequence mining and cluster analysis as generalizable methods for understanding student higher-order competencies through log data in digital gaming and learning environments. It also suggests future directions on how to provide just-in-time, in-game feedback to enhance student problem-solving competences.
- 18 March 2024
- Published by Association for Computing Machinery (ACM)
- p. 498-506
Open AccessDiscovering Players’ Problem-Solving Behavioral Characteristics in a Puzzle Game through Sequence Mining
Digital games offer promising platforms for assessing student higher-order competencies such as problem-solving. However, processing and analyzing the large volume of interaction log data generated in these platforms to uncover meaningful behavioral patterns remain a complex research challenge. In this study, we employ sequence mining and clustering techniques to examine students’ log data in an interactive puzzle game that requires player to change rules to win the game. Our goal is to identify behavioral characteristics associated with the problem-solving practices adopted by individual students. The findings indicate that the most effective problem solvers made fewer rule changes and took longer time to make those changes across both an introductory and a more advanced level of the game. Conversely, rapid rule change actions were linked to ineffective problem-solving. This research underscores the potential of sequence mining and cluster analysis as generalizable methods for understanding student higher-order competencies through log data in digital gaming and learning environments. It also suggests future directions on how to provide just-in-time, in-game feedback to enhance student problem-solving competences.
- Open Access
- 30 November 2023
- Published by JMIR Publications Inc. in JMIR Preprints
Exploring the Proficiency of ChatGPT 3.5, 4, and 4 with Vision in the Chile Medical Licensing Exam (Preprint)
BACKGROUND The deployment of OpenAI's ChatGPT 3.5 and its subsequent versions, ChatGPT 4 and 4 with Vision (4V), has notably influenced the medical field. Demonstrating remarkable performance in medical exams globally, these models show potential for educational applications. However, their effectiveness in non-English contexts, particularly in Chile's Medical Licensing Exam, a critical step for medical practitioners in Chile, is less explored. This gap highlights the need to evaluate ChatGPT's adaptability to diverse linguistic and cultural scenarios. OBJECTIVE This study aims to evaluate the proficiency of ChatGPT versions 3.5, 4, and 4V in answering EUNACOM (Examen Único Nacional de Conocimientos de Medicina), a major medical examination in Chile, format questions. METHODS Three official drills from the University of Chile, mirroring the EUNACOM exam structure and difficulty, were used to test ChatGPT versions 3.5, 4, and 4V. The three ChatGPT versions underwent three rounds of answering each drill. Responses to questions during each round were systematically categorized and analyzed to assess the accuracy rate of the responses. RESULTS All versions of ChatGPT successfully passed EUNACOM-style exams, with version 4 outperforming 3.5 and 4V. A detailed analysis revealed a higher accuracy rate in questions related to Surgery and Psychiatry for all versions, while performance dipped in the areas of Internal Medicine and Public Health. Version 4V didn't demonstrate a better performance compared to the two other versions, despite access to figures of the questions. CONCLUSIONS The study reveals ChatGPT's ability to pass the EUNACOM test, with distinct proficiencies across versions 3.5, 4, and 4V. Notably, advancements in artificial intelligence (AI) don't significantly enhance image-based question performance. The variations in proficiency across medical fields suggest the need for more nuanced AI training. While AI shows promise in medical education, its limitations in depth and variability of expertise highlight the necessity for medical curricula to focus on critical thinking and reflective practices, ensuring the effective integration of AI in patient-centered care.
- 30 November 2023
- Published by JMIR Publications Inc. in JMIR Preprints
Open AccessExploring the Proficiency of ChatGPT 3.5, 4, and 4 with Vision in the Chile Medical Licensing Exam (Preprint)
BACKGROUND The deployment of OpenAI's ChatGPT 3.5 and its subsequent versions, ChatGPT 4 and 4 with Vision (4V), has notably influenced the medical field. Demonstrating remarkable performance in medical exams globally, these models show potential for educational applications. However, their effectiveness in non-English contexts, particularly in Chile's Medical Licensing Exam, a critical step for medical practitioners in Chile, is less explored. This gap highlights the need to evaluate ChatGPT's adaptability to diverse linguistic and cultural scenarios. OBJECTIVE This study aims to evaluate the proficiency of ChatGPT versions 3.5, 4, and 4V in answering EUNACOM (Examen Único Nacional de Conocimientos de Medicina), a major medical examination in Chile, format questions. METHODS Three official drills from the University of Chile, mirroring the EUNACOM exam structure and difficulty, were used to test ChatGPT versions 3.5, 4, and 4V. The three ChatGPT versions underwent three rounds of answering each drill. Responses to questions during each round were systematically categorized and analyzed to assess the accuracy rate of the responses. RESULTS All versions of ChatGPT successfully passed EUNACOM-style exams, with version 4 outperforming 3.5 and 4V. A detailed analysis revealed a higher accuracy rate in questions related to Surgery and Psychiatry for all versions, while performance dipped in the areas of Internal Medicine and Public Health. Version 4V didn't demonstrate a better performance compared to the two other versions, despite access to figures of the questions. CONCLUSIONS The study reveals ChatGPT's ability to pass the EUNACOM test, with distinct proficiencies across versions 3.5, 4, and 4V. Notably, advancements in artificial intelligence (AI) don't significantly enhance image-based question performance. The variations in proficiency across medical fields suggest the need for more nuanced AI training. While AI shows promise in medical education, its limitations in depth and variability of expertise highlight the necessity for medical curricula to focus on critical thinking and reflective practices, ensuring the effective integration of AI in patient-centered care.
- Open Access
Examining the Potential and Pitfalls of ChatGPT in Science and Engineering Problem-Solving
The study explores the capabilities of OpenAI's ChatGPT in solving different types of physics problems. ChatGPT (with GPT-4) was queried to solve a total of 40 problems from a college-level engineering physics course. These problems ranged from well-specified problems, where all data required for solving the problem was provided, to under-specified, real-world problems where not all necessary data were given. Our findings show that ChatGPT could successfully solve 62.5% of the well-specified problems, but its accuracy drops to 8.3% for under-specified problems. Analysis of the model's incorrect solutions revealed three distinct failure modes: 1) failure to construct accurate models of the physical world, 2) failure to make reasonable assumptions about missing data, and 3) calculation errors. The study offers implications for how to leverage LLM-augmented instructional materials to enhance STEM education. The insights also contribute to the broader discourse on AI's strengths and limitations, serving both educators aiming to leverage the technology and researchers investigating human-AI collaboration frameworks for problem-solving and decision-making.
Open AccessExamining the Potential and Pitfalls of ChatGPT in Science and Engineering Problem-Solving
The study explores the capabilities of OpenAI's ChatGPT in solving different types of physics problems. ChatGPT (with GPT-4) was queried to solve a total of 40 problems from a college-level engineering physics course. These problems ranged from well-specified problems, where all data required for solving the problem was provided, to under-specified, real-world problems where not all necessary data were given. Our findings show that ChatGPT could successfully solve 62.5% of the well-specified problems, but its accuracy drops to 8.3% for under-specified problems. Analysis of the model's incorrect solutions revealed three distinct failure modes: 1) failure to construct accurate models of the physical world, 2) failure to make reasonable assumptions about missing data, and 3) calculation errors. The study offers implications for how to leverage LLM-augmented instructional materials to enhance STEM education. The insights also contribute to the broader discourse on AI's strengths and limitations, serving both educators aiming to leverage the technology and researchers investigating human-AI collaboration frameworks for problem-solving and decision-making.
- Open Access
- 3 October 2023
- Published by International Society of the Learning Sciences
- p. 1462-1465
Humanizing College Student Success: The Role of Brave Space, Validation, and Social-Emotional Competencies
Higher education inequities are caused by multiple factors within and beyond the classroom; yet, notions of college student success beyond course grades in formal classes are understudied. Using a sociocultural framework called the zone of proximal self, the purpose of this study is to deepen understanding of how to advance equity in informal learning settings such as counseling and advising spaces. This paper presents three forms of effective equityoriented and relational practices, drawing from two contrasting student cases from a study of 50 students at a public four-year U.S. institution. The cases highlight how counselors' creation of a brave space, validation, and support for social-emotional competencies can enhance students' pathways towards their possible selves in humanizing, relational ways. These findings illuminate how the ZPS framework can offer broader ways to examine college student success by including students' personal and professional achievements across formal and informal learning environments.
- 3 October 2023
- Published by International Society of the Learning Sciences
- p. 1462-1465
Open AccessHumanizing College Student Success: The Role of Brave Space, Validation, and Social-Emotional Competencies
Higher education inequities are caused by multiple factors within and beyond the classroom; yet, notions of college student success beyond course grades in formal classes are understudied. Using a sociocultural framework called the zone of proximal self, the purpose of this study is to deepen understanding of how to advance equity in informal learning settings such as counseling and advising spaces. This paper presents three forms of effective equityoriented and relational practices, drawing from two contrasting student cases from a study of 50 students at a public four-year U.S. institution. The cases highlight how counselors' creation of a brave space, validation, and support for social-emotional competencies can enhance students' pathways towards their possible selves in humanizing, relational ways. These findings illuminate how the ZPS framework can offer broader ways to examine college student success by including students' personal and professional achievements across formal and informal learning environments.
- 14 June 2023
- Published by Springer Nature
- p. 31-52
Applying Log Data Analytics to Measure Problem Solving in Simulation-Based Learning Environments
- 14 June 2023
- Published by Springer Nature
- p. 31-52
Applying Log Data Analytics to Measure Problem Solving in Simulation-Based Learning Environments
- 14 September 2022
- Published by American Chemical Society (ACS) in Journal of Chemical Education
- Vol. 99 (10) , 3461-3470
Differential Impact of the COVID-19 Pandemic on Female Graduate Students and Postdocs in the Chemical Sciences
Over the past two and a half years, the COVID-19 pandemic severely disrupted almost all aspects of life as people throughout the world were instructed to work-from-home. Scientific researchers, whose work is reliant on access to laboratory equipment, have been acutely impacted by these global changes. In this study, we surveyed graduate students and postdocs in the chemical sciences at a selected number of academic institutions in the United States. We found that many survey participants, especially women, experienced severely diminished research progress and increased anxiety levels during the COVID-19 pandemic. Through factor analysis and multiple regression modeling, we found that during this challenging time participants who reported greater levels of professional support also reported greater research progress and lower levels of anxiety. We also found that, although advisors and departments provide some forms of professional support, there are other types of support that students and postdocs still desire. This phenomenon is magnified for female and underrepresented minority participants, as they need greater levels of professional support and place immense value on the quality of their work environments. Based on these results, we have identified some ways in which departments and advisors can provide the needed support for their graduate students and postdocs, thereby providing timeless advice that is applicable to improving academic work conditions not only during a global pandemic but also in a postpandemic world.
- 14 September 2022
- Published by American Chemical Society (ACS) in Journal of Chemical Education
- Vol. 99 (10) , 3461-3470
Differential Impact of the COVID-19 Pandemic on Female Graduate Students and Postdocs in the Chemical Sciences
Over the past two and a half years, the COVID-19 pandemic severely disrupted almost all aspects of life as people throughout the world were instructed to work-from-home. Scientific researchers, whose work is reliant on access to laboratory equipment, have been acutely impacted by these global changes. In this study, we surveyed graduate students and postdocs in the chemical sciences at a selected number of academic institutions in the United States. We found that many survey participants, especially women, experienced severely diminished research progress and increased anxiety levels during the COVID-19 pandemic. Through factor analysis and multiple regression modeling, we found that during this challenging time participants who reported greater levels of professional support also reported greater research progress and lower levels of anxiety. We also found that, although advisors and departments provide some forms of professional support, there are other types of support that students and postdocs still desire. This phenomenon is magnified for female and underrepresented minority participants, as they need greater levels of professional support and place immense value on the quality of their work environments. Based on these results, we have identified some ways in which departments and advisors can provide the needed support for their graduate students and postdocs, thereby providing timeless advice that is applicable to improving academic work conditions not only during a global pandemic but also in a postpandemic world.
- Open Access
- 22 August 2022
- Published by Taylor & Francis in International Journal of Science Education
- Vol. 44 (13) , 2061-2084
An accurate and practical method for assessing science and engineering problem-solving expertise
The ability to solve authentic real-world problems in science, engineering, and medicine is an important goal in post-secondary education. Despite extensive research on problem solving and expertise, the teaching and assessing of advanced problem-solving skills in post-secondary students remains a challenge. We present a template for creating assessments of advanced problem-solving skills that is applicable across science, engineering, and medical disciplines. It is based on a cognitive model of the problem-solving process that is empirically grounded in the study of skilled practitioners (‘experts’) solving authentic problems in their disciplines. These assessments have three key features that overcome shortcomings of current assessment approaches: 1) a more authentic amount and sequence of information provided, 2) opportunities for students to make decisions, and 3) scoring based on comparison with skilled practitioners. This provides a more complete and accurate assessment of authentic problem-solving skills than currently exists for science, engineering, and medicine. Such assessments will be valuable to instructors and curriculum designers for evaluating and improving the teaching of problem solving in these disciplines. We provide examples of problem-solving assessments that illustrate the use of this template in several disciplines.
- 22 August 2022
- Published by Taylor & Francis in International Journal of Science Education
- Vol. 44 (13) , 2061-2084
Open AccessAn accurate and practical method for assessing science and engineering problem-solving expertise
The ability to solve authentic real-world problems in science, engineering, and medicine is an important goal in post-secondary education. Despite extensive research on problem solving and expertise, the teaching and assessing of advanced problem-solving skills in post-secondary students remains a challenge. We present a template for creating assessments of advanced problem-solving skills that is applicable across science, engineering, and medical disciplines. It is based on a cognitive model of the problem-solving process that is empirically grounded in the study of skilled practitioners (‘experts’) solving authentic problems in their disciplines. These assessments have three key features that overcome shortcomings of current assessment approaches: 1) a more authentic amount and sequence of information provided, 2) opportunities for students to make decisions, and 3) scoring based on comparison with skilled practitioners. This provides a more complete and accurate assessment of authentic problem-solving skills than currently exists for science, engineering, and medicine. Such assessments will be valuable to instructors and curriculum designers for evaluating and improving the teaching of problem solving in these disciplines. We provide examples of problem-solving assessments that illustrate the use of this template in several disciplines.
- Open Access
- 1 October 2021
- Published by Frontiers Media SA in Frontiers in Education
- Vol. 6
Inclusive Instructional Practices: Course Design, Implementation, and Discourse
As national efforts strive to make STEM more inclusive, it is important to identify instructional practices that maximize effective learning for all and provide students from different demographic and educational backgrounds equal opportunities to excel. Here, we present a guideline for inclusive instructional practices based on findings from 1) cognitive psychology about learning and memory, 2) social psychology about creating inclusive discourse, and 3) discipline-based education research (DBER) about effective learning practices in STEM higher education. Our aim is to promote equity across STEM education by providing researchers and instructors across different STEM fields with concrete suggestions for implementing inclusive instructional practices in their courses.
- 1 October 2021
- Published by Frontiers Media SA in Frontiers in Education
- Vol. 6
Open AccessInclusive Instructional Practices: Course Design, Implementation, and Discourse
As national efforts strive to make STEM more inclusive, it is important to identify instructional practices that maximize effective learning for all and provide students from different demographic and educational backgrounds equal opportunities to excel. Here, we present a guideline for inclusive instructional practices based on findings from 1) cognitive psychology about learning and memory, 2) social psychology about creating inclusive discourse, and 3) discipline-based education research (DBER) about effective learning practices in STEM higher education. Our aim is to promote equity across STEM education by providing researchers and instructors across different STEM fields with concrete suggestions for implementing inclusive instructional practices in their courses.