Subin Lee
Current Organizations / Institutes
The organizations/institutes from the scholar's latest article
Previous Organizations / Institutes
The organizations / institutes from the scholar's other articles in the past ten years
Research Subjects
Subin Lee has a total of 21 co-authors. Here are the top co-authors by number of shared publications:
- Hocheon Yoo
Department of Electronic Engineering Hanyang University Seoul Republic of Korea
5 shared publications - Chaitanya Malaviya
University of Pennsylvania
4 shared publications - Dan Roth
University of Pennsylvania
4 shared publications - Mark Yatskar
University of Pennsylvania
4 shared publications - Elizabeth Sieber
University of Pennsylvania University of Washington
2 shared publications - Sihao Chen
University of Pennsylvania University of Washington
2 shared publications
Latest Publications
- Open Access
What if you said that differently?: How Explanation Formats Affect Human Feedback Efficacy and User Perception
Eliciting feedback from end users of NLP models can be beneficial for improving models. However, how should we present model responses to users so they are most amenable to be corrected from user feedback? Further, what properties do users value to understand and trust responses? We answer these questions by analyzing the effect of rationales (or explanations) generated by QA models to support their answers. We specifically consider decomposed QA models that first extract an intermediate rationale based on a context and a question and then use solely this rationale to answer the question. A rationale outlines the approach followed by the model to answer the question. Our work considers various formats of these rationales that vary according to well-defined properties of interest. We sample rationales from language models using few-shot prompting for two datasets, and then perform two user studies. First, we present users with incorrect answers and corresponding rationales in various formats and ask them to provide natural language feedback to revise the rationale. We then measure the effectiveness of this feedback in patching these rationales through in-context learning. The second study evaluates how well different rationale formats enable users to understand and trust model answers, when they are correct. We find that rationale formats significantly affect how easy it is (1) for users to give feedback for rationales, and (2) for models to subsequently execute this feedback. In addition, formats with attributions to the context and in-depth reasoning significantly enhance user-reported understanding and trust of model outputs.
Open AccessWhat if you said that differently?: How Explanation Formats Affect Human Feedback Efficacy and User Perception
Eliciting feedback from end users of NLP models can be beneficial for improving models. However, how should we present model responses to users so they are most amenable to be corrected from user feedback? Further, what properties do users value to understand and trust responses? We answer these questions by analyzing the effect of rationales (or explanations) generated by QA models to support their answers. We specifically consider decomposed QA models that first extract an intermediate rationale based on a context and a question and then use solely this rationale to answer the question. A rationale outlines the approach followed by the model to answer the question. Our work considers various formats of these rationales that vary according to well-defined properties of interest. We sample rationales from language models using few-shot prompting for two datasets, and then perform two user studies. First, we present users with incorrect answers and corresponding rationales in various formats and ask them to provide natural language feedback to revise the rationale. We then measure the effectiveness of this feedback in patching these rationales through in-context learning. The second study evaluates how well different rationale formats enable users to understand and trust model answers, when they are correct. We find that rationale formats significantly affect how easy it is (1) for users to give feedback for rationales, and (2) for models to subsequently execute this feedback. In addition, formats with attributions to the context and in-depth reasoning significantly enhance user-reported understanding and trust of model outputs.
- Open Access
ExpertQA: Expert-Curated Questions and Attributed Answers
As language models are adopted by a more sophisticated and diverse set of users, the importance of guaranteeing that they provide factually correct information supported by verifiable sources is critical across fields of study. This is especially the case for high-stakes fields, such as medicine and law, where the risk of propagating false information is high and can lead to undesirable societal consequences. Previous work studying attribution and factuality has not focused on analyzing these characteristics of language model outputs in domain-specific scenarios. In this work, we conduct human evaluation of responses from a few representative systems along various axes of attribution and factuality, by bringing domain experts in the loop. Specifically, we collect expert-curated questions from 484 participants across 32 fields of study, and then ask the same experts to evaluate generated responses to their own questions. In addition, we ask experts to improve upon responses from language models. The output of our analysis is ExpertQA, a high-quality long-form QA dataset with 2177 questions spanning 32 fields, along with verified answers and attributions for claims in the answers.
Open AccessExpertQA: Expert-Curated Questions and Attributed Answers
As language models are adopted by a more sophisticated and diverse set of users, the importance of guaranteeing that they provide factually correct information supported by verifiable sources is critical across fields of study. This is especially the case for high-stakes fields, such as medicine and law, where the risk of propagating false information is high and can lead to undesirable societal consequences. Previous work studying attribution and factuality has not focused on analyzing these characteristics of language model outputs in domain-specific scenarios. In this work, we conduct human evaluation of responses from a few representative systems along various axes of attribution and factuality, by bringing domain experts in the loop. Specifically, we collect expert-curated questions from 484 participants across 32 fields of study, and then ask the same experts to evaluate generated responses to their own questions. In addition, we ask experts to improve upon responses from language models. The output of our analysis is ExpertQA, a high-quality long-form QA dataset with 2177 questions spanning 32 fields, along with verified answers and attributions for claims in the answers.
- Open Access
- 22 January 2024
- Published by MDPI AG in Micromachines
- Vol. 15 (1) , 164
Split-Gate: Harnessing Gate Modulation Power in Thin-Film Electronics
With the increase in electronic devices across various applications, there is rising demand for selective carrier control. The split-gate consists of a gate electrode divided into multiple parts, allowing for the independent biasing of electric fields within the device. This configuration enables the potential formation of both p- and n-channels by injecting holes and electrons owing to the presence of the two gate electrodes. Applying voltage to the split-gate allows for the control of the Fermi level and, consequently, the barrier height in the device. This facilitates band bending in unipolar transistors and allows ambipolar transistors to operate as if unipolar. Moreover, the split-gate serves as a revolutionary tool to modulate the contact resistance by controlling the barrier height. This approach enables the precise control of the device by biasing the partial electric field without limitations on materials, making it adaptable for various applications, as reported in various types of research. However, the gap length between gates can affect the injection of the electric field for the precise control of carriers. Hence, the design of the gap length is a critical element for the split-gate structure. The primary investigation in this review is the introduction of split-gate technology applied in various applications by using diverse materials, the methods for forming the split-gate in each device, and the operational mechanisms under applied voltage conditions.
- 22 January 2024
- Published by MDPI AG in Micromachines
- Vol. 15 (1) , 164
Open AccessSplit-Gate: Harnessing Gate Modulation Power in Thin-Film Electronics
With the increase in electronic devices across various applications, there is rising demand for selective carrier control. The split-gate consists of a gate electrode divided into multiple parts, allowing for the independent biasing of electric fields within the device. This configuration enables the potential formation of both p- and n-channels by injecting holes and electrons owing to the presence of the two gate electrodes. Applying voltage to the split-gate allows for the control of the Fermi level and, consequently, the barrier height in the device. This facilitates band bending in unipolar transistors and allows ambipolar transistors to operate as if unipolar. Moreover, the split-gate serves as a revolutionary tool to modulate the contact resistance by controlling the barrier height. This approach enables the precise control of the device by biasing the partial electric field without limitations on materials, making it adaptable for various applications, as reported in various types of research. However, the gap length between gates can affect the injection of the electric field for the precise control of carriers. Hence, the design of the gap length is a critical element for the split-gate structure. The primary investigation in this review is the introduction of split-gate technology applied in various applications by using diverse materials, the methods for forming the split-gate in each device, and the operational mechanisms under applied voltage conditions.
- Open Access
- 1 January 2024
- Published by Association for Computational Linguistics (ACL)
- p. 3046-3065
What if you said that differently?: How Explanation Formats Affect Human Feedback Efficacy and User Perception
Chaitanya Malaviya, Subin Lee, Dan Roth, Mark Yatskar. Proceedings of the 2024 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies (Volume 1: Long Papers). 2024.
- 1 January 2024
- Published by Association for Computational Linguistics (ACL)
- p. 3046-3065
Open AccessWhat if you said that differently?: How Explanation Formats Affect Human Feedback Efficacy and User Perception
Chaitanya Malaviya, Subin Lee, Dan Roth, Mark Yatskar. Proceedings of the 2024 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies (Volume 1: Long Papers). 2024.
- Top Cited PapersOpen Access
- 1 January 2024
- Published by Association for Computational Linguistics (ACL)
- p. 3025-3045
ExpertQA: Expert-Curated Questions and Attributed Answers
Chaitanya Malaviya, Subin Lee, Sihao Chen, Elizabeth Sieber, Mark Yatskar, Dan Roth. Proceedings of the 2024 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies (Volume 1: Long Papers). 2024.
- 1 January 2024
- Published by Association for Computational Linguistics (ACL)
- p. 3025-3045
Top Cited PapersOpen AccessExpertQA: Expert-Curated Questions and Attributed Answers
Chaitanya Malaviya, Subin Lee, Sihao Chen, Elizabeth Sieber, Mark Yatskar, Dan Roth. Proceedings of the 2024 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies (Volume 1: Long Papers). 2024.
- 13 November 2023
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Transactions on Electron Devices
- Vol. 71 (1) , 759-761
Bridge Transistors: A Study of Bridging Effects and Early Saturation Behaviors
This brief proposes the use of a bridge transistor as a means of achieving early saturation characteristics of drain current. To investigate the electrical properties of the bridge transistor, we introduce materials capable of forming homojunction 2,7-Dioctyl1benzothieno3,2-b1 benzothiophene (C8-BTBT), heterojunction dinaphtho2,3-b:2’,3’-fthieno3,2 -bthiophene (DNTT), and Schottky junction (Au) as bridge layers, along with the C8-BTBT-based channel. Our findings reveal early saturation characteristics of drain current in bridge transistors with Schottky junction. Through contact resistance extraction and 2-D numerical simulation, we suggest that the early current saturation behavior is due to the additional contact resistance and depletion region.
- 13 November 2023
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Transactions on Electron Devices
- Vol. 71 (1) , 759-761
Bridge Transistors: A Study of Bridging Effects and Early Saturation Behaviors
This brief proposes the use of a bridge transistor as a means of achieving early saturation characteristics of drain current. To investigate the electrical properties of the bridge transistor, we introduce materials capable of forming homojunction 2,7-Dioctyl1benzothieno3,2-b1 benzothiophene (C8-BTBT), heterojunction dinaphtho2,3-b:2’,3’-fthieno3,2 -bthiophene (DNTT), and Schottky junction (Au) as bridge layers, along with the C8-BTBT-based channel. Our findings reveal early saturation characteristics of drain current in bridge transistors with Schottky junction. Through contact resistance extraction and 2-D numerical simulation, we suggest that the early current saturation behavior is due to the additional contact resistance and depletion region.
- 20 October 2022
- Published by Institute of Navigation
- p. 852-858
Assessment of RFI Robustness Performance for New RNSS Signal Design
This paper presents the results of anti-jamming performance assessment for the design of new Radio Navigation Satellite System (RNSS) signals in the L6 band. For this, new RNSS signal scenarios are proposed, and these signals are assessed along with legacy signals in the L6 band. The assessment is performed using anti-jamming performance Figures of Merit (FoMs) such as jamming resistance quality factor, jamming to signal power ratio, tolerable jamming power, and range to jammer. The assessment results are used to analyze the anti-jamming performance of new RNSS signal scenarios and to derive waveform design conditions for high anti-jamming performance.
- 20 October 2022
- Published by Institute of Navigation
- p. 852-858
Assessment of RFI Robustness Performance for New RNSS Signal Design
This paper presents the results of anti-jamming performance assessment for the design of new Radio Navigation Satellite System (RNSS) signals in the L6 band. For this, new RNSS signal scenarios are proposed, and these signals are assessed along with legacy signals in the L6 band. The assessment is performed using anti-jamming performance Figures of Merit (FoMs) such as jamming resistance quality factor, jamming to signal power ratio, tolerable jamming power, and range to jammer. The assessment results are used to analyze the anti-jamming performance of new RNSS signal scenarios and to derive waveform design conditions for high anti-jamming performance.
- Open Access
Contribution of Polymers to Electronic Memory Devices and Applications
Electronic memory devices, such as memristors, charge trap memory, and floating-gate memory, have been developed over the last decade. The use of polymers in electronic memory devices enables new opportunities, including easy-to-fabricate processes, mechanical flexibility, and neuromorphic applications. This review revisits recent efforts on polymer-based electronic memory developments. The versatile contributions of polymers for emerging memory devices are classified, providing a timely overview of such unconventional functionalities with a strong emphasis on the merits of polymer utilization. Furthermore, this review discusses the opportunities and challenges of polymer-based memory devices with respect to their device performance and stability for practical applications.
Open AccessContribution of Polymers to Electronic Memory Devices and Applications
Electronic memory devices, such as memristors, charge trap memory, and floating-gate memory, have been developed over the last decade. The use of polymers in electronic memory devices enables new opportunities, including easy-to-fabricate processes, mechanical flexibility, and neuromorphic applications. This review revisits recent efforts on polymer-based electronic memory developments. The versatile contributions of polymers for emerging memory devices are classified, providing a timely overview of such unconventional functionalities with a strong emphasis on the merits of polymer utilization. Furthermore, this review discusses the opportunities and challenges of polymer-based memory devices with respect to their device performance and stability for practical applications.
- Open Access
Synthesis of Lead-Free CaTiO3 Oxide Perovskite Film through Solution Combustion Method and Its Thickness-Dependent Hysteresis Behaviors within 100 mV Operation
Perovskite is attracting considerable interest because of its excellent semiconducting properties and optoelectronic performance. In particular, lead perovskites have been used extensively in photovoltaic, photodetectors, thin-film transistors, and various electronic applications. On the other hand, the elimination of lead is essential because of its strong toxicity. This paper reports the synthesis of lead-free calcium titanate perovskite (CaTiO3) using a solution-processed combustion method. The chemical and morphological properties of CaTiO3 were examined as a function of its thickness by scanning electron microscopy, X-ray diffraction (XRD), atomic force microscopy, X-ray photoelectron spectroscopy, and ultraviolet–visible spectrophotometry. The analysis showed that thicker films formed by a cumulative coating result in larger grains and more oxygen vacancies. Furthermore, thickness-dependent hysteresis behaviors were examined by fabricating a metal-CaTiO3-metal structure. The electrical hysteresis could be controlled over an extremely low voltage operation, as low as 100 mV, by varying the grain size and oxygen vacancies.
Open AccessSynthesis of Lead-Free CaTiO3 Oxide Perovskite Film through Solution Combustion Method and Its Thickness-Dependent Hysteresis Behaviors within 100 mV Operation
Perovskite is attracting considerable interest because of its excellent semiconducting properties and optoelectronic performance. In particular, lead perovskites have been used extensively in photovoltaic, photodetectors, thin-film transistors, and various electronic applications. On the other hand, the elimination of lead is essential because of its strong toxicity. This paper reports the synthesis of lead-free calcium titanate perovskite (CaTiO3) using a solution-processed combustion method. The chemical and morphological properties of CaTiO3 were examined as a function of its thickness by scanning electron microscopy, X-ray diffraction (XRD), atomic force microscopy, X-ray photoelectron spectroscopy, and ultraviolet–visible spectrophotometry. The analysis showed that thicker films formed by a cumulative coating result in larger grains and more oxygen vacancies. Furthermore, thickness-dependent hysteresis behaviors were examined by fabricating a metal-CaTiO3-metal structure. The electrical hysteresis could be controlled over an extremely low voltage operation, as low as 100 mV, by varying the grain size and oxygen vacancies.