Gary M. Weiss

Recent advances in Artificial Intelligence (AI), such as the development of large language models like ChatGPT, have blurred the boundaries between human and AI-generated text. This has led to a pressing need for tools that can determine whether text has been created or revised using AI. A general and universally effective detection model would be extremely useful, but appears to be beyond the reach of current technology and detection methods. The research described in this study adopts a domain and task specific approach and shows that specialized detection models can attain high accuracy. The study focuses on the higher education graduate admissions process, with the specific goal of identifying AI-generated and AI-revised Letters of Recommendation (LORs) and Statements of Intent (SOIs). Detecting such application materials is essential to ensure that applicants are evaluated on their true merits and abilities, and to foster an equitable and trustworthy admissions process. Our research is based on 3755 LORs and 1973 SOIs extracted from the application records of Fordham University’s Master’s programs in Computer Science and Data Science. To facilitate the construction and evaluation of detection models, we generated AI counterparts for each LOR and SOI using the GPT-3.5 Turbo API. The prompts for AI-generation text were derived from the admission data of the respective applicants, and the AI-revised LORs and SOIs were generated directly from the human-authored versions. We also utilize an open-access GPT-wiki-intro dataset to further validate our hypothesis regarding the feasibility of constructing domain-specific AI content detectors. Our experiments yield promising results in developing classifiers tailored to a specific domain when provided with sufficient training samples. Additionally, we present a comparative analysis of the word frequency and statistical characteristics of the text, which provides convincing evidence that ChatGPT employs distinctive vocabulary and paragraph structure compared to human-authored text. The code for this study is available on GitHub, and the models can be executed on user-provided data via an interactive web interface.

Educational Data Mining (EDM) is concerned with exploring large-scale educational data using computational and statistical methods in order to gain a better understanding of the educational process. EDM can be used to improve decision making across all aspects of the educational process, including but not limited to admissions; planning; and assessment and evaluation. EDM can also be applied to student data, course data, and instructor data. There are also substantial opportunities when student learning is done online, since in such cases a wealth of low-level data is readily available. While the education domain has utilized data mining technology for some time, it still lags other fields, such as business, in the use of such technology. The purpose of this minitrack is to bring together EDM researchers and practitioners from academia and industry to discuss the latest advances in the field that support data-driven decision making. Descriptions of relevant deployed applications are especially encouraged, as is research related to large language models (e.g., ChatGPT) for education that relates to decision making. Note that topics that cover learning analytics, which is a major component of EDM but do not facilitate decision making, are not the focus of this minitrack.

Letters of Recommendation (LORs) are widely utilized for admission to both undergraduate and graduate programs, and are becoming even more important with the decreasing role that standardized tests play in the admissions process. However, LORs are highly subjective and thus can inject recommender bias into the process, leading to an inequitable evaluation of the candidates’ competitiveness and competence. Our study utilizes natural language processing methods and manually determined ratings to investigate gender and cultural differences and biases in LORs written for STEM Master’s program applicants. We generate features to measure important characteristics of the LORs and then compare these characteristics across groups based on recommender gender, applicant gender, and applicant country of origin. One set of features, which measure the underlying sentiment, tone, and emotions associated with each LOR, is automatically generated using IBM Watson’s Natural Language Understanding (NLU) service. The second set of features is measured manually by our research team and quantifies the relevance, specificity, and positivity of each LOR. We identify and discuss features that exhibit statistically significant differences across gender and culture study groups. Our analysis is based on approximately 4000 applications for the MS in Data Science and MS in Computer Science programs at Fordham University. To our knowledge, no similar study has been performed on these graduate programs.

Smartphones and smartwatches, which include powerful sensors, provide a readily available platform for implementing and deploying mobile motion-based behavioral biometrics. However, the few studies that utilize these commercial devices for motion-based biometrics are quite limited in terms of the sensors and physical activities that they evaluate. In many such studies, only the smartwatch accelerometer is utilized and only one physical activity, walking, is investigated. In this study we consider the accelerome-ter and gyroscope sensor on both the smartphone and smartwatch, and determine which combination of sensors performs best. Furthermore, eighteen diverse activities of daily living are evaluated for their bio-metric efficacy and, unlike most other studies, biometric identification is evaluated in addition to biometric authentication. The results presented in this article show that motion-based biometrics using smartphones and/or smartwatches yield good results, and that these results hold for the eighteen activities. This suggests that zero-effort continuous biometrics based on normal activities of daily living is feasible, and also demon-strates that certain easy-to-perform activities, such as clapping, may be a viable alternative (or supplement) to gait-based biometrics.

A Wireless Body Area Network (WBAN) is a network of wirelessly connected sensing and actuating devices. WBANs used for recording biometric information and administering medication are classified as part of a Cyber Physical System (CPS). Preserving user security and privacy is a fundamental concern of WBANs, which introduces the notion of using biometric readings as a mechanism for authentication. Extensive research has been conducted regarding the various methodologies (e.g., ECG, EEG, gait, head/arm motion, skin impedance). This paper seeks to analyze and evaluate the most prominent biometric authentication techniques based on accuracy, cost, and feasibility of implementation. We suggest several authentication schemes which incorporate multiple biometric properties.

A Wireless Body Area Network (WBAN) is a network of wirelessly connected sensing and actuating devices. WBANs used for recording biometric information and administering medication are classified as part of a Cyber Physical System (CPS). Preserving user security and privacy is a fundamental concern of WBANs, which introduces the notion of using biometric readings as a mechanism for authentication. Extensive research has been conducted regarding the various methodologies (e.g. ECG, EEG, gait, head/arm motion, skin impedance). This paper seeks to analyze and evaluate the most prominent biometric authentication techniques based on accuracy, cost, and feasibility of implementation. We suggest several authentication schemes which incorporate multiple biometric properties.

Sunni extremism poses a significant danger to society, yet it is relatively easy for these extremist organizations to spread jihadist propaganda and recruit new members via the Internet, Darknet, and social media. The sheer volume of these sites make them very difficult to police. This paper discusses an approach that can assist with this problem, by automatically identifying a subset of web pages and social media content (or any text) that contains extremist content. The approach utilizes machine learning, specifically neural networks and deep learning, to classify text as containing “extremist” or “benign” (i.e., not extremist) content. This method is robust and can effectively learn to classify extremist multilingual text of varying length. This study also involved the construction of a high quality dataset for training and testing, put together by a team of 40 people (some with fluency in Arabic) who expended 9,500 hours of combined effort. This dataset should facilitate future research on this topic.