Vicki H. Allan

Large language models (LLMs) can provide formative feedback in programming to help students improve the code they have written. We investigate the use of LLMs (GPT-4) to provide formative code feedback in a sophomore-level computer science (CS) course on data structures and algorithms. In three quizzes on recursion, half of the students randomly received GPT-4's feedback, while the other half received feedback from the course instructor. Students resubmitted their code based on the provided feedback. We found that students in the LLM-feedback condition scored higher in resubmissions than those receiving feedback from the instructor. Students perceived the two types of feedback as equally supportive of guiding resubmissions. We discuss the implications of using LLMs to provide formative feedback at scale in CS instruction.

Large language models (LLMs) have shown promise in generating sensible code explanation and feedback in programming exercises. In this experience report, we discuss the process of using one of these models (OpenAI's GPT-4) to generate individualized feedback for students' Java code and pseudocode. We instructed GPT-4 to generate feedback for 113 submissions to four programming problems in an Algorithms and Data Structures class. We prompted the model with example feedback (few-shot learning) and instruction to (1) give feedback on conceptual understanding, syntax, and time complexity, and (2) suggest follow-up actions based on students' code or provide guiding questions. Overall, GPT-4 provided accurate feedback and successfully built on students' ideas in most submissions. Human evaluators (computer science instructors and tutors) rated GPT-4's hints as useful in guiding students' next steps. Model performance varied with programming problems but not submission quality. We reflect on where the model performed well and fell short, and discuss the potential of integrating LLM-generated, individualized feedback into computer science instruction.

In this paper, we study the challenging problem of how to balance taxi distribution across a city in a dynamic ridesharing service. First, we introduce the architecture of the dynamic ridesharing system and formally define the performance metrics indicating the efficiency of the system. Then, we propose a hybrid solution involving a series of algorithms: the Correlated Pooling collects correlated rider requests, the Adjacency Ride-Matching based on Demand Learning assigns taxis to riders and balances taxi distribution locally, the Greedy Idle Movement aims to direct taxis without a current assignment to the areas with riders in need of service. In the experiment, we apply city-scale data sets from the city of Chicago and complete a case study analyzing the threshold of correlated rider requests and the average online running time of each algorithm. We also compare our hybrid solution with multiple other methods. The results of our experiment show that our hybrid solution improves customer serving rate without increasing the number of taxis in operation, allows both drivers to earn more and riders to save more per trip, and all with a small increase in calling and extra trip time.

In this paper, we study the challenging problem of how to balance taxi distribution across a city in a dynamic ridesharing service. First, we introduce the architecture of the dynamic ridesharing system and formally define the performance metrics indicating the efficiency of the system. Then, we propose a hybrid solution involving a series of algorithms: the Correlated Pooling collects correlated rider requests, the Adjacency Ride-Matching based on Demand Learning assigns taxis to riders and balances taxi distribution locally, the Greedy Idle Movement aims to direct taxis without a current assignment to the areas with riders in need of service. In the experiment, we apply city-scale data sets from the city of Chicago and complete a case study analyzing the threshold of correlated rider requests and the average online running time of each algorithm. We also compare our hybrid solution with multiple other methods. The results of our experiment show that our hybrid solution improves customer serving rate without increasing the number of taxis in operation, allows both drivers to earn more and riders to save more per trip, and all with a small increase in calling and extra trip time.

In this paper, we study a challenging problem of how to pool multiple ride-share trip requests in real time under an uncertain environment. The goals are better performance metrics of efficiency and acceptable satisfaction of riders. To solve the problem effectively, an objective function that compromises the benefits and losses of dynamic ridesharing service is proposed. The Polar Coordinates based Ride-Matching strategy (PCRM) that can adapt to the satisfaction of riders on board is also addressed. In the experiment, large scale data sets from New York City (NYC) are applied. We do a case study to identify the best set of parameters of the dynamic ridesharing service with a training set of 135,252 trip requests. In addition, we also use a testing set containing 427,799 trip requests and two state-of-the-art approaches as baselines to estimate the effectiveness of our method. The experimental results show that on average 38% of traveling distance can be saved, nearly 100% of passengers can be served and each rider only spends an additional 3.8 minutes in ridesharing trips compared to single rider service.