Morgan Heisler

Large Language Models (LLMs) require substantial computational resources, making efficient inference a critical challenge. As the demand for LLMs grows in serverless and large-scale deployments, scheduling techniques play a pivotal role in optimizing inference performance, balancing latency, cost, and throughput. This survey provides a comprehensive overview of scheduling strategies for LLM serving across both the control and data planes. We explore key challenges such as cold start latency, HOL blocking, resource utilization, and dynamic workload variations. We analyze scheduling techniques, including reinforcement learning-based workload management, adaptive batching, speculative execution, and multi-tenant scheduling. Additionally, we compare scheduling frameworks across latency reduction, cost efficiency, and scalability. Finally, we identify open research directions, including ML-driven adaptive scheduling, energy-aware inference, and heterogeneous hardware utilization. This survey aims to serve as a foundational reference for researchers and practitioners optimizing LLM inference at scale and is maintained as a live document, updated monthly to reflect ongoing developments.

Précis: Glaucoma was associated with axial bowing and rotation of Bruchs membrane opening (BMO) and anterior laminar insertion (ALI), skewed neural canal, and deeper anterior lamina cribrosa surface (ALCS). Longer axial length was associated with wider, longer, and more skewed neural canal and flatter ALCS. Purpose: Investigate the effects of myopia and glaucoma in the prelaminar neural canal and anterior lamina cribrosa using 1060-nm swept-source optical coherence tomography. Patients: 19 control (38 eyes) and 38 glaucomatous subjects (63 eyes). Materials and Methods: Participants were imaged with swept-source optical coherence tomography, and the images were analyzed for the BMO and ALI dimensions, prelaminar neural canal dimensions, and ALCS depth. Results: Glaucomatous eyes had more bowed and nasally rotated BMO and ALI, more horizontally skewed prelaminar neural canal, and deeper ALCS than the control eyes. Increased axial length was associated with a wider, longer, and more horizontally skewed neural canal and a decrease in the ALCS depth and curvature. Conclusion: Our findings suggest that glaucomatous posterior bowing or cupping of lamina cribrosa can be significantly confounded by the myopic expansion of the neural canal. This may be related to higher glaucoma risk associated with myopia from decreased compliance and increased susceptibility to IOP-related damage of LC being pulled taut.

High resolution visualization of optical coherence tomography (OCT) and OCT angiography (OCT-A) data is required to fully take advantage of the imaging modality’s three-dimensional nature. However, artifacts induced by patient motion often degrade OCT-A data quality. This is especially true for patients with deteriorated focal vision, such as those with diabetic retinopathy (DR). We propose a novel methodology for software-based OCT-A motion correction achieved through serial acquisition, volumetric registration, and averaging. Motion artifacts are removed via a multi-step 3D registration process, and visibility is significantly enhanced through volumetric averaging. We demonstrate that this method permits clear 3D visualization of retinal pathologies and their surrounding features, 3D visualization of inner retinal capillary connections, as well as reliable visualization of the choriocapillaris layer.

Optical coherence tomography (OCT) has become an essential tool in the evaluation of glaucoma, typically through analyzing retinal nerve fiber layer changes in circumpapillary scans. Three-dimensional OCT volumes enable a much more thorough analysis of the optic nerve head (ONH) region, which may be the site of initial glaucomatous optic nerve damage. Automated analysis of this region is of great interest, though large anatomical variations and the termination of layers make the requisite peripapillary layer and Bruch’s membrane opening (BMO) segmentation a challenging task. Several machine learning-based segmentation methods have been proposed for retinal layer segmentation, and a few for the ONH region, but they typically depend on either heavily averaged or pre-processed B-scans or a large amount of annotated data, which is a tedious task and resource-intensive. We evaluated a semi-supervised adversarial deep learning method for segmenting peripapillary retinal layers in OCT B-scans to take advantage of unlabeled data. We show that the use of a generative adversarial network and unlabeled data can improve the performance of segmentation. Additionally, we use a Faster R-CNN architecture to automatically segment the BMO. The proposed methods are then used for the 3D morphometric analysis of both control and glaucomatous ONH volumes to demonstrate the potential for clinical utility.

Automated measurements of the human cone mosaic requires the identification of individual cone photoreceptors. The current gold standard, manual labeling, is a tedious process and can not be done in a clinically useful timeframe. As such, we present an automated algorithm for identifying cone photoreceptors in adaptive optics optical coherence tomography (AO-OCT) images. Our approach fine-tunes a pre-trained convolutional neural network originally trained on AO scanning laser ophthalmoscope (AO-SLO) images, to work on previously unseen data from a different imaging modality. On average, the automated method correctly identified 94% of manually labeled cones when compared to manual raters, from twenty different AO-OCT images acquired from five normal subjects. Voronoi analysis confirmed the general hexagonal-packing structure of the cone mosaic as well as the general cone density variability across portions of the retina. The consistency of our measurements demonstrates the high reliability and practical utility of having an automated solution to this problem.

We demonstrate the utility of a novel scanning method for optical coherence tomography angiography (OCTA). Although raster scanning is commonly used for OCTA imaging, a bidirectional approach would lessen the distortion caused by galvanometer-based scanners as sources continue to increase sweep rates. As shown, a unidirectional raster scan approach has a lower effective scanning time than bidirectional approaches; however, a strictly bidirectional approach causes contrast variation along the B-scan direction due to the non-uniform time interval between B-scans. Therefore, a stepped bidirectional approach is introduced and successfully applied to retinal imaging in normal controls and in a pathological subject with diabetic retinopathy.

We present a multiscale sensorless adaptive optics (SAO) OCT system capable of imaging retinal structure and vasculature with various fields-of-view (FOV) and resolutions. Using a single deformable mirror and exploiting the polarization properties of light, the SAO-OCT-A was implemented in a compact and easy to operate system. With the ability to adjust the beam diameter at the pupil, retinal imaging was demonstrated at two different numerical apertures with the same system. The general morphological structure and retinal vasculature could be observed with a few tens of micrometer-scale lateral resolution with conventional OCT and OCT-A scanning protocols with a 1.7-mm-diameter beam incident at the pupil and a large FOV (15 deg× 15 deg). Changing the system to a higher numerical aperture with a 5.0-mm-diameter beam incident at the pupil and the SAO aberration correction, the FOV was reduced to 3 deg× 3 deg for fine detailed imaging of morphological structure and microvasculature such as the photoreceptor mosaic and capillaries. Multiscale functional SAO-OCT imaging was performed on four healthy subjects, demonstrating its functionality and potential for clinical utility.