LSLalith Sharan

Lalith Sharan

*This is an algorithmically generated author record based on publicly available information.

Current Organizations / Institutes

The organizations/institutes from the scholar's latest article

Previous Organizations / Institutes

The organizations / institutes from the scholar's other articles in the past ten years


Lalith Sharan has a total of 23 co-authors. Here are the top co-authors by number of shared publications:


Latest Publications

  • Open Access

    Surgical phase and instrument recognition: how to identify appropriate dataset splits

    by Georgii Kostiuchik, Lalith Sharan, Benedikt Mayer, Ivo Wolf, Bernhard Preim, Sandy Engelhardt

    Purpose: Machine learning approaches can only be reliably evaluated if training, validation, and test data splits are representative and not affected by the absence of classes. Surgical workflow and instrument recognition are two tasks that are complicated in this manner, because of heavy data imbalances resulting from different length of phases and their potential erratic occurrences. Furthermore, sub-properties like instrument (co-)occurrence are usually not particularly considered when defining the split. Methods: We present a publicly available data visualization tool that enables interactive exploration of dataset partitions for surgical phase and instrument recognition. The application focuses on the visualization of the occurrence of phases, phase transitions, instruments, and instrument combinations across sets. Particularly, it facilitates assessment of dataset splits, especially regarding identification of sub-optimal dataset splits. Results: We performed analysis of the datasets Cholec80, CATARACTS, CaDIS, M2CAI-workflow, and M2CAI-tool using the proposed application. We were able to uncover phase transitions, individual instruments, and combinations of surgical instruments that were not represented in one of the sets. Addressing these issues, we identify possible improvements in the splits using our tool. A user study with ten participants demonstrated that the participants were able to successfully solve a selection of data exploration tasks. Conclusion: In highly unbalanced class distributions, special care should be taken with respect to the selection of an appropriate dataset split because it can greatly influence the assessments of machine learning approaches. Our interactive tool allows for determination of better splits to improve current practices in the field. The live application is available at https://cardio-ai.github.io/endovis-ml/.


  • Surgical training of minimally invasive mitral valve repair on a patient-specific simulator improves surgical skills

    by Christina Wang, Roger Karl, Lalith Sharan, Andela Grizelj, Samantha Fischer, Matthias Karck, Raffaele De Simone, Gabriele Romano, Sandy Engelhardt

    OBJECTIVES: Minimally invasive mitral valve repair (MVR) is considered one of the most challenging operations in cardiac surgery and requires much practice and experience. Simulation-based surgical training might be a method to support the learning process and help to flatten the steep learning curve of novices. The purpose of this study was to show the possible effects on learning of surgical training using a high-fidelity simulator with patient-specific mitral valve replicas. METHODS: Twenty-five participants were recruited to perform MVR on anatomically realistic valve models during different training sessions. After every session their performance was evaluated by a surgical expert regarding accuracy and duration for each step. A second blinded rater similarly assessed the performance after the study. Through repeated documentation of those parameters, their progress in learning was analysed, and gains in proficiency were evaluated. RESULTS: Participants showed significant performance enhancements in terms of both accuracy and time. Their surgical skills showed sizeable improvements after only 1 session. For example, the time to implant neo-chordae decreased by 24.64% (354 s-264 s, P < 0.001) and the time for annuloplasty by 4.01% (54 s-50 s, P = 0.165), whereas the number of irregular stitches for annuloplasty decreased from 52% to 24%. The significance of simulation-based surgical training as a tool for acquiring and training surgical skills was reviewed positively. CONCLUSIONS: The results of this study indicate that simulation-based surgical training is a valuable and effective method for learning reconstructive techniques of minimally invasive MVR and overall general dexterity. The novel learning and training options should be implemented in the surgical traineeship for systematic teaching of various surgical skills.


  • Open Access

    Comparative evaluation of three commercially available markerless depth sensors for close-range use in surgical simulation

    by Lukas Burger, Lalith Sharan, Roger Karl, Christina Wang, Matthias Karck, Raffaele De Simone, Ivo Wolf, Gabriele Romano, Sandy Engelhardt

    Purpose: Minimally invasive surgeries have restricted surgical ports, demanding a high skill level from the surgeon. Surgical simulation potentially reduces this steep learning curve and additionally provides quantitative feedback. Markerless depth sensors show great promise for quantification, but most such sensors are not designed for accurate reconstruction of complex anatomical forms in close-range. Methods: This work compares three commercially available depth sensors, namely the Intel D405, D415, and the Stereolabs Zed-Mini in the range of 12–20 cm, for use in surgical simulation. Three environments are designed that closely mimic surgical simulation, comprising planar surfaces, rigid objects, and mitral valve models of silicone and realistic porcine tissue. The cameras are evaluated on Z-accuracy, temporal noise, fill rate, checker distance, point cloud comparisons, and visual inspection of surgical scenes, across several camera settings. Results: The Intel cameras show sub-mm accuracy in most static environments. The D415 fails in reconstructing valve models, while the Zed-Mini provides lesser temporal noise and higher fill rate. The D405 could reconstruct anatomical structures like the mitral valve leaflet and a ring prosthesis, but performs poorly for reflective surfaces like surgical tools and thin structures like sutures. Conclusion: If a high temporal resolution is needed and lower spatial resolution is acceptable, the Zed-Mini is the best choice, whereas the Intel D405 is the most suited for close-range applications. The D405 shows potential for applications like deformable registration of surfaces, but is not yet suitable for applications like real-time tool tracking or surgical skill assessment.


  • Open Access

    mvHOTA: A multi-view higher order tracking accuracy metric to measure spatial and temporal associations in multi-point detection

    by Lalith Sharan, Halvar Kelm, Gabriele Romano, Matthias Karck, Raffaele De Simone, Sandy Engelhardt

    Multi-point tracking is a challenging task that involves detecting points in the scene and tracking them across a sequence of frames. Computing detection-based measures like the F-measure on a frame-by-frame basis is not sufficient to assess the overall performance, as it does not interpret performance in the temporal domain. The main evaluation metric available comes from Multi-object tracking (MOT) methods to benchmark performance on datasets such as KITTI with the recently proposed higher order tracking accuracy (HOTA) metric, which is capable of providing a better description of the performance over metrics such as MOTA, DetA, and IDF1. While the HOTA metric takes into account temporal associations, it does not provide a tailored means to analyse the spatial associations of a dataset in a multi-camera setup. Moreover, there are differences in evaluating the detection task for points when compared to objects (point distances vs. bounding box overlap). Therefore in this work, we propose a multi-view higher order tracking metric (mvHOTA) to determine the accuracy of multi-point (multi-instance and multi-class) tracking methods, while taking into account temporal and spatial associations.mvHOTA can be interpreted as the geometric mean of detection, temporal, and spatial associations, thereby providing equal weighting to each of the factors. We demonstrate the use of this metric to evaluate the tracking performance on an endoscopic point detection dataset from a previously organised surgical data science challenge. Furthermore, we compare with other adjusted MOT metrics for this use-case, discuss the properties of mvHOTA, and show how the proposed multi-view Association and the Occlusion index (OI) facilitate analysis of methods with respect to handling of occlusions. The code is available at https://github.com/Cardio-AI/mvhota.

  • mvHOTA: A multi-view higher order tracking accuracy metric to measure temporal and spatial associations in multi-point tracking

    by Lalith Sharan, Halvar Kelm, Gabriele Romano, Matthias Karck, Raffaele De Simone, Sandy Engelhardt

    Multi-point tracking is a challenging task that involves detecting points in the scene and tracking them across time. Here, metrics from Multi-object tracking (MOT) methods are shown to perform better than frame-based F-measures. The recently proposed HOTA metric, used for benchmarks such as the KITTI dataset, better evaluates the performance over metrics like MOTA, DetA, and IDF1. While HOTA takes into account temporal associations, it does not provide a tailored means to analyse the spatial associations of a dataset in a multi-camera setup. Moreover, there are differences in evaluating the detection task for points vs. objects (point distances vs. bounding box overlap). Therefore, we propose a multi-view higher-order tracking metric mvHOTA , to determine the accuracy of multi-point (multi-instance and multi-class) tracking methods while taking into account temporal and spatial associations. We demonstrate its use in evaluating the tracking performance on an endoscopic point detection dataset from a previously organised surgical data science challenge. Furthermore, we compare with other adjusted MOT metrics for this use-case, discuss the properties of mvHOTA, and show how the proposed multi-view Association and the Occlusion index (OI) facilitate analysis of methods with respect to handling of occlusions. The code is available at https://github.com/Cardio-AI/mvhota.


  • Open Access

    Comparison of Depth Estimation Setups from Stereo Endoscopy and Optical Tracking for Point Measurements

    by Lukas Burger, Lalith Sharan, Samantha Fischer, Julian Brand, Maximillian Hehl, Gabriele Romano, Matthias Karck, Raffaele De Simone, Ivo Wolf, Sandy Engelhardt

    To support minimally-invasive intraoperative mitral valve repair, quantitative measurements from the valve can be obtained using an infra-red tracked stylus. It is desirable to view such manually measured points together with the endoscopic image for further assistance. Therefore, hand-eye calibration is required that links both coordinate systems and is a prerequisite to project the points onto the image plane. A complementary approach to this is to use a vision-based endoscopic stereo-setup to detect and triangulate points of interest, to obtain the 3D coordinates. In this paper, we aim to compare both approaches on a rigid phantom and two patient-individual silicone replica which resemble the intraoperative scenario. The preliminary results indicate that 3D landmark estimation, either labeled manually or through partly automated detection with a deep learning approach, provides more accurate triangulated depth measurements when performed with a tailored image-based method than with stylus measurements.

  • Open Access

    Point detection through multi-instance deep heatmap regression for sutures in endoscopy

    by Lalith Sharan, Gabriele Romano, Julian Brand, Halvar Kelm, Matthias Karck, Raffaele De Simone, Sandy Engelhardt

    Purpose: Mitral valve repair is a complex minimally invasive surgery of the heart valve. In this context, suture detection from endoscopic images is a highly relevant task that provides quantitative information to analyse suturing patterns, assess prosthetic configurations and produce augmented reality visualisations. Facial or anatomical landmark detection tasks typically contain a fixed number of landmarks, and use regression or fixed heatmap-based approaches to localize the landmarks. However in endoscopy, there are a varying number of sutures in every image, and the sutures may occur at any location in the annulus, as they are not semantically unique. Method: In this work, we formulate the suture detection task as a multi-instance deep heatmap regression problem, to identify entry and exit points of sutures. We extend our previous work, and introduce the novel use of a 2D Gaussian layer followed by a differentiable 2D spatial Soft-Argmax layer to function as a local non-maximum suppression. Results: We present extensive experiments with multiple heatmap distribution functions and two variants of the proposed model. In the intra-operative domain, Variant 1 showed a mean $$F_1$$ F 1 of $$+ 0.0422$$ + 0.0422 over the baseline. Similarly, in the simulator domain, Variant 1 showed a mean $$F_1$$ F 1 of $$+ 0.0865$$ + 0.0865 over the baseline. Conclusion: The proposed model shows an improvement over the baseline in the intra-operative and the simulator domains. The data is made publicly available within the scope of the MICCAI AdaptOR2021 Challenge https://adaptor2021.github.io/, and the code at https://github.com/Cardio-AI/suture-detection-pytorch/.


  • Mutually Improved Endoscopic Image Synthesis and Landmark Detection in Unpaired Image-to-Image Translation

    by Lalith Sharan, Gabriele Romano, Sven Koehler, Halvar Kelm, Matthias Karck, Raffaele De Simone, Sandy Engelhardt

    The CycleGAN framework allows for unsupervised image-to-image translation of unpaired data. In a scenario of surgical training on a physical surgical simulator, this method can be used to transform endoscopic images of phantoms into images which more closely resemble the intra-operative appearance of the same surgical target structure. This can be viewed as a novel augmented reality approach, which we coined Hyperrealism in previous work. In this use case, it is of paramount importance to display objects like needles, sutures or instruments consistent in both domains while altering the style to a more tissue-like appearance. Segmentation of these objects would allow for a direct transfer, however, contouring of these, partly tiny and thin foreground objects is cumbersome and perhaps inaccurate. Instead, we propose to use landmark detection on the points when sutures pass into the tissue. This objective is directly incorporated into a CycleGAN framework by treating the performance of pre-trained detector models as an additional optimization goal. We show that a task defined on these sparse landmark labels improves consistency of synthesis by the generator network in both domains. Comparing a baseline CycleGAN architecture to our proposed extension (DetCycleGAN), mean precision (PPV) improved by $+61.32$, mean sensitivity (TPR) by $+37.91$, and mean $F_1$ score by $+0.4743$. Furthermore, it could be shown that by dataset fusion, generated intra-operative images can be leveraged as additional training data for the detection network itself.