Qiang Li

Accurate segmentation of pulmonary nodules in computed tomography (CT) is an important and difficult task for computer‐aided diagnosis of lung cancer. Therefore, the authors developed a novel automated method for accurate segmentation of nodules in three‐dimensional (3D) CT. First, a volume of interest (VOI) was determined at the location of a nodule. To simplify nodule segmentation, the 3D VOI was transformed into a two‐dimensional (2D) image by use of a key “spiral‐scanning” technique, in which a number of radial lines originating from the center of the VOI spirally scanned the VOI from the “north pole” to the “south pole.” The voxels scanned by the radial lines provided a transformed 2D image. Because the surface of a nodule in the 3D image became a curve in the transformed 2D image, the spiral‐scanning technique considerably simplified the segmentation method and enabled reliable segmentation results to be obtained. A dynamic programming technique was employed to delineate the “optimal” outline of a nodule in the 2D image, which corresponded to the surface of the nodule in the 3D image. The optimal outline was then transformed back into 3D image space to provide the surface of the nodule. An overlap between nodule regions provided by computer and by the radiologists was employed as a performance metric for evaluating the segmentation method. The database included two Lung Imaging Database Consortium (LIDC) data sets that contained 23 and 86 CT scans, respectively, with 23 and 73 nodules that were 3 mm or larger in diameter. For the two data sets, six and four radiologists manually delineated the outlines of the nodules as reference standards in a performance evaluation for nodule segmentation. The segmentation method was trained on the first and was tested on the second LIDC data sets. The mean overlap values were and for the nodules in the first and second LIDC data sets, respectively, which represented a higher performance level than those of two existing segmentation methods that were also evaluated by use of the LIDC data sets. The segmentation method provided relatively reliable results for pulmonary nodule segmentation and would be useful for lung cancer quantification, detection, and diagnosis.

Computer-aided diagnostic (CAD) schemes have been developed for assisting radiologists in the detection of various lesions in medical images. Many evaluation approaches, such as the resubstitution, leave-one-out, cross-validation, and hold-out methods, have been employed for the assessment of the performance of various CAD schemes. For these evaluation methods, some investigators have studied their bias in the estimated performance levels of CAD schemes trained with finite samples. However, systematical study has not been conducted for the comparison of these common evaluation methods in terms of multiple important characteristics such as the bias of the estimated performance, the generalization performance, and the uniqueness of the trained CAD scheme. Therefore, in this study, we examined and compared these important characteristics for various evaluation methods and attempted to provide a guideline for investigators to select appropriate evaluation methods for the assessment of CAD schemes in typical practical situations.

Computer‐aided diagnostic (CAD) schemes have been developed to assist radiologists in detecting various lesions in medical images. In addition to the development, an equally important problem is the reliable evaluation of the performance levels of various CAD schemes. It is good to see that more and more investigators are employing more reliable evaluation methods such as leave‐one‐out and cross validation, instead of less reliable methods such as resubstitution, for assessing their CAD schemes. However, the common applications of leave‐one‐out and cross‐validation evaluation methods do not necessarily imply that the estimated performance levels are accurate and precise. Pitfalls often occur in the use of leave‐one‐out and cross‐validation evaluation methods, and they lead to unreliable estimation of performance levels. In this study, we first identified a number of typical pitfalls for the evaluation of CAD schemes, and conducted a Monte Carlo simulation experiment for each of the pitfalls to demonstrate quantitatively the extent of bias and/or variance caused by the pitfall. Our experimental results indicate that considerable bias and variance may exist in the estimated performance levels of CAD schemes if one employs various flawed leave‐one‐out and cross‐validation evaluation methods. In addition, for promoting and utilizing a high standard for reliable evaluation of CAD schemes, we attempt to make recommendations, whenever possible, for overcoming these pitfalls. We believe that, with the recommended evaluation methods, we can considerably reduce the bias and variance in the estimated performance levels of CAD schemes.

Computer‐aided diagnostic (CAD) schemes have been developed to assist radiologists detect various lesions in medical images. In CAD schemes, classifiers play a key role in achieving a high lesion detection rate and a low false‐positive rate. Although many popular classifiers such as linear discriminant analysis and artificial neural networks have been employed in CAD schemes for reduction of false positives, a rule‐based classifier has probably been the simplest and most frequently used one since the early days of development of various CAD schemes. However, with existing rule‐based classifiers, there are major disadvantages that significantly reduce their practicality and credibility. The disadvantages include manual design, poor reproducibility, poor evaluation methods such as resubstitution, and a large overtraining effect. An automated rule‐based classifier with a minimized overtraining effect can overcome or significantly reduce the extent of the above‐mentioned disadvantages. In this study, we developed an “optimal” method for the selection of cutoff thresholds and a fully automated rule‐based classifier. Experimental results performed with Monte Carlo simulation and a real lung nodule CT data set demonstrated that the automated threshold selection method can completely eliminate overtraining effect in the procedure of cutoff threshold selection, and thus can minimize overall overtraining effect in the constructed rule‐based classifier. We believe that this threshold selection method is very useful in the construction of automated rule‐based classifiers with minimized overtraining effect.

We have been developing a computerized scheme to assist radiologists in improving the diagnostic accuracy for lung cancers on low-dose computed tomography (LDCT) scans by use of similar images for malignant nodules and benign nodules. A database of 415 LDCT scans including 73 cases with 76 confirmed cancers and 342 cases with 413 confirmed benign nodules was first collected in an LDCT screening program for early detection of lung cancers in Nagano, Japan. An observer study by use of receiver operating characteristics analysis was first conducted with five radiologists to demonstrate that presenting similar images for malignant nodules and benign nodules can significantly improve radiologists' performance in the diagnosis of unknown nodules. Another observer study was then conducted for obtaining reliable data on subjective similarity ratings by 10 radiologists. Based on the subjective similarity ratings, three important features were selected from a number of nodule features, and four different techniques for the determination of similarity measures, namely, a feature-based technique, a pixel-value-difference based technique, a cross-correlation-based technique, and a neural-network-based technique, were investigated and evaluated in terms of the correlation coefficient with the subjective similarity ratings. The experimental results in this study indicated that the neural-network-based technique can provide a reliable psychophysical similarity measure which is comparable to the subjective similarity ratings for a single radiologist when evaluated by use of correlation with the average similarity ratings for the other nine radiologists.