UABDivulga - Barcelona recerca i innovació

The techniques of medical diagnosis based on images have had a great technological development in the last decades, achieving a quality and definition unthinkable in the past. However, whatever the technology used to obtain the images (X-rays, magnetic resonance, ultrasound), their interpretation, validation and discrimination between normality and illness is, ultimately, the responsibility of a professional who will apply their knowledge and experience, but with an inevitable component of subjectivity. Although the agreement among professionals in the interpretation of medical images is very high, this subjectivity based on the optical capabilities of the human eye and its vulnerability to visual illusions must be taken into account.

Ultrasound is currently the test of choice for the exploration of thyroid nodules. Although only one in 10 is malignant, they are so common in the general population (between 30 and 50%) that they become an important health problem in number and importance. Ultrasound determines the size and suspicious of malignancy characteristics of the nodule and, according to this, those that should be evaluated by cytology will be selected. Therefore, the careful evaluation of the characteristics of each nodule is of paramount importance and, for that reason, risk classifications have been developed that attribute to each nodule a certain probability of malignancy according to the sonographic findings. The most commonly used are the American Thyroid Association (ATA) classification, the classification of the American Association of Clinical Endocrinologists/American College of Endocrinology/Associazione Medici Endocrinologi (AACE / ACE / AME) and the European classification (EU-TIRADS).

For years now, computer image processing programs have been developed in the field of design and engineering that have led to the creation of the so-called CAD (computer-aided diagnosis) systems. They are programs that use mathematical algorithms to analyze images in an attempt to improve interpretation by reducing subjectivity. These CAD systems are based on different computer analysis technologies, such as the detection of shades of gray and their numerical transformation to analyze the contrast and also the latest artificial intelligence or machine learning technologies, more specifically artificial neural networks, in which a large number of images with the diagnosis of malignancy or benignity (from thousands to tens of thousands) are introduced in the system to create the diagnostic algorithm. This system allows to increase the library of images over time (learning). The vast majority of CAD programs applicable to thyroid ultrasound are in the process of being created or validated and, therefore, not available for clinical practice.

Our group of the Endocrinology and Nutrition Service of the Germans Trias i Pujol Hospital in Badalona (Department of Medicine, Autonomous University of Barcelona), had the possibility of validating in clinical practice the first commercially available CAD program. The company provided us with a limited version and we were able to analyze the images of a series of thyroid nodules in a completely autonomous and independent way, guaranteeing by previous contract that the results would be published integrally and without interference.

The system evaluated is called AmCAD-UT® (AmCAD Biomed, Taiwan). It is a program that without installation on the computer opens up a very intuitive workstation. When acquiring the image in a JEPG or DICOM format, it allows to delimit the nodulation in a semi-automatic way and subsequently analyzes and highlights the risk characteristics, such as the shape, the limits of the lesion, the internal structure, the vascularization or the presence of microcalcifications. Finally, it offers the risk of malignancy probability expressed in seven of the existing classifications.

We collected the images of 300 thyroid nodules that had been operated in our hospital: 135 with a definitive biopsy of cancer and 165 benign lesions. We recorded the probability of malignancy according to the ATA classification established before cytology and operation. Subsequently, these images were anonymized and analyzed using the AmCAD-UT system and the risk classification was established in accordance with the ATA, AACE/ACE/AME and EU-TIRADS classifications. Sensitivity (the ability of a test to detect a sick person), specificity (the ability to identify effectively people who are not sick), positive predictive value (probability that a person with a positive result does have the disease) and a negative predictive value (probability that a person with a negative result does not have the disease) according to the clinical expert and the CAD system were calculated. Finally, the area under the ROC curve was calculated and compared, which indicates the discriminatory capacity of the obtained results.

The results obtained showed that the evaluations carried out by the expert observer were very accurate, with a sensitivity of 87%, a specificity of 91.2%, a positive predictive value of 90.5% and a negative predictive value of 90.9%. When applying the CAD program, the best results were obtained when the ATA risk classification was used, with a sensitivity of 87% and a negative predictive value of 86.3%. However, the specificity (68.8%) and the positive predictive value (64.5%) were significantly lower. The calculation of the area under the ROC curve was 0.88 (when closer to 1.00, better discrimination) for the expert and 0.72 for the CAD system applying the ATA classification, being the difference statistically significant

Remarkably, when applying the ATA classification, the CAD system could not offer a probability of risk in 29% of the cases, considering them as indeterminate, so it was necessary to use some of the other classifications that showed a lower diagnostic efficiency.

This is the first validation study of a CAD system commercially available in ultrasound images for thyroid nodules. Our results show that sonographic exploration performed by an expert has a very good performance in the classification of risks. Regarding the program, its good sensitivity and its good negative predictive value make it acceptably reliable as a screening system to rule out malignant tumors, which could be useful in order to establish an initial system for evaluating cases of nodular thyroid disease in nearby primary care to the patient. It should be applied by technical personnel that could safely filter the cases that should be attended to a specialized level or that require a cytological examination. Other utilities of the CAD system could be support for diagnosis and as a learning tool for professionals interested in thyroid ultrasound.

In summary, the CAD program evaluated showed a diagnostic efficiency lower than that of an expert professional in the echographic evaluation of thyroid nodules. However, it can be a good screening tool and also for teaching purposes. It is expected that the development and improvement of algorithms will allow the utility and reliability of these systems to be expanded.