Susanne Kimeswenger, Philipp Tschandl, Petar Noack, Markus Hofmarcher, Elisabeth Rumetshofer, Harald Kindermann, Rene Silye, Sepp Hochreiter, Martin Kaltenbrunner, Emmanuella Guenova, Guenter Klambauer, and Wolfram Hoetzenecker

Attention weight matrices and eye tracking of BCCs.

Regions of interest according to attention weight matrices and eye tracking (pathologists).

Recent advances in artificial intelligence, particularly in the field of deep learning, have enabled researchers to create compelling algorithms for medical image analysis. Histological slides of basal cell carcinomas (BCCs), the most frequent skin tumor, are accessed by pathologists on a daily basis and are therefore well suited for automated prescreening by neural networks for the identification of cancerous regions and swift tumor classification.

In this proof-of-concept study, we implemented an accurate and intuitively interpretable artificial neural network (ANN) for the detection of BCCs in histological whole-slide images (WSIs). Furthermore, we identified and compared differences in the diagnostic histological features and recognition patterns relevant for machine learning algorithms vs. expert pathologists.

An attention-ANN was trained with WSIs of BCCs to identify tumor regions (n = 820). The diagnosis-relevant regions used by the ANN were compared to regions of interest for pathologists, detected by eye-tracking techniques.

This ANN accurately identified BCC tumor regions on images of histologic slides (area under the ROC curve: 0.993, 95% CI: 0.990–0.995; sensitivity: 0.965, 95% CI: 0.951–0.979; specificity: 0.910, 95% CI: 0.859–0.960). The ANN implicitly calculated a weight matrix, indicating the regions of a histological image that are important for the prediction of the network. Interestingly, compared to pathologists’ eye-tracking results, machine learning algorithms rely on significantly different recognition patterns for tumor identification (p < 10−4).

To conclude, we found on the example of BCC WSIs, that histopathological images can be efficiently and interpretably analyzed by state-of-the-art machine learning techniques. Neural networks and machine learning algorithms can potentially enhance diagnostic precision in digital pathology and uncover hitherto unused classification patterns.

E.G., G.K., and W.H. contributed equally.

Modern Pathology, 34, 5, 895–903, 2020-11-13.

View paper
IARAI Authors
Dr Sepp Hochreiter
Health and Well-being
Deep Learning, Image Classification


Imprint | Privacy Policy

Stay in the know with developments at IARAI

We can let you know if there’s any

updates from the Institute.
You can later also tailor your news feed to specific research areas or keywords (Privacy)

Log in with your credentials

Forgot your details?

Create Account