Classification of Cardiomyopathies from MR Cine Images Using Convolutional Neural Network with Transfer Learning
Abstract
:1. Introduction
2. Materials and Methods
2.1. Study Population
2.2. Cine-MR Acquisitions
2.3. Image Preparation
2.4. Deep Learning Process
2.5. Independent Reader Analysis
2.6. Saliency Maps
2.7. Evaluation and Statistical Analysis
3. Results
3.1. Classification According to the Four Orientation Planes
3.2. Classification According to the Pathology
3.3. Analysis of Misclassified Cases
3.4. Comparison with Human Reader Classification
3.5. Influence of Image Preparation Parameters on Classification Accuracy
3.6. Analysis of the Saliency Maps
4. Discussion
4.1. CNN Models
4.2. Importance of Data Preparation
4.3. Sources of Human Errors
4.4. Sources of Errors Related to the Algorithm
4.5. Limitations
4.6. Perspective
5. Conclusions
Author Contributions
Funding
Institutional Review Board Statement
Informed Consent Statement
Data Availability Statement
Conflicts of Interest
References
- Litjens, G.; Ciompi, F.; Wolterink, J.M.; de Vos, B.D.; Leiner, T.; Teuwen, J.; Isgum, I. State-of-the-Art Deep Learning in Cardiovascular Image Analysis. JACC Cardiovasc. Imaging 2019, 12, 1549–1565. [Google Scholar] [CrossRef] [PubMed]
- Liu, X.; Faes, L.; Kale, A.U.; Wagner, S.K.; Fu, D.J.; Bruynseels, A.; Mahendiran, T.; Moraes, G.; Shamdas, M.; Kern, C.; et al. A comparison of deep learning performance against health-care professionals in detecting diseases from medical imaging: A systematic review and meta-analysis. Lancet Digit. Health 2019, 1, e271–e297. [Google Scholar] [CrossRef]
- Martin-Isla, C.; Campello, V.M.; Izquierdo, C.; Raisi-Estabragh, Z.; Baebler, B.; Petersen, S.E.; Lekadir, K. Image-Based Cardiac Diagnosis With Machine Learning: A Review. Front Cardiovasc. Med. 2020, 7, 1. [Google Scholar] [CrossRef] [PubMed] [Green Version]
- Jiang, B.; Guo, N.; Ge, Y.; Zhang, L.; Oudkerk, M.; Xie, X. Development and application of artificial intelligence in cardiac imaging. Br. J. Radiol. 2020, 93, 1113. [Google Scholar] [CrossRef]
- Tan, L.K.; McLaughlin, R.A.; Lim, E.; Abdul Aziz, Y.F.; Liew, Y.M. Fully automated segmentation of the left ventricle in cine cardiac MRI using neural network regression. J. Magn. Reason. Imaging. 2018, 48, 140–152. [Google Scholar] [CrossRef] [PubMed]
- Chen, C.; Qin, C.; Qiu, H.; Tarroni, G.; Duan, J.; Bai, W.; Rueckert, D. Deep Learning for Cardiac Image Segmentation: A Review. Front Cardiovasc. Med. 2020, 7, 25. [Google Scholar] [CrossRef]
- Madani, A.; Ong, J.R.; Tibrewal, A.; Mofrad, M.R.K. Deep echocardiography: Data-efficient supervised and semi-supervised deep learning towards automated diagnosis of cardiac disease. NPJ Digit. Med. 2018, 1, 59. [Google Scholar] [CrossRef]
- Zhang, J.; Gajjala, S.; Agrawal, P.; Tison, G.H.; Hallock, L.A.; Beussink-Nelson, L.; Lassen, M.H.; Fan, E.; Aras, M.A.; Jordan, C.; et al. Fully Automated Echocardiogram Interpretation in Clinical Practice. Circulation 2018, 138, 1623–1635. [Google Scholar] [CrossRef]
- Betancur, J.; Commandeur, F.; Motlagh, M.; Sharir, T.; Einstein, A.J.; Bokhari, S.; Fish, M.B.; Ruddy, T.D.; Kaufmann, P.; Sinusas, A.J.; et al. Deep Learning for Prediction of Obstructive Disease From Fast Myocardial Perfusion SPECT: A Multicenter Study. J. Am. Coll. Cardiol. Img. 2018, 11, 1654–1663. [Google Scholar] [CrossRef]
- Togo, R.; Hirata, K.; Manabe, O.; Ohira, H.; Tsujino, I.; Magota, K.; Ogawa, T.; Haseyama, M.; Shiga, T. Cardiac sarcoidosis classification with deep convolutional neural network-based features using polar maps. Comput. Biol. Med. 2019, 104, 81–86. [Google Scholar] [CrossRef] [Green Version]
- Martini, N.; Aimo, A.; Barison, A.; Latta, D.D.; Vergaro, G.; Aquaro, G.D.; Ripoli, A.; Emdin, M.; Chiappino, D. Deep learning to diagnose cardiac amyloidosis from cardiovascular magnetic resonance. J. Cardiovasc. Magn. Reson. 2020, 22, 84. [Google Scholar] [CrossRef]
- Zhou, H.; Li, L.; Liu, Z.; Zhao, K.; Chen, X.; Lu, M.; Yin, G.; Song, L.; Zhao, S.; Zheng, H.; et al. Deep learning algorithm to improve hypertrophic cardiomyopathy mutation prediction using cardiac cine images. Eur. Radiol. 2020, 31, 3931–3940. [Google Scholar] [CrossRef]
- Patel, A.R.; Kramer, C.M. Role of Cardiac Magnetic Resonance in the Diagnosis and Prognosis of Nonischemic Cardiomyopathy. J. Am. Coll. Cardiol. Img. 2017, 10, 1180–1193. [Google Scholar] [CrossRef]
- Gersh, B.J.; Maron, B.J.; Bonow, R.O.; Dearani, J.A.; Fifer, M.A.; Link, M.S.; Naidu, S.S.; Nishimura, R.A.; Ommen, S.R.; Rakowski, H.; et al. 2011 ACCF/AHA guideline for the diagnosis and treatment of hypertrophic cardiomyopathy. Circulation 2011, 124, e783–831. [Google Scholar]
- Yeon, S.B.; Salton, C.J.; Gona, P.; Chuang, M.L.; Blease, S.J.; Han, Y.; Tsao, C.W.; Danias, P.G.; Levy, D.; O′Donnell, C.J.; et al. Impact of age, sex, and indexation method on MR left ventricular reference values in the Framingham Heart Study offspring cohort. J. Magn. Reason. Imaging 2015, 41, 1038–1045. [Google Scholar] [CrossRef] [PubMed] [Green Version]
- Kawel-Boehm, N.; Hetzel, S.J.; Ambale-Venkatesh, B.; Captur, G.; Francois, C.J.; Jerosch-Herold, M.; Salerno, M.; Teague, S.D.; Valsangiacomo-Beuchel, E.; van der Geest, R.J.; et al. Reference ranges (“normal values”) for cardiovascular magnetic resonance (CMR) in adults and children: 2020 update. J. Cardiovasc. Magn. Reson. 2020, 22, 87. [Google Scholar] [CrossRef] [PubMed]
- Chollet, F. Deep Learning with Python; Manning Publications: New York, NY, USA, 2017; Chapter 5; pp. 117–178. [Google Scholar]
- Simonyan, K.; Zisserman, A. Very Deep Convolutional Network for Large Scale Image Recognition. arXiv 2014, arXiv:1409.1556. [Google Scholar]
- Selvaraju, R.R.; Das, A.; Vedantam, R.; Cogswell, M.; Parikh, D.; Batra, D. Grad-CAM: Visual Explanations from Deep Networks via Gradient-Based Localization. Int. J. Comput. Vis. 2019, 128, 336–359. [Google Scholar] [CrossRef] [Green Version]
- Ho, N.; Kim, Y.C. Evaluation of transfer learning in deep convolutional neural network models for cardiac short axis slice classification. Sci. Rep. 2021, 11, 1839. [Google Scholar] [CrossRef]
- Kim, I.; Rajaraman, S.; Antani, S. Visual Interpretation of Convolutional Neural Network Predictions in Classifying Medical Image Modalities. Diagnostics 2019, 9, 38. [Google Scholar] [CrossRef] [Green Version]
- Geirhos, R.; Jacobsen, J.H.; Michaelis, C.; Zemel, R.; Brendel, W.; Bethge, M.; Wichmann, F.A. Shortcut Learning in Deep Neural Networks. Nat. Mach. Intell. 2020, 2, 665–673. https://arxiv.org/abs/2004.07780. [CrossRef]
- Oakden-Rayner, L.; Dunnmon, J.; Carneiro, G.; Ré, C. Hidden Stratification Causes Clinically Meaningful Failures in Machine Learning for Medical Imaging. In Proceedings of the ACM Conference of Health, Inference, and Learning, Toronto, ON, Canada, April 2020; https://arxiv.org/abs/1909.12475.
- Zheng, Q.; Delingette, H.; Ayache, N. Explainable cardiac pathology classification on cine MRI with motion characterization by semi-supervised learning of apparent flow. Med. Image Anal. 2019, 56, 80–95. [Google Scholar] [CrossRef] [PubMed] [Green Version]
- Yan, W.; Wang, Y.; van der Geest, R.J.; Tao, Q. Cine MRI analysis by deep learning of optical flow: Adding the temporal dimension. Comput. Biol. Med. 2019, 111, 103356. [Google Scholar] [CrossRef]
- Neisius, U.; El-Rewaidy, H.; Nakamori, S.; Rodriguez, J.; Manning, W.J.; Nezafat, R. Radiomic Analysis of Myocardial Native T1 Imaging Discriminates Between Hypertensive Heart Disease and Hypertrophic Cardiomyopathy. J. Am. Coll. Cardiol. Img. 2019, 12, 1946–1954. [Google Scholar] [CrossRef]
- Larroza, A.; Materka, A.; Lopez-Lereu, M.; Monmeneu, J.V.; Bodi, V.; Moratal, D. Differentiation between acute and chronic myocardial infarction by means of texture analysis of late gadolinium enhancement and cine cardiac magnetic resonance imaging. Eur. J. Radiol. 2017, 92, 78–83. [Google Scholar] [CrossRef]
- Shao, X.N.; Sun, Y.J.; Xiao, K.T.; Zhang, Y.; Zhang, W.-B.; Kou, Z.-F.; Cheng, J.-L. Texture analysis of magnetic resonance T1 mapping with dilated cardiomyopathy: A machine learning approach. Medicine 2018, 97, e12246. [Google Scholar] [CrossRef]
Normal | HCM | DCM | Total | p | |
---|---|---|---|---|---|
n patients | 209 | 175 | 150 | 534 | |
n frames | 395 | 411 | 394 | 1200 | |
Sex (F/M) | 148/247 | 112/299 | 103/291 | 363/837 | 0.0007 |
Age (years) | 45.6 ± 15.8 | 52.5 ± 17.8 | 56.4 ± 14.3 | 51.5 ± 16.7 | 0.01 |
VLAl | 39 | 39 | 66 | 144 | <0.0001 |
VLAr | 106 | 49 | 50 | 205 | |
4-chamber | 132 | 143 | 142 | 417 | |
Short axis | 118 | 180 | 136 | 434 | |
Systolic time | 329 ± 35 | 346 ± 39 | 337 ± 29 | 338 ± 33 | 0.001 |
Model | Frames | Classification of Orientation Planes (4 Classes) | Classification of Pathology (3 Classes) |
---|---|---|---|
VGG-single | diastole | 0.999 ± 0.002 (ns) | 0.961 ± 0.011 (p = 0.016) |
VGG-single | systole | 0.998 ± 0.002 (ns) | 0.952 ± 0.012 (p = 0.0092) |
VGG-concat | D + S | 0.999 ± 0.002 | 0.982 ± 0.009 |
VGG-Single Diastole | VGG-Single Systole | VGG-Concat (Diastole + Systole) | ||||||
---|---|---|---|---|---|---|---|---|
369 | 4 | 21 | 359 | 26 | 10 | 390 | 3 | 2 |
9 | 396 | 6 | 14 | 394 | 3 | 9 | 400 | 2 |
6 | 1 | 388 | 1 | 0 | 393 | 4 | 0 | 388 |
47/1200 misclassified inputs (3.92%) | 54/1200 misclassified inputs (4.50%) | 22/1200 misclassified inputs (1.83%) |
VGG-Single (S) | Average Validation Accuracy |
---|---|
128 × 128 | 0.954 ± 0.011 (ns) |
160 × 160 | 0.959 ± 0.009 |
256 × 256 | 0.921 ± 0.008 (p = 0.0001) |
Raw | 0.915 ± 0.007 (p = 0.0001) |
Publisher’s Note: MDPI stays neutral with regard to jurisdictional claims in published maps and institutional affiliations. |
© 2021 by the authors. Licensee MDPI, Basel, Switzerland. This article is an open access article distributed under the terms and conditions of the Creative Commons Attribution (CC BY) license (https://creativecommons.org/licenses/by/4.0/).
Share and Cite
Germain, P.; Vardazaryan, A.; Padoy, N.; Labani, A.; Roy, C.; Schindler, T.H.; El Ghannudi, S. Classification of Cardiomyopathies from MR Cine Images Using Convolutional Neural Network with Transfer Learning. Diagnostics 2021, 11, 1554. https://0-doi-org.brum.beds.ac.uk/10.3390/diagnostics11091554
Germain P, Vardazaryan A, Padoy N, Labani A, Roy C, Schindler TH, El Ghannudi S. Classification of Cardiomyopathies from MR Cine Images Using Convolutional Neural Network with Transfer Learning. Diagnostics. 2021; 11(9):1554. https://0-doi-org.brum.beds.ac.uk/10.3390/diagnostics11091554
Chicago/Turabian StyleGermain, Philippe, Armine Vardazaryan, Nicolas Padoy, Aissam Labani, Catherine Roy, Thomas Hellmut Schindler, and Soraya El Ghannudi. 2021. "Classification of Cardiomyopathies from MR Cine Images Using Convolutional Neural Network with Transfer Learning" Diagnostics 11, no. 9: 1554. https://0-doi-org.brum.beds.ac.uk/10.3390/diagnostics11091554