Indexed In : Science Citation Index Expanded(SCIE), MEDLINE,
Pubmed/Pubmed Central, Elsevier Bibliographic, Google Scholar,
Databases(Scopus & Embase), KCI, KoreaMed, DOAJ
Gut and Liver is an international journal of gastroenterology, focusing on the gastrointestinal tract, liver, biliary tree, pancreas, motility, and neurogastroenterology. Gut atnd Liver delivers up-to-date, authoritative papers on both clinical and research-based topics in gastroenterology. The Journal publishes original articles, case reports, brief communications, letters to the editor and invited review articles in the field of gastroenterology. The Journal is operated by internationally renowned editorial boards and designed to provide a global opportunity to promote academic developments in the field of gastroenterology and hepatology. +MORE
Yong Chan Lee |
Professor of Medicine Director, Gastrointestinal Research Laboratory Veterans Affairs Medical Center, Univ. California San Francisco San Francisco, USA |
Jong Pil Im | Seoul National University College of Medicine, Seoul, Korea |
Robert S. Bresalier | University of Texas M. D. Anderson Cancer Center, Houston, USA |
Steven H. Itzkowitz | Mount Sinai Medical Center, NY, USA |
All papers submitted to Gut and Liver are reviewed by the editorial team before being sent out for an external peer review to rule out papers that have low priority, insufficient originality, scientific flaws, or the absence of a message of importance to the readers of the Journal. A decision about these papers will usually be made within two or three weeks.
The remaining articles are usually sent to two reviewers. It would be very helpful if you could suggest a selection of reviewers and include their contact details. We may not always use the reviewers you recommend, but suggesting reviewers will make our reviewer database much richer; in the end, everyone will benefit. We reserve the right to return manuscripts in which no reviewers are suggested.
The final responsibility for the decision to accept or reject lies with the editors. In many cases, papers may be rejected despite favorable reviews because of editorial policy or a lack of space. The editor retains the right to determine publication priorities, the style of the paper, and to request, if necessary, that the material submitted be shortened for publication.
Kyeong Ok Kim1 , Eun Young Kim2
Correspondence to: Eun Young Kim
ORCID https://orcid.org/0000-0003-3965-9964
E-mail kimey@cu.ac.kr
This is an Open Access article distributed under the terms of the Creative Commons Attribution Non-Commercial License (http://creativecommons.org/licenses/by-nc/4.0) which permits unrestricted non-commercial use, distribution, and reproduction in any medium, provided the original work is properly cited.
Gut Liver 2021;15(3):346-353. https://doi.org/10.5009/gnl20186
Published online August 12, 2020, Published date May 15, 2021
Copyright © Gut and Liver.
Endoscpists always have tried to pursue a perfect colonoscopy, and application of artificial intelligence (AI) using deep-learning algorithms is one of the promising supportive options for detection and characterization of colorectal polyps during colonoscopy. Many retrospective studies conducted with real-time application of AI using convolutional neural networks have shown improved colorectal polyp detection. Moreover, a recent randomized clinical trial reported additional polyp detection with shorter analysis time. Studies conducted regarding polyp characterization provided additional promising results. Application of AI with narrow band imaging in real-time prediction of the pathology of diminutive polyps resulted in high diagnostic accuracy. In addition, application of AI with endocytoscopy or confocal laser endomicroscopy was investigated for realtime cellular diagnosis, and the diagnostic accuracy of some studies was comparable to that of pathologists. With AI technology, we can expect a higher polyp detection rate with reduced time and cost by avoiding unnecessary procedures, resulting in enhanced colonoscopy efficiency. However, for AI application in actual daily clinical practice, more prospective studies with minimized selection bias, consensus on standardized utilization, and regulatory approval are needed.
Keywords: Polyp, Colon, Colonoscopy, Artificial intelligence, Convolutional neural network
The incidence of colorectal cancer (CRC), the third most common cancer worldwide,1 has been steadily increasing in the Republic of Korea recently. Because most of the CRC arise from adenomas, detection and complete removal of these precancerous lesions can reduce the incidence and mortality associated with CRC.2,3 However, for the effective prevention of CRC, high-quality colonoscopy that detects all the polyps, is a prerequisite. Adenoma detection rate (ADR) has been considered as one of the important quality indicators of colonoscopy, and the inverse association between ADR and incidence of interval CRC has been established.4,5 Because colonoscopy may not be perfect, many efforts to enhance ADR have been made.6 Artificial intelligence (AI), one of the promising technologies, can mitigate the shortcomings of colonoscopy. As it is easy to obtain polyp images to provide enough data for AI training, many studies using this technology have been reported lately.7 Well-trained AI modelling can increase polyp identification and optical diagnosis. Especially, its use can increase the detection rate of polyps in the right colon, where a higher miss rate is understandably anticipated with conventional colonoscopy.8
In this review, we aim to contemplate the current status and future directions of AI applications, including technologies, applications, efficacy, and unmet needs, regarding colorectal neoplasm.
AI, having machine intelligence that is different from the natural intelligence displayed by humans and other animals,9 can learn and solve problems.10 Earlier, machine learning (ML) had been the main focus of AI. A computer-vision algorithm, developed by computer scientists based on earlier ML research for the detection of colorectal polyps was “hand-crafted,” based on the features adopted by the designers.11 In other words, human efforts or instructions were needed for extracting features, such as color, shape, or texture of polyps, because the earlier ML could learn only the classification of the extracted images.12 Although the “hand-crafted” algorithm showed high accuracy, the risk of missing the lesions without those extracted features or obtaining false positive results was of concern because it was designed to detect the lesions with certain features chosen by designers. In addition, actual clinical application was limited due to the difference in image quality and also slow processing time.11
Deep-learning (DL) algorithm, one of the subtype of ML introduced in the 1980s, could overcome the limitation of earlier ML by combining both the extraction and classification of image features using deep neural networks (DNN).13 The innovative method of DNN gained significant attention because of self-learning capability of DL that could automatically identify polyp and non-polyp features from the huge dataset, instead of capturing specific features of a polyp using several networks. Self-extraction, the key feature, was achieved using backpropagation algorithm and changing the internal parameters of each neural network layer.14
Among the variable classes of DNN for image and video applications, convolutional neural network (CNN), the most popular method, can carry out layers of convolutions and can completely connect layers to unite all features in the final outcome.15,16 Provided with sufficient annotated data, CNNs can be trained to describe, in detail, what they see and discriminate polyps from non-polypoid lesions.11 Currently, AI can be trained with enough input data, thanks to the easy accessibility of big data, aiding rapid progress in the research and application of AI in colonoscopic polyp detection and characterization.17
Since the initial study regarding computer-aided detection based on “hand-crafted” data,18 the performance of AI has improved, especially with the introduction of DL, for colorectal neoplasm detection.19,20 Table 1 shows the summary of the clinical studies of AI for the detection of colorectal polyps.18-26
Urban
Wang
Accurate histologic diagnosis of the colorectal polyp before resection is desired because endoscopic resection and pathologic evaluation of lesions with very low risk of malignant potential may result in waste of time and cost. On the other hand, incomplete resection of lesions with high risk of malignancy should be avoided. Therefore, various advanced endoscopic techniques with image enhancement for optical diagnosis have been introduced, and various studies concerning the application of AI to these endoscopic systems have also been steadily reported to date (Table 2).
Narrow band imaging (NBI; Olympus Corp., Tokyo, Japan), an image-enhanced endoscopy used for the observation of microstructure or capillaries of colorectal neoplasm, could discriminate each polyp well, based on the pit and vascular pattern.30 Tischendorf
Another Japanese group developed a real-time image recognition system that could achieve 93.2% accuracy in real-time prediction of diminutive polyp pathology.33 In addition, the follow-up recommendation based on the prediction by that model showed 92.7% consistency with Preservation and Incorporation of Valuable Endoscopic Innovations (PIVI)-2 criteria of American Society for Gastrointestinal Endoscopy for “resect-and-discard” strategy.34 Resect and discard strategy can bring a substantial economic benefit.
Two groups carried out retrospective studies based on DL. Byrne
Although there’s a lack of recent further studies, CAD of pit pattern by magnifying chromoendoscopy was done by quantitative analysis of pit structure or texture analysis of endoscopic images.37,38 Takemura
Recently, novel introduction of
AI application with confocal endomicroscopy was assessed in several studies, in experimental setting, with promising accuracy.47 50 AI in other advanced endoscopies, including laser-induced fluorescence spectroscopy and autofluorescence endoscopy, was also evaluated retrospectively or prospectively.51-57 We can expect that the use of AI in these advanced endoscopies for optical diagnosis could aid the real-time decision making of endoscopists.
For an endoscopist, both polyp detection and characterization are essential in clinical practice. An ideal scenario would be an AI-assisted immediate detection and characterization of the colorectal polyp. A Japanese group developed novel technologies that included two algorithm systems–one, based on DL algorithm, for the detection of polyps in white light images, and the other, for the prediction of pathology by endocytoscopic images generated by a photograph.58 According to the most recent study using CNN, AI system (Single Shot Multibox Detector) could detect 1,246 polyps with 92% sensitivity, 86% PPV, and 83% accuracy in polyp classification.59 Although more studies are needed, an ideal colonoscopy for the detection and characterization of colorectal neoplasm seems to be achieved with AI assistance.
Prior to DL, SVM was a highly efficient computational tool that classified and regressed best by optimizing a hyperplane with largest functional margin.15 Ichimasa
Although AI-assisted detection and diagnosis of colorectal neoplasm are promising, most of the studies are retrospective and covered lesions which might have been selected with bias. Well-designed prospective studies that present more reliable data compared to the previous retrospective studies are needed. Most of the studies regarding efficacy of AI deal with polypoid lesions. However, for practical utilization, the efficacy of AI needs to be consistent irrespective of the shape of the polyps. Therefore, more studies with all types of polyps, including polypoid, depressed, or flat type, are needed because these non-polypoid type lesions are often more aggressive.61
In addition, as AI is trained with high-quality images, the system has to overcome the blurry vision, inadequate preparation status, and variable unpredictable hurdles observed in actual practice. There is a need for real-time application of CAD and randomized controlled comparative study between usage and avoidance of AI.
Previous studies, conducted in a variable design, resulted in different primary outcomes. Because AI development needed both engineers, who dealt with the software, and clinicians, who contributed to the clinical use, the outcomes and study design could be different depending on the study conductors.11 Communication and collaboration between these different groups are also needed.
To apply AI in real clinical practice, a regulatory approval of AI-based decision making, the rules of which vary for each country, is an essential step. For that, we need to prove the minimal risk of treatment failure due to the misdiagnosis by AI.22,62 So, we need the evaluation of risk stratification for AI system through well-designed randomized clinical trials.
Because colonoscopy cannot completely prevent CRC, AI application in the field of colorectal neoplasm could be one of the promising options for enhancing the efficiency of colonosocpy. Owing to easy accessibility of big data and computer science, the AI technologies for the detection and classification of colorectal polyps have developed rapidly, with studies supporting the advantage of AI use in colonoscopy. However, obstacles, such as insufficient evidence of practical clinical usefulness, lack of consensus on standardized utilization, and need for regulatory approval, exist. For the ideal implementation of AI in actual clinical practice, comprehensive understanding of the strengths and weaknesses of the technology, qualified real-time studies, and accumulation of experience are warranted.
E.Y.K. is an editorial board member of the journal but did not involve in the peer reviewer selection, evaluation, or decision process of this article. No other potential conflicts of interest relevant to this article were reported.
Clinical Studies of Artificial Intelligence for the Detection of Colorectal Polyps
Author (year) | Study design | Algorithm type | Dataset | Processing time | Results |
---|---|---|---|---|---|
Wang (2019) 21 |
Randomized controlled study |
Convolutional neural network | 5,545 Images | 25 fps with 77 ms latency | 9% Increase of ADR |
Klare (2019) 22 |
Prospective |
Convolutional neural network | 55 Live colonoscopies | 50 ms latency | Sensitivity 75%/polyp ADR 29% (31% in endoscopist) |
Urban (2018) 20 |
Retrospective |
Convolutional neural network | Image dataset: 8,641 polyps Video: 20 colonoscopies |
10 ms/frame (real-time) |
Image dataset: accuracy 96.4% AUROC 0.991 |
Misawa (2018)19 |
Retrospective |
Convolutional neural network | 135 Video clips | No description | Sensitivity 90% Specificity 63.3% Accuracy 76.5% |
Zhang (2017) 23 |
Retrospective |
Convolutional neural network | 150 Random+30 NBI images | No description | Sensitivity 98% PPV 99% AUROC 1.00 |
Yu (2017) 24 |
Retrospective |
Convolutional neural network | ASU-Mayo 20 videos | 1.23 s/frame | Sensitivity 7% PPV 88% |
Angermann 25 |
Retrospective |
Hand-crafted | No description | 20–185 ms 0.3-1.8 s delay |
Sensitivity 100%/polyp PPV50% |
Tajbakhsh 26 |
Retrospective |
Hand-crafted | No description | 2.6 s/frame | Sensitivity 48% on proprietary database Sensitivity 88% in CVC-colon DB |
Karkanis 18 |
Retrospective |
Hand-crafted | 180 Still images | 1.5 m/video | Sensitivity 94% Specificity 99% |
ADR, adenoma detection rate; AUROC, area under the receiver operating characteristics; NBI, narrow band imaging; PPV, positive predictive value; ASU, Arizona State University; CVC, computer vision center; DB, data base.
Clinical Studies of Artificial Intelligence for Characterization of Colorectal Polyps
Author (year) | Study design | Classification target and base | Algorithm type | Image modality | Dataset | Results |
---|---|---|---|---|---|---|
Byrne 35 |
Retrospective | Histology of diminutive polyp | Convolutional neural network | NBI video frames | 125 Diminutive polyp videos | Sensitivity 98% Specificity 83% Accuracy 94% |
Chen 36 |
Retrospective | Neoplastic or hyperplastic polyp <5 mm | Convolutional neural network | Magnifying NBI | 284 Diminutive polyps image | Sensitivity 96.3% Specificity 78.1% Accuracy 90.1% |
Mori 44 |
Prospective | Diagnosis of neoplastic diminutive polyp | SVM | Endocytoscopy with NBI and stained images | 466 Diminutive polyps from 325 patients |
Prediction rate 98.1% |
Takeda 43 |
Retrospective | Invasive CRC | SVM | Endocytoscopy with NBI and stained images | 200 Images | Sensitivity 89.4% Specificity 98.9% Accuracy 94.1% |
Kominami 33 |
Prospective | Histology | SVM with logistic regression | Magnifying NBI | 118 Colorectal lesions | Sensitivity 95.9% Specificity 93.3% Accuracy 94.9% |
Misawa 46 |
Retrospective | Microvascular findings | SVM | Endocytoscopy with NBI | 100 Images | Sensitivity 84.5% Specificity 97.6% Accuracy 90.0% |
Mori 42 |
Retrospective | Neoplastic changes in small polyps | Multivariate regression analysis | Endocytoscopy | 176 Polyps from 152 patients | Sensitivity 92% Specificity 79.5% Accuracy 89.2% |
Takemura 57 |
Retrospective | Pit pattern | SVM | Magnifying NBI | 371 Images | Sensitivity 97.8% Specificity 97.9% Accuracy 97.8% |
Gross 32 |
Prospective | Small colonic polyp <10 mm | SVM | Magnifying NBI | 434 Polyps from 214 patients | Sensitivity 95% Specificity 90.3% Accuracy 93.1% |
Tischendorf 31 |
Prospective pilot |
Vascularization features | SVM | Magnifying NBI | 209 Polyps from 128 patients | Sensitivity 90% Specificity 70.2% Accuracy 85.3% |
Takemura 39 |
Retrospective | Pit pattern | HuPAS software version 1.3 | Magnifying NBI with chromoendoscopy (crystal violet) | 134 Images | Accuracy 98.5% |
NBI, narrow band imaging; SVM, support vector machine; CRC, colorectal cancer.
Gut and Liver 2021; 15(3): 346-353
Published online May 15, 2021 https://doi.org/10.5009/gnl20186
Copyright © Gut and Liver.
Kyeong Ok Kim1 , Eun Young Kim2
1Division of Gastroenterology and Hepatology, Department of Internal Medicine, Yeungnam University College of Medicine, and 2Division of Gastroenterology and Hepatology, Department of Internal Medicine, Daegu Catholic University School of Medicine, Daegu, Korea
Correspondence to:Eun Young Kim
ORCID https://orcid.org/0000-0003-3965-9964
E-mail kimey@cu.ac.kr
This is an Open Access article distributed under the terms of the Creative Commons Attribution Non-Commercial License (http://creativecommons.org/licenses/by-nc/4.0) which permits unrestricted non-commercial use, distribution, and reproduction in any medium, provided the original work is properly cited.
Endoscpists always have tried to pursue a perfect colonoscopy, and application of artificial intelligence (AI) using deep-learning algorithms is one of the promising supportive options for detection and characterization of colorectal polyps during colonoscopy. Many retrospective studies conducted with real-time application of AI using convolutional neural networks have shown improved colorectal polyp detection. Moreover, a recent randomized clinical trial reported additional polyp detection with shorter analysis time. Studies conducted regarding polyp characterization provided additional promising results. Application of AI with narrow band imaging in real-time prediction of the pathology of diminutive polyps resulted in high diagnostic accuracy. In addition, application of AI with endocytoscopy or confocal laser endomicroscopy was investigated for realtime cellular diagnosis, and the diagnostic accuracy of some studies was comparable to that of pathologists. With AI technology, we can expect a higher polyp detection rate with reduced time and cost by avoiding unnecessary procedures, resulting in enhanced colonoscopy efficiency. However, for AI application in actual daily clinical practice, more prospective studies with minimized selection bias, consensus on standardized utilization, and regulatory approval are needed.
Keywords: Polyp, Colon, Colonoscopy, Artificial intelligence, Convolutional neural network
The incidence of colorectal cancer (CRC), the third most common cancer worldwide,1 has been steadily increasing in the Republic of Korea recently. Because most of the CRC arise from adenomas, detection and complete removal of these precancerous lesions can reduce the incidence and mortality associated with CRC.2,3 However, for the effective prevention of CRC, high-quality colonoscopy that detects all the polyps, is a prerequisite. Adenoma detection rate (ADR) has been considered as one of the important quality indicators of colonoscopy, and the inverse association between ADR and incidence of interval CRC has been established.4,5 Because colonoscopy may not be perfect, many efforts to enhance ADR have been made.6 Artificial intelligence (AI), one of the promising technologies, can mitigate the shortcomings of colonoscopy. As it is easy to obtain polyp images to provide enough data for AI training, many studies using this technology have been reported lately.7 Well-trained AI modelling can increase polyp identification and optical diagnosis. Especially, its use can increase the detection rate of polyps in the right colon, where a higher miss rate is understandably anticipated with conventional colonoscopy.8
In this review, we aim to contemplate the current status and future directions of AI applications, including technologies, applications, efficacy, and unmet needs, regarding colorectal neoplasm.
AI, having machine intelligence that is different from the natural intelligence displayed by humans and other animals,9 can learn and solve problems.10 Earlier, machine learning (ML) had been the main focus of AI. A computer-vision algorithm, developed by computer scientists based on earlier ML research for the detection of colorectal polyps was “hand-crafted,” based on the features adopted by the designers.11 In other words, human efforts or instructions were needed for extracting features, such as color, shape, or texture of polyps, because the earlier ML could learn only the classification of the extracted images.12 Although the “hand-crafted” algorithm showed high accuracy, the risk of missing the lesions without those extracted features or obtaining false positive results was of concern because it was designed to detect the lesions with certain features chosen by designers. In addition, actual clinical application was limited due to the difference in image quality and also slow processing time.11
Deep-learning (DL) algorithm, one of the subtype of ML introduced in the 1980s, could overcome the limitation of earlier ML by combining both the extraction and classification of image features using deep neural networks (DNN).13 The innovative method of DNN gained significant attention because of self-learning capability of DL that could automatically identify polyp and non-polyp features from the huge dataset, instead of capturing specific features of a polyp using several networks. Self-extraction, the key feature, was achieved using backpropagation algorithm and changing the internal parameters of each neural network layer.14
Among the variable classes of DNN for image and video applications, convolutional neural network (CNN), the most popular method, can carry out layers of convolutions and can completely connect layers to unite all features in the final outcome.15,16 Provided with sufficient annotated data, CNNs can be trained to describe, in detail, what they see and discriminate polyps from non-polypoid lesions.11 Currently, AI can be trained with enough input data, thanks to the easy accessibility of big data, aiding rapid progress in the research and application of AI in colonoscopic polyp detection and characterization.17
Since the initial study regarding computer-aided detection based on “hand-crafted” data,18 the performance of AI has improved, especially with the introduction of DL, for colorectal neoplasm detection.19,20 Table 1 shows the summary of the clinical studies of AI for the detection of colorectal polyps.18-26
Urban
Wang
Accurate histologic diagnosis of the colorectal polyp before resection is desired because endoscopic resection and pathologic evaluation of lesions with very low risk of malignant potential may result in waste of time and cost. On the other hand, incomplete resection of lesions with high risk of malignancy should be avoided. Therefore, various advanced endoscopic techniques with image enhancement for optical diagnosis have been introduced, and various studies concerning the application of AI to these endoscopic systems have also been steadily reported to date (Table 2).
Narrow band imaging (NBI; Olympus Corp., Tokyo, Japan), an image-enhanced endoscopy used for the observation of microstructure or capillaries of colorectal neoplasm, could discriminate each polyp well, based on the pit and vascular pattern.30 Tischendorf
Another Japanese group developed a real-time image recognition system that could achieve 93.2% accuracy in real-time prediction of diminutive polyp pathology.33 In addition, the follow-up recommendation based on the prediction by that model showed 92.7% consistency with Preservation and Incorporation of Valuable Endoscopic Innovations (PIVI)-2 criteria of American Society for Gastrointestinal Endoscopy for “resect-and-discard” strategy.34 Resect and discard strategy can bring a substantial economic benefit.
Two groups carried out retrospective studies based on DL. Byrne
Although there’s a lack of recent further studies, CAD of pit pattern by magnifying chromoendoscopy was done by quantitative analysis of pit structure or texture analysis of endoscopic images.37,38 Takemura
Recently, novel introduction of
AI application with confocal endomicroscopy was assessed in several studies, in experimental setting, with promising accuracy.47 50 AI in other advanced endoscopies, including laser-induced fluorescence spectroscopy and autofluorescence endoscopy, was also evaluated retrospectively or prospectively.51-57 We can expect that the use of AI in these advanced endoscopies for optical diagnosis could aid the real-time decision making of endoscopists.
For an endoscopist, both polyp detection and characterization are essential in clinical practice. An ideal scenario would be an AI-assisted immediate detection and characterization of the colorectal polyp. A Japanese group developed novel technologies that included two algorithm systems–one, based on DL algorithm, for the detection of polyps in white light images, and the other, for the prediction of pathology by endocytoscopic images generated by a photograph.58 According to the most recent study using CNN, AI system (Single Shot Multibox Detector) could detect 1,246 polyps with 92% sensitivity, 86% PPV, and 83% accuracy in polyp classification.59 Although more studies are needed, an ideal colonoscopy for the detection and characterization of colorectal neoplasm seems to be achieved with AI assistance.
Prior to DL, SVM was a highly efficient computational tool that classified and regressed best by optimizing a hyperplane with largest functional margin.15 Ichimasa
Although AI-assisted detection and diagnosis of colorectal neoplasm are promising, most of the studies are retrospective and covered lesions which might have been selected with bias. Well-designed prospective studies that present more reliable data compared to the previous retrospective studies are needed. Most of the studies regarding efficacy of AI deal with polypoid lesions. However, for practical utilization, the efficacy of AI needs to be consistent irrespective of the shape of the polyps. Therefore, more studies with all types of polyps, including polypoid, depressed, or flat type, are needed because these non-polypoid type lesions are often more aggressive.61
In addition, as AI is trained with high-quality images, the system has to overcome the blurry vision, inadequate preparation status, and variable unpredictable hurdles observed in actual practice. There is a need for real-time application of CAD and randomized controlled comparative study between usage and avoidance of AI.
Previous studies, conducted in a variable design, resulted in different primary outcomes. Because AI development needed both engineers, who dealt with the software, and clinicians, who contributed to the clinical use, the outcomes and study design could be different depending on the study conductors.11 Communication and collaboration between these different groups are also needed.
To apply AI in real clinical practice, a regulatory approval of AI-based decision making, the rules of which vary for each country, is an essential step. For that, we need to prove the minimal risk of treatment failure due to the misdiagnosis by AI.22,62 So, we need the evaluation of risk stratification for AI system through well-designed randomized clinical trials.
Because colonoscopy cannot completely prevent CRC, AI application in the field of colorectal neoplasm could be one of the promising options for enhancing the efficiency of colonosocpy. Owing to easy accessibility of big data and computer science, the AI technologies for the detection and classification of colorectal polyps have developed rapidly, with studies supporting the advantage of AI use in colonoscopy. However, obstacles, such as insufficient evidence of practical clinical usefulness, lack of consensus on standardized utilization, and need for regulatory approval, exist. For the ideal implementation of AI in actual clinical practice, comprehensive understanding of the strengths and weaknesses of the technology, qualified real-time studies, and accumulation of experience are warranted.
E.Y.K. is an editorial board member of the journal but did not involve in the peer reviewer selection, evaluation, or decision process of this article. No other potential conflicts of interest relevant to this article were reported.
Clinical Studies of Artificial Intelligence for the Detection of Colorectal Polyps
Author (year) | Study design | Algorithm type | Dataset | Processing time | Results |
---|---|---|---|---|---|
Wang (2019) 21 |
Randomized controlled study |
Convolutional neural network | 5,545 Images | 25 fps with 77 ms latency | 9% Increase of ADR |
Klare (2019) 22 |
Prospective |
Convolutional neural network | 55 Live colonoscopies | 50 ms latency | Sensitivity 75%/polyp ADR 29% (31% in endoscopist) |
Urban (2018) 20 |
Retrospective |
Convolutional neural network | Image dataset: 8,641 polyps Video: 20 colonoscopies |
10 ms/frame (real-time) |
Image dataset: accuracy 96.4% AUROC 0.991 |
Misawa (2018)19 |
Retrospective |
Convolutional neural network | 135 Video clips | No description | Sensitivity 90% Specificity 63.3% Accuracy 76.5% |
Zhang (2017) 23 |
Retrospective |
Convolutional neural network | 150 Random+30 NBI images | No description | Sensitivity 98% PPV 99% AUROC 1.00 |
Yu (2017) 24 |
Retrospective |
Convolutional neural network | ASU-Mayo 20 videos | 1.23 s/frame | Sensitivity 7% PPV 88% |
Angermann 25 |
Retrospective |
Hand-crafted | No description | 20–185 ms 0.3-1.8 s delay |
Sensitivity 100%/polyp PPV50% |
Tajbakhsh 26 |
Retrospective |
Hand-crafted | No description | 2.6 s/frame | Sensitivity 48% on proprietary database Sensitivity 88% in CVC-colon DB |
Karkanis 18 |
Retrospective |
Hand-crafted | 180 Still images | 1.5 m/video | Sensitivity 94% Specificity 99% |
ADR, adenoma detection rate; AUROC, area under the receiver operating characteristics; NBI, narrow band imaging; PPV, positive predictive value; ASU, Arizona State University; CVC, computer vision center; DB, data base.
Clinical Studies of Artificial Intelligence for Characterization of Colorectal Polyps
Author (year) | Study design | Classification target and base | Algorithm type | Image modality | Dataset | Results |
---|---|---|---|---|---|---|
Byrne 35 |
Retrospective | Histology of diminutive polyp | Convolutional neural network | NBI video frames | 125 Diminutive polyp videos | Sensitivity 98% Specificity 83% Accuracy 94% |
Chen 36 |
Retrospective | Neoplastic or hyperplastic polyp <5 mm | Convolutional neural network | Magnifying NBI | 284 Diminutive polyps image | Sensitivity 96.3% Specificity 78.1% Accuracy 90.1% |
Mori 44 |
Prospective | Diagnosis of neoplastic diminutive polyp | SVM | Endocytoscopy with NBI and stained images | 466 Diminutive polyps from 325 patients |
Prediction rate 98.1% |
Takeda 43 |
Retrospective | Invasive CRC | SVM | Endocytoscopy with NBI and stained images | 200 Images | Sensitivity 89.4% Specificity 98.9% Accuracy 94.1% |
Kominami 33 |
Prospective | Histology | SVM with logistic regression | Magnifying NBI | 118 Colorectal lesions | Sensitivity 95.9% Specificity 93.3% Accuracy 94.9% |
Misawa 46 |
Retrospective | Microvascular findings | SVM | Endocytoscopy with NBI | 100 Images | Sensitivity 84.5% Specificity 97.6% Accuracy 90.0% |
Mori 42 |
Retrospective | Neoplastic changes in small polyps | Multivariate regression analysis | Endocytoscopy | 176 Polyps from 152 patients | Sensitivity 92% Specificity 79.5% Accuracy 89.2% |
Takemura 57 |
Retrospective | Pit pattern | SVM | Magnifying NBI | 371 Images | Sensitivity 97.8% Specificity 97.9% Accuracy 97.8% |
Gross 32 |
Prospective | Small colonic polyp <10 mm | SVM | Magnifying NBI | 434 Polyps from 214 patients | Sensitivity 95% Specificity 90.3% Accuracy 93.1% |
Tischendorf 31 |
Prospective pilot |
Vascularization features | SVM | Magnifying NBI | 209 Polyps from 128 patients | Sensitivity 90% Specificity 70.2% Accuracy 85.3% |
Takemura 39 |
Retrospective | Pit pattern | HuPAS software version 1.3 | Magnifying NBI with chromoendoscopy (crystal violet) | 134 Images | Accuracy 98.5% |
NBI, narrow band imaging; SVM, support vector machine; CRC, colorectal cancer.
Table 1 Clinical Studies of Artificial Intelligence for the Detection of Colorectal Polyps
Author (year) | Study design | Algorithm type | Dataset | Processing time | Results |
---|---|---|---|---|---|
Wang (2019) 21 | Randomized controlled study | Convolutional neural network | 5,545 Images | 25 fps with 77 ms latency | 9% Increase of ADR |
Klare (2019) 22 | Prospective | Convolutional neural network | 55 Live colonoscopies | 50 ms latency | Sensitivity 75%/polyp ADR 29% (31% in endoscopist) |
Urban (2018) 20 | Retrospective | Convolutional neural network | Image dataset: 8,641 polyps Video: 20 colonoscopies | 10 ms/frame (real-time) | Image dataset: accuracy 96.4% AUROC 0.991 |
Misawa (2018)19 | Retrospective | Convolutional neural network | 135 Video clips | No description | Sensitivity 90% Specificity 63.3% Accuracy 76.5% |
Zhang (2017) 23 | Retrospective | Convolutional neural network | 150 Random+30 NBI images | No description | Sensitivity 98% PPV 99% AUROC 1.00 |
Yu (2017) 24 | Retrospective | Convolutional neural network | ASU-Mayo 20 videos | 1.23 s/frame | Sensitivity 7% PPV 88% |
Angermann 25 | Retrospective | Hand-crafted | No description | 20–185 ms 0.3-1.8 s delay | Sensitivity 100%/polyp PPV50% |
Tajbakhsh 26 | Retrospective | Hand-crafted | No description | 2.6 s/frame | Sensitivity 48% on proprietary database Sensitivity 88% in CVC-colon DB |
Karkanis 18 | Retrospective | Hand-crafted | 180 Still images | 1.5 m/video | Sensitivity 94% Specificity 99% |
ADR, adenoma detection rate; AUROC, area under the receiver operating characteristics; NBI, narrow band imaging; PPV, positive predictive value; ASU, Arizona State University; CVC, computer vision center; DB, data base.
Table 2 Clinical Studies of Artificial Intelligence for Characterization of Colorectal Polyps
Author (year) | Study design | Classification target and base | Algorithm type | Image modality | Dataset | Results |
---|---|---|---|---|---|---|
Byrne 35 | Retrospective | Histology of diminutive polyp | Convolutional neural network | NBI video frames | 125 Diminutive polyp videos | Sensitivity 98% Specificity 83% Accuracy 94% |
Chen 36 | Retrospective | Neoplastic or hyperplastic polyp <5 mm | Convolutional neural network | Magnifying NBI | 284 Diminutive polyps image | Sensitivity 96.3% Specificity 78.1% Accuracy 90.1% |
Mori 44 | Prospective | Diagnosis of neoplastic diminutive polyp | SVM | Endocytoscopy with NBI and stained images | 466 Diminutive polyps from 325 patients | Prediction rate 98.1% |
Takeda 43 | Retrospective | Invasive CRC | SVM | Endocytoscopy with NBI and stained images | 200 Images | Sensitivity 89.4% Specificity 98.9% Accuracy 94.1% |
Kominami 33 | Prospective | Histology | SVM with logistic regression | Magnifying NBI | 118 Colorectal lesions | Sensitivity 95.9% Specificity 93.3% Accuracy 94.9% |
Misawa 46 | Retrospective | Microvascular findings | SVM | Endocytoscopy with NBI | 100 Images | Sensitivity 84.5% Specificity 97.6% Accuracy 90.0% |
Mori 42 | Retrospective | Neoplastic changes in small polyps | Multivariate regression analysis | Endocytoscopy | 176 Polyps from 152 patients | Sensitivity 92% Specificity 79.5% Accuracy 89.2% |
Takemura 57 | Retrospective | Pit pattern | SVM | Magnifying NBI | 371 Images | Sensitivity 97.8% Specificity 97.9% Accuracy 97.8% |
Gross 32 | Prospective | Small colonic polyp <10 mm | SVM | Magnifying NBI | 434 Polyps from 214 patients | Sensitivity 95% Specificity 90.3% Accuracy 93.1% |
Tischendorf 31 | Prospective pilot | Vascularization features | SVM | Magnifying NBI | 209 Polyps from 128 patients | Sensitivity 90% Specificity 70.2% Accuracy 85.3% |
Takemura 39 | Retrospective | Pit pattern | HuPAS software version 1.3 | Magnifying NBI with chromoendoscopy (crystal violet) | 134 Images | Accuracy 98.5% |
NBI, narrow band imaging; SVM, support vector machine; CRC, colorectal cancer.