11.6 KB
Newer Older
jenia jitsev's avatar
jenia jitsev committed
### Overview COVID X-Ray detection deep neural networks and open datasets

jenia jitsev's avatar
jenia jitsev committed
Author: Jenia Jitsev (initiated 29.04.2019) \
4 5
(Helmholtz AI Local "Information", Juelich Supercomputing Center (JSC))

jenia jitsev's avatar
jenia jitsev committed
6 7 8 9 10 11
Further contributors:

#### Relevant work and papers

General remark: Datasets available are either X-Ray scans or CT scans

12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30
##### COVID-Net: A Tailored Deep Convolutional Neural Network Design for Detection of COVID-19 Cases from Chest X-Ray Images

* paper:

The COVID-19 pandemic continues to have a devastating effect on the health and well-being of the global population. A critical step in the fight against COVID-19 is effective screening of infected patients, with one of the key screening approaches being radiological imaging using chest radiography. Motivated by this, a number of artificial intelligence (AI) systems based on deep learning have been proposed and results have been shown to be quite promising in terms of accuracy in detecting patients infected with COVID-19 using chest radiography images. However, to the best of the authors' knowledge, these developed AI systems have been closed source and unavailable to the research community for deeper understanding and extension, and unavailable for public access and use. Therefore, in this study we introduce COVID-Net, a deep convolutional neural network design tailored for the detection of COVID-19 cases from chest X-ray (CXR) images that is open source and available to the general public. We also describe the CXR dataset leveraged to train COVID-Net, which we will refer to as COVIDx and is comprised of 13,800 chest radiography images across 13,725 patient patient cases from three open access data repositories, one of which we introduced. Furthermore, we investigate how COVID-Net makes predictions using an explainability method in an attempt to gain deeper insights into critical factors associated with COVID cases, which can aid clinicians in improved screening. By no means a production-ready solution, the hope is that the open access COVID-Net, along with the description on constructing the open source COVIDx dataset, will be leveraged and build upon by both researchers and citizen data scientists alike to accelerate the development of highly accurate yet practical deep learning solutions for detecting COVID-19 cases and accelerate treatment of those who need it the most.

* University Waterloo, Waterloo Artificial Intelligence Institute, DarwinAI Corp, Canada
* Network architecture is not hand-made; generated via generative synthesis approach (variant of Neural Architecture Search), put forward by the same group, FermiNets :; related, by the same group, GenSynth :
    - "What is most interesting is that, once a generator has been learned through generative synthesis, it can be used to generate not just one but a large variety of different, unique highly efficient deep neural networks that satisfy operational requirements."

* code :

* Data used : COVIDx
    * COVIDx:
    * 16,756 chest radiography images across 13,645 patient cases
        - 121 COVID-19 patients (104 train, 14 test)
        - 183 COVID-19 X-Ray images in total (152 train, 31 test)
    * Created as a combination and modification of three open access data repositories
        - University Montreal Dataset (Joseph Paul Cohen) : COVID-19 Image Data Collection,;
jenia jitsev's avatar
jenia jitsev committed

        Project Summary: To build a public open dataset of chest X-ray and CT images of patients which are positive or suspected of COVID-19 or other viral and bacterial pneumonias (MERS, SARS, and ARDS.). Data will be collected from public sources as well as through indirect collection from hospitals and physicians. All images and data will be released publicly in this GitHub repo.
jenia jitsev's avatar
jenia jitsev committed
33 34 35 36

        Persons / instituions involved:

        - Joseph Paul Cohen. Postdoctoral Fellow, Mila, University of Montreal
        - University Waterloo, DarwinAI Corp Dataset (authors of the COVID-net paper):
jenia jitsev's avatar
jenia jitsev committed
38 39 40

        Related :
41 42 43 44
        - Data from Radiological  Society  of  North  America. RSNA  pneumonia  detection  challenge.

##### Towards an Efficient Deep Learning Model for COVID-19 Patterns Detection in X-ray Images
* paper:
jenia jitsev's avatar
jenia jitsev committed
* using EfficientNet Architecture - evolved by Neural Architecture Search (see below for details)
46 47 48 49 50 51 52 53

Confronting the pandemic of COVID-19 caused by the new coronavirus, the SARS-CoV-2, is nowadays one of the most prominent challenges of the human species. A key factor in slowing down the virus propagation is rapid diagnosis and isolation of infected patients. Nevertheless, the standard method for COVID-19 identification, the RT-PCR, is time-consuming and in short supply due to the pandemic.

Researchers around the world have been trying to find alternative screening methods. In this context, deep learning applied to chest X-rays of patients has been showing a lot of promise for the identification of COVID-19. Despite their success, the computational cost of these methods remains high, which imposes difficulties in their accessibility and availability. Thus, in this work, we address the hypothesis that better performance in terms of overall accuracy and COVID-19 sensitivity can be achieved with much more compact models. In order to test this hypothesis, we propose a modification of the EfficientNet family of models. By doing this we were able to produce a high-quality model with an overall accuracy of 91.4%, COVID-19, sensitivity of 90% and positive prediction of 100% while having about 30 times fewer parameters than the baseline model, 28 and 5 times fewer parameters than the popular VGG16 and ResNet50 architectures, respectively.

* UFOP, Computer Science Department, University of Outro Preto, Brazil
* beats COVID-Net, reaching 100% COVID positive recognition rate
* code :,
jenia jitsev's avatar
jenia jitsev committed
* Data: COVIDx (see above)
55 56 57 58 59 60 61
* Network architecture based on EfficientNet - derived from Neural Architecture Search by Google Brain group
    - EfficientNet: Rethinking Model Scaling for Convolutional Neural Networks,

    Convolutional Neural Networks (ConvNets) are commonly developed at a fixed resource budget, and then scaled up for better accuracy if more resources are available. In this paper, we systematically study model scaling and identify that carefully balancing network depth, width, and resolution can lead to better performance. Based on this observation, we propose a new scaling method that uniformly scales all dimensions of depth/width/resolution using a simple yet highly effective compound coefficient. We demonstrate the effectiveness of this method on scaling up MobileNets and ResNet.
    To go even further, we use neural architecture search to design a new baseline network and scale it up to obtain a family of models, called EfficientNets, which achieve much better accuracy and efficiency than previous ConvNets. In particular, our EfficientNet-B7 achieves state-of-the-art 84.4% top-1 / 97.1% top-5 accuracy on ImageNet, while being 8.4x smaller and 6.1x faster on inference than the best existing ConvNet. Our EfficientNets also transfer well and achieve state-of-the-art accuracy on CIFAR-100 (91.7%), Flowers (98.8%), and 3 other transfer learning datasets, with an order of magnitude fewer parameters. Source code is at this https URL.

    - code:
jenia jitsev's avatar
jenia jitsev committed
62 63 64
    - related  : EfficientDet

    Object detection and segmentation, multi-scale processing automatically balanced across network
65 66 67 68 69 70 71 72 73 74 75
    EfficientDet: Scalable and Efficient Object Detection

    Model efficiency has become increasingly important in computer vision. In this paper, we systematically study neural network architecture design choices for object detection and propose several key optimizations to improve efficiency. First, we propose a weighted bi-directional feature pyramid network (BiFPN), which allows easy and fast multi-scale feature fusion; Second, we propose a compound scaling method that uniformly scales the resolution, depth, and width for all backbone, feature network, and box/class prediction networks at the same time. Based on these optimizations and EfficientNet backbones, we have developed a new family of object detectors, called EfficientDet, which consistently achieve much better efficiency than prior art across a wide spectrum of resource constraints. In particular, with single-model and single-scale, our EfficientDet-D7 achieves state-of-the-art 52.2 AP on COCO test-dev with 52M parameters and 325B FLOPs, being 4x - 9x smaller and using 13x - 42x fewer FLOPs than previous detectors. Code is available at this https URL.

    Code:, \
    Also appeared at CVPR, 2020: Mingxing Tan, Ruoming Pang, Quoc V. Le. EfficientDet: Scalable and Efficient Object Detection. CVPR 2020. Arxiv link:

#####  Sample-Efficient Deep Learning for COVID-19 Diagnosis Based on CT Scans
* paper:
* University of California San Diego;
jenia jitsev's avatar
jenia jitsev committed
* CT Scan Images
* using contrastive loss for self-supervised learning; transfer learning using different strategies
jenia jitsev's avatar
jenia jitsev committed
78 79 80
* pre-training with different datasets: standard ImageNet; Lung Nodule Malignancy (LNM)
* code will be open-sourced according to the authors
  - UPDATE: code is available :
81 82 83 84
* dataset: UCSD
    - dataset paper:
    - COVID patients in total : 216 (Training : 130, Validation : 32, Test: 54)
jenia jitsev's avatar
jenia jitsev committed
    - COVID CT images total : 349 (Training : 191, Validation : 60, Test: 98)
    - low resolution (224x224)
87 88 89 90 91 92 93 94 95 96 97 98 99 100 101

#####  Chester, the AI Radiology Assistant (MILA Medical, MLMED, Montreal)

* Demo for an X-Ray based diagnostics (MILA Medical, MLMED)
* paper: Joseph Paul Cohen, Paul Bertin, Vincent Frappier. “Chester: A Web Delivered Locally Computed Chest X-Ray Disease Prediction System”. Jan. 2019
* Machine Learning and Medicine Lab (MLMED), MILA Medical (Montreal Institute for Learning Algorithms)
* Tool with Web Interface:
* Code:
* Core components already described:

"The system contains three main components":
  - **Out Of Distribution error** (*transferability / compatibility*): This is a heatmap showing how the image differs from our training data. If the heatmap is too bright then the image is very different from our training data and the model will likely not work. We will prevent an image from being processed if it is not similar enough to our training data in order to prevent errors in predictions.
  - **Predictive image regions** (*explainability*): The brighter each pixel is in the heatmap the more influence it can have on the predictions. If the color is bright it means that a change in these pixels will change the prediction.
  - **Disease Predictions** (*uncertainty* representation): A probability indicating how likely the image contains the disease. 50\% means the network is not sure.