Learning Deconvolution Network for Semantic Segmentation

Graph Neural Networks (GNNs) have emerged as a powerful and flexible framework for representation learning on irregular data. As they generalize the operations of classical CNNs on grids to arbitrary topologies, GNNs also bring much of the implementation challenges of their Euclidean counterparts. Model size, memory footprint, and energy consumption are common concerns for many real-world applications. Network binarization allocates a single bit to parameters and activations, thus dramatically reducing the memory requirements (up to 32x compared to single-precision floating-point numbers) and maximizing the benefits of fast SIMD instructions on modern hardware for measurable speedups. However, in spite of the large body of work on binarization for classical CNNs, this area remains largely unexplored in geometric deep learning. In this paper, we present and evaluate different strategies for the binarization of graph neural networks. We show that through careful design of the models, and control of the training process, binary graph neural networks can be trained at only a moderate cost in accuracy on challenging benchmarks. In particular, we present the first dynamic graph neural network in Hamming space, able to leverage efficient k-NN search on binary vectors to speed-up the construction of the dynamic graph. We further verify that the binary models offer significant savings on embedded devices. Our code is publicly available on Github1.

/pdf/binary-graph-neural-networks-h1f45f6kew.pdf

Binary Graph Neural Networks

Graph neural networks (GNNs) have emerged as a series of competent graph learning methods for diverse real-world scenarios, ranging from daily applications like recommendation systems and question answering to cutting-edge technologies such as drug discovery in life sciences and n-body simulation in astrophysics. However, task performance is not the only requirement for GNNs. Performance-oriented GNNs have exhibited potential adverse effects like vulnerability to adversarial attacks, unexplainable discrimination against disadvantaged groups, or excessive resource consumption in edge computing environments. To avoid these unintentional harms, it is necessary to build competent GNNs characterised by trustworthiness. To this end, we propose a comprehensive roadmap to build trustworthy GNNs from the view of the various computing technologies involved. In this survey, we introduce basic concepts and comprehensively summarise existing efforts for trustworthy GNNs from six aspects, including robustness, explainability, privacy, fairness, accountability, and environmental well-being. Additionally, we highlight the intricate cross-aspect relations between the above six aspects of trustworthy GNNs. Finally, we present a thorough overview of trending directions for facilitating the research and industrialisation of trustworthy GNNs.

Trustworthy Graph Neural Networks: Aspects, Methods and Trends

The U-Net architecture is a state-of-the-art neural network for semantic image segmentation that is widely used in biomedical research. It is based on an encoder-decoder framework and its vanilla version shows already high performance in terms of segmentation quality. Due to its large parameter space, however, it has high computational costs on both, CPUs and GPUs. In a research setting, inference time is relevant, but not crucial for the results. However, especially in mobile, clinical applications a light and fast variant would allow deep-learning assisted, objective diagnosis at the point of care. In this work, we suggest an optimized, tiny-weight U-Net for an inexpensive hardware accelerator. We first mined the U-Net architecture to reduce computational complexity to increase runtime performance by simultaneously keeping the accuracy on a high level. Using an open, biomedical dataset for high-speed videoendoscopy (BAGLS), we show that we can dramatically reduce the parameter space and computations by over 99.8% while keeping the segmentation performance at 95% of our baseline. Using a custom upscaling routine, we further successfully deployed our optimized U-Net to an EdgeTPU hardware accelerator to gain cost-effective speed improvements on conventional computers and to showcase the applicability of EdgeTPUs for biomedical imaging processing of large images on portable devices. Combining the optimized architecture and the EdgeTPU, we gain a speedup of >79-times compared to our initial baseline while keeping high accuracy. This combination allows to provide immediate results to the clinician, especially in constrained computational environments, and an objective diagnosis at the point of care.

/pdf/efficient-biomedical-image-segmentation-on-edgetpus-at-point-48u3ld7o03.pdf

Efficient Biomedical Image Segmentation on EdgeTPUs at Point of Care

Timely detection of flooding is paramount for saving lives as well as evaluating levels of damage. Floods generally occur under specific weather conditions, such as excessive precipitation, which makes the presence of clouds very likely. For this reason, radar-based sensors are most suitable for near-real-time flood mapping. The public dataset Sen1Floods11 recently released by the Cloud to Street is one example of ongoing beneficial initiatives to employ deep learning for flood detection with synthetic aperture radar. The present study used this dataset to improve flood detection using well-known segmentation architectures, such as SegNet and UNet, as networks. In addition, this study provided a deeper understanding of which set of polarized band combination is more suitable for distinguishing permanent water, as well as flooded areas from the SAR image. The overall performance of the models with various kinds of labels and a combination of bands to detect all surface water areas were also assessed. Finally, the trained models were tested on a completely different location at Kerala, India, during the 2018 flood for verifying their performance in the real-world situation of a flood event outside of the given test set in the dataset. The results prove that trained models can be used as off-the-shelf models to achieve an intersection over union (IoU) as high as 0.88 in comparison with optical images. The omission and commission error were less than 6%. However, the most important result is that the processing time for the whole satellite image was less than 1 min. This will help significantly for providing analysis and near-real-time flood mapping services to first responder organizations during flooding disasters.

Near-Real-Time Flood Mapping Using Off-the-Shelf Models with SAR Imagery and Deep Learning

Semantic segmentation methods have made impressive progress with deep learning. However, while achieving higher and higher accuracy, state-of-the-art neural networks overlook the complexity of architectures, which typically feature dozens of millions of trainable parameters. Consequently, these networks requires high computational resources and are mostly not suited to perform on edge devices with tight resource constraints, such as phones, drones, or satellites. In this work, we propose two highly compact neural network architectures for semantic segmentation of images, which are up to 100 000 times less complex than state-of-the-art architectures while approaching their accuracy. To decrease the complexity of existing networks, our main ideas consist in exploiting lightweight encoders and decoders with depth-wise separable convolutions and decreasing memory usage with the removal of skip connections between encoder and decoder. Our architectures are designed to be implemented on a basic FPGA such as the one featured on the Intel Altera Cyclone V family of SoCs. We demonstrate the potential of our solutions in the case of binary segmentation of remote sensing images, in particular for extracting clouds and trees from RGB satellite images.

/pdf/low-power-neural-networks-for-semantic-segmentation-of-foju479u32.pdf

Low-Power Neural Networks for Semantic Segmentation of Satellite Images

Introduction The first patient-specific biomechanical model for planning the surgical reduction of acetabular fractures was developed in our institution and validated retrospectively. There are no prior studies showing its effectiveness in terms of reduction quality, operative duration and intraoperative bleeding. Therefore, we performed a case control study aiming to: 1) evaluate the effect of preoperative simulation by patient-specific biomechanical simulator on the operating time and intraoperative bleeding; 2) evaluate the effect of preoperative simulation by patient-specific biomechanical simulator on the quality of reduction. Method All patients operated on between January 2019 and June 2019 after planning by biomechanical simulation were included in this case-control study. Each patient included was matched to 2 controls from our database (2015–2018) according to age and fracture-type. DICOM data were extracted from the preoperative high-resolution scanners to build a three-dimensional model of the fracture by semi-automatic segmentation. A biomechanical model was built to virtually simulate the different stages of surgical reduction. Surgery was then performed according to simulation data. Surgical duration, blood loss, radiological findings and intraoperative complications were recorded, analysed and compared. Results Thirty patients were included, 10 in the simulation group and 20 in the control group. The two groups were comparable in terms of age, time from accident to surgery, fracture-type and surgical approach. The mean operative time was significantly reduced in the simulation group: 113 min ± 33 (60–180) versus 196 min ± 32 (60–260) (p = 0.01). Mean blood loss was significantly reduced in the simulation group: 505 mL ± 189 (100–750) versus 745 mL ± 130 (200–850) (p  Conclusion This study confirms the promising results of preoperative planning in acetabular trauma surgery based on patient-specific biomechanical simulation as well as its feasibility in routine clinical practice. By providing a better understanding of the fracture and its behavior, a reduction in intraoperative bleeding and in operative duration is achieved. Level of evidence III; case-control study.

Virtual preoperative planning of acetabular fractures using patient-specific biomechanical simulation: A case-control study.

Automatic road graph extraction from aerial and satellite images is a long-standing challenge. Existing algorithms are either based on pixel-level segmentation followed by vectorization, or on iterative graph construction using next move prediction. Both of these strategies suffer from severe drawbacks, in particular high computing resources and incomplete outputs. By contrast, we propose a method that directly infers the final road graph in a single pass. The key idea consists in combining a Fully Convolutional Network in charge of locating points of interest such as intersections, dead ends and turns, and a Graph Neural Network which predicts links between these points. Such a strategy is more efficient than iterative methods and allows us to streamline the training process by removing the need for generation of starting locations while keeping the training end-to-end. We evaluate our method against existing works on the popular RoadTracer dataset and achieve competitive results. We also benchmark the speed of our method and show that it outperforms existing approaches. Our method opens the possibility of in-flight processing on embedded devices for applications such as real-time road network monitoring and alerts for disaster response.

Single-Shot End-to-end Road Graph Extraction

A simple, patient-specific biomechanical model (PSBM) is proposed in which the main surgical tools and actions can be simulated, which enables clinicians to evaluate different strategies for an optimal surgical planning. A prospective and comparative clinical study was performed to assess early clinical and radiological results. From January 2019 to July 2019, a PSBM was created for every operated acetabular fracture (simulation group). DICOM data were extracted from the pre-operative high-resolution CT scans to build a 3D model of the fracture using segmentation methods. A PSBM was implemented in a custom software allowing a biomechanical simulation of the surgery in terms of reduction sequences. From July 2019 to December 2019, every patient with an operated for acetabular fracture without PSBM was included in the standard group. Surgery duration, blood loss, radiological results and per-operative complications were recorded and compared between the two groups. Twenty-two patients were included, 10 in the simulation group and 12 in the standard group. The two groups were comparable regarding age, time to surgery, fracture pattern distribution and surgical approaches. The mean operative time was significantly lower in the simulation group: 113 min ± 33 (60–180) versus 184 ± 58 (90–260), p = 0.04. The mean blood loss was significantly lower in the simulation group, p = 0.01. No statistical significant differences were found regarding radiological results (p = 0.16). No per-operative complications were recorded. This study confirms that pre-operative planning in acetabular surgery based on a PSBM results in a shorter operative time and a reduction of blood loss during surgery. This study also confirms the feasibility of PSBM planning in daily clinical routine. II: prospective study.

Gaetan Bahl

Papers

Binary Graph Neural Networks

Low-Power Neural Networks for Semantic Segmentation of Satellite Images

Virtual preoperative planning of acetabular fractures using patient-specific biomechanical simulation: A case-control study.

Single-Shot End-to-end Road Graph Extraction

Planning acetabular fracture reduction using a patient-specific biomechanical model: a prospective and comparative clinical study