Showing papers in "IEEE Journal of Oceanic Engineering in 2022"

Underwater Image Enhancement by Attenuated Color Channel Correction and Detail Preserved Contrast Enhancement

Abstract: An underwater image often suffers from quality degradation issues, such as color deviations, low contrast, and blurred details, due to the absorption and scattering of light. In this article, we propose to address the aforementioned degradation issues via attenuated color channel correction and detail preserved contrast enhancement. Concretely, we first propose an underwater image color correction method. Considering the differences between superior and inferior color channels of an underwater image, the inferior color channels are compensated via especially designed attenuation matrices. We then employ a dual-histogram-based iterative threshold method and a limited histogram method with Rayleigh distribution to improve the global and local contrast of the color-corrected image, thus achieving a global contrast-enhanced version and a local contrast-enhanced version, respectively. To integrate the complementary merits between the global contrast-enhanced version and the local contrast-enhanced version, we adopt a multiscale fusion strategy to fuse them. Finally, we propose a multiscale unsharp masking strategy to further sharpen the fused image for better visual quality. Extensive experiments on four underwater image enhancement benchmark data sets demonstrate that our method effectively enhances underwater images qualitatively and quantitatively. Besides, our method also generalizes well to the enhancement of low-light images and hazy images.

Comprehensive Underwater Object Tracking Benchmark Dataset and Underwater Image Enhancement With GAN

Abstract: Current state-of-the-art object tracking methods have largely benefited from the public availability of numerous benchmark datasets. However, the focus has been on open-air imagery and much less on underwater visual data. Inherent underwater distortions, such as color loss, poor contrast, and underexposure, caused by attenuation of light, refraction, and scattering, greatly affect the visual quality of underwater data, and as such, existing open-air trackers perform less efficiently on such data. To help bridge this gap, this article proposes a first comprehensive underwater object tracking (UOT100) benchmark dataset to facilitate the development of tracking algorithms well-suited for underwater environments. The proposed dataset consists of 104 underwater video sequences and more than 74 000 annotated frames derived from both natural and artificial underwater videos, with great varieties of distortions. We benchmark the performance of 20 state-of-the-art object tracking algorithms and further introduce a cascaded residual network for underwater image enhancement model to improve tracking accuracy and success rate of trackers. Our experimental results demonstrate the shortcomings of existing tracking algorithms on underwater data and how our generative adversarial network (GAN)-based enhancement model can be used to improve tracking performance. We also evaluate the visual quality of our model's output against existing GAN-based methods using well-accepted quality metrics and demonstrate that our model yields better visual data.

IA-Net$:$ An Inception–Attention-Module-Based Network for Classifying Underwater Images From Others

Abstract: To distinguish underwater images from natural images is one of the challenge of collecting and generation of underwater image data. Common image classification and recognition models classify the objects in an image depending on the saliency while suppressing the background. In this article, an inception–attention network (IA-Net), a convolutional neural network (CNN)-based model to classify the underwater images from natural images is reported, in which an inception–attention (I-A) module is constructed to simulate the visual correlation mechanism of classifying images taken from special environments such as fog, nighttime and under water. It is illustrated that the context background is as important as the salient object when understanding the underwater images. We executed experiments on a data set, which consists of 4000 underwater images and 5000 nonunderwater images, and demonstrate that the proposed IA-Net achieves an accuracy of 99.3$\%$ on underwater image classification, which is significantly better than classical image classification networks, such as AlexNet, InceptionV3, and ResNet. In addition, the comparative experiments prove that the IA-Net is superior to other networks when distinguishing underwater images from foggy, nighttime images and fish images taken in nonunderwater environments, although these images have indistinguishable characteristics with underwater images. Moreover, we demonstrate the I-A structure we proposed can be used to boost the performance of the existing object recognition networks. By substituting the inception module with the I-A module, the Inception-ResNetV2 network achieves a 10.7$\%$ top-1 error rate on the subset of ILSVRC-2012, which further illustrates the effectiveness of the correlation between the image background and subjective perception in improving the performance of the visual analysis tasks.

Underwater Image Enhancement With Reinforcement Learning

Abstract: In this article, we develop an underwater image enhancement framework based on reinforcement learning. To do this, we model the underwater image enhancement as a Markov decision process (MDP), in which states are represented by image feature maps, actions are represented by image enhancement methods, and rewards are represented by image quality improvements. The MDP trained with reinforcement learning can characterize a sequence of enhanced results for an underwater image. At each step of the MDP, a state transitions from one to another according to an action of image enhancement selected by a deep Q network. The final enhanced image in the sequence is obtained with respect to the biggest overall image quality improvement. In this manner, our reinforcement learning framework effectively organizes a sequence of image enhancement methods in a principled manner. In contrast to the black box processing schemes of deep learning methods, our reinforcement learning framework gives a sequence of specific actions, which are transparent from the implementation perspective. Benefiting from the exploration and exploitation training fashion, our reinforcement learning framework possibly generates enhanced images that are of better quality than reference images. Experimental results validate the effectiveness of our reinforcement learning framework in underwater image enhancement. The code and detailed results are available at https://gitee.com/sunshixin_upc/underwater-image-enhancement-with-reinforcement-learning.

Underwater Image Enhancement via Physical-Feedback Adversarial Transfer Learning

Abstract: This article proposes a domain adaptive learning framework based on physical model feedback for underwater image enhancement. Underwater image enhancement involves mapping from low-quality underwater images to their dewatered counterparts. Due to the lack of dewatered images as ground truth, most learning-based methods are trained using synthetic datasets. However, they usually ignored the domain gap between synthetic training data and real-world testing data, which seriously reduces the generalization ability of those models when testing on real underwater images. We solve the problem by embedding a domain adaptive mechanism in a learning framework to eliminate the domain gap. However, the basic formulation of a domain adaptive-based learning framework does not generate realistic images in color and details. Motivated by an observation that the estimated results should be consistent with the physical model of underwater imaging, we propose a physics constraint as a feedback controller so that it can guide the estimation of underwater image enhancement. Extensive experiments validate the superiority of the proposed framework.

A Terminology Standard for Underwater Acoustics and the Benefits of International Standardization

Abstract: Applications of underwater acoustics include sonar, communication, geophysical imaging, acoustical oceanography, and bioacoustics. Specialists typically work with little interdisciplinary interaction, and the terminology they employ has evolved separately in each discipline, to the point that transdisciplinary misunderstandings are common. Furthermore, increasing societal concern about possible detrimental effects of underwater noise on aquatic animals has led national and international regulators to require monitoring of underwater noise, with a consequent need for interdisciplinary harmonization of terminology. By adopting a common language, we facilitate the effective communication of concepts and information in underwater acoustics, whether for research, technology, or regulation. In the words of William H. Taft, “Don't write so that you can be understood, write so that you can't be misunderstood.” Clear definitions of widely used terms are needed, such as those used for the characterization of sound fields (e.g., “soundscape” and “ambient noise”), sound sources (“source level” and “source waveform”), sound propagation (“transmission loss” and “propagation loss”), and sound reception (“hearing threshold” and “frequency weighting function”). Terms that are used synonymously in one application have different meanings in another (examples include “hearing threshold” versus “detection threshold” and “transmission loss” versus “propagation loss”). Distinct definitions for these and many other acoustic terms are provided in a standard published in April 2017 by the International Organization for Standardization, ISO 18405. This article summarizes ISO 18405 and the process that led to the published definitions, including the reasons for omitting some terms.

Development of a Buoy-Borne Underwater Imaging System for <i>In Situ</i> Mesoplankton Monitoring of Coastal Waters

Abstract: This article reports the development of an underwater imaging system and its trial on a moored surface buoy for in situ plankton monitoring of coastal waters. The imager features shadowless white light illumination by an orthogonal lamellar lighting design, resulting in high-quality underwater darkfield color imaging of planktonic particles in the size range of ∼200 μm to 40 mm and effective reduction of zooplankton phototaxis. Through raft and buoy trials, 46 804 plankton and suspending particle images have been annotated through a human–machine mutually assisted effort into a data set with 90 categories. In the meanwhile, a deep learning model based on a triclassification VGGNet-11 and multiclassification ResNet-18 convolutional neuron networks in a two-staged hierarchy has also been trained and developed. The model has been applied with human supervision to semiautomatically analyze a total of 1 545 187 images obtained from a buoy trial for six months from late spring to early winter of 2020. The high temporal resolution results well documented the variation of the mesoplankton community structure in two time series of 38 days in summer and 54 days in autumn of the target sea region. In addition, the dominant species in the trial period and a zooplankton outbreak that had threatened the safety of the nearby nuclear power plants were quantitatively analyzed. The system is expected to become a new entry into the toolkit for marine surface buoy platforms to upscale their capabilities for more comprehensive in situ plankton monitoring.

Automatic Target Recognition for Mine Countermeasure Missions Using Forward-Looking Sonar Data

Abstract: The detection of objects on the seafloor is a complex task. The domain of the detection and classification of naval mines is additionally complicated by the high risk nature of the task. Autonomous underwater vehicles (AUVs) have been used in naval mine countermeasures (MCM) operations to search large areas using sensors such as sidescan or synthetic aperture sonars. These sensors generally have a high coverage rate, while sacrificing spatial resolution. Conversely, sensors with higher resolution but lower coverage (such as forward-looking sonars and electro–optical cameras) are employed for the later classification and identification stages of the MCM mission. However, to autonomously execute a target reacquisition mission, it is important to be able to collect and process data automatically and, in near real time, onboard an AUV. For this purpose, an automatic target recognition (ATR) system is required. This article proposes an ATR, which can be used onboard an autonomous vehicle, capable of detecting mine-like objects in forward-looking sonar data. The ATR combines a detector and a classifier, based on convolutional neural network models, with a probabilistic grid map that filters out false positives and combines reported detections at nearby locations. A strategy, combining a survey pattern with target-mapping maneuvers automatically activated by the ATR, has been designed to maximize the performance of this ATR. The whole system has been tested in simulation as well as using data from previous MCM exercises, the results of which are presented here.

Measurements of Geologic Characteristics and Geophysical Properties of Sediments From the New England Mud Patch

Abstract: The characterization of physical, geological, and geophysical properties of sediments within the New England Mud Patch (NEMP) was undertaken to provide a physical basis for acoustic inversions associated with the SeaBed Characterization EXperiment 2017 (SBCEX17). Using a suite of 89 sediment cores (piston/trigger, gravity [acoustic], and vibracore), a comprehensive database of laboratory-based sediment analyses, geophysical core logs, and the results of seismic reflection profiling, we formulate a three-layer lithostratigraphic model of the area within and immediately adjacent to the SBCEX17 focus area, referred to as the seabed experiment area (SEA). The uppermost lithostratigraphic unit, Unit 1, is relatively homogenous clayey- to sandy silt, with consistent downcore textural, mineralogical, and physical property attributes. Unit 2 is a variable-thickness transitional layer between Unit 1 and Unit 3, whose properties reflect a decrease in proximal erosion and transition to a lower energy depositional environment. Unit 3 is clean quartz sand containing abundant shells and shell fragments that was regionally deposited during Holocene sea-level rise. ²¹⁰ Pb and ¹⁴ C radiocarbon geochronologies spanning the past 13 000 years are used to facilitate intercore comparison across the SEA. Analytical results and laboratory methods used in the derivation of those results are described in detail, serving as a reference for ongoing and future investigation of the SEA and entire NEMP. Although the derived lithostratigraphic model of the SEA is in good agreement with past evaluations of the regional sedimentology, comparisons of the lithostratigraphic and seismostratigraphic models highlight several significant incompatibilities that remain to be satisfactorily explained.

Takeoff and Landing Control of a Hybrid Aerial Underwater Vehicle on Disturbed Water’s Surface

Abstract: This article proposes a disturbance observer-based nonlinear control strategy for the accurate takeoff and landing control of a novel hybrid aerial underwater vehicle, Nezha III, subjected to wave and wind. The approach consists of the dynamic surface controller (DSC) and the nonlinear disturbance observers (NDOs). The problem is first modeled with full consideration of diverse external forces, including the capsizing buoyant moment, the added mass effect, and wave and wind loads. The DSC forms the base of the controller and benefits the tracking behavior by handling the nonlinearity of the system. Meanwhile, the integrated NDOs enhance the robustness of the closed-loop system by estimating the unmeasurable external forces. The proposed method is proved theoretically to stabilize the vehicle when it tracks the reference trajectory across different media in the presence of environmental disturbances. Finally, the proposed controller is tested numerically by challenging it under the wind and wave disturbances. The results show the controller makes Nezha III achieve successful takeoff and landing on the disturbed water’s surface in spite of the hazardous environment, which strongly evidence the outstanding performance of the proposed method.

Underwater Optical 3-D Reconstruction of Photometric Stereo Considering Light Refraction and Attenuation

Abstract: Photometric stereo is a widely used optical 3-D reconstruction method, which has advantages for high-resolution and well-detailed 3-D reconstruction even in weak texture regions. However, in underwater photometric stereo, the light refraction and attenuation using the camera and point light sources usually result in significant errors in shape recovery. In this article, we present a novel solution to address these challenges for improving the performance of underwater photometric stereo by combining it with underwater laser triangulation. First, we employ underwater laser triangulation as references to correct the global surface-shape distortion caused by nonuniform close-range illumination. Second, we propose to use an underwater camera refractive model to eliminate nonlinear refraction distortion. Third, we build a device implementing the proposed method for 3-D seabed reconstruction in on-site sea trials. Experimental results demonstrate that the proposed approach is able to produce accurate 3-D reconstruction results in the underwater environment.

Performance Measures for Validation of Oil Spill Dispersion Models Based on Satellite and Coastal Data

Abstract: This article presents a set of performance metrics, whose purpose is to provide a quantitative measure of the ability of oil spill dispersion models to simulate real-world oil spills. The metrics are described in detail and are applied to the output from an existing oil spill model for two specific case studies. The metrics in question make use of both satellite imagery and coastal impact reports as the basis of the validation. Specifically, we recommend the 2-D measure of effectiveness as a means of quantifying model performance based on the extent of overlap between the observations and the model output. Additionally, we show that it is advantageous to supplement the 2-D measure of effectiveness with a newly proposed set of skill scores, based on the geometric area and centroid of a given oil spill. We also demonstrate how the metrics can be used to assess the sensitivity of a model to its input parameters and the impact this has on the accuracy of the resultant forecast. Finally, we offer a real-world interpretation for each metric introduced and suggest ways that they can be used to assist in cleanup operations of actual oil spills.

Coherence-Induced Bias Reduction in Synthetic Aperture Sonar Along-Track Micronavigation

Abstract: Subwavelength motion estimation is vital for the production of focused synthetic aperture sonar (SAS) imagery. The required precision is obtainable from the sonar data itself through a process termed micronavigation. Along-track micronavigation is achieved by a similar technique to that used in correlation velocity logs (CVLs), where sparse estimates of the spatial coherence function are interpolated to estimate the location of the peak coherence and hence estimate the interping vehicle motion. However, along-track micronavigation estimates made using this technique are biased, which limits the utility of these measurements for long-term navigation of autonomous underwater vehicles (AUVs). Three sources of along-track motion estimation bias are considered in this article. First, imperfect temporal registration between the signals results in coherence estimates that are negatively biased as a function of the temporal offset. Second, the sparse estimates of the spatial coherence function are obtained by cross-correlation of complex baseband signals, a process which is known to result in positively biased coherence estimates, especially when the true coherence is low. Finally, mismatches between the underlying spatial coherence function and the interpolation kernel used to estimate the peak coherence location also result in along-track micronavigation bias. In this article, we describe and evaluate three methods for reducing along-track micronavigation bias. We introduce a temporal registration of the signals before coherence estimation, which reduces the impact of negative coherence bias due to temporal offsets. The remaining coherence estimation bias is reduced by combining multiple coherence estimates in a Bayesian coherence estimator. Additionally, an improved interpolation kernel is derived with a significantly improved fit compared to the current gold standard Gaussian interpolation kernel. The improvements in along-track micronavigation accuracy are demonstrated using two simulated data sets, which both allow comparison with ground truth. The first involves direct simulation of the spatial coherence from a given interping geometry using the pulse-echo formulation of the van Cittert–Zernike theorem, while the second involves simulation of raw sonar echo data using a point-scatterer model. Using these simulations, a reduction in along-track micronavigation bias of 48.5%–99.5% is demonstrated, with reductions in along-track micronavigation error standard deviation of up to 34%. This improvement expands the potential for SAS-equipped AUVs to reduce their long-term navigation drift, facilitating longer underwater transits, improved target localization, and reduced track misalignment in repeat-pass operations.

Tracking Multiple Surface Vessels With an Autonomous Underwater Vehicle: Field Results

Abstract: This article describes the development and testing of a passive sonar, multitarget tracker, and adaptive behavior that enable an autonomous underwater vehicle (AUV) to detect and actively track nearby surface vessels. A planar hull-mounted hydrophone array, originally designed for active sonar, is repurposed for passive sonar use and provides acoustic data to a time-delay-and-sum beamformer that generates multiple angle-only contacts. A particle filter tracker assimilates these contacts with a single-hypothesis data association strategy to estimate the position and velocity of targets. Summary statistics of each track are periodically reported to an onboard database along with qualitative labels. To improve tracking performance, detections trigger an adaptive behavior that maneuvers the AUV to maintain multiple targets in the field of view by minimizing the worst case aspect angle deviation from broadside (across all targets). The tracking system is demonstrated through at-sea experiments in which a Bluefin-21 AUV adaptively tracks multiple surface vessels, including another autonomous platform, in the approaches to Boston Harbor.

Range-Dependent Inversion for Seabed Parameters Using Vector Acoustic Measurements of Underwater Ship Noise

Abstract: The Intensity Vector Autonomous Recorder (IVAR) measures acoustic particle velocity and pressure simultaneously. IVAR was deployed on the seabed during the 2017 Seabed Characterization Experiment (SBCEX) with the primary objective to study sound propagation within underwater waveguides for which the seabed consists of fine-grained, muddy sediments. In this study, a Bayesian framework is applied to underwater noise recorded by IVAR from a cargo ship traversing the central region of the SBCEX2017 area for the purpose of inversion to characterize sediment properties. The vector acoustic data are in the form of a bounded, nondimensional form known as circularity, a quantity that is independent of the ship noise-source spectrum and that can be interpreted as the normalized curl of active intensity. The inversion model space for the seabed consists of a low-compressional speed layer and underlying basement half-space, with each having compressional and shear components. The interpretative model for producing a replica of the data is based on the plane wave reflection coefficient for a layered, elastic seabed in conjunction with the depth-dependent Green’s function that is integrated in the complex wave number plane to obtain pressure and particle velocity fields. The small change in water depth between the location of the ship source and IVAR is addressed using adiabatic mode theory. The inversion results exhibit slow variation over the 20-min observation period, representing approximately 5 km of travel by the ship source.

Transdimensional Inversion on the New England Mud Patch Using High-Order Modes

Abstract: This article presents geoacoustic inversion results for modal-dispersion data collected during the 2017 Seabed Characterization Experiment on the New England Mud Patch, an area where the seabed is characterized by an upper layer of mud. The experiment utilized a combustive sound source and a vertical line array of receivers at 5.4-km range. Using a careful combination of source deconvolution and warping time–frequency analysis, modal dispersion data (arrival time as a function of frequency) are estimated for 18 modes between modes 1 and 21. The modal dispersion data are then used to estimate seabed geoacoustic profiles and uncertainties via transdimensional Bayesian inversion. This article demonstrates the capacity to estimate high-order modes using warping. Comparing inversion results obtained with subsets of (lower order) modes to those obtained with the full set of available modes highlights the rich data information content carried by high-order modes. The results suggest a small sound-speed increase over the first 8 m of the seabed, the upper portion of the mud layer, which some earlier studies found to be isospeed. Overall, the inversion results are consistent with in situ measurements, as well as with previous geoacoustic inversion results.

DRW-AE: A Deep Recurrent-Wavelet Autoencoder for Underwater Target Recognition

Abstract: The sonar received signals may contain various artifacts such as noise, reverberation, and clutter that affect the proper target recognition and classification capabilities of sonar systems. This performance reduction is exacerbated by multidirectional propagation, underwater heterogeneity, variable sound channels, and variable sound-speed profiles. To address these problems, this article develops a deep recurrent-wavelet autoencoder (DRW-AE) coupled with some machine learning classifiers to design an end-to-end underwater target classifier. In this approach, autoencoders play automatic feature extractors’ role in choosing the best feature composition in terms of types and dimensions without human intervention. Wavelet networks extract the vessels’ periodic frequency signatures, which are changed by various machinery conditions; finally, the recurrent network addresses the effects of time-varying and time-dependent inhomogeneous underwater environment. To investigate the efficiency of the hybrid DRW-AE, ShipsEar data set is exploited. Before comparing with other methods, the symmetric and asymmetric mother wavelet families were investigated to choose the proper wavelet function. Then, the efficiency of different combinations of deep autoencoders and proposed classifiers is investigated. Finally, the performance of the DRW-AE is compared with ten benchmark methods that have used this data set. The results show that the proposed algorithm with 94.49% accuracy and giga-multiplier–accumulators equal to 0.02 represents the best performance in terms of network accuracy and complexity compared to benchmark models.

Prediction of Mobility and Burial of Objects on Sandy Seafloor

Abstract: A numerical model was developed to predict mobility and burial of seafloor cylindrical objects. The model contains four components: 1) object's physical parameters such as diameter, length, mass, and rolling moment; 2) dynamics of rolling cylinder around its major axis; 3) empirical sediment scour model; and 4) seabed environmental characteristics such as currents, waves (peak period, significant wave height), sediment density, and medium sediment grain size. Under the sponsorship of the Department of Defense Strategic Environmental Research and Development Program, a field experiment was conducted from April 21 to May 23, 2013 off the coast of Panama City, FL, USA to measure both objects' mobility using sector scanning and pencil beam sonars and simultaneous environmental time-series data of the boundary layer hydrodynamics and sediment transport conditions for driving mobility. Comparison between modeled and observed data shows the model capability. Future work needs to consider more realistic object shapes and motions such as pitch and yaw, and wavy seabed.

Transdimensional Geoacoustic Inversion Using Prior Information on Range-Dependent Seabed Layering

Abstract: This article proposes a transdimensional (trans-D) geoacoustic inversion method adapted to range-dependent (RD) propagation tracks based on prior information from a high-resolution seismic survey. Most trans-D inversions to date model the seabed as a stack of range-independent homogeneous layers, with unknown geoacoustic parameters and an unknown number of layers. The proposed method models the seabed as an unknown number of homogeneous sediment layers with an RD thickness structure and applies an adiabatic normal-mode model to predict acoustic propagation. To do so, the method extrapolates trans-D seabed models proposed at the receiver position over the range of the propagation track using reflector-interface information from a seismic survey. The method is applied successfully to modal time–frequency dispersion data collected over an RD track during the 2017 Seabed Characterization Experiment (SBCEX).

Seabed Observations at the New England Mud Patch: Reflection and Scattering Measurements and Direct Geoacoustic Information

Abstract: Seabed reflection and scattering measurements were conducted at the New England Mud Patch to better understand the acoustic properties of fine-grained (muddy) sediments. The measurement philosophy and the measurements themselves are summarized. In addition, geoacoustic information accessed directly from the data in the time and frequency domains is presented. The main result is the existence of an angle of intromission. This observation proves that the mud sound speed is less than that of the water and yields a sediment sound speed ratio 0.9865 with outer bounds {0.985 0.989}. Another result is the observation of strong scattered arrivals from within the mud volume at/near normal incidence but not at low grazing angles. These are likely due to anisotropic sediment heterogeneities with a large horizontal to vertical scale. Evidence is also presented for a highly heterogeneous mud–sand horizon with lateral variability down to scales of order meters. Finally, the reflection measurements successfully capture Bragg interference patterns. Their importance is their substantial geoacoustic information content, which can be accessed by several inversion methods.

Seabed Classification Using a Convolutional Neural Network on Explosive Sounds

Abstract: The Seabed Characterization Experiment 2017 yielded a rich set of environmental and acoustical data, and subsequent geoacoustic inversions have estimated seabed properties. Seventeen of these seabed parameterizations are now used to define seabed classes, and a convolutional neural network (CNN) is trained to select a seabed class using explosive sounds. The CNN is trained on synthetic 1-s pressure time series and then applied to measured data samples from a single pressure sensor. While the environmental variability in the training data impacts the seabed classification, physical insights are gained by considering the classification results as a function of the sound speed ratio across the sediment–water interface and the interval velocity of the top sediment layer. These results indicate that the selected seabed classes for the data samples with longer propagation distances consistently have similar sediment–water sound speed ratios that are less than unity, while the data samples with the shortest propagation distances consistently have higher interval velocities. These classification results indicate that the CNN has learned physical features associated with acoustic sound propagation and points to future work that needs to be considered if the seabed classes have acoustically distinct signatures in the data.

Hybrid Seabed Parameterization to Investigate Geoacoustic Gradients at the New England Mud Patch

Abstract: This article applies Bayesian geoacoustic inversion with a hybrid seabed-model parameterization to modal-dispersion data from the New England Mud Patch to estimate gradient structure in the upper mud layer. The hybrid parameterization comprises an upper layer with general smooth (continuous) gradients represented by Bernstein polynomials (BPs) for the mud, above an unknown number of discrete (uniform) layers. The Bayesian information criterion is applied to estimate BP orders for sound-speed and density profiles in the mud, and trans-dimensional (trans-D) inversion is applied for the underlying layered structure. The data, collected during the 2017 Seabed Characterization Experiment, include high-order modes (up to mode 21) extracted via warping time-frequency analysis from recordings of a combustive sound source at a vertical hydrophone array. Inversion results for the hybrid parameterization are compared to those from trans-D inversion with no gradient layer. Hybrid-inversion results indicate a nearly iso-speed mud layer with a rapidly increasing gradient near its base, consistent with increasing sand content in the mud above a sand interface, as indicated by cores. The sound-speed ratio of surficial sediments to bottom seawater is found to be $<$ 1 with high probability, which differs from trans-D inversion results, indicating the significance of the choice of parameterization in interpreting structure.

Reinforcement Learning Based Underwater Wireless Optical Communication Alignment for Autonomous Underwater Vehicles

Abstract: With the developments in underwater wireless optical communication (UWOC) technology, UWOC can be used in conjunction with autonomous underwater vehicles (AUVs) for high-speed data sharing among the vehicle formation during underwater exploration. A beam alignment problem arises during communication due to the transmission range, external disturbances and noise, and uncertainties in the AUV dynamic model. In this article, we propose an acoustic navigation method to guide the alignment process without requiring beam directors, light intensity sensors, and/or scanning algorithms as used in previous research. The AUVs need stably maintain a specific relative position and orientation for establishing an optical link. We model the alignment problem as a partially observable Markov decision process (POMDP) that takes manipulation, navigation, and energy consumption of underwater vehicles into account. However, finding an efficient policy for the POMDP under high partial observability and environmental variability is challenging. Therefore, for successful policy optimization, we utilize the soft actor–critic reinforcement learning algorithm together with AUV-specific belief updates and reward shaping based curriculum learning. Our approach outperformed baseline approaches in a simulation environment and successfully performed the beam alignment process from one AUV to another on the real AUV Tri-TON 2.

DRW-AE: A Deep Recurrent-Wavelet Autoencoder for Underwater Target Recognition

Abstract: The sonar received signals may contain various artifacts such as noise, reverberation, and clutter that affect the proper target recognition and classification capabilities of sonar systems. This performance reduction is exacerbated by multidirectional propagation, underwater heterogeneity, variable sound channels, and variable sound-speed profiles. To address these problems, this article develops a deep recurrent-wavelet autoencoder (DRW-AE) coupled with some machine learning classifiers to design an end-to-end underwater target classifier. In this approach, autoencoders play automatic feature extractors’ role in choosing the best feature composition in terms of types and dimensions without human intervention. Wavelet networks extract the vessels’ periodic frequency signatures, which are changed by various machinery conditions; finally, the recurrent network addresses the effects of time-varying and time-dependent inhomogeneous underwater environment. To investigate the efficiency of the hybrid DRW-AE, ShipsEar data set is exploited. Before comparing with other methods, the symmetric and asymmetric mother wavelet families were investigated to choose the proper wavelet function. Then, the efficiency of different combinations of deep autoencoders and proposed classifiers is investigated. Finally, the performance of the DRW-AE is compared with ten benchmark methods that have used this data set. The results show that the proposed algorithm with 94.49% accuracy and giga-multiplier–accumulators equal to 0.02 represents the best performance in terms of network accuracy and complexity compared to benchmark models.

Detecting Marine Organisms Via Joint Attention-Relation Learning for Marine Video Surveillance

Abstract: The better way to understand marine life and ecosystems is to surveil and analyze the activities of marine organisms. Recently, research on marine video surveillance is becoming increasingly popular. With the rapid development of deep learning (DL), convolutional neural networks (CNNs) have made remarkable progresses in image/video understanding tasks. In this article, we explore a visual attention and relation mechanism for marine organism detection, and propose a new way to apply an improved attention-relation (AR) module on an efficient marine organism detector (EMOD), which can well enhance the discrimination of organisms in complex underwater environments. We design our EMOD via integrating current state-of-the-art (SOTA) detection methods in order to detect organisms and surveil marine environments in a real time and fast fashion for high-resolution marine video surveillance. We implement our EMOD and AR on the annotated video data sets provided by the public data challenges in conjunction with the workshops (CVPR 2018 and 2019), which are supported by National Oceanic and Atmospheric Administration (NOAA) and their research works (NMFS-PIFSC-83). Experimental results and visualizations demonstrate that our application of AR module is effective and efficient, and our EMOD equipped with AR modules can outperform SOTA performance on the experimental data sets. For application requirements, we also provide the application suggestions of EMOD framework. Our code is publicly available at https://github.com/zhenglab/EMOD.

Virtual Maps for Autonomous Exploration of Cluttered Underwater Environments

Abstract: We consider the problem of autonomous mobile robot exploration in an unknown environment, taking into account a robot’s coverage rate, map uncertainty and state estimation uncertainty. In this article, we present a novel exploration framework for underwater robots operating in cluttered environments, built upon simultaneous localization and mapping with imaging sonar. The proposed system comprises path generation, place recognition forecasting, belief propagation and utility evaluation using a virtual map, which estimates the uncertainty associated with map cells throughout a robot’s workspace. We evaluate the performance of this framework in simulated experiments, showing that our algorithm maintains a high coverage rate during exploration while also maintaining low mapping and localization error. The real-world applicability of our framework is also demonstrated on an underwater remotely operated vehicle exploring a harbor environment.

Reinforcement Learning Based Underwater Wireless Optical Communication Alignment for Autonomous Underwater Vehicles

Abstract: With the developments in underwater wireless optical communication (UWOC) technology, UWOC can be used in conjunction with autonomous underwater vehicles (AUVs) for high-speed data sharing among the vehicle formation during underwater exploration. A beam alignment problem arises during communication due to the transmission range, external disturbances and noise, and uncertainties in the AUV dynamic model. In this article, we propose an acoustic navigation method to guide the alignment process without requiring beam directors, light intensity sensors, and/or scanning algorithms as used in previous research. The AUVs need stably maintain a specific relative position and orientation for establishing an optical link. We model the alignment problem as a partially observable Markov decision process (POMDP) that takes manipulation, navigation, and energy consumption of underwater vehicles into account. However, finding an efficient policy for the POMDP under high partial observability and environmental variability is challenging. Therefore, for successful policy optimization, we utilize the soft actor–critic reinforcement learning algorithm together with AUV-specific belief updates and reward shaping based curriculum learning. Our approach outperformed baseline approaches in a simulation environment and successfully performed the beam alignment process from one AUV to another on the real AUV Tri-TON 2.

Cross-Medium Communication Combining Acoustic Wave and Millimeter Wave: Theoretical Channel Model and Experiments

Abstract: Communication across the water–air interface is an open research problem. A common solution is to place a relay on the water–air interface, which transmits received underwater acoustic signals to shore via electromagnetic waves. However, this solution has limitations, such as surface relays easily floating away or underwater relays becoming resource-intensive due to frequent surfacing, which make water–air communications unstable and costly. In this article, we investigate a recently proposed cross-medium communication technique, translational acoustic radio frequency (TARF) communications, in which underwater nodes can directly communicate with airborne nodes via acoustic waves and millimeter waves. For the first time, we introduce a closed-form, end-to-end channel and present its experimental validation. The high-frequency TARF communication is achievable with a frequency of up to 4.4 kHz. In the experiment deployment, underwater acoustic signals traveled as pressure waves, which produced water surface waves when they impinged on the water–air boundary. Airborne radar was employed to measure and decode these water surface waves. Experimental results and theoretical analysis demonstrated that 1) the cross-medium communication channel is frequency selective and time invariant; 2) cross-medium communications with both single-carrier and multicarrier signals are feasible; and 3) high-frequency signals from 3.6 to 4.4 kHz can be detected, which was considered difficult previously. This study provides a theoretical and experimental basis for achieving cross-medium communications between deep-sea moving objects, such as autonomous underwater vehicles and airborne nodes.

Target Detection and Segmentation in Circular-Scan Synthetic Aperture Sonar Images Using Semisupervised Convolutional Encoder–Decoders

Abstract: We propose a framework for saliency-based, multi-target detection and segmentation of circular-scan, synthetic-aperture-sonar (CSAS) imagery. Our framework relies on a multi-branch, convolutional encoder-decoder network (MB-CEDN). The encoder portion of the MB-CEDN extracts visual contrast features from CSAS images. These features are fed into dual decoders that perform pixel-level segmentation to mask targets. Each decoder provides different perspectives as to what constitutes a salient target. These opinions are aggregated and cascaded into a deep-parsing network to refine the segmentation. We evaluate our framework using real-world CSAS imagery consisting of five broad target classes. We compare against existing approaches from the computer-vision literature. We show that our framework outperforms supervised, deep-saliency networks designed for natural imagery. It greatly outperforms unsupervised saliency approaches developed for natural imagery. This illustrates that natural-image-based models may need to be altered to be effective for this imaging-sonar modality.