Structured illumination microscopy (SIM) surpasses the optical diffraction limit and offers a two-fold enhancement in resolution over diffraction limited microscopy. However, it requires both intense illumination and multiple acquisitions to produce a single high-resolution image. Using deep learning to augment SIM, we obtain a five-fold reduction in the number of raw images required for super-resolution SIM, and generate images under extreme low light conditions (at least 100× fewer photons). We validate the performance of deep neural networks on different cellular structures and achieve multi-color, live-cell super-resolution imaging with greatly reduced photobleaching. Super-resolution microscopy typically requires high laser powers which can induce photobleaching and degrade image quality. Here the authors augment structured illumination microscopy (SIM) with deep learning to reduce the number of raw images required and boost its performance under low light conditions.

/pdf/deep-learning-enables-structured-illumination-microscopy-2akrn4sde9.pdf

Deep learning enables structured illumination microscopy with low light levels and enhanced speed.

Structured illumination microscopy (SIM) has become a widely used tool for insight into biomedical challenges due to its rapid, long-term, and super-resolution (SR) imaging. However, artifacts that often appear in SIM images have long brought into question its fidelity, and might cause misinterpretation of biological structures. We present HiFi-SIM, a high-fidelity SIM reconstruction algorithm, by engineering the effective point spread function (PSF) into an ideal form. HiFi-SIM can effectively reduce commonly seen artifacts without loss of fine structures and improve the axial sectioning for samples with strong background. In particular, HiFi-SIM is not sensitive to the commonly used PSF and reconstruction parameters; hence, it lowers the requirements for dedicated PSF calibration and complicated parameter adjustment, thus promoting SIM as a daily imaging tool.

High-fidelity structured illumination microscopy by point-spread-function engineering

Correlating data from different microscopy techniques holds the potential to discover new facets of signaling events in cellular biology. Here we report for the first time a hardware set-up capable of achieving simultaneous co-localized imaging of spatially correlated far-field super-resolution fluorescence microscopy and atomic force microscopy, a feat only obtained until now by fluorescence microscopy set-ups with spatial resolution restricted by the Abbe diffraction limit. We detail system integration and demonstrate system performance using sub-resolution fluorescent beads and applied to a test sample consisting of human bone osteosarcoma epithelial cells, with plasma membrane transporter 1 (MCT1) tagged with an enhanced green fluorescent protein (EGFP) at the N-terminal.

/pdf/simultaneous-co-localized-super-resolution-fluorescence-1en5mvhde1.pdf

Simultaneous co-localized super-resolution fluorescence microscopy and atomic force microscopy: combined SIM and AFM platform for the life sciences.

Super-resolution structured illumination microscopy (SR-SIM) provides an up to twofold enhanced spatial resolution of fluorescently labeled samples. The reconstruction of high-quality SR-SIM images critically depends on patterned illumination with high modulation contrast. Noisy raw image data (e.g., as a result of low excitation power or low exposure time), result in reconstruction artifacts. Here, we demonstrate deep-learning based SR-SIM image denoising that results in high-quality reconstructed images. A residual encoding–decoding convolutional neural network (RED-Net) was used to successfully denoise computationally reconstructed noisy SR-SIM images. We also demonstrate the end-to-end deep-learning based denoising and reconstruction of raw SIM images into high-resolution SR-SIM images. Both image reconstruction methods prove to be very robust against image reconstruction artifacts and generalize very well across various noise levels. The combination of computational image reconstruction and subsequent denoising via RED-Net shows very robust performance during inference after training even if the microscope settings change.

/pdf/deep-learning-based-denoising-and-reconstruction-of-super-2e2exoootu.pdf

Deep-learning based denoising and reconstruction of super-resolution structured illumination microscopy images

This paper reports a study on linear and nonlinear optical properties in a series of lithium‑calcium tetraborate (CaLiBO) glasses in the presence of two rare earth ions, Tb3+ and Yb3+. The nonlinear index of refraction (n2) was determined in a wide wavelength range (470–1500 nm) through femtosecond Z-scan technique. Such results were modeled by BGO approach, in which the optical nonlinearity was related to the oxygens present in the glasses. Also, the molar electronic polarizability evidenced that there is a strong influence of the polarizability of the rare earth ions in the nonlinear optical properties. Thus, we can infer that the cooperation of oxygens and the polarizability of rare earth ions are responsible for the nonlinearity present in the investigated vitreous system.

Effect of Tb3+/Yb3+ in the nonlinear refractive spectrum of CaLiBO glasses

Successful optimization of reconstruction parameters in structured illumination microscopy – A practical guide

The nonlinear refractive index n2 of SiO2-Al2O3-La2O3 (SAL) glasses of 10 to 24 mol% La2O3 is determined via Z-scan technique at 800 nm in the sub-100 fs time regime. n2 (5.8 to 9.3 × 10−16 cm2/W) correlates linearly with the La2O3 concentration scaling by the factor (0.264 ± 0.007)×10−16 cm2/W per mol% of La2O3. The relation between n2 and the linear refractive index n0 of the SAL systems is successfully described by the theoretical model of Boling, Glass and Owyoung (BGO theory). Increasing concentrations of heavy La3+ ions and non-bridging oxygen accompanied by a rising volume of the O2− ions are considered to be responsible for the n0 and n2 increase as well as a slight Urbach energy decrease. The upper limit of the La2/3O hyperpolarizability is estimated to amount to (2.2 ± 0.2)×10−36 esu.

Nonlinear refractive index study on SiO_2-Al_2O_3-La_2O_3 glasses

Selected semiconductor nanostructures provide extremely localized coherent light sources. Here an ensemble of CdS nanostructures was excited by UV/vis femtosecond laser pulses and their ultrafast luminescence characteristics were investigated as functions of the pulse energy fluence and the photon quantum energy. All optical Kerr gating enabled studies of the emission dynamics with a time resolution of 150 fs avoiding any influence on the CdS emission. The initially observed emission built up after a delay of 0.1–3 ps and decayed rapidly in a biexponential way, strongly dependent on both the laser energy fluence and the quantum energy. The central wavelength of the emission spectrum revealed a significant blue-shift within the first few ps followed by a transient red-shift relative to spontaneous excitonic emission of CdS. All findings are mainly attributed to stimulated radiative carrier recombination in the laser excited electron–hole plasma after its thermalization with the CdS lattice.

Excitation Energy Dependent Ultrafast Luminescence Behavior of CdS Nanostructures

Novel SiO2-Al2O3-La2O3 (SAL) glass shows a switching efficiency η ≤ 70% for dual wavelengths optical Kerr gating (OKG) at 0.7 TW/cm2 peak intensity IP of the gate pulse. It steadily increases with IP, is highest for the highest La2O3 content (24 mol%) and then superior to OKG media of large nonlinear refractive index n2 above 150 GW/cm2 (ZnS), 350 GW/cm2 (tellurite glass), and 450 GW/cm2 (N-SF56 glass). This superiority is attributed to negligible nonlinear counter processes in SAL glass which in the large n2 valued media make η(IP) decrease in the high IP region.

SiO_2-Al_2O_3-La_2O_3 glass - a superior medium for optical Kerr gating at moderate pump intensity

Avalanche ionization plays a crucial role in the photoionization of dielectric materials and as such is important for
optical damage. Although it has been investigated closely during the last years, there is little experimental evidence
of how the impact ionization parameter changes with electron density and/or incident pulse intensity. One reason is
that in most dielectric materials there are several competing ionization and relaxation processes. Here we present an
UV-pump IR-probe experiment that allowed us to isolate the avalanche ionization from other major ionization
processes, especially multiphoton ionization, electron tunneling, and relaxation into traps and their re-excitation. We
have measured the intensity dependence of a transmitted IR pulse, propagating through a thin sample of UV-grade
 sapphire (α-Al2O3), after seeding electrons in the conduction band with a UV pulse. We show that the assumption of
an intensity independent impact ionization factor α cannot explain the results. Application of a simple avalanche
ionization model within the flux-doubling approximation requires an intensity dependent coefficient a(I) to explain
the data. We also determined the two photon absorption coefficient of sapphire at 266 nm (β(2) = (2.7 ± 0.1) • 10-11cm/W) as well as the "free" electron absorption cross section for 800 nm of conduction band electrons in sapphire
(δ0 = (12.5 ± 0.2) •10-18 cm2).

Christian Karras

Papers

Successful optimization of reconstruction parameters in structured illumination microscopy – A practical guide

Nonlinear refractive index study on SiO_2-Al_2O_3-La_2O_3 glasses

Excitation Energy Dependent Ultrafast Luminescence Behavior of CdS Nanostructures

SiO_2-Al_2O_3-La_2O_3 glass - a superior medium for optical Kerr gating at moderate pump intensity

The impact ionization coefficient in dielectric materials revisited