Instant multicolor super-resolution microscopy with deep convolutional neural network

Songyue Wang; Chang Qiao; Amin Jiang; Di Li; Dong Li

doi:10.52601/bpr.2021.210017

Volume 7 Issue 4

Sep. 2021

Turn off MathJax

Article Contents

Article Navigation > Biophysics Reports > 2021 > 7(4): 304-312

Songyue Wang, Chang Qiao, Amin Jiang, Di Li, Dong Li. Instant multicolor super-resolution microscopy with deep convolutional neural network[J]. Biophysics Reports, 2021, 7(4): 304-312. doi: 10.52601/bpr.2021.210017

Citation:

Songyue Wang, Chang Qiao, Amin Jiang, Di Li, Dong Li. Instant multicolor super-resolution microscopy with deep convolutional neural network[J]. Biophysics Reports, 2021, 7(4): 304-312. doi: 10.52601/bpr.2021.210017

Citation:

PDF (6212 KB)

Instant multicolor super-resolution microscopy with deep convolutional neural network

doi: 10.52601/bpr.2021.210017

Songyue Wang^{1, 2, *},
Chang Qiao^{3, *},
Amin Jiang^{1, 4, *},
Di Li¹,
Dong Li^{1, 2, 5
,
,}

1.
National Laboratory of Biomacromolecules, CAS Center for Excellence in Biomacromolecules, Institute of Biophysics, Chinese Academy of Sciences, Beijing 100101, China
2.
College of Life Sciences, University of Chinese Academy of Sciences, Beijing 100049, China
3.
Department of Automation, Tsinghua University, Beijing 100084, China
4.
Hefei National Laboratory for Physical Sciences at Microscale and School of Life Sciences, University of Science and Technology of China, Hefei 230026, China
5.
Bioland Laboratory (Guangzhou Regenerative Medicine and Health Guangdong Laboratory), Guangzhou 510005, China

Funds: This work was financially supported by grants from Ministry of Science and Technology (2017YFA0505301), the National Natural Science Foundation of China (31827802, 31770930), the Chinese Academy of Sciences (ZDBS-LY-SM004).

More Information

Corresponding author: lidong@ibp.ac.cn (D. Li)
Received Date: 09 June 2021
Accepted Date: 21 July 2021
Publish Date: 17 September 2021

Abstract

Abstract

Multicolor super-resolution (SR) microscopy plays a critical role in cell biology research and can visualize the interactions between different organelles and the cytoskeleton within a single cell. However, more color channels bring about a heavier budget for imaging and sample preparation, and the use of fluorescent dyes of higher emission wavelengths leads to a worse spatial resolution. Recently, deep convolutional neural networks (CNNs) have shown a compelling capability in cell segmentation, super-resolution reconstruction, image restoration, and many other aspects. Taking advantage of CNN’s strong representational ability, we devised a deep CNN-based instant multicolor super-resolution imaging method termed IMC-SR and demonstrated that it could be used to separate different biological components labeled with the same fluorophore, and generate multicolor images from a single super-resolution image in silico. By IMC-SR, we achieved fast three-color live-cell super-resolution imaging with ~100 nm resolution over a long temporal duration, revealing the complicated interactions between multiple organelles and the cytoskeleton in a single COS-7 cell.
- Multicolor imaging,
- Super-resolution,
- Convolutional neural network

*Songyue Wang, Chang Qiao and Amin Jiang have contributed equally to this study

FullText(HTML)

References(22)

References

[1]	Gustafsson MG, Shao L, Carlton PM, Wang CJ, Golubovskaya IN, Cande WZ, Agard DA, Sedat JW (2008) Three-dimensional resolution doubling in wide-field fluorescence microscopy by structured illumination. Biophys J 94: 4957−4970 doi: 10.1529/biophysj.107.120345
[2]	Guo Y, Li D, Zhang S, Yang Y, Liu JJ, Wang X, Liu C, Milkie DE, Moore RP, Tulu US, Kiehart DP, Hu J, Lippincott-Schwartz J, Betzig E, Li D (2018) Visualizing intracellular organelle and cytoskeletal interactions at nanoscale resolution on millisecond timescales. Cell 175: 1430−1442
[3]	Hendrycks D, Gimpel K (2016) Gaussian error linear units (gelus). arXiv preprint arXiv: 160608415
[4]	Hu J, Shen L, Sun G (2018) Squeeze-and-excitation networks. Paper presented at: Proceedings of the IEEE conference on computer vision and pattern recognition
[5]	Isensee F, Jaeger PF, Kohl SA, Petersen J, Maier-Hein KH (2021) nnU-Net: a self-configuring method for deep learning-based biomedical image segmentation. Nat Methods 18: 203−211 doi: 10.1038/s41592-020-01008-z
[6]	Kermany DS, Goldbaum M, Cai W, Valentim CCS, Liang H, Baxter SL, McKeown A, Yang G, Wu X, Yan F, Dong J, Prasadha MK, Pei J, Ting MYL, Zhu J, Li C, Hewett S, Dong J, Ziyar I, Shi A, Zhang R, Zheng L, Hou R, Shi W, Fu X, Duan Y, Huu VAN, Wen C, Zhang WD, Zhang CL, Li O, Wang X, Singer MA, Sun X, Xu J, Tafreshi A, Lewis MA, Xia H, Zhang K (2018) Identifying medical diagnoses and treatable diseases by image-based deep learning. Cell 172: 1122−1131 doi: 10.1016/j.cell.2018.02.010
[7]	Li D, Shao L Chen BC, Zhang X, Zhang M, Moses B, Milkie DE, Beach JR, Hammer JA, 3rd Pasham M, Kirchhausen T, Baird MA, Davidson MW, Xu P, Betzig E (2015) Extended-resolution structured illumination imaging of endocytic and cytoskeletal dynamics. Science 349: aab3500 doi: 10.1126/science.aab3500
[8]	Lim B, Son S, Kim H, Nah S, Mu Lee K (2017) Enhanced deep residual networks for single image super-resolution. Paper presented at: Proceedings of the IEEE conference on computer vision and pattern recognition workshops
[9]	Moen E, Bannon D, Kudo T, Graf W, Covert M, Van Valen D (2019) Deep learning for cellular image analysis. Nat methods 16: 1233−1246 doi: 10.1038/s41592-019-0403-1
[10]	Pan C, Schoppe O, Parra-Damas A, Cai R, Todorov MI, Gondi G, von Neubeck B, Böğürcü-Seidel N, Seidel S, Sleiman K (2019) Deep learning reveals cancer metastasis and therapeutic antibody targeting in the entire body. Cell 179: 1661−1676 doi: 10.1016/j.cell.2019.11.013
[11]	Phillips MJ, Voeltz GK (2016) Structure and function of ER membrane contact sites with other organelles. Nat Rev Mol Cell Biol 17: 69−82 doi: 10.1038/nrm.2015.8
[12]	Prinz WA (2014) Bridging the gap: membrane contact sites in signaling, metabolism, and organelle dynamics. J Cell Biol 205: 759−769 doi: 10.1083/jcb.201401126
[13]	Qiao C, Chen X, Zhang S, Li D, Guo Y, Dai Q, Li D (2021a) 3D Structured illumination microscopy via channel attention generative adversarial network. IEEE J Selec Topics Quant Electr 27: 1−11 doi: 10.1109/JSTQE.2021.3060762
[14]	Qiao C, Li D, Guo Y, Liu C, Jiang T, Dai Q, Li D (2021b) Evaluation and development of deep neural networks for image super-resolution in optical microscopy. Nat Methods 18: 194−202 doi: 10.1038/s41592-020-01048-5
[15]	Ronneberger O, Fischer P, Brox T (2015) U-net: convolutional networks for biomedical image segmentation. Paper presented at: International Conference on Medical image computing and computer-assisted intervention (Springer)
[16]	Rust MJ, Bates M, Zhuang X (2006) Sub-diffraction-limit imaging by stochastic optical reconstruction microscopy (STORM). Nat Methods 3: 793−795 doi: 10.1038/nmeth929
[17]	Shroff H, Galbraith CG, Galbraith JA, Betzig E (2008) Live-cell photoactivated localization microscopy of nanoscale adhesion dynamics. Nat Methods 5: 417−423 doi: 10.1038/nmeth.1202
[18]	Szegedy C, Ioffe S, Vanhoucke V, Alemi AA (2017) Inception-v4, inception-resnet and the impact of residual connections on learning. Paper presented at: Thirty-first AAAI conference on artificial intelligence
[19]	Wang H, Rivenson Y, Jin Y, Wei Z, Gao R, Gunaydin H, Bentolila LA, Kural C, Ozcan A (2019) Deep learning enables cross-modality super-resolution in fluorescence microscopy. Nat Methods 16: 103−110 doi: 10.1038/s41592-018-0239-0
[20]	Weigert M, Schmidt U, Boothe T, Muller A, Dibrov A, Jain A, Wilhelm B, Schmidt D, Broaddus C, Culley S, Rocha-Martins M, Segovia-Miranda F, Norden C, Henriques R, Zerial M, Solimena M, Rink J, Tomancak P, Royer L, Jug F, Myers EW (2018) Content-aware image restoration: pushing the limits of fluorescence microscopy. Nat Methods 15: 1090−1097 doi: 10.1038/s41592-018-0216-7
[21]	Willig KI, Rizzoli SO, Westphal V, Jahn R, Hell SW (2006) STED microscopy reveals that synaptotagmin remains clustered after synaptic vesicle exocytosis. Nature 440: 935−939 doi: 10.1038/nature04592
[22]	Zhang Y, Li K, Li K, Wang L, Zhong B, Fu Y (2018) Image super-resolution using very deep residual channel attention networks. Paper presented at: Proceedings of the European Conference on Computer Vision (ECCV)