Advancing biological super-resolution microscopy through deep learning: a brief review

Tianjie Yang; Yaoru Luo; Wei Ji; Ge Yang

doi:10.52601/bpr.2021.210019

Volume 7 Issue 4

Sep. 2021

Turn off MathJax

Article Contents

Article Navigation > Biophysics Reports > 2021 > 7(4): 253-266

Tianjie Yang, Yaoru Luo, Wei Ji, Ge Yang. Advancing biological super-resolution microscopy through deep learning: a brief review[J]. Biophysics Reports, 2021, 7(4): 253-266. doi: 10.52601/bpr.2021.210019

Citation:

Tianjie Yang, Yaoru Luo, Wei Ji, Ge Yang. Advancing biological super-resolution microscopy through deep learning: a brief review[J]. Biophysics Reports, 2021, 7(4): 253-266. doi: 10.52601/bpr.2021.210019

Citation:

PDF (2777 KB)

Advancing biological super-resolution microscopy through deep learning: a brief review

doi: 10.52601/bpr.2021.210019

Tianjie Yang^{1, 2, 3},
Yaoru Luo^{3, 4},
Wei Ji^{1, 2},
Ge Yang^{3, 4
,
,}

1.
Institute of Biophysics, Chinese Academy of Sciences, Beijing 100101, China
2.
College of Life Sciences, University of Chinese Academy of Sciences, Beijing 100049, China
3.
Laboratory of Computational Biology and Machine Intelligence, School of Artificial Intelligence, University of Chinese Academy of Sciences, Beijing 100190, China
4.
National Laboratory of Pattern Recognition, Institute of Automation, Chinese Academy of Sciences, Beijing 100190, China

Funds: The authors thank Yuting Zhao for her assistance with figure preparation. The work was supported in part by the Strategic Priority Research Program of Chinese Academy of Sciences (XDB37040402 and XDB37040104), the National Natural Science Foundation of China (91954201 under the major research program “Organellar interactomes for cellular homeostasis” and 31971289), the Chinese Academy of Sciences (292019000056), and the University of Chinese Academy of Sciences (115200M001).

More Information

Corresponding author: yangge@ucas.edu.cn (G. Yang)
Received Date: 20 June 2021
Accepted Date: 22 August 2021
Publish Date: 17 September 2021

Abstract

Abstract

Biological super-resolution microscopy is a new generation of imaging techniques that overcome the ~200 nm diffraction limit of conventional light microscopy in spatial resolution. By providing novel spatial or spatiotemporal information on biological processes at nanometer resolution with molecular specificity, it plays an increasingly important role in biomedical sciences. However, its technical constraints also require trade-offs to balance its spatial resolution, temporal resolution, and light exposure of samples. Recently, deep learning has achieved breakthrough performance in many image processing and computer vision tasks. It has also shown great promise in pushing the performance envelope of biological super-resolution microscopy. In this brief review, we survey recent advances in using deep learning to enhance the performance of biological super-resolution microscopy, focusing primarily on computational reconstruction of super-resolution images. Related key technical challenges are discussed. Despite the challenges, deep learning is expected to play an important role in the development of biological super-resolution microscopy. We conclude with an outlook into the future of this new research area.
- Super-resolution microscopy,
- Image super-resolution,
- Deep learning,
- Image reconstruction,
- Fluorescence microscopy

FullText(HTML)

References(120)

References

[1]	Anwar S, Khan S, Barnes N (2020) A deep journey into super-resolution: a Survey. ACM Comput Surv 53(3): 1−34 doi: 10.1145/3390462
[2]	Badrinarayanan V, Kendall A, Cipolla R (2017) Segnet: A deep convolutional encoder-decoder architecture for image segmentation. IEEE T Pattern AnalMachIntell 39(12): 2481−2495 doi: 10.1109/TPAMI.2016.2644615
[3]	Balzarotti F, Eilers Y, Gwosch KC, Gynnå AH, Westphal V, Stefani FD, Elf J, Hell SW (2017) Nanometer resolution imaging and tracking of fluorescent molecules with minimal photon fluxes. Science 355(6325): 606−612 doi: 10.1126/science.aak9913
[4]	Belthangady C, Royer LA (2019) Applications, promises, and pitfalls of deep learning for fluorescence image reconstruction. Nat Methods 16(12): 1215−1225 doi: 10.1038/s41592-019-0458-z
[5]	Betzig E, Patterson GH, Sougrat R, Lindwasser OW, Olenych S, Bonifacino JS, Davidson MW, Lippincott-Schwartz J, Hess HF (2006) Imaging intracellular fluorescent proteins at nanometer resolution. Science 313(5793): 1642−1645 doi: 10.1126/science.1127344
[6]	Boulanger J, Kervrann C, Bouthemy P, Elbau P, Sibarita J, Salamero J (2010) Patch-based nonlocal functional for denoising fluorescence microscopy image sequences. IEEE T Med Imaging 29(2): 442−454 doi: 10.1109/TMI.2009.2033991
[7]	Brown T, Mann B, Ryder N, Subbiah M, Kaplan J, Dhariwal P, Neelakantan A, Shyam P, Sastry G, Askell A, Agarwal S, Herbert-Voss A, Krueger G, Henighan T, Child R, Ramesh A, Ziegler DM, Wu J, Winter C, Hesse C, Chen M, Sigler E, Litwin M, Gray S, Chess B, Clark J, Berner C, McCandlish S, Radford A, Sutskever I, Amodei D (2020) Language models are few-shot learners. https://arxiv.org/abs/2005.14165v4
[8]	Buades A, Coll B, Morel J (2005) A non-local algorithm for image denoising. Proceedings of 2005 IEEE Conference on Computer Vision and Pattern Recognition (CVPR'05). Vol. 2: 60-65. https://doi.org/10.1109/CVPR.2005.38
[9]	Caicedo JC, Goodman A, Karhohs KW, Cimini BA, Ackerman J, Haghighi M, Heng C, Becker T, Doan M, McQuin C, Rohban M, Singh S, Carpenter AE (2019) Nucleus segmentation across imaging experiments: the 2018 Data Science Bowl. Nat Methods 16(12): 1247−1253 doi: 10.1038/s41592-019-0612-7
[10]	Chai X, Ba Q, Yang G (2018) Characterizing robustness and sensitivity of convolutional neural networks for quantitative analysis of mitochondrial morphology. Quant Biol 6(4): 344−358 doi: 10.1007/s40484-018-0156-3
[11]	Chen B, Gong W, Yang Z, Pan W, Verwilst P, Shin J, Yan W, Liu L, Qu J, Kim JS (2020) STORM imaging of mitochondrial dynamics using a vicinal-dithiol-proteins-targeted probe. Biomaterials 243: 119938 doi: 10.1016/j.biomaterials.2020.119938
[12]	Cheng Y, Wang D, Zhou P, Zhang T (2017) A survey of model compression and acceleration for deep neural networks. https://arxiv.org/abs/1710.09282v9
[13]	Ciaparrone G, Luque Sánchez F, Tabik S, Troiano L, Tagliaferri R, Herrera F (2020) Deep learning in video multi-object tracking: a survey. Neurocomputing 381: 61−88 doi: 10.1016/j.neucom.2019.11.023
[14]	Culley S, Albrecht D, Jacobs C, Pereira PM, Leterrier C, Mercer J, Henriques R (2018) Quantitative mapping and minimization of super-resolution optical imaging artifacts. Nat Methods 15(4): 263−266 doi: 10.1038/nmeth.4605
[15]	Dai J, Li Y, He K, Sun J (2016) R-FCN: Object detection via region-based fully convolutional networks. https://arxiv.org/abs/1605.06409v2
[16]	Dertinger T, Colyer R, Iyer G, Weiss S, Enderlein J (2009) Fast, background-free, 3D super-resolution optical fluctuation imaging (SOFI). Proc Natl Acad Sci USA 106(52): 22287−22292 doi: 10.1073/pnas.0907866106
[17]	Dong C, Loy CC, He KM, Tang XO (2016) Image super-resolution using deep convolutional networks. IEEE T Pattern AnalMachIntell 38(2): 295−307 doi: 10.1109/TPAMI.2015.2439281
[18]	Fang LJ, Monroe F, Novak SW, Kirk L, Schiavon CR, Yu SB, Zhang T, Wu M, Kastner K, Latif AA, Lin ZJ, Shaw A, Kubota Y, Mendenhall J, Zhang Z, Pekkurnaz G, Harris K, Howard J, Manor U (2021) Deep learning-based point-scanning super-resolution imaging. Nat Methods 18(4): 406−416 doi: 10.1038/s41592-021-01080-z
[19]	Farsiu S, Robinson D, Elad M, Milanfar P (2004) Advances and challenges in super-resolution. Int J Imaging Syst Technol 14(2): 47−57 doi: 10.1002/ima.20007
[20]	Feng Y, Chai X, Ba Q, Yang G (2019) Quality assessment of synthetic fluorescence microscopy images for image segmentation. Proceedings of 2019 IEEE International Conference on Image Processing, pp. 814-818. https://doi.org/10.1109/ICIP.2019.8802971
[21]	Fu Y, Lei Y, Wang T, Curran WJ, Liu T, Yang X (2020) Deep learning in medical image registration: a review. Phys Med Biol 65(20): 20TR01 doi: 10.1088/1361-6560/ab843e
[22]	Gaire SK, Zhang Y, Li H, Yu R, Zhang HF, Ying L (2020) Accelerating multicolor spectroscopic single-molecule localization microscopy using deep learning. Biomed Opt Express 11(5): 2705−2721 doi: 10.1364/BOE.391806
[23]	Garcia-Garcia A, Orts S, Oprea S, Villena-Martinez V, Rodríguez J (2017) A review on deep learning techniques applied to semantic segmentation. https://arxiv.org/abs/1704.06857 a
[24]	Giepmans BNG, Adams SR, Ellisman MH, Tsien RY (2006) The fluorescent toolbox for assessing protein location and function. Science 312(5771): 217−224 doi: 10.1126/science.1124618
[25]	Girshick R (2015) Fast R-CNN. Proceedings of the IEEE International Conference on Computer Vision (ICCV), pp: 1440-1448. https://doi.org/10.1109/ICCV.2015.169
[26]	Girshick R, Donahue J, Darrell T, Malik (2014) Rich feature hierarchies for accurate object detection and semantic segmentation. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp: 580-587. https://doi.org/10.1109/CVPR.2014.81
[27]	Gu L, Li Y, Zhang S, Xue Y, Li W, Li D, Xu T, Ji W (2019) Molecular resolution imaging by repetitive optical selective exposure. Nat Methods 16(11): 1114−1118 doi: 10.1038/s41592-019-0544-2
[28]	Gu L, Li Y, Zhang S, Zhou M, Xue Y, Li W, Xu T, Ji W (2021) Molecular-scale axial localization by repetitive optical selective exposure. Nat Methods 18(4): 369−373 doi: 10.1038/s41592-021-01099-2
[29]	Gu L, Sheng Y, Chen Y, Chang H, Zhang Y, Lv P, Ji W, Xu T (2014) High-density 3D single molecular analysis based on compressed sensing. Biophys J 106(11): 2443−2449 doi: 10.1016/j.bpj.2014.04.021
[30]	Guo YT, Li D, Zhang SW, Yang YR, Liu JJ, Wang XY, Liu C, Milkie DE, Moore RP, Tulu US, Kiehart DP, Hu JJ, Lippincott-Schwartz J, Betzig E, Li D (2018) Visualizing intracellular organelle and cytoskeletal interactions at nanoscale resolution on millisecond timescales. Cell 175(5): 1430−1442 doi: 10.1016/j.cell.2018.09.057
[31]	Gupta A, Harrison PJ, Wieslander H, Pielawski N, Kartasalo K, Partel G, Solorzano L, Suveer A, Klemm AH, Spjuth O, Sintorn I-M, Wählby C (2019) Deep learning in image cytometry: aReview. Cytometry A 95(4): 366−380 doi: 10.1002/cyto.a.23701
[32]	Gustafsson MGL (2000) Surpassing the lateral resolution limit by a factor of two using structured illumination microscopy. J Microsc 198(2): 82−87 doi: 10.1046/j.1365-2818.2000.00710.x
[33]	Gustafsson N, Culley S, Ashdown G, Owen DM, Pereira PM, Henriques R (2016) Fast live-cell conventional fluorophore nanoscopy with ImageJ through super-resolution radial fluctuations. Nat Commun 7(1): 12471 doi: 10.1038/ncomms12471
[34]	Haskins G, Kruger U, Yan P (2020) Deep learning in medical image registration: a survey. Mach Vis Appl 31(1): 8 doi: 10.1007/s00138-020-01060-x
[35]	He F, Tao D (2020) Recent advances in deep learning theory. https://www.arxiv-vanity.com/papers/2012.10931/
[36]	He J, Deng Z, Zhou L, Wang Y, Qiao Y (2019) Adaptive pyramid context network for semantic segmentation. Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pp: 7519-7528. https://doi.org/10.1109/CVPR.2019.00770
[37]	He K, Gkioxari G, Dollár P, Girshick R (2017) Mask R-CNN. Proceedings of the IEEE International Conference on Computer Vision (ICCV), pp: 2961-2969. https://doi.org/10.1109/ICCV.2017.322
[38]	He K, Zhang X, Ren S, Sun J (2016) Deep residual learning for image recognition. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp: 770-778. https://doi.org/10.1109/CVPR.2016.90
[39]	Hell SW, Wichmann J (1994) Breaking the diffraction resolution limit by stimulated-emission - stimulated-emission-depletion fluorescence microscopy. Opt Lett 19(11): 780−782 doi: 10.1364/OL.19.000780
[40]	Hershko E, Weiss LE, Michaeli T, Shechtman Y (2019) Multicolor localization microscopy and point-spread-function engineering by deep learning. Opt Express 27(5): 6158−6183 doi: 10.1364/OE.27.006158
[41]	Hoffman DP, Slavitt I, Fitzpatrick CA (2021) The promise and peril of deep learning in microscopy. Nat Methods 18(2): 131−132 doi: 10.1038/s41592-020-01035-w
[42]	Hofmann M, Eggeling C, Jakobs S, Hell SW (2005) Breaking the diffraction barrier in fluorescence microscopy at low light intensities by using reversibly photoswitchable proteins. Proc Natl Acad Sci USA 102(49): 17565−17569 doi: 10.1073/pnas.0506010102
[43]	Huang F, Hartwich TMP, Rivera-Molina FE, Lin Y, Duim WC, Long JJ, Uchil PD, Myers JR, Baird MA, Mothes W, Davidson MW, Toomre D, Bewersdorf J (2013) Video-rate nanoscopy using sCMOS camera–specific single-molecule localization algorithms. Nat Methods 10(7): 653−658 doi: 10.1038/nmeth.2488
[44]	Jadon S (2020) A survey of loss functions for semantic segmentation. 2020 IEEE Conference on Computational Intelligence in Bioinformatics and Computational Biology (CIBCB), pp. 1-7. https://arxiv.org/abs/2006.14822
[45]	Jin L, Liu B, Zhao F, Hahn S, Dong B, Song R, Elston TC, Xu Y, Hahn KM (2020) Deep learning enables structured illumination microscopy with low light levels and enhanced speed. Nat Commun 11(1): 1934 doi: 10.1038/s41467-020-15784-x
[46]	Jones SA, Shim S-H, He J, Zhuang X (2011) Fast, three-dimensional super-resolution imaging of live cells. Nat Methods 8(6): 499−505 doi: 10.1038/nmeth.1605
[47]	Karhunen J, Raiko T, Cho K (2015) Chapter 7 - Unsupervised deep learning: ashort review. In: Bingham E, Kaski S, Laaksonen J and Lampinen J (eds). Advances in Independent Component Analysis and Learning Machines. Academic Press. pp 125-142
[48]	Khan S, Naseer M, Hayat M, Zamir SW, Khan F, Shah M (2021) Transformers in vision: a survey. https://export.arxiv.org/abs/2101.01169
[49]	Klar TA, Jakobs S, Dyba M, Egner A, Hell SW (2000) Fluorescence microscopy with diffraction resolution barrier broken by stimulated emission. Proc Natl Acad Sci USA 97(15): 8206−8210 doi: 10.1073/pnas.97.15.8206
[50]	Kolesnikov A, Beyer L, Zhai X, Puigcerver J, Yung J, Gelly S, Houlsby N (2019) Big transfer (bit): general visual representation learning. https://arxiv.org/abs/1912.11370v3
[51]	Krizhevsky A, Sutskever I, Hinton GE (2012) Imagenet classification with deep convolutional neural networks. Adv Neural Inform Process Syst 25: 1097−1105
[52]	Laissue PP, Alghamdi RA, Tomancak P, Reynaud EG, Shroff H (2017) Assessing phototoxicity in live fluorescence imaging. Nat Methods 14(7): 657−661 doi: 10.1038/nmeth.4344
[53]	Law H, Deng J (2018) Cornernet: Detecting objects as paired keypoints. Proceedings of the European Conference on Computer Vision (ECCV), pp: 734-750. https://doi.org/10.1007/s11263-019-01204-1
[54]	LeCun Y, Bengio Y, Hinton G (2015) Deep learning. Nature 521(7553): 436−444 doi: 10.1038/nature14539
[55]	Le QV, Ngiam J, Coates A, Lahiri A, Prochnow B, Ng A (2011) On optimization methods for deep learning. Proceedings of the 2011 International Conference on Machine Learning, pp: 265-272. https://icml.cc/2011/papers/210_icmlpaper.pdf
[56]	Leshno M, Lin VY, Pinkus A, Schocken S (1993) Multilayer feedforward networks with a nonpolynomial activation function can approximate any function. Neural Netw 6(6): 861−867 doi: 10.1016/S0893-6080(05)80131-5
[57]	Li D, Shao L, Chen BC, Zhang X, Zhang MS, Moses B, Milkie DE, Beach JR, Hammer JA, Pasham M, Kirchhausen T, Baird MA, Davidson MW, Xu PY, Betzig E (2015) Extended-resolution structured illumination imaging of endocytic and cytoskeletal dynamics. Science 349(6251): aab3500 doi: 10.1126/science.aab3500
[58]	Li M, Shan H, Pryshchep S, Lopez MM, Wang G (2020) Deep adversarial network for super stimulated emission depletion imaging. J Nanophoton 14(1): 016009 doi: 10.1117/1.JNP.14.016009
[59]	Li P, Wang D, Wang L, Lu H (2018) Deep visual tracking: Review and experimental comparison. Pattern Recogn 76: 323−338 doi: 10.1016/j.patcog.2017.11.007
[60]	Lichtman JW, Conchello J-A (2005) Fluorescence microscopy. Nat Methods 2(12): 910−919 doi: 10.1038/nmeth817
[61]	Lim B, Son S, Kim H, Nah S, Lee KM (2017) Enhanced deep residual networks for single image super-resolution. 2017 IEEE Conference on Computer Vision and Pattern Recognition Workshops(CVPRW), pp. 1132−1140. https://doi.org/10.1109/CVPRW.2017.151
[62]	Lin D, Ji Y, Lischinski D, Cohen-Or D, Huang H (2018) Multi-scale context intertwining for semantic segmentation. Proceedings of the European Conference on Computer Vision (ECCV), pp: 603-619. https://doi.org/10.1007/978-3-030-01219-9_37
[63]	Lin T-Y, Goyal P, Girshick R, He K, Dollár P (2017) Focal loss for dense object detection. Proceedings of the IEEE International Conference on Computer Vision (ICCV), pp: 2980-2988. https://doi.org/10.1109/ICCV.2017.324
[64]	Lin Y, Long JJ, Huang F, Duim WC, Kirschbaum S, Zhang Y, Schroeder LK, Rebane AA, Velasco MGM, Virrueta A, Moonan DW, Jiao J, Hernandez SY, Zhang Y, Bewersdorf J (2015) Quantifying and optimizing single-molecule switching nanoscopy at high speeds. PLoS One 10(5): e0128135 doi: 10.1371/journal.pone.0128135
[65]	Liu L, Ouyang W, Wang X, Fieguth P, Chen J, Liu X, Pietikäinen M (2020) Deep learning for generic object detection: a survey. Int J Comput Vis 128(2): 261−318 doi: 10.1007/s11263-019-01247-4
[66]	Liu W, Anguelov D, Erhan D, Szegedy C, Reed S, Fu C-Y, Berg AC (2016) Ssd: Single shot multibox detector. Proceedings of the European Conference on Computer Vision (ECCV). Springer. pp: 21-37. https://doi.org/10.1007/978-3-319-46448-0_2
[67]	Liu Z, Lin Y, Cao Y, Hu H, Wei Y, Zhang Z, Lin S, Guo B (2021) Swin transformer: hierarchical vision transformer using shifted windows. https://arxiv.org/abs/2103.14030v1?ref=hackernoon.com
[68]	Long J, Shelhamer E, Darrell T (2015) Fully convolutional networks for semantic segmentation. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp: 3431-3440. https://doi.org/10.1109/TPAMI.2016.2572683
[69]	Luisier F, Blu T, Unser M (2011) Image denoising in mixed Poisson–Gaussian noise. IEEE T Image Process 20(3): 696−708 doi: 10.1109/TIP.2010.2073477
[70]	Lu Z, Pu H, Wang F, Hu Z, Wang L (2017) The expressive power of neural networks: a view from the width. Proceedings of International Conference on Neural Information Processing Systems (NIPS), pp: 6234-6240. https://proceedings.neurips.cc/paper/2017/file/32cbf687880eb1674a07bf717761dd3a-Paper.pdf
[71]	McCann MT, Jin KH, Unser M (2017) Convolutional neural networks for inverse problems in imaging: a review. IEEE Signal Process Mag 34(6): 85−95 doi: 10.1109/MSP.2017.2739299
[72]	Möckl L, Roy AR, Moerner WE (2020) Deep learning in single-molecule microscopy: fundamentals, caveats, and recent developments. Biomed Opt Express 11(3): 1633−1661 doi: 10.1364/BOE.386361
[73]	Moen E, Bannon D, Kudo T, Graf W, Covert M, Van Valen D (2019) Deep learning for cellular image analysis. Nat Methods 16(12): 1233−1246 doi: 10.1038/s41592-019-0403-1
[74]	Nehme E, Freedman D, Gordon R, Ferdman B, Weiss LE, Alalouf O, Naor T, Orange R, Michaeli T, Shechtman Y (2020) DeepSTORM3D: dense 3D localization microscopy and PSF design by deep learning. Nat Methods 17(7): 734−740 doi: 10.1038/s41592-020-0853-5
[75]	Nehme E, Weiss LE, Michaeli T, Shechtman Y (2018) Deep-STORM: super-resolution single-molecule microscopy by deep learning. Optica 5(4): 458−464 doi: 10.1364/OPTICA.5.000458
[76]	Ouyang W, Aristov A, Lelek M, Hao X, Zimmer C (2018) Deep learning massively accelerates super-resolution localization microscopy. Nat Biotechnol 36(5): 460−468 doi: 10.1038/nbt.4106
[77]	Ovesný M, Křížek P, Švindrych Z, Hagen GM (2014) High density 3D localization microscopy using sparse support recovery. Opt Express 22(25): 31263−31276 doi: 10.1364/OE.22.031263
[78]	Qiao C, Li D, Guo Y, Liu C, Jiang T, Dai Q, Li D (2021) Evaluation and development of deep neural networks for image super-resolution in optical microscopy. Nat Methods 18(2): 194−202 doi: 10.1038/s41592-020-01048-5
[79]	Redmon J, Divvala S, Girshick R, Farhadi A (2016) You only look once: unified, real-time object detection. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp: 779-788. https://doi.org/10.1109/CVPR.2016.91
[80]	Rawat W, Wang Z (2017) Deep convolutional neural networks for image classification: a comprehensive review. Neural Comput 29(9): 2352−2449 doi: 10.1162/neco_a_00990
[81]	Rego EH, Shao L, Macklin JJ, Winoto L, Johansson GA, Kamps-Hughes N, Davidson MW, Gustafsson MGL (2012) Nonlinear structured-illumination microscopy with a photoswitchable protein reveals cellular structures at 50-nm resolution. Proc Natl Acad Sci USA 109(3): E135−E143 doi: 10.1073/pnas.1107547108
[82]	Ren S, He K, Girshick R, Sun J (2015) Faster R-CNN: towards real-time object detection with region proposal networks. https://arxiv.org/abs/1506.01497v1
[83]	Ronneberger O, Fischer P, Brox T (2015) U-net: convolutional networks for biomedical image segmentation. International Conference on Medical Image Computing and Computer-Assisted Intervention (MICCAI). Springer. pp: 234-241. https://doi.org/10.1007/978-3-319-24574-4_28
[84]	Russakovsky O, Deng J, Su H, Krause J, Satheesh S, Ma S, Huang Z, Karpathy A, Khosla A, Bernstein M (2015) Imagenet large scale visual recognition challenge. Int J Comput Vis 115(3): 211−252 doi: 10.1007/s11263-015-0816-y
[85]	Rust MJ, Bates M, Zhuang XW (2006) Sub-diffraction-limit imaging by stochastic optical reconstruction microscopy (STORM). Nat Methods 3(10): 793−795 doi: 10.1038/nmeth929
[86]	Sage D, Pham T-A, Babcock H, Lukes T, Pengo T, Chao J, Velmurugan R, Herbert A, Agrawal A, Colabrese S, Wheeler A, Archetti A, Rieger B, Ober R, Hagen GM, Sibarita J-B, Ries J, Henriques R, Unser M, Holden S (2019) Super-resolution fight club: assessment of 2D and 3D single-molecule localization microscopy software. Nat Methods 16(5): 387−395 doi: 10.1038/s41592-019-0364-4
[87]	Sahl SJ, Hell SW, Jakobs S (2017) Fluorescence nanoscopy in cell biology. Nat Rev Mol Cell Biol 18(11): 685−701
[88]	Schmarje L, Santarossa M, Schröder SM, Koch R (2021) A survey on semi-, self- and unsupervised learning for image classification. IEEE Access 9: 82146−82168 doi: 10.1109/ACCESS.2021.3084358
[89]	Shorten C, Khoshgoftaar TM (2019) A survey on image data augmentation for deep learning. J Big Data 6(1): 60 doi: 10.1186/s40537-019-0197-0
[90]	Sigal YM, Zhou R, Zhuang X (2018) Visualizing and discovering cellular structures with super-resolution microscopy. Science 361(6405): 880−887 doi: 10.1126/science.aau1044
[91]	Simonyan K, Zisserman A (2014) Very deep convolutional networks for large-scale image recognition. https://arxiv.org/abs/1409.1556v4
[92]	Sun R (2019) Optimization for deep learning: theory and algorithms. https://arxiv.org/abs/1912.08957v1
[93]	Szegedy C, Liu W, Jia Y, Sermanet P, Reed S, Anguelov D, Erhan D, Vanhoucke V, Rabinovich A (2015) Going deeper with convolutions. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp: 1-9. https://doi.org/10.1109/CVPR.2015.7298594
[94]	Tian C, Fei L, Zheng W, Xu Y, Zuo W, Lin C-W (2020) Deep learning on image denoising: an overview. Neural Netw 131: 251-275
[95]	Valli J, Garcia-Burgos A, Rooney LM, Vale de Melo e Oliveira B, Duncan RR, Rickman C (2021) Seeing beyond the limit: a guide to choosing the right super resolution microscopy technique. J Biol Chem 297(1): 100791 doi: 10.1016/j.jbc.2021.100791
[96]	van Engelen JE, Hoos HH (2020) A survey on semi-supervised learning. Mach Learn 109(2): 373−440 doi: 10.1007/s10994-019-05855-6
[97]	van Ouwerkerk JD (2006) Image super-resolution survey. Image Vis Comput 24(10): 1039−1052 doi: 10.1016/j.imavis.2006.02.026
[98]	Wagner N, Beuttenmueller F, Norlin N, Gierten J, Boffi JC, Wittbrodt J, Weigert M, Hufnagel L, Prevedel R, Kreshuk A (2021) Deep learning-enhanced light-field imaging with continuous validation. Nat Methods 18(5): 557−563 doi: 10.1038/s41592-021-01136-0
[99]	Wang HD, Rivenson Y, Jin YY, Wei ZS, Gao R, Gunaydin H, Bentolila LA, Kural C, Ozcan A (2019a) Deep learning enables cross-modality super-resolution in fluorescence microscopy. Nat Methods 16(1): 103−110 doi: 10.1038/s41592-018-0239-0
[100]	Wang J, Sun K, Cheng T, Jiang B, Deng C, Zhao Y, Liu D, Mu Y, Tan M, Wang X (2020a) Deep high-resolution representation learning for visual recognition. IEEE T Pattern Anal Mach Intell doi: 10.1109/TPAMI.2020.2983686
[101]	Wang Q, Ma Y, Zhao K, Tian Y (2020b) A comprehensive survey of loss functions in machine learning. Ann Data Sci doi: 10.1007/s40745-020-00253-5
[102]	Wang Z, Chen J, Hoi SCH (2020c) Deep learning for image super-resolution: a survey. IEEE T Pattern Anal Mach Intell doi: 10.1109/TPAMI.2020.2982166
[103]	Wang Z, She Q, Ward T (2019b) Generative adversarial networks: a survey and taxonomy. https://arxiv.org/abs/1906.01529v1
[104]	Weigert M, Schmidt U, Boothe T, Müller A, Dibrov A, Jain A, Wilhelm B, Schmidt D, Broaddus C, Culley S, Rocha-Martins M, Segovia-Miranda F, Norden C, Henriques R, Zerial M, Solimena M, Rink J, Tomancak P, Royer L, Jug F, Myers EW (2018) Content-aware image restoration: pushing the limits of fluorescence microscopy. Nat Methods 15(12): 1090−1097 doi: 10.1038/s41592-018-0216-7
[105]	Wolf T, Debut L, Sanh V, Chaumond J, Delangue C, Moi A, Cistac P, Rault T, Louf Re, Funtowicz M, Brew J (2019) HuggingFace's transformers: state-of-the-art natural language processing. https://arxiv.org/abs/1910.03771
[106]	Wu Y, Rivenson Y, Wang H, Luo Y, Ben-David E, Bentolila LA, Pritz C, Ozcan A (2019) Three-dimensional virtual refocusing of fluorescence microscopy images using deep learning. Nat Methods 16(12): 1323−1331 doi: 10.1038/s41592-019-0622-5
[107]	Wu Z, Pan S, Chen F, Long G, Zhang C, Yu PS (2021) A comprehensive survey on graph neural networks. IEEE T Neural Netw Learn Syst 32: 4−24 doi: 10.1109/TNNLS.2020.2978386
[108]	Xing F, Xie Y, Su H, Liu F, Yang L (2018) Deep learning in microscopy image analysis: a survey. IEEE T Neural Netw Learn Syst 29: 4550−4568 doi: 10.1109/TNNLS.2017.2766168
[109]	Yang J, Huang T (2017) Image super-resolution: historical overview and future challenges. In: Milanfar P (eds). Super-resolution imaging. CRC Press. pp 1-33
[110]	Yang L, Shami A (2020) On hyperparameter optimization of machine learning algorithms: theory and practice. Neurocomputing 415: 295−316 doi: 10.1016/j.neucom.2020.07.061
[111]	Yang WM, Zhang XC, Tian YP, Wang W, Xue JH, Liao QM (2019) Deep learning for single image super-resolution: a brief review. IEEE T Multimedia 21(12): 3106−3121 doi: 10.1109/TMM.2019.2919431
[112]	Yu T, Zhu H (2020) Hyper-parameter optimization: a review of algorithms and applications. https://arxiv.org/abs/2003.05689
[113]	Yu Y, Si X, Hu C, Zhang J (2019) A review of recurrent neural networks: LSTM cells and network architectures. Neural Comput 31(7): 1235−1270 doi: 10.1162/neco_a_01199
[114]	Zelger P, Kaser K, Rossboth B, Velas L, Schütz GJ, Jesacher A (2018) Three-dimensional localization microscopy using deep learning. Opt Express 26(25): 33166−33179 doi: 10.1364/OE.26.033166
[115]	Zhang P, Liu S, Chaurasia A, Ma D, Mlodzianoski MJ, Culurciello E, Huang F (2018) Analyzing complex single-molecule emission patterns with deep learning. Nat Methods 15(11): 913−916 doi: 10.1038/s41592-018-0153-5
[116]	Zhang Q-S, Zhu S-C (2018) Visual interpretability for deep learning: a survey. Front Inform Technol Electron Eng 19(1): 27−39 doi: 10.1631/FITEE.1700808
[117]	Zhang Y, Zhu Y, Nichols E, Wang Q, Zhang S, Smith CJ, Howard S (2019) A Poisson-Gaussian denoising dataset with real fluorescence microscopy images. 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pp. 11702-11710. https://doi.org.10.1109/CVPR.2019.01198
[118]	Zhao H, Shi J, Qi X, Wang X, Jia J (2017) Pyramid scene parsing network. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp: 2881-2890. https://doi.org/10.1109/CVPR.2017.660
[119]	Zhao Z, Zheng P, Xu S, Wu X (2019) Object detection with deep learning: a review. IEEE T Neural Netw Learn Syst 30(11): 3212−3232 doi: 10.1109/TNNLS.2018.2876865
[120]	Zhou J, Cui G, Hu S, Zhang Z, Yang C, Liu Z, Wang L, Li C, Sun M (2020) Graph neural networks: a review of methods and applications. AI Open 1: 57−81 doi: 10.1016/j.aiopen.2021.01.001