U-Net: deep learning for cell counting, detection, and morphometry (2024)

Table of Contents
Data availability Change history References Acknowledgements Author information Authors and Affiliations Contributions Corresponding author Ethics declarations Competing interests Additional information Integrated supplementary information Supplementary Figure 1 The U-Net architecture in the example of a 2D cell segmentation network. Supplementary Figure 2 Separation of touching cells by using pixelwise loss weights. Supplementary Figure 3 Training data augmentation through random smooth elastic deformation. Supplementary information Supplementary Text and Figures Reporting Summary Supplementary Software 1 Supplementary Software 2 Supplementary Software 3 Supplementary Software 4 Rights and permissions About this article Cite this article References

Data availability

Datasets F1-MSC, F2-GOWT1, F3-SIM, F4-HeLa, DIC1-HeLa, PC1-U373 and PC2-PSC are from the ISBI Cell Tracking Challenge 2015 (ref. 17). Information on how to obtain the data can be found at http://celltrackingchallenge.net/datasets.html, and free registration for the challenge is currently required. Datasets PC3-HKPV, BF1-POL, BF2-PPL and BF3-MiSp are custom and are available from the corresponding author upon reasonable request. Datasets for the detection experiments partially contain unpublished sample-preparation protocols and are currently not freely available. After protocol publication, datasets will be made available on an as-requested basis. Details on sample preparation for our life science experiments can be found in Supplementary Note 3 and the Life Sciences Reporting Summary.

Change history

References

  1. Sommer, C, Strähle, C, Koethe, U. & Hamprecht, F. A. in Ilastik: interactive learning and segmentation toolkit in IEEE Int. Symp. Biomed. Imaging. 230–233 (IEEE: Piscataway, NJ, USA, 2011).

  2. Arganda-Carreras, I. et al. Bioinformatics 33, 2424–2426 (2017).

    Article CAS Google Scholar

  3. Ronneberger, O., Fischer, P. & Brox, T. U-Net: convolutional networks for biomedical image segmentation. in Medical Image Computing and Computer-Assisted Intervention—MICCAI 2015 Vol. 9351, 234–241 (Springer, Cham, Switzerland, 2015).

  4. Rusk, N. Nat. Methods 13, 35 (2016).

    Article CAS Google Scholar

  5. Webb, S. Nature 554, 555–557 (2018).

    Article CAS Google Scholar

  6. Sadanandan, S. K., Ranefall, P., Le Guyader, S. & Wählby, C. Sci. Rep. 7, 7860 (2017).

    Article Google Scholar

  7. Weigert, M. et al. Nat. Methods https://doi.org/10.1038/s41592-018-0216-7 (2018).

  8. Haberl, M. G. et al. Nat. Methods 15, 677–680 (2018).

    Article CAS Google Scholar

  9. Ulman, V. et al. Nat. Methods 14, 1141–1152 (2017).

    Article CAS Google Scholar

  10. Schneider, C. A., Rasband, W. S. & Eliceiri, K. W. Nat. Methods 9, 671–675 (2012).

    Article CAS Google Scholar

  11. Long, J., Shelhamer, E. & Darrell, T. Fully convolutional networks for semantic segmentation. in IEEE Conf. Comput. Vis. Pattern Recognit. (CVPR) 3431–3440 (IEEE, Piscataway, NJ, USA, 2015).

  12. Simonyan, K. & Zisserman, A. Preprint at https://arxiv.org/abs/1409.1556 (2014)

  13. Çiçek, Ö., Abdulkadir, A., Lienkamp, S. S., Brox, T. & Ronneberger, O. 3D U-Net: learning dense volumetric segmentation from sparse annotation. in Medical Image Computing and Computer-Assisted Intervention—MICCAI 2016 Vol. 9901, 424–432 (Springer, Cham, Switzerland, 2016).

  14. Jia, Y. et al. Preprint at https://arxiv.org/abs/1408.5093 (2014).

  15. He, K., Zhang, X., Ren, S. & Sun, J. Preprint at https://arxiv.org/abs/1502.01852 (2015).

  16. Everingham, M., Van Gool, L., Williams, C. K. I., Winn, J. & Zisserman, A. Int. J. Comput. Vis. 88, 303–338 (2010).

    Article Google Scholar

  17. Maška, M. et al. Bioinformatics 30, 1609–1617 (2014).

    Article Google Scholar

Download references

Acknowledgements

This work was supported by the German Federal Ministry for Education and Research (BMBF) through the MICROSYSTEMS project (0316185B) to T.F. and A.D.; the Bernstein Award 2012 (01GQ2301) to I.D.; the Federal Ministry for Economic Affairs and Energy (ZF4184101CR5) to A.B.; the Deutsche Forschungsgemeinschaft (DFG) through the collaborative research center KIDGEM (SFB 1140) to D.M., Ö.Ç., T.F. and O.R., and (SFB 746, INST 39/839,840,841) to K.P.; the Clusters of Excellence BIOSS (EXC 294) to T.F., D.M., R.B., A.A., Y.M., D.S., T.L.T., M.P., K.P., M.S., T.B. and O.R.; BrainLinks-Brain-Tools (EXC 1086) to Z.J., K.S., I.D. and T.B.; grants DI 1908/3-1 to J.D., DI 1908/6-1 to Z.J. and K.S., and DI 1908/7-1 to I.D.; the Swiss National Science Foundation (SNF grant 173880) to A.A.; the ERC Starting grant OptoMotorPath (338041) to I.D.; and the FENS-Kavli Network of Excellence (FKNE) to I.D. We thank F. Prósper, E. Bártová, V. Ulman, D. Svoboda, G. van Cappellen, S. Kumar, T. Becker and the Mitocheck consortium for providing a rich diversity of datasets through the ISBI segmentation challenge. We thank P. Fischer for manual image annotations. We thank S. Wrobel for tobacco microspore preparation.

Author information

Author notes

  1. Dominic Mai

    Present address: SICK AG, Waldkirch, Germany

  2. Robert Bensch

    Present address: ANavS GmbH, München, Germany

  3. Alexander Dovzhenko,Olaf Tietz&Sean Walsh

    Present address: ScreenSYS GmbH, Freiburg, Germany

  4. Olaf Ronneberger

    Present address: DeepMind, London, UK

  5. These authors contributed equally: Thorsten Falk, Dominic Mai, Robert Bensch.

Authors and Affiliations

  1. Department of Computer Science, Albert-Ludwigs-University, Freiburg, Germany

    Thorsten Falk,Dominic Mai,Robert Bensch,Özgün Çiçek,Ahmed Abdulkadir,Yassine Marrakchi,Anton Böhm,Thomas Brox&Olaf Ronneberger

  2. BIOSS Centre for Biological Signalling Studies, Freiburg, Germany

    Thorsten Falk,Dominic Mai,Robert Bensch,Yassine Marrakchi,Deniz Saltukoglu,Marco Prinz,Klaus Palme,Matias Simons,Thomas Brox&Olaf Ronneberger

  3. CIBSS Centre for Integrative Biological Signalling Studies, Albert-Ludwigs-University, Freiburg, Germany

    Thorsten Falk,Yassine Marrakchi,Marco Prinz&Thomas Brox

  4. Life Imaging Center, Center for Biological Systems Analysis, Albert-Ludwigs-University, Freiburg, Germany

    Dominic Mai

  5. University Hospital of Old Age Psychiatry and Psychotherapy, University of Bern, Bern, Switzerland

    Ahmed Abdulkadir

    See Also
    U-NET: Alles über das neuronale Netzwerk von Computer VisionU-Net: Convolutional Networks for Biomedical Image Segmentation- SummarizedU-Net: A Comprehensive Guide to Its Architecture and Applications - viso.aiU-Net Image Segmentation in Keras - PyImageSearch

  6. Optophysiology Lab, Institute of Biology III, Albert-Ludwigs-University, Freiburg, Germany

    Jan Deubner,Zoe Jäckel,Katharina Seiwald&Ilka Diester

  7. BrainLinks-BrainTools, Albert-Ludwigs-University, Freiburg, Germany

    Jan Deubner,Zoe Jäckel,Tuan Leng Tay,Ilka Diester&Thomas Brox

  8. Institute of Biology II, Albert-Ludwigs-University, Freiburg, Germany

    Alexander Dovzhenko,Olaf Tietz,Cristina Dal Bosco,Sean Walsh&Klaus Palme

  9. Center for Biological Systems Analysis (ZBSA), Albert-Ludwigs-University, Freiburg, Germany

    Deniz Saltukoglu&Matias Simons

  10. Renal Division, University Medical Centre, Freiburg, Germany

    Deniz Saltukoglu&Matias Simons

  11. Spemann Graduate School of Biology and Medicine (SGBM), Albert-Ludwigs-University, Freiburg, Germany

    Deniz Saltukoglu

  12. Institute of Neuropathology, University Medical Centre, Freiburg, Germany

    Tuan Leng Tay&Marco Prinz

  13. Institute of Biology I, Albert-Ludwigs-University, Freiburg, Germany

    Tuan Leng Tay

  14. Paris Descartes University-Sorbonne Paris Cité, Imagine Institute, Paris, France

    Matias Simons

  15. Bernstein Center Freiburg, Albert-Ludwigs-University, Freiburg, Germany

    Ilka Diester

Authors

  1. Thorsten Falk

    View author publications

    You can also search for this author in PubMedGoogle Scholar

  2. Dominic Mai

    View author publications

    You can also search for this author in PubMedGoogle Scholar

  3. Robert Bensch

    View author publications

    You can also search for this author in PubMedGoogle Scholar

  4. Özgün Çiçek

    View author publications

    You can also search for this author in PubMedGoogle Scholar

  5. Ahmed Abdulkadir

    View author publications

    You can also search for this author in PubMedGoogle Scholar

  6. Yassine Marrakchi

    View author publications

    You can also search for this author in PubMedGoogle Scholar

  7. Anton Böhm

    View author publications

    You can also search for this author in PubMedGoogle Scholar

  8. Jan Deubner

    You can also search for this author in PubMedGoogle Scholar

  9. Zoe Jäckel

    View author publications

    You can also search for this author in PubMedGoogle Scholar

  10. Katharina Seiwald

    View author publications

    You can also search for this author in PubMedGoogle Scholar

  11. Alexander Dovzhenko

    View author publications

    You can also search for this author in PubMedGoogle Scholar

  12. Olaf Tietz

    View author publications

    You can also search for this author in PubMedGoogle Scholar

  13. Cristina Dal Bosco

    View author publications

    You can also search for this author in PubMedGoogle Scholar

  14. Sean Walsh

    View author publications

    You can also search for this author in PubMedGoogle Scholar

  15. Deniz Saltukoglu

    View author publications

    You can also search for this author in PubMedGoogle Scholar

  16. Tuan Leng Tay

    View author publications

    You can also search for this author in PubMedGoogle Scholar

  17. Marco Prinz

    View author publications

    You can also search for this author in PubMedGoogle Scholar

  18. Klaus Palme

    View author publications

    You can also search for this author in PubMedGoogle Scholar

  19. Matias Simons

    View author publications

    You can also search for this author in PubMedGoogle Scholar

  20. Ilka Diester

    View author publications

    You can also search for this author in PubMedGoogle Scholar

  21. Thomas Brox

    View author publications

    You can also search for this author in PubMedGoogle Scholar

  22. Olaf Ronneberger

    View author publications

    You can also search for this author in PubMedGoogle Scholar

Contributions

T.F., D.M., R.B., Y.M., Ö.Ç., T.B. and O.R. selected and designed the computational experiments. T.F., R.B., D.M., Y.M., A.B. and Ö.Ç. performed the experiments: R.B., D.M., Y.M. and A.B. (2D), and T.F. and Ö.Ç. (3D). R.B., Ö.Ç., A.A., T.F. and O.R. implemented the U-Net extensions into caffe. T.F. designed and implemented the Fiji plugin. D.S. and M.S. selected, prepared and recorded the keratinocyte dataset PC3-HKPV. T.F. and O.R. prepared the airborne-pollen dataset BF1-POL. A.D., S.W., O.T., C.D.B. and K.P. selected, prepared and recorded the protoplast and microspore datasets BF2-PPL and BF3-MiSp. T.L.T. and M.P. prepared, recorded and annotated the data for the microglial proliferation experiment. J.D., K.S. and Z.J. selected, prepared and recorded the optogenetic dataset. I.D., J.D. and Z.J. manually annotated the optogenetic dataset. I.D., T.F., D.M., R.B., Ö.Ç., T.B. and O.R. wrote the manuscript.

Corresponding author

Correspondence to Olaf Ronneberger.

Ethics declarations

Competing interests

The authors declare no competing interests.

Additional information

Publisher’s note: Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Integrated supplementary information

Supplementary Figure 1 The U-Net architecture in the example of a 2D cell segmentation network.

(left) Input: An image tile with 540×540 pixels and C channels (blue box). (right) Output: The K-class soft-max segmentation with 356×356 pixels (yellow box). Blocks show the computed feature hierarchy. Numbers atop each network block: number of feature channels; numbers left to each block: spatial feature map shape in pixels. Yellow arrows: Data flow

Supplementary Figure 2 Separation of touching cells by using pixelwise loss weights.

(a) Generated segmentation mask with one-pixel wide background ridge between touching cells (white: foreground, black: background). (b) Map showing pixel-wise loss weights to enforce the network to separate touching cells

Supplementary Figure 3 Training data augmentation through random smooth elastic deformation.

(a) Upper left: Raw image; Upper right: Labels; Lower Left: Loss Weights; Lower Right: 20μm grid (for illustration purpose only) (b) Deformation field (black arrows) generated using bicubic interpolation from a coarse grid of displacement vectors (blue arrows; magnification: 5×). Vector components are drawn from a Gaussian distribution (σ = 10px). (c) Backwarp-transformed images of (a) using the deformation field

Supplementary information

Supplementary Text and Figures

Supplementary Figures 1–3 and Supplementary Notes 1–3

Supplementary Software 1

Caffe_unet binary package (GPU version with cuDNN, Recommended). Pre-compiled binary version of the caffe_unet backend software for Ubuntu 16.04, cuda 8.0.61 (https://developer.nvidia.com/cuda-80-ga2-download-archive) and cuDNN 7.1.4 for cuda 8.0 (https://developer.nvidia.com/rdp/cudnn-archive). At time of publication cuDNN download from the nVidia website requires free registration as nVidia developer

Supplementary Software 2

Caffe_unet binary package (GPU version no cuDNN). Pre-compiled binary version of the caffe_unet backend software for Ubuntu 16.04 and cuda 8.0.61 (https://developer.nvidia.com/cuda-80-ga2-downloadarchive)

Supplementary Software 3

Caffe_unet binary package (CPU version). Pre-compiled binary version of the caffe_unet backend software for Ubuntu 16.04

Supplementary Software 4

The source code difference (patch file) to the open source caffe deep learning software (https://github.com/BVLC/caffe.git commit hash d1208dbf313698de9ef70b3362c89cfddb51c520). Checkout the correspondingly tagged commit and apply the patch using “git apply” to get the full source for custom builds

Rights and permissions

About this article

U-Net: deep learning for cell counting, detection, and morphometry (1)

Cite this article

Falk, T., Mai, D., Bensch, R. et al. U-Net: deep learning for cell counting, detection, and morphometry. Nat Methods 16, 67–70 (2019). https://doi.org/10.1038/s41592-018-0261-2

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1038/s41592-018-0261-2

U-Net: deep learning for cell counting, detection, and morphometry (2024)

References

Top Articles
Italian Chili ~ Vegan Recipe with VIDEO ~ This Wife Cooks
The Perfect Dumpling Sauce Recipe - The Woks of Life
New Power Generation Shows Itself At Sharjah Masters
Expert Picks: Commercial Bank Qatar Masters - Articles
MovieFire - Movies & WebSeries v1.1.4 Free purchase
8 ET, ESPN: Sho, Alonso highlight Derby
Introduction: Takeovers - A Game-Changing NBA 2K23 Mechanic - DowneLink
دانلود بازی جدید کامپیوتر | مرجع دانلود بهترین بازی‌ها + رایگان | دانلود فارسی
North Park Produce Poway Weekly Ad
Best of the Money blog
Uic Printing Funds
Discover the Magic: Elemental Showtimes You Can't Afford to Miss!
Latest Posts
Recipe This | Instant Pot Popcorn
25 Easy, Last Minute Thanksgiving Side Dish Recipes
Article information

Author: Rubie Ullrich

Last Updated:

Views: 6574

Rating: 4.1 / 5 (72 voted)

Reviews: 87% of readers found this page helpful

Author information

Name: Rubie Ullrich

Birthday: 1998-02-02

Address: 743 Stoltenberg Center, Genovevaville, NJ 59925-3119

Phone: +2202978377583

Job: Administration Engineer

Hobby: Surfing, Sailing, Listening to music, Web surfing, Kitesurfing, Geocaching, Backpacking

Introduction: My name is Rubie Ullrich, I am a enthusiastic, perfect, tender, vivacious, talented, famous, delightful person who loves writing and wants to share my knowledge and understanding with you.