AccScience Publishing / IJB / Online First / DOI: 10.36922/ijb.4035
REVIEW

Deep learning for polymer scaffold bioimage analysis: Opportunities and challenges                                

Jie Sun1,2* Kai Yao1 Hui Zhu3 Kaizhu Huang4* Dejian Huang5
Show Less
1 Department of Mechatronics and Robotics, School of Advanced Technology, Xi’an Jiaotong- Liverpool University, Suzhou, Jiangsu, China
2 State Key Laboratory for Manufacturing Systems Engineering, Xi’an Jiaotong University, Xi’an, Shaanxi, China
3 Department of Mechanical Engineering, School of Mechanical Engineering, Xi’an Jiaotong University, Xi’an, Shaanxi, China
4 Data Science Research Centre, Duke Kunshan University, Kunshan, Jiangsu, China
5 Department of Food Science and Technology, Faculty of Science, National University of Singapore, Singapore
Submitted: 24 June 2024 | Accepted: 18 December 2024 | Published: 18 December 2024
© 2024 by the Author(s). This article is an open access article distributed under the terms and conditions of the Creative Commons Attribution 4.0 International License ( https://creativecommons.org/licenses/by/4.0/ )
Abstract

Significant efforts have been made to advance bioprinted scaffold research in cell biology, tissue engineering, and drug screening studies. Ideal scaffolds should demonstrate suitable mechanical properties, excellent biocompatibility, and bioactivities. However, the design and preparation of such scaffolds are challenging. Imaging modalities, including magnetic resonance imaging, micro-computed tomography, ultrasound imaging, optical coherence tomography, and confocal laser scanning microscopy, are commonly used to visualize the interior architecture of bioprinted scaffolds, as well as the surrounding cells and tissues. The obtained bioimages provide direct insight into the biological functionalities of the scaffold, though their interpretation may lead to differing viewpoints and even debates. This review explores deep learning (DL) methods employed for image analysis, including restoration, segmentation, and classification. First, current DL methods for biological image processing are summarized, such as convolutional neural network, U-Net, and generative adversarial network. The corresponding outcomes of these methods reveal cell–scaffold and tissue–scaffold interactions, providing guidance for scaffold design in specific applications. Thereafter, the challenges and limitations of DL applications are highlighted, such as building DL models using smaller bioimage datasets, interpreting DL models, vision-language model-guided bioimage analysis, and developing intelligent analysis platforms. Hence, this review would mark a paradigm shift in polymer scaffold designs and the associated performance.

Graphical abstract
Keywords
Bioimage analysis
Deep learning
Imaging modalities
Machine learning
Polymer scaffold
Funding
This work was financially supported by the Xi’an Jiaotong- Liverpool University Grant (REF-21-02-001).
Conflict of interest
Jie Sun serves as the Editorial Board Member of the journal but was not in any way involved in the editorial and peer-review process conducted for this paper, directly or indirectly.
References
  1. Jing L, Wang X, Leng B, et al. Engineered nanotopography on the microfibers of 3D-printed PCL scaffolds to modulate cellular responses and establish an in vitro tumor model. ACS Appl Bio Mater. 2021;4(2):1381-1394. doi: 10.1021/acsabm.0c01243
  2. Tian S, Stevens R, McInnes BT, Lewinski NA. Machine assisted experimentation of extrusion-based bioprinting systems. Micromachines (Basel). 2021;12(7):780. doi: 10.3390/mi12070780
  3. Ritzau-Reid KI, Callens SJP, Xie R, et al. Microfibrous scaffolds guide stem cell lumenogenesis and brain organoid engineering. Adv Mater. 2023;35(41):e2300305. doi: 10.1002/adma.202300305
  4. Sun J, Vijayavenkataraman S, Liu H. An overview of scaffold design and fabrication technology for engineered knee meniscus. Materials (Basel). 2017;10(1):29. doi: 10.3390/ma10010029
  5. Gupta V, Khan Y, Berkland CJ, Laurencin CT, Detamore MS. Microsphere-based scaffolds in regenerative engineering. Annu Rev Biomed Eng. 2017;19:135-161. doi: 10.1146/annurev-bioeng-071516-044712
  6. Shepherd DV, Shepherd JH, Best SM, Cameron RE. 3D imaging of cells in scaffolds: direct labelling for micro CT. J Mater Sci Mater Med. 2018;29(6):86. doi: 10.1007/s10856-018-6089-6
  7. Finamore TA, Curtis TE, Tedesco JV, Grandfield K, Roeder RK. Nondestructive, longitudinal measurement of collagen scaffold degradation using computed tomography and gold nanoparticles. Nanoscale. 2019;11(10):4345-4354. doi: 10.1039/c9nr00313d
  8. Zielińska A, Karczewski J, Eder P, et al. Scaffolds for drug delivery and tissue engineering: the role of genetics. J Control Release. 2023;359:207-223. doi: 10.1016/j.jconrel.2023.05.042
  9. Mertens ME, Hermann A, Bühren A, et al. Iron oxide-labelled collagen scaffolds for non-invasive MR imaging in tissue engineering. Adv Funct Mater. 2014;24(6): 754-762. doi: 10.1002/adfm.201301275
  10. Farooq V, Onuma Y, Radu M, et al. Optical coherence tomography (OCT) of overlapping bioresorbable scaffolds: from benchwork to clinical application. EuroIntervention. 2011;7(3):386-399. doi: 10.4244/EIJV7I3A64
  11. Holmes C, Tabrizian M, Bagnaninchi PO. Motility imaging via optical coherence phase microscopy enables label-free monitoring of tissue growth and viability in 3D tissue-engineering scaffolds. J Tissue Eng Regen Med. 2015;9(5):641-645. doi: 10.1002/term.1687
  12. Chen J, Huang D, Wang L, et al. 3D bioprinted multiscale composite scaffolds based on gelatin methacryloyl (GelMA)/ chitosan microspheres as a modular bioink for enhancing 3D neurite outgrowth and elongation. J Colloid Interface Sci. 2020;574:162-173. doi: 10.1016/j.jcis.2020.04.040
  13. Serpooshan V, Zhao M, Metzler SA, et al. The effect of bioengineered acellular collagen patch on cardiac remodeling and ventricular function post myocardial infarction. Biomaterials. 2013;34(36):9048-9055. doi: 10.1016/j.biomaterials.2013.08.017
  14. Loai S, Szulc DA, Cheng HM. Three-dimensional bioprinted MR-trackable regenerative scaffold for postimplantation monitoring on T1-weighted MRI. J Magn Reson Imaging. 2022;56(2):570-578. doi: 10.1002/jmri.28057
  15. Cunha FB, Pomini KT, Plepis AMG, et al. In vivo biological behavior of polymer scaffolds of natural origin in the bone repair process. Molecules. 2021;26(6):1598. doi: 10.3390/molecules26061598
  16. Sun Y, Responte D, Xie H, et al. Nondestructive evaluation of tissue engineered articular cartilage using time-resolved fluorescence spectroscopy and ultrasound backscatter microscopy. Tissue Eng Part C Methods. 2012; 18(3):215-226. doi: 10.1089/ten.TEC.2011.0343
  17. Mueller KMA, Topping GJ, Schwaminger SP, et al. Visualization of USPIO-labelled melt-electrowritten scaffolds by non-invasive magnetic resonance imaging. Biomater Sci. 2021;9(13):4607-4612. doi: 10.1039/d1bm00461a
  18. Chen W, Yang J, Liao W, et al. In vitro remodeling and structural characterization of degradable polymer scaffold-based tissue-engineered vascular grafts using optical coherence tomography. Cell Tissue Res. 2017; 370(3):417-426. doi: 10.1007/s00441-017-2683-z
  19. Yao K, Sun J, Huang K, et al. Analyzing cell-scaffold interaction through unsupervised 3D nuclei segmentation. Int J Bioprint. 2021;8(1):495. doi: 10.18063/ijb.v8i1.495
  20. Monllau JC, Poggioli F, Erquicia J, et al. Magnetic resonance imaging and functional outcomes after a polyurethane meniscal scaffold implantation: minimum 5-year follow-up. Arthroscopy. 2018;34(5):1621-1627. doi: 10.1016/j.arthro.2017.12.019
  21. Li X, Zou Q, Wei J, et al. The degradation regulation of 3D printed scaffolds for promotion of osteogenesis and in vivo tracking. Compos B Eng. 2021;222:109084. doi: 10.1016/j.compositesb.2021.109084
  22. Talukdar Y, Avti P, Sun J, Sitharaman B. Multimodal ultrasound-photoacoustic imaging of tissue engineering scaffolds and blood oxygen saturation in and around the scaffolds. Tissue Eng Part C Methods. 2014;20(5): 440-449. doi: 10.1089/ten.TEC.2013.0203
  23. Wang L, Xu ME, Luo L, Zhou Y, Si P. Iterative feedback bio-printing-derived cell-laden hydrogel scaffolds with optimal geometrical fidelity and cellular controllability. Sci Rep. 2018;8(1):2802. doi: 10.1038/s41598-018-21274-4
  24. Theus AS, Ning L, Kabboul G, et al. 3D bioprinting of nanoparticle-laden hydrogel scaffolds with enhanced antibacterial and imaging properties. iScience. 2022;25(9): 104947. doi: 10.1016/j.isci.2022.104947
  25. Li L, Gil CJ, Finamore TA, et al. Methacrylate-modified gold nanoparticles enable non-invasive monitoring of photocrosslinked hydrogel scaffolds. Adv Nanobiomed Res. 2022;2(7):2200022. doi: 10.1002/anbr.202200022
  26. Mercado KP, Helguera M, Hocking DC, Dalecki D. Estimating cell concentration in three-dimensional engineered tissues using high frequency quantitative ultrasound. Ann Biomed Eng. 2014;42(6):1292-1304. doi: 10.1007/s10439-014-0994-8
  27. Rivero RE, Capella V, Cecilia Liaudat A, et al. Mechanical and physicochemical behavior of a 3D hydrogel scaffold during cell growth and proliferation. RSC Adv. 2020;10(10):5827-5837. doi: 10.1039/c9ra08162c
  28. Sandker MJ, Petit A, Redout EM, et al. In situ forming acyl-capped PCLA-PEG-PCLA triblock copolymer based hydrogels. Biomaterials. 2013;34(32):8002-8011. doi: 10.1016/j.biomaterials.2013.07.046
  29. Zhu W, Chu C, Kuddannaya S, et al. In vivo imaging of composite hydrogel scaffold degradation using CEST MRI and two-color NIR imaging. Adv Funct Mater. 2019;29(36):1903753. doi: 10.1002/adfm.201903753
  30. Leng X, Liu B, Su B, et al. In situ ultrasound imaging of silk hydrogel degradation and neovascularization. J Tissue Eng Regen Med. 2017;11(3):822-830. doi: 10.1002/term.1981
  31. Yao K, Huang K, Sun J, et al. Scaffold-A549: a benchmark 3D fluorescence image dataset for unsupervised nuclei segmentation. Cogn Comput. 2021;13:1603-1608. doi: 10.1007/s12559-021-09944-4
  32. Rafieyan S, Vasheghani-Farahani E, Baheiraei N, Keshavarz H. MLATE: machine learning for predicting cell behavior on cardiac tissue engineering scaffolds. Comput Biol Med. 2023;158:106804. doi: 10.1016/j.compbiomed.2023.106804
  33. Huang K, Hussain A, Wang, Q, et al. Deep Learning: Fundamentals, Theory and Applications. Springer; 2019:2. ISBN:3030060721. doi: 10.1007/978-3-030-06073-2
  34. You S, Fan Y, Chen Y, et al. Advancements and prospects of deep learning in biomaterials evolution. Cell Rep Phys Sci. 2024;5(9):102116. doi: 10.1016/j.xcrp.2024.102116
  35. Huang J, Yang G, Li B, et al. Segmentation of cervical cell images based on generative adversarial networks. IEEE Access. 2021;9:115415-115428. doi: 10.1109/access.2021.3104609
  36. Niioka H, Asatani S, Yoshimura A, Ohigashi H, Tagawa S, Miyake J. Classification of C2C12 cells at differentiation by convolutional neural network of deep learning using phase contrast images. Hum Cell. 2018;31(1):87-93. doi: 10.1007/s13577-017-0191-9
  37. Appel AA, Anastasio MA, Larson JC, Brey EM. Imaging challenges in biomaterials and tissue engineering. Biomaterials. 2013;34(28):6615-6630. doi: 10.1016/j.biomaterials.2013.05.033
  38. Nam SY, Ricles LM, Suggs LJ, Emelianov SY. Imaging strategies for tissue engineering applications. Tissue Eng Part B Rev. 2015;21(1):88-102. doi: 10.1089/ten.TEB.2014.0180
  39. Voigt SP, Ravikumar K, Basu B, et al. Automated image processing workflow for morphological analysis of fluorescence microscopy cell images. JOM. 2021;73(8):2356-2365. doi: 10.1007/s11837-021-04707-w
  40. Kérourédan O, Ribot EJ, Fricain JC, Devillard R, Miraux S. Magnetic resonance imaging for tracking cellular patterns obtained by laser-assisted bioprinting. Sci Rep. 2018;8(1):15777. doi: 10.1038/s41598-018-34226-9
  41. Catanzaro V, Digilio G, Capuana F, et al. Gadolinium-labelled cell scaffolds to follow-up cell transplantation by magnetic resonance imaging. J Funct Biomater. 2019;10(3):28. doi: 10.3390/jfb10030028
  42. Constantinides C, Basnett P, Lukasiewicz B, et al. In vivo tracking and 1H/19F magnetic resonance imaging of biodegradable polyhydroxyalkanoate/polycaprolactone blend scaffolds seeded with labelled cardiac stem cells. ACS Appl Mater Interfaces. 2018;10(30):25056-25068. doi: 10.1021/acsami.8b06096
  43. El-Fiqi A, Kim JH, Kim HW. Novel bone-mimetic nanohydroxyapatite/collagen porous scaffolds biomimetically mineralized from surface silanized mesoporous nanobioglass/collagen hybrid scaffold: physicochemical, mechanical and in vivo evaluations. Mater Sci Eng C Mater Biol Appl. 2020;110:110660. doi: 10.1016/j.msec.2020.110660
  44. Palmroth A, Pitkänen S, Hannula M, et al. Evaluation of scaffold microstructure and comparison of cell seeding methods using micro-computed tomography-based tools. J R Soc Interface. 2020;17(165):20200102. doi: 10.1098/rsif.2020.0102
  45. Ning L, Zhu N, Smith A, et al. Noninvasive three-dimensional in situ and in vivo characterization of bioprinted hydrogel scaffolds using the X-ray propagation-based imaging technique. ACS Appl Mater Interfaces. 2021;13(22):25611-25623. doi: 10.1021/acsami.1c02297
  46. Zhou Y. The application of ultrasound in 3D bio-printing. Molecules. 2016;21(5):590. doi: 10.3390/molecules21050590
  47. Deng CX, Hong X, Stegemann JP. Ultrasound imaging techniques for spatiotemporal characterization of composition, microstructure, and mechanical properties in tissue engineering. Tissue Eng Part B Rev. 2016;22(4):311-321. doi: 10.1089/ten.TEB.2015.0453
  48. Chiu YC, Brey EM, Pérez-Luna VH. A study of the intrinsic autofluorescence of poly (ethylene glycol)-co-(L-lactic acid) diacrylate. J Fluoresc. 2012;22(3):907-913. doi: 10.1007/s10895-011-1029-6
  49. Boutin ME, Voss TC, Titus SA, Cruz-Gutierrez K, Michael S, Ferrer M. A high-throughput imaging and nuclear segmentation analysis protocol for cleared 3D culture models. Sci Rep. 2018;8(1):11135. doi: 10.1038/s41598-018-29169-0
  50. Sun J, Yao K, An J, Jing L, Huang K, Huang D. Machine learning and 3D bioprinting. Int J Bioprint. 2023;9(4):717. doi: 10.18063/ijb.717
  51. Malatesta M. Transmission electron microscopy as a powerful tool to investigate the interaction of nanoparticles with subcellular structures. Int J Mol Sci. 2021;22(23):12789. doi: 10.3390/ijms222312789
  52. Younesi M, Islam A, Kishore V, Anderson JM, Akkus O. Tenogenic induction of human MSCs by anisotropically aligned collagen biotextiles. Adv Funct Mater 2014;24(36):5762-5770. doi: 10.1002/adfm.201400828
  53. Iandolo D, Pennacchio FA, Mollo V, et al. Electron microscopy for 3D scaffolds-cell biointerface characterization. Adv Biosyst. 2019;3(2):e1800103. doi: 10.1002/adbi.201800103
  54. Peddie CJ, Genoud C, Kreshuk A, et al. Volume electron microscopy. Nat Rev Methods Primers. 2022;2:51. doi: 10.1038/s43586-022-00131-9
  55. Guaccio A, Guarino V, Perez MA, Cirillo V, Netti PA, Ambrosio L. Influence of electrospun fiber mesh size on hMSC oxygen metabolism in 3D collagen matrices: experimental and theoretical evidences. Biotechnol Bioeng. 2011;108(8):1965-1976. doi: 10.1002/bit.23113
  56. McQuin C, Goodman A, Chernyshev V, et al. CellProfiler 3.0: next-generation image processing for biology. PLoS Biol. 2018;16(7):e2005970. doi: 10.1371/journal.pbio.2005970
  57. Gómez-de-Mariscal E, García-López-de-Haro C, Ouyang W, et al. DeepImageJ: a user-friendly environment to run deep learning models in ImageJ. Nat Methods. 2021;18(10): 1192-1195. doi: 10.1038/s41592-021-01262-9
  58. Berg S, Kutra D, Kroeger T, et al. Ilastik: interactive machine learning for (bio)image analysis. Nat Methods. 2019;16(12):1226-1232. doi: 10.1038/s41592-019-0582-9
  59. Goodfellow I, Pouget-Abadie J, Mirza M, et al. Generative adversarial nets. In: Advances in Neural Information Processing Systems;2014:27. ISBN:9781510800410. doi: 10.1145/3422622
  60. Zhang S, Qian Z, Huang K, et al. Robust generative adversarial network. Mach Learn. 2023;112(12):5135-5161. doi: 10.1007/s10994-023-06367-0.
  61. Shahin AI, Guo Y, Amin KM, Sharawi AA. White blood cells identification system based on convolutional deep neural learning networks. Comput Methods Programs Biomed. 2019;168:69-80. doi: 10.1016/j.cmpb.2017.11.015
  62. Sun J, Jing L, Fan X, Gao X, Liang YC. Electrohydrodynamic printing process monitoring by microscopic image identification. Int J Bioprint. 2018;5(1):164. doi: 10.18063/ijb.v5i1.164.
  63. Zheng Z, Sun B, He S, et al. Unsupervised deep learning of bright-field images for apoptotic cell classification. SIViP. 2023;17:3657-3664. doi: 10.1007/s11760-023-02592-1
  64. Ronneberger O, Fischer P, Brox T. U-net: convolutional networks for biomedical image segmentation. MICCAI. 2015;9351:234-241. doi: 10.1007/978-3-319-24574-4_28
  65. Pfister M, Schützenberger K, Pfeiffenberger U, et al. Automated segmentation of dermal fillers in OCT images of mice using convolutional neural networks. Biomed Opt Express. 2019;10(3):1315-1328. doi: 10.1364/boe.10.001315
  66. Lugagne JB, Lin H, Dunlop MJ. DeLTA: automated cell segmentation, tracking, and lineage reconstruction using deep learning. PLoS Comput Biol. 2020;16(4):e1007673. doi: 10.1371/journal.pcbi.1007673
  67. Rautaray SS, Dey S, Pandey M, et al. Nuclei segmentation in cell images using fully convolutional neural networks. Int J Emerg Technol. 2020;11(3):731-737.
  68. Yao K, Rochman ND, Sun SX. Cell type classification and unsupervised morphological phenotyping from low-resolution images using deep learning. Sci Rep. 2019;9(1): 13467. doi: 10.1038/s41598-019-50010-9
  69. Jin J, Lu JQ, Wen Y, Tian P, Hu XH. Deep learning of diffraction image patterns for accurate classification of five cell types. J Biophotonics. 2020;13(3):e201900242. doi: 10.1002/jbio.201900242
  70. Wang H, Rivenson Y, Jin Y, et al. Deep learning enables cross-modality super-resolution in fluorescence microscopy. Nat Methods. 2019;16(1):103-110. doi: 10.1038/s41592-018-0239-0
  71. Bailo O, Ham DS, Min Shin Y. Red blood cell image generation for data augmentation using conditional generative adversarial networks. In: Proceedings of the IEEE/ CVF Conference on Computer Vision and Pattern Recognition Workshops; 2019: 1039-1048. doi: 10.1109/CVPRW.2019.00136
  72. Moen E, Bannon D, Kudo T, Graf W, Covert M, Van Valen D. Deep learning for cellular image analysis. Nat Methods. 2019;16(12):1233-1246. doi: 10.1038/s41592-019-0403-1
  73. Al-Kofahi Y, Zaltsman A, Graves R, Marshall W, Rusu M. A deep learning-based algorithm for 2-D cell segmentation in microscopy images. BMC Bioinformatics. 2018;19(1):365. doi: 10.1186/s12859-018-2375-z
  74. Lux F, Matula P. DIC image segmentation of dense cell populations by combining deep learning and watershed. In: IEEE 16th International Symposium on Biomedical Imaging; 2019: 236-239. doi: 10.1109/ISBI.2019.8759594
  75. Inés A, Domínguez C, Heras J, Mata E, Pascual V. Biomedical image classification made easier thanks to transfer and semi-supervised learning. Comput Methods Programs Biomed. 2021;198:105782. doi: 10.1016/j.cmpb.2020.105782
  76. Laine RF, Jacquemet G, Krull A. Imaging in focus: an introduction to denoising bioimages in the era of deep learning. Int J Biochem Cell Biol. 2021;140:106077. doi: 10.1016/j.biocel.2021.106077
  77. Li X, Li Y, Zhou Y, et al. Real-time denoising enables high-sensitivity fluorescence time-lapse imaging beyond the shot-noise limit. Nat Biotechnol. 2023;41(2):282-292. doi: 10.1038/s41587-022-01450-8
  78. Rizk A, Paul G, Incardona P, et al. Segmentation and quantification of subcellular structures in fluorescence microscopy images using Squassh. Nat Protoc. 2014;9(3): 586-596. doi: 10.1038/nprot.2014.037
  79. Wen T, Tong B, Liu Y, et al. Review of research on the instance segmentation of cell images. Comput Methods Programs Biomed. 2022;227:107211. doi: 10.1016/j.cmpb.2022.107211
  80. Wen C, Miura T, Fujie Y, et al. Deep-learning-based flexible pipeline for segmenting and tracking cells in 3D image time series for whole brain imaging. bioRxiv. 2018. doi: 10.1101/385567
  81. Wu C, Entezari A, Zheng K, et al. A machine learning-based multiscale model to predict bone formation in scaffolds. Nat Comput Sci. 2021;1(8):532-541. doi: 10.1038/s43588-021-00115-x
  82. Wang Y, Mao H, Yi Z. Stem cell motion-tracking by using deep neural networks with multi-output[J]. Neural Comput Appl. 2019;31:3455-3467. doi: 10.1007/s00521-017-3291-2
  83. Helaehil JV, Lourenço CB, Huang B, et al. In vivo investigation of polymer-ceramic PCL/HA and PCL/β-TCP 3D composite scaffolds and electrical stimulation for bone regeneration. Polymers (Basel). 2021;14(1):65. doi: 10.3390/polym14010065
  84. Su Z, Yao K, Yang X, et al. Rethinking data augmentation for single-source domain generalization in medical image segmentation. Proc AAAI Conf Artif Intell. 2023;37(2):2366-2374. doi: 10.1609/aaai.v37i2.25332
  85. Hassija V, Chamola V, Mahapatra A, et al. Interpreting black-box models: a review on explainable artificial intelligence. Cogn Comput. 2024;16(1):45-74. doi: 10.1007/s12559-023-10179-8
  86. Achiam J, Adler S, Agarwal S, et al. Gpt-4 technical report. arXiv preprint arXiv:2303.08774. doi: 10.48550/arXiv.2303.08774
  87. Luo R, Sun L, Xia Y, et al. BioGPT: generative pre-trained transformer for biomedical text generation and mining. Brief Bioinform. 2022;23(6):bbac409. doi: 10.1093/bib/bbac409

 

 

 

 

Share
Back to top
International Journal of Bioprinting, Electronic ISSN: 2424-8002 Print ISSN: 2424-7723, Published by AccScience Publishing