AccScience Publishing / IJB / Volume 8 / Issue 4 / DOI: 10.18063/ijb.v8i4.620
Cite this article
54
Download
1121
Views
Journal Browser
Volume | Year
Issue
Search
News and Announcements
View All
RESEARCH ARTICLE

A Deep Learning Quality Control Loop of the Extrusion-based Bioprinting Process

Amedeo Franco Bonatti1 Giovanni Vozzi1 Chee Kai Chua2 Carmelo De Maria1*
Show Less
1 Department of Information Engineering and Research Center “Enrico Piaggio,” University of Pisa, Pisa, Italy
2 Engineering Product Development Pillar University of Technology and Design, Singapore
Submitted: 26 August 2022 | Accepted: 6 September 2022 | Published: 11 October 2022
(This article belongs to the Special Issue Related to 3D printing technology and materials)
© 2022 by the Author(s). This article is an open access article distributed under the terms and conditions of the Creative Commons Attribution 4.0 International License ( https://creativecommons.org/licenses/by/4.0/ )
Abstract

Extrusion-based bioprinting (EBB) represents one of the most used deposition technologies in the field of bioprinting, thanks to key advantages such as the easy-to-use hardware and the wide variety of materials that can be successfully printed. In recent years, research efforts have been focused on implementing a quality control loop for EBB, which can reduce the trial-and-error process necessary to optimize the printing parameters for a specific ink, standardize the results of a print across multiple laboratories, and so accelerate the translation of extrusion bioprinted products to more impactful clinical applications. Due to its capacity to acquire relevant features from a training dataset and generalize to unseen data, machine learning (ML) is currently being studied in literature as a relevant enabling technology for quality control in EBB. In this context, we propose a robust, deep learning-based control loop to automatically optimize the printing parameters and monitor the printing process online. We collected a comprehensive dataset of EBB prints by recording the process with a high-resolution webcam. To model multiple printing scenarios, each video represents a combination of multiple parameters, including printing set-up (either mechanical or pneumatic extrusion), material color, layer height, and infill density. After pre-processing, the collected dataset was used to thoroughly train and evaluate an ad hoc defined convolutional neural network by controlling over-fitting and optimizing the prediction time of the network. Finally, the ML model was used in a control loop to: (i) monitor the printing process and detect if a print with an error could be stopped before completion to save material and time and (ii) automatically optimize the printing parameters by combining the ML model with a previously published mathematical model of the EBB process. Altogether, we demonstrated for the first time how ML techniques can be used to automatize the EBB process, paving the way for a total quality control loop of the printing process.

Keywords
Extrusion-based bioprinting
Quality control
Convolutional neuronal network
Automatic parameter optimization
References

1. Santoni S, Gugliandolo SG, Sponchioni M, et al., 2021, 3D bioprinting: Current status and trends a guide to the literature and industrial practice. Biodes Manuf, 5: 14–42. https://doi.org/10.1007/s42242-021-00165-0

2. Moroni L, Boland T, Burdick JA, et al., 2018, Biofabrication: A guide to technology and terminology. Trends Biotechnol, 36: 384–402. https://doi.org/10.1016/j.tibtech.2017.10.015

3. Bonatti AF, Fortunato GM, de Maria C, et al., 2022, Bioprinting technologies: An overview. Bioprinting, 19-49. https://doi.org/10.1016/B978-0-323-85430-6.00006-6

4. Pati F, Jang J, Lee JW, et al., 2015, Extrusion bioprinting. In: Essentials of 3D Biofabrication and Translation. Academic Press, Cambridge, Massachusetts, p123–152.

5. Choudhury D, Anand S, Naing MW, 2018, The arrival of commercial bioprinters-Towards 3D bioprinting revolution! Int J Bioprint, 4: 139. https://doi.org/10.18063/IJB.v4i2.139

6. Groll J, Burdick JA, Cho DW, et al., 2018, A definition of bioinks and their distinction from biomaterial inks. Biofabrication, 11: 013001.

7. Bonatti AF, Chiesa I, Vozzi G, et al., 2021, Open-source CAD-CAM simulator of the extrusion-based bioprinting process. Bioprinting, 24: e00172. https://doi.org/10.1016/j.bprint.2021.e00172

8. Schwab A, Levato R, D’Este M, et al., 2020, Printability and shape fidelity of bioinks in 3D bioprinting. Chem Rev, 120: 11028–11055. https://doi.org/10.1021/acs.chemrev.0c00084

9. Paxton N, Smolan W, Böck T, et al., 2017, Proposal to assess printability of bioinks for extrusion-based bioprinting and evaluation of rheological properties governing bioprintability. Biofabrication, 9: 044107. https://doi.org/10.1088/1758-5090/aa8dd8

10. Di Pietro L, Ravizza A, Vozzi G, et al., 2019, European regulatory framework for the clinical translation of bioprinted scaffolds and tissues. Biomed Sci Eng, 3: 108. https://doi.org/10.4081/bse.2019.108

11. Yu C, Jiang J, 2020, A perspective on using machine learning in 3D bioprinting. Int J Bioprint, 6: 253. https://doi.org/10.18063/ijb.v6i1.253

12. An J, Chua CK, Mironov V, 2021, Application of machine learning in 3D bioprinting: Focus on development of big data and digital twin. Int J Bioprint, 7: 342. https://doi.org/10.18063/ijb.v7i1.342

13. Freeman S, Calabro S, Williams R, et al., 2022, Bioink formulation and machine learning-empowered bioprinting optimization. Front Bioeng Biotechnol, 10: 913579. https://doi.org/10.3389/fbioe.2022.913579

14. Bone JM, Childs CM, Menon A, et al., 2020, Hierarchical machine learning for high-fidelity 3D printed biopolymers. ACS Biomater Sci Eng, 6: 7021–7031. https://doi.org/10.1021/acsbiomaterials.0c00755

15. Fu Z, Angeline V, Sun W, 2021, Evaluation of printing parameters on 3D extrusion printing of pluronic hydrogels and machine learning guided parameter recommendation. Int J Bioprint, 7: 434. https://doi.org/10.18063/ijb.v7i4.434

16. Ruberu K, Senadeera M, Rana S, et al., 2021, Coupling machine learning with 3D bioprinting to fast track optimization of extrusion printing. Appl Mater Today, 22: 100914. https://doi.org/10.1016/j.apmt.2020.100914

17. Jin Z, Zhang Z, Shao X, et al., 2021, Monitoring anomalies in 3D bioprinting with deep neural networks. ACS Biomater Sci Eng. https://doi.org/10.1021/acsbiomaterials.0c01761

18. Chandola V, Banerjee A, Kumar V, 2009, Anomaly detection: A survey. ACM Comput Surv (CSUR), 41: 1–58. https://doi.org/10.1145/1541880.1541882

19. Wang R, Nie K, Wang T, et al., 2020, Deep learning for anomaly detection. In: Proceedings of the 13th International Conference on Web Search and Data Mining. p894–896. https://doi.org/10.1145/3336191.3371876

20. Pang G, Shen C, Cao L, et al., 2021, Deep learning for anomaly detection: A review. ACM Comput Surv (CSUR), 54: 1–38. https://doi.org/10.1145/3439950

21. Goodfellow I, Bengio Y, Courville A, 2016, Deep Learning. Cambridge, Massachusetts, MIT Press.

22. Chalapathy R, Chawla S, 2019, Deep learning for anomaly detection: A survey. arXiv preprint arXiv, 1901: 03407. https://doi.org/10.48550/arXiv.1901.03407

23. LeCun Y, Bengio Y, Hinton G, 2015, Deep learning. Nature, 521: 436–444.

24. Wang C, Tan XP, Tor SB, et al., 2020, Machine learning in additive manufacturing: State-of-the-art and perspectives. Addit Manuf, 36: 101538. https://doi.org/10.1016/j.addma.2020.101538

25. Meng L, McWilliams B, Jarosinski W, et al., 2020, Machine learning in additive manufacturing: A review. JOM, 72: 2363–2377. https://doi.org/10.1007/s11837-020-04155-y

26. Wu M, Phoha VV, Moon YB, et al., 2016, Detecting malicious defects in 3D printing process using machine learning and image classification. In: ASME International Mechanical Engineering Congress and Exposition. Vol. 50688. American Society of Mechanical Engineers, New York, pV014T07A004.

27. Liu C, Roberson D, Kong Z, 2017, Textural analysis-based online closed-loop quality control for additive manufacturing processes. In: IIE Annual Conference. Proceedings. Institute of Industrial and Systems Engineers, Peachtree Corners, Georgia, p1127–1132.

28. Ogunsanya M, Isichei J, Parupelli SK, et al., 2021, In-situ droplet monitoring of inkjet 3D printing process using image analysis and machine learning models. Procedia Manuf, 53: 427–434. https://doi.org/10.1016/j.promfg.2021.06.045

29. Gobert C, Reutzel EW, Petrich J, et al., 2018, Application of supervised machine learning for defect detection during metallic powder bed fusion additive manufacturing using high resolution imaging. Addit Manuf, 21: 517–528. https://doi.org/10.1016/j.addma.2018.04.005

30. Sing SL, Kuo CN, Shih CT, et al., 2021, Perspectives of using machine learning in laser powder bed fusion for metal additive manufacturing. Virtual Phys Prototyp, 16: 372–386. https://doi.org/10.1080/17452759.2021.1944229

31. Jin Z, Zhang Z, Gu GX, 2019, Autonomous in-situ correction of fused deposition modeling printers using computer vision and deep learning. Manuf Lett, 22, 11–15. https://doi.org/10.1016/j.mfglet.2019.09.005

32. Tonnaer L, Li J, Osin V, et al., 2019, Anomaly detection for visual quality control of 3D-printed products. In: 2019 International Joint Conference on Neural Networks (IJCNN). IEEE, Piscataway, New Jersey, p1–8.

33. Zhang J, Wang P, Gao RX, 2019, Deep learning-based tensile strength prediction in fused deposition modeling. Comput Ind, 107: 11–21. https://doi.org/10.1016/j.compind.2019.01.011

34. Zhang B, Liu S, Shin YC, 2019, In-process monitoring of porosity during laser additive manufacturing process. Addit Manuf, 28: 497–505. https://doi.org/10.1016/j.addma.2019.05.030

35. Wang T, Kwok TH, Zhou C, et al., 2018. In-situ droplet inspection and closed-loop control system using machine learning for liquid metal jet printing. J Manuf Syst, 47: 83–92. https://doi.org/10.1016/j.jmsy.2018.04.003

36. McKinney W, 2011, pandas: A foundational Python library for data analysis and statistics. Python High Perform Sci Comput, 14: 1–9.

37. Bradski G, 2000, The openCV library. Dr Dobbs J Softw Tools, 25: 120–123.

38. Abadi M, Agarwal A, Barham P, et al., 2016, Tensorflow: Large-scale machine learning on heterogeneous distributed systems. arXiv preprint arXiv, 1603: 04467. https://doi.org/10.48550/arXiv.1603.04467

39. Chollet F, 2018, Keras: The python deep learning library. Astrophysics Source Code Library, ascl-1806.

40. Pedregosa F, Varoquaux G, Gramfort A, et al., 2011, Scikit learn: Machine learning in Python. J Mach Learn Res, 12: 2825–2830.

41. Bonatti AF, Fortunato GM, Lapomarda A, et al., 2021, Advanced Firmware and Hardware for Multiscale and Multimaterial Bioprinting. Proceedings of the Seventh National Congress of Bioengineering.

42. Zhao M, Bu J, Chen C, 2002, Robust background subtraction in HSV color space. In: Multimedia systems and Applications, Vol. 4861. International Society for Optics and Photonics, United States, p325–332. https://doi.org/10.1117/12.456333

43. Simonyan K, Zisserman A, 2014, Very deep convolutional networks for large-scale image recognition. arXiv preprint arXiv, 1409: 1556. https://doi.org/10.48550/arXiv.1409.1556

44. He K, Zhang X, Ren S, et al., 2016, Deep residual learning for image recognition. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. p770–778.

45. He K, Zhang X, Ren S, et al., 2015, Delving deep into rectifiers: Surpassing human-level performance on imagenet classification. In: Proceedings of the IEEE International Conference on Computer Vision. p1026–1034.

46. Kingma DP, Ba J, 2014, Adam: A method for stochastic optimization. arXiv preprint arXiv, 1412: 6980. https://doi.org/10.48550/arXiv.1412.6980

47. Selvaraju RR, Das A, Vedantam R, et al., 2016, Grad-CAM: Why did you say that? arXiv preprint arXiv, 1611: 07450. https://doi.org/10.48550/arXiv.1611.07450

Share
Back to top
International Journal of Bioprinting, Electronic ISSN: 2424-8002 Print ISSN: 2424-7723, Published by AccScience Publishing