AccScience Publishing / IJAMD / Online First / DOI: 10.36922/ijamd.3919
Submit to IJAMD
Cite this article
36
Download
255
Views
Journal Browser
Volume | Year
Issue
Search
Forthcoming Issue
Current Issue
View All
News and Announcements
View All
ORIGINAL RESEARCH ARTICLE

AMTransformer: A Koopman theory-based transformer for learning additive manufacturing dynamics in laser processes

Suk Ki Lee1 ,  Hyunwoong Ko1*
IJAMD 2024, 1(2), 76–91; https://doi.org/10.36922/ijamd.3919
Submitted: 12 June 2024 | Accepted: 9 August 2024 | Published: 2 September 2024
© 2024 by the Author(s). This article is an open access article distributed under the terms and conditions of the Creative Commons Attribution 4.0 International License ( https://creativecommons.org/licenses/by/4.0/ )
Download PDF
Cite
XML
Abstract

Recent advancements in machine learning (ML) have shown unprecedented promise in understanding and predicting additive manufacturing (AM) dynamics. However, existing ML studies on AM often lack a comprehensive approach to address the multi-scale complexities inherent in AM processes and tend to employ context-specific methods. To address these limitations, we present a foundational method for formulating AM dynamics suitable for ML modeling. We then introduce a novel approach, the AMTransformer, designed to comprehend complex spatiotemporal dynamical dependencies among physical entities and their properties within the AM process. To enhance the understanding of AM dynamics, our method adapts Koopman’s theory to generate latent embeddings of AM states and their transitions, effectively extracting hidden features related to physical properties and dynamical dependencies. In addition, by utilizing the transformer’s attention mechanism, the proposed approach enhances the learning of non-local, non-linear dynamical dependencies across multiple scales. Our experiments, conducted using melt pool data from a laser powder bed fusion process, demonstrate that the AMTransformer outperforms traditional transformer and convolutional long short-term memory models. Specifically, the AMTransformer achieved structural similarity, mean absolute error, and accuracy metric values of 0.9206, 0.0009 mm2, and 92.73%, respectively. These results indicate the AMTransformer’s superior ability to predict future AM states, attributed to its improved learning of complex AM dynamics. By combining linear Koopman-based methods with non-linear transformer-based approaches, the AMTransformer significantly improves data-driven modeling for AM, providing a more comprehensive understanding of AM dynamics. Furthermore, the generalizability of the proposed method facilitates the expansion of the model’s scope and enhances its applicability across various fields.

Keywords
Additive manufacturing
Koopman theory
Laser AM dynamics
Machine learning
Predictive modeling
Transformer
Funding
This research was supported by Arizona State University startup funds (Award number: CC1379 PG14421), as well as by PADT and the Arizona State University Science and Technology Centers (Award number: AWD00037762).
Conflict of interest
Hyunwoong Ko is an Editorial Board Member of this journal, but was not in any way involved in the editorial and peer-review process conducted for this paper, directly or indirectly. Separately, other authors declared that they have no known competing financial interests or personal relationships that could have influenced the work reported in this paper.
References
  1. Gibson I, Stucker B, Khorasani M. Additive Manufacturing Technologies. Vol. 17. Berlin: Springer; 2010. doi: 10.1007/978-1-4419-1120-9

 

  1. Ko H, Moon SK, Hwang J. Design for additive manufacturing in customized products. Int J Precis Eng Manuf. 2015;16:2369-2375. doi: 10.1007/s12541-015-0305-9

 

  1. Chua CK, Leong KF. 3D Printing and Additive Manufacturing: Principles and Applications (with Companion Media Pack)-of Rapid Prototyping. Singapore: World Scientific Publishing Company; 2014. doi: 10.1142/9008

 

  1. Gao W, Zhang Y, Ramanujan D, et al. The status, challenges, and future of additive manufacturing in engineering. Comput Aided Des. 2015;69:65-89. doi: 10.1016/j.cad.2015.04.001

 

  1. King WE, Anderson AT, Ferencz RM, et al. Laser powder bed fusion additive manufacturing of metals; physics, computational, and materials challenges. Appl Phys Rev. 2015;2(4):041304. doi: 10.1063/1.4937809

 

  1. Bikas H, Stavropoulos P, Chryssolouris G. Additive manufacturing methods and modelling approaches: A critical review. Int J Adv Manuf Technol. 2016;83:389-405. doi: 10.1007/s00170-015-7576-2

 

  1. Bourell DL, Frazier WE, Kuhn HA, Seifi M. Additive Manufacturing Processes. Vol. 24. OH, USA: ASM International Novelty; 2020.

 

  1. Kruth JP, Levy G, Klocke F, Childs T. Consolidation phenomena in laser and powder-bed based layered manufacturing. CIRP Ann. 2007;56(2):730-759. doi: 10.1016/j.cirp.2007.10.004

 

  1. Ko H, Kim J, Lu Y, Shin D, Yang Z, Oh Y. Spatial-Temporal Modeling using Deep Learning for Real-Time Monitoring of Additive Manufacturing. In: ASME 2022 International Design Engineering Technical Conferences & Computers and Information in Engineering Conference; 2022. doi: 10.1115/DETC2022-91021

 

  1. Ko H, Lu Y, Yang Z, Ndiaye NY, Witherell P. A framework driven by physics-guided machine learning for process-structure-property causal analytics in additive manufacturing. J Manuf Syst. 2023;67:213-228. doi: 10.1016/j.jmsy.2022.09.010

 

  1. Yan W, Lin S, Kafka OL, et al. Data-driven multi-scale multi-physics models to derive process-structure-property relationships for additive manufacturing. Comput Mech. 2018;61:521-541. doi: 10.1007/s00466-018-1539-z

 

  1. Marshall GJ, Thompson SM, Shamsaei N. Data indicating temperature response of Ti-6Al-4V thin-walled structure during its additive manufacture via Laser Engineered Net Shaping. Data Brief. 2016;7:697-703. doi: 10.1016/j.dib.2016.02.084

 

  1. Mitchell JA, Ivanoff TA, Dagel D, Madison JD, Jared B. Linking pyrometry to porosity in additively manufactured metals. Addit Manuf. 2020;31:100946. doi: 10.1016/j.addma.2019.100946

 

  1. Fathizadan S, Ju F, Lu Y. Deep representation learning for process variation management in laser powder bed fusion. Addit Manuf. 2021;42:101961. doi: 10.1016/j.addma.2021.101961

 

  1. Yang Z, Lu Y, Yeung H, Krishnamurty S. From scan strategy to melt pool prediction: A neighboring-effect modeling method. J Comput Inf Sci Eng. 2020;20(5):051001. doi: 10.1115/1.4046335

 

  1. Zhang Z, Sahu CK, Singh SK, Rai R, Yang Z, Lu Y. Machine learning based prediction of melt pool morphology in a laser-based powder bed fusion additive manufacturing process. Int J Prod Res. 2024;62(5):1803-1817. doi: 10.1080/00207543.2023.2201860

 

  1. Tempelman JR, Wachtor AJ, Flynn EB, et al. Detection of keyhole pore formations in laser powder-bed fusion using acoustic process monitoring measurements. Addit Manuf. 2022;55:102735. doi: 10.1016/j.addma.2022.102735

 

  1. Kononenko DY, Nikonova V, Seleznev M, van den Brink J, Chernyavsky D. An in situ crack detection approach in additive manufacturing based on acoustic emission and machine learning. Addit Manuf Lett. 2023;5:100130. doi: 10.1016/j.addlet.2023.100130

 

  1. Lough CS, Escano LI, Qu M, et al. In-situ optical emission spectroscopy of selective laser melting. J Manuf Process. 2020;53:336-341.

 

  1. Montazeri M, Nassar AR, Dunbar AJ, Rao P. In-process monitoring of porosity in additive manufacturing using optical emission spectroscopy. IISE Trans. 2020;52(5):500-515. doi: 10.1080/24725854.2019.1659525

 

  1. Liu J, Ye J, Silva Izquierdo D, Vinel A, Shamsaei N, Shao S. A review of machine learning techniques for process and performance optimization in laser beam powder bed fusion additive manufacturing. J Intell Manuf. 2023;34(8):3249-3275. doi: 10.1007/s10845-022-02012-0

 

  1. Cai Y, Xiong J, Chen H, Zhang G. A review of in-situ monitoring and process control system in metal-based laser additive manufacturing. J Manuf Syst. 2023;70:309-326. doi: 10.1016/j.jmsy.2023.07.018

 

  1. Yang Z, Lu Y, Yeung H, Krishnamurty S. Investigation of Deep Learning for Real-Time Melt Pool Classification in Additive Manufacturing. United States: IEEE; 2019. p. 640-647. doi: 10.1109/COASE.2019.8843291

 

  1. Zhang Y, Soon HG, Ye D, Fuh JYH, Zhu K. Powder-bed fusion process monitoring by machine vision with hybrid convolutional neural networks. IEEE Trans Ind Inf. 2019;16(9):5769-5779. doi: 10.1109/TII.2019.2956078

 

  1. Larsen S, Hooper PA. Deep semi-supervised learning of dynamics for anomaly detection in laser powder bed fusion. J Intell Manuf. 2022;33(2):457-471. doi: 10.1007/s10845-021-01842-8

 

  1. Fernandez-Zelaia P, Dryepondt SN, Ziabari AK, Kirka MM. Self-supervised learning of spatiotemporal thermal signatures in additive manufacturing using reduced order physics models and transformers. Comput Mater Sci. 2024;232:112603. doi: 10.1016/j.commatsci.2023.112603

 

  1. Guirguis D, Tucker C, Beuth J. Accelerating process development for 3D printing of new metal alloys. Nat Commun. 2024;15(1):1-12. doi: 10.1038/s41467-024-44783-5

 

  1. Vaswani A, Shazeer N, Parmar N, et al. Attention is all you need. Adv Neural Inf Process Syst. 2017;30:5998-6008. doi: 10.48550/arXiv.1706.03762

 

  1. Koopman BO. Hamiltonian systems and transformation in Hilbert space. Proc Natl Acad Sci. 1931;17(5):315-318.

 

  1. Rowley CW, Mezić I, Bagheri S, Schlatter P, Henningson DS. Spectral analysis of nonlinear flows. J Fluid Mech. 2009;641:115-127. doi: 10.1017/S0022112009992059

 

  1. Brunton SL, Kutz JN. Data-Driven Science and Engineering: Machine Learning, Dynamical Systems, and Control. Cambridge, UK: Cambridge University Press; 2019.

 

  1. Geneva N, Zabaras N. Transformers for modeling physical systems. Neural Netw. 2022;146:272-289. doi: 10.1016/j.neunet.2021.11.022

 

  1. Mezić I. Analysis of fluid flows via spectral properties of the Koopman operator. Annu Rev Fluid Mech. 2013;45:357-378. doi: 10.1146/annurev-fluid-011212-140652

 

  1. Schmid PJ. Dynamic mode decomposition and its variants. Annu Rev Fluid Mech. 2022;54:225-254. doi: 10.1146/annurev-fluid-030121-015835

 

  1. Schmid PJ. Dynamic mode decomposition of numerical and experimental data. J Fluid Mech. 2010;656:5-28. doi: 10.1017/S0022112010001217

 

  1. Radford A, Narasimhan K, Salimans T, Sutskever I. Improving language understanding by generative pre-training. 2018, OpenAI Blog. Available from: https:// openai-assets.s3.amazonaws.com/research-covers/ languageunsupervised/language_understanding_paper.pdf [Last accessed: August 29, 2024].

 

  1. Radford A, Wu J, Child R, Luan D, Amodei D, Sutskever I. Language models are unsupervised multitask learners. Open AI Blog. 2019;1(8):9.

 

  1. Wolf T, Debut L, Sanh V, et al. Hugging face’s transformers: State-of-the-art natural language processing. arXiv preprint arXiv:191003771. 2019. doi: 10.48550/arXiv.1910.03771

 

  1. Lane B, Yeung H. Process monitoring dataset from the additive manufacturing metrology testbed (ammt): Overhang part x4. J Res Natl Inst Stand Technol. 2020;125:1- 18. doi: 10.6028/jres.125.027

 

  1. Glorot X, Bordes A, Bengio Y. Deep Sparse Rectifier Neural Networks. In: JMLRWorkshop and Conference Proceedings; 2011. p. 315-323.

 

  1. Glorot X, Bordes A, Bengio Y. Domain Adaptation for Large- Scale Sentiment Classification: A Deep Learning Approach. In: Proceedings of the 28th International Conference on Machine Learning; 2011. p. 513-520.

 

  1. Hendrycks D, Gimpel K. Gaussian error linear units (gelus). arXiv preprint arXiv:160608415. 2016. doi: 10.48550/arXiv.1606.08415

 

  1. Wang Z, Simoncelli EP, Bovik AC. Multiscale Structural Similarity for Image Quality Assessment. IEEE: IEEE; 2003. p. 1398-1402. doi: 10.1109/ACSSC.2003.1292216
Next article in this issue
Share
Back to top
International Journal of AI for Materials and Design, Electronic ISSN: 3029-2573 Print ISSN: 3041-0746, Published by AccScience Publishing
We use cookies on our website to ensure you get the best experience.
Read more about our cookies here
Accept