Physics-informed machine learning models for the prediction of transient temperature distribution of ferritic steel in directed energy deposition by cold metal transfer

In-situ monitoring of the additive layer characteristics in the directed energy deposition (DED) process by any contact technology is cumbersome. A well-tested finite element (FE) model is often employed to extract transient temperature distribution during deposition. However, the numerical model pertaining to each deposition attribute is computationally expensive. In the present work, we have generated a dataset through an experimentally validated thermal model, and further multiple machine learning (ML) algorithms are applied to train datasets. Models with an accuracy of more than 99% are utilised for the prediction of transient temperature distribution. The validation of deposition attributes using experiments and numerical model suggests that the physics-informed machine learning models for cold metal transfer can be applied in the DED process.


Introduction
Additive manufacturing (AM) and data science are the most significant research areas in the digital manufacturing process.By utilising a data analytics concept in AM industry, meaningful insights can be drawn, and suitable recommendations can be made for fabricating a structurally sound component.The transient temperature profile during the deposition of material has a significant impact on the microstructure and residual stresses evolution that decides the mechanical properties of the additively manufactured component.The generation of data through the experimentally validated finite element (FE) model and fitting with appropriate machine learning (ML) algorithms on training datasets to understand the underlying physics involved in melting and solidifion of deposits is known as physicsinformed machine learning (PIML) model.
In a directed energy deposition (DED) based AM process, the fabrication of a component is performed by melting of feedstock materials (wire, powder, or sheet) using a suitable heat source (laser, electric arc, or electron beam).The wire-arc-DED process is able to manufacture large and complex parts with a high deposition rate in an economical way compared to other competitive AM processes [1].In this DED process, any pertubration in deposition process experiences an abrupt change in temperature distribution.During the solidification of the molten pool, it may experience several defects like gas entrapment, lack of fusion, and solidification cracking [1,2].The phase transformation leads to microstructural changes and the evolution of type -II residual stress due to spatial variation of temperature gradients during cooling of molten material [2].The analysis of thermal behavior, such as the accumulation of heat and its variation with different deposition parameters, plays a significant role in making a defectfree component with high degree of functionality.
Measurement of temperature by Infrared camera for a movable object and multi-deposited tracks on the liquid molten pool is cumbersome or almost impossible [3,4].The thermocouple measures temperature by a contact method, but it is associated with a response time lag.Hence, in-situ monitoring of the rapid melting and cooling is not always possible with available measurement techniques.To evaluate the temperature in AM deposits, a physics-based model (analytical and numerical) is more appropriately used [3,4].It requires solving of mass, momentum, and energy conservation equations with suitable boundary condition.The solution of 3D flow field is computationally intensive.When a large number of deposition variables are correlated to each other in a non-linear fashion, it is quite difficult to identify such co-relation and their combined effect on a printed component.Hence, ML algorithms have the unique capabilities of getting useful insights from the data given for training and providing the correlation between input features and output variables [5].Few of the ML algorithms like extreme gradient boosting (XGBOOST), Random forest (RF), and artificial neural network (ANN) models fit well with non-linear datasets and can provide relatively accurate predictions without any prior knowledge of physics applied for generating such data in AM processes [6][7][8].
The application of ML algorithms in the AM sector is limited because large number of experimental data is required for training and interpretation of the same as feature.Even the data that have physical significance consists of a number of pre-processing steps to create a complete dataset [9].A well-tested FE model is a good source for generating data for ML models for a large number of process variables.PIML model (combined FE and ML) has been reported in the existing literature for analyzing different deposition attributes on additively manufactured components [10][11][12].Hajializadeh et al. [10] developed an ANN model integrated with the FE model for the computation of residual stresses in different deposited structures.The transient temperature evolution for multi-layer deposition with varying heat input levels for a WAAM process is analysed by ANN [13].Mukherjee et al. [7] predicted the maximum longitudinal residual stress of a deposit by ANN and RF models, and FE model.Ren et al. [12] conducted a combined study of the FE model and recurrent neural networks and deep neural networks (RNN-DNN) at different scanning strategies.However, most of the reported work in AM technologies is aligned to predict single data points as compared to complete non-linear profiles by ML algorithms.
It is obvious that a non-linear dataset can be handled by using suitable ML algorithms.Hence, the present work analyzes the determination of the transient temperature profile (multiple data points) and peak temperature (single data point) for the deposition of single layer using PIML model.The DED process is developed for P91 on SS316L substrate.The simulations are performed by considering the deposition velocity and power as process parameters for 121 different cases.Five different ML algorithms associated with different degrees of non-linearity are applied to analyze the dataset obtained from a physics-based FE model.Finally, the performance of the models are evaluated and compared in terms of computed error.The models having the highest accuracy are utilised for the prediction of temperature at multiple data points (transient temperature profile) and single data point (peak temperature) at validation attribute.

Materials and methods
A 3D thermal model is developed following the FE method where a double-ellipsoidal heat source model is implemented [14].The heat source model parameters are calibrated from single-bead experiment (Table S1).The elemental birth and death technique is utilised for material deposition following a scanning path activated by the user-defined Python program [15].The numerically calculated deposited track is parabolic in nature (Figure 3).It is obtained by creating a geometry of the same dimension as observed from cold metal transfer (CMT) deposits.In the part module, the deactivating and activating of the element is performed using a model change option present in the interaction module.The substrate and deposited material are set to ambient condition (25°C) as initial condition.During the deposition of the material, heat loss from the deposited component consists of radiation and convection mode of heat transfer.A convective heat transfer coefficient of 800 Wm −2 C −1 is considered for the bottom surface of the substrate and 25 Wm −2 C −1 for the remaining part.In the radiative mode of heat transfer, the emissivity of a 'grey' body is considered as 0.8.A set of numerical simulations are performed to obtain the transient temperature profile and peak temperature during deposition.This simulation consists of depositing a single layer on a substrate of dimension 120 × 90 × 10mm 3 as shown in Figure 1(a).The temperature dependent thermo-physical properties of P91 (wire) and 316L (substrate) are utilised for the numerical model [16,17].These properties are depicted in Figure S1 and Table S3.It has been realised that the significant impact on the variation of temperature are power and velocity [18].For each deposition attribute, eleven different levels are considered where the dataset in depicted in Table S2.A total 121 cases of numerical simulations are performed and accordingly the results are modified.Hence, the dataset contains 33,462 rows and four columns (power, velocity, time as a feature, and temperature as output variables) (Table S4).However, for the prediction of peak temperature, time is not considered as a feature.Further, the dataset is split into the training (80%) and testing (20%) parts that contain all the features.The fitting of ML models are performed on the training dataset, and the predicted outputs are compared with the testing dataset.Step by step procedure utilised in this study for predicting the temperature profile for any deposition is shown in Figure 1.A fine mesh is associated with the deposited part.The activation of the element is performed as per the deposition rate.The transient temperature profile is calculated with respect to power and velocity (Figure 1c).The performance of the ML algorithms was evaluated to find the best one suited for the highly non-linear dataset.Finally, the comparison between the best-fitted ML algorithm and the numerically calculated temperature profile is performed.The mesh size with shape function (linear and quadratic) is optimized with respect to the peak temperature of the system.The maximum temperature obtained for linear and quadratic type of elements for each mesh is depicted in Figure S2.There is not much difference in the peak temperature of linear and quadratic elements, however, the simulation time with quadratic mesh is much more.Therefore, the linear mesh is considered in the present work with total number of 108000 elements.
The pre-processing steps are followed before training the ML model.Over the specified range of power, velocity, and time, the weightage of each feature is mapped with a scale ranging from 0 to 1.In the present analysis, five supervised ML models such as Artificial neural network (ANN), Extreme Gradient Boosting (XGB), K-nearest neighbors (KNN), Decision Tree (DT), and Random Forest (RF) are applied to identify the most efficient one for evaluating transient temperature profile.All these models are developed using the open-source library (sci-kit-learn in Python) [19].A DT model consists of tree-like structures with roots, branches, leaves, and nodes, and it is more efficient for data points with non-linearity [20].However, DT generally undergoes overfitting that can be overcome by the RF model.RF is an ensemble learning technique that adopts a bagging methodology coupled with the decision of multiple trees [21].XGB model is generally applied for regression problems where the requirement of accuracy is more [22].KNN model is based on the concept of evaluation of euclidean distance and groups the output of similar features together and provides the mean value [23].ANN model has the capability of identifying the complex relationship among the non-linear data points during training [24].The basic architecture of the ANN model consisting of input layers, hidden layers, and output layers is shown in Figure S3.The hidden layers contain the number of neurons having a weight and bias to get higher accuracy by adjusting the errors in each epoch, called back-propagation technique.Neurons comprise transfer and activation functions [25].In the current study, the number of hidden layers are considered as three, the number of neurons in each layer is thirty-five, and the number of epochs are six thousand.The activation functions used in the input and output layers are relu and linear whereas, only relu is used in the three hidden layers.
The ML models are hyper-tuned to get the best accuracy on fitting and provide better prediction on unseen data.In the present work, GridSearchCV (GSCV) technique is used to hyper-tune the models that utilise all the possible permutations and combinations of parameters to train the model [26].RF, DT, and XGB utilise similar hyperparameters.In DT, the maximum depth, minimum sample leaf, and minimum sample split are significant parameters.For RF and XGB, the number of estimators plays a significant role, whereas the number of neighbors plays a key role for the KNN model.The performance of models are evaluated by R 2 -score (coefficient of determination) and root mean square error (RMSE) for a regression problem.The definition of error term utilises the difference betweem the data from FE model and ML models.
In order to select the optimal process parameters (current and velocity) for better deposition quality, single-bead experiments are designed.A total of 17 beads are deposited by considering different levels of current and velocity (120 A to 200 A and 3.3 mm/s to 15 mm/s).On deposition, dimensions (bead width, bead height, and depth of penetration) of continuous beads are measured.By considering five different process parameters ranging from 1.28 kW to 3.16 kW and 3.3 mm/s to 15 mm/s, a single track of dimensions, 120 mm length, is deposited using a CMT machine and 6-axis Fanuc robot (Figure 2a).The deposited track is shown in Figure 2(b).The feedstock material is ER90S-B9 (0.09% C, 0.59% Mn, 0.2% Si, 0.63% Ni, 8.93% Cr, 0.87% Mo, 0.21% V, 0.05% Cu, 0.07% Nb, 0.045% N) of 1.2 mm diameter, and SS316 is used as substrate material.The composition of shielding gas is 80% argon and 20% CO 2 with a flow rate of 15 L/min, and the offset of 12 mm is utilised between the CMT torch and deposited material.While the deposition of a material using CMT, the co-relation between current, wire feed speed (WFS), and voltage is known as a synergic line that varies with filler materials and shielding gas composition.For this study, the synergic line number 1220 is utilised for the deposition.The oxide layers of the substrate are removed by acetone.Two sets of deposition performed at a process parameter of 2.2 kW and 9.2 mm/s (240 J/mm) and 2.8 kW and 5.7 mm/s (490 J/mm) are utilised for the validation of FE model.

Results and discussions
It is obvious that the accuracy of the ML prediction relies on the correctness of FE results.Hence, the thermal model is validated with the experimentally measured temperature at selected thermocouple points.Figure 2(c-d) depicts the transient temperature distribution of deposits at thermocouple points Y1 and Y2.The numerically computed temperature is close to the experimental measurement.The temperature increases first and reaches to a peak point, and gradually decreases when the consumable wire is away from the thermocouple point.The maximum temperature at Y2 is more than that of Y1 since the thermocouple point is closer to the molten pool, and the arc power utilised for the first case is more than that of the second case (2.2 and 2.8 kW).The variation of peak temperature with respect to power and velocity is depicted in Figure 2(e-f).The weld bead profile is also measured experimentally and compared with the numerically obtained profile (Figure 3).The deposited track is assumed as parabolic in nature.Hence, the geometric profile of track is predefined by maintaining the volume flow rate of the process.The similar kind of parabolic profile is also observed from experimental measurement.The peak temperature increases with enhancing  power while it reduces on increasing the velocity at any step time.When the deposition speed is low, the concentrated heat source remains close to the molten pool, and it can effectively complement the dissipated heat that leads to a higher temperature.At the increased deposition speed, the heat dissipation is faster, and the elongated molten pool leads to a decrease in peak temperature.Figure 3 represents the cross-sectional view of the deposited bead at high and low heat inputs (240 and 490 J/mm).Both the profile and the dimension of the simulated result are well agreed with the experimentally obtained macrograph.The temperature between 867°C and 920°C (AC 1 and AC 3 ) illustrates the austenitization temperature range that depicts the transformation of deposits from tempered martensite to austenite.The difference between the solidus and liquidus temperatures of ferritic steel (80°C) is depicted in red color.However, the mushy zone is found wider at the trailing edge for higher deposition speed.With a high degree of constitutional supercooling, this may promote equiaxed grain size.The molten pool is represented by grey color in the ellipsoidal shape.It is obvious that the depth of the molten pool of the deposited material with higher arc power is more than that of the lower one (increased by 29%).However, the 26.3% increase in length attributed to higher deposition speed leads to more heat dissipation.It is obvious that the developed thermal model can be used to generate several other data that are not possible to measure directly from a simple experimental set-up.The datasets generated utilised for training and testing of a model are considered in the range of 1.28 kW to3.16 kW arc power and deposition velocity between 3.3 mm/s to 15 mm/s at eleven different levels.

Prediction of the transient temperature profile
Each ML model is hyper-tuned for a given range of hyperparameters in order to get the best R 2 score.The range of parameters used in the present investigation is depicted in Table S5.The results from ML algorithms are compared with FE results, and accordingly, the performance of the models are evaluated.Further, the correlation matrix for both the transient temperature distribution and peak temperature are analyzed to know the dependency between features and output variables.Figure 4(a) indicates that the accuracy of the RF, ANN, XGBOOST, and DT models are close to 100% while the accuracy for the KNN model is the lowest (88%) for this particular non-linear dataset.All the models are free from the most common issue (underfitting and overfitting), as the difference between the accuracy on training and testing datasets is less than 10%.After fitting a model, an error between the actual and predicted value of temperatures are evaluated using the RMSE metrics (Figure 4b).Computed errors between training and testing data for all the models are less than 16°C for the prediction of transient thermal profile.The difference in RMSE values of ANN and XGBOOST models is less than 4°C followed by RF (9°C), DT (15°C), and KNN (16°C).Figure 5 depicts the accuracy of both the training and testing datasets for all the models to predict the temperature at multiple data points.The predicted data points are closer to the diagonal line, which indicates the degree of accuracy for a particular model (R 2 closer to 1).First model comprises of low loss (ANN, XGBOOST, RF, DT, KNN) between the actual and predicted value.The data points of all these models are closer to the diagonal line (Figure 5(a-h)).It depicts the reliability of the ML models.Figure 5(i-j) indicate that the data points for the KNN models are scattered and away from the diagonal line (95% accurate on training and 88% on testing).Overall, the performance of the ANN model is better and it is suited for non-linear data points in the present case.Hence, the ANN algorithm is further utilised for predicting the distribution of transient temperature profile at validation deposition attributes by considering two different heat input levels (155 and 185 J/mm).These levels of validation attributes are considered outside the process parameter range so that the predicted output by ML models are compared with FE model at the same attributes (Figure 6).ANN algorithm is also capable to consider the effect of changing the deposition attributes of power for the same deposition speed (9.7 mm/s) on changing heat input.The peak temperature difference of 188°C is observed for ferritic steel (P91) deposits on increasing arc power from 1.5 kW to 1.8 kW.

Bidirectional prediction of maximum temperature
ML models with the highest accuracy are utilised for both forward and inverse prediction between the deposition attributes and peak temperature (Figure S4).To evaluate the maximum temperature for a deposition attribute the models (ANN, DT, RF, and XGBOOST) are considered and hyper-tuned to get better accuracy.Table 1 indicates the comparison between FE and ML predicted maximum temperature by all four models.The predicted peak temperature is closer to the FE values with an error of less than 10%.For the prediction of deposition attributes pertaining to the targeted peak temperature, both ANN and XGBOOST models are used.First, the velocity and temperature are considered as features and power is predicted.Further, power and temperature are utilised as input and velocity as a output.On these two independent datasets, ANN model performs quite well (R 2 = 0.99 for power and velocity) as compared to XGBOOST (R 2 = 0.95 for power and R 2 = 0.85) (Figure S5).Table 2 depicts the output of inverse prediction of power and velocity and their close resemblance with actual values.Hence, this study reveals that the in-situ monitoring of single-track deposition of ferritic on austenitic steel substrate at either single or multiple data points (transient thermal profile) for any deposition attribute by ANN, and XGBOOST model can be done with reasonable accuracy.
The correlation between the process parameters is represented by Pearson's correlation matrix (Figure S6).For the transient temperature profile, enhancement of continuing deposition value of temperature is analogous to the practical scenario of fabricating a component by the WAAM process.On increasing arc power, the maximum temperature increases, while on increasing velocity, it is negatively correlated (−0.55).It indicates that the generated datasets from a well-tested FE-model is correlated with the physics of the process behind the deposition process.Hence an efficient PIML model can be developed for any deposition attributes in AM industry.

Conclusions
In the present work, a hybrid modelling approach by combining ML algorithms and an experimentally validated FE model is deployed.The thermal model predicts the significant deposition attributes and their influence on the peak temperature.The key findings from this study are as follows: • Among all the five algorithms fitted on training dataset, the accuracy of the ANN, RF, XGBOOST, and DT are more than 99%, while KNN has the lowest accuracy (88%).The difference in error (between training and testing datasets) is less than 5°C by the most efficient ML models (ANN and XGBOOST).• The typical transient temperature profile in DED with an error of less than 10% using the best-suited ANN model is observed on validation attributes.The discrete peak temperature predicted by ANN, XGBOOST, RF, and DT is also having the highest accuracy (error less than 10%) and is in close resemblance with the output of the FE model.• The inverse predictions are performed for deposition attributes (power and velocity) having an accuracy of more than 99% using ANN model.• For the transient temperature analysis, time is found to be strongly correlated with temperature as the heating and cooling cycle varied with respect to the movement of the heat source.While for the maximum temperature, power is positively correlated and velocity is negatively correlated with heat input.• The accuracy of the ML model depends on the quality of the datasets required for training along with hypertuning of parameters.The similar methodology can be used for developing a more efficient PIML model for the prediction of residual stress and distortion such that the optimal features of AM process is beneficially used to fabricate a defect-free component.

Figure 1 .
Figure 1.Sequential steps involved in digital twin model: (a) Schematic representation of the solution geometry, (b) 3D temperature profile, (c) time-temperature profile at different process parameters, (d) steps involved in integrating with ML algorithms, (e) output predicted by ML algorithm for training, and (f) testing of ML model.

Figure 2 .
Figure 2. (a) Experimental set-up; (b) Deposited tracks for an arc power ranging from 1.28 kW to 3.16 kW and deposition speed of 3.3-15 mm/s.(c,d) Comparison between the simulated transient thermal profiles with experimentally measured data at thermocouple points Y 1 and Y 2 ; Variation of peak temperature at the end of deposition for investigated range of (e) arc power and (f) velocity.

Figure 3 .
Figure 3.Comparison between computed and experimentally measured thermal profile: (b) at an arc power of 2.2 kW and 9.2 mm/s and (c) 2.8 kW and 5.7 mm/s.

Figure 4 .
Figure 4. Evaluation metrics utilised for different models: (a) R 2 score, (b) RMSE of both training and testing datasets for an arc power ranging from 1.28 kW to 3.16 kW and deposition speed of 3.3-15 mm/s.

Figure 5 .
Figure 5.Comparison of the predicted temperature ML models and FE-based numerical model on training and testing datasets for an arc power ranging from 1.28 kW to 3.16 kW and deposition speed of 3.3-15 mm/s.

Figure 6 .
Figure 6.Transient temperature distribution prediction by digital twin model at different deposition variants.

Table 1 .
Prediction of the peak temperature on validation deposition attributes with an error by ML model.

Table 2 .
Comparison of the actual and predicted power by the ANN and XGBOOST models.