Smoothing-Aided Support Vector Machine Based Nonstationary Video Traffic Prediction Towards B5G Networks

Video services have hold a surprising proportion of the whole network traffic in wireless communication networks. Accurate prediction of video traffic can endow networks with intelligence in resource management, especially for the forthcoming beyond the fifth-generation (B5G) networks. However, the existing approaches fail to accurately predict video traffic with all types of frames, due to the natures of strong long-range dependence, self-similarity and burstiness. Obviously, it is unable to meet the QoS and QoE requirements of dynamic bandwidth allocation. In this paper, we propose the feasibility of advanced machine learning methodology applied in nonstationary video traffic prediction, i.e., smoothing-aided support vector machine (SSVM) model. The model utilizes classical smoothing methods to preprocess video traffic by relieving the drastic fluctuation of video stream. It can provide an effective association for the subsequent support vector regression, as the preprocessed data becomes more smooth and continuous than the original unprocessed one. Experimental results show that our proposed model significantly outperforms the state of the art model, i.e., logistic smooth transition autoregressive, in prediction performance. The superior nonlinear approximation capacity is further demonstrated by visualized statistical analysis.


I. INTRODUCTION
B Eyond the fifth-generation (B5G) networks are expected to provide rather reliable services with large-scale connectivity, superhigh transmission rate, ultra-low latency, much enhanced security, very little energy loss, and excellent quality of experience (QoE) [1].Given these superiorities, the rising B5G networks should be intelligent enough to cope with more autonomous [2]- [4], self-organizing and dynamic situations, such as dynamic topologies, application-oriented storage, autonomic and scalable computing, and various quality of service (QoS) requirements.For the design of such smart networks, machine learning is deemed as a critical element and it has been adopted in many applications [5]- [16].
With the deployment of 5G mobile networks [17], traffic has been governed by explosive video applications including IPTV, video assisted surgery, video conference and augmented reality.For the following B5G networks, this situation can be further exacerbated.Network operators are confronted with a significant issue: if they intend to reduce the bandwidth provision for a users requirement to enable more users to access the network, it may lead to potential packet loss that has a significant impact on the communication quality; on the other hand, if they oversupply bandwidth to cope with possible bursts in data, bandwidth wastage occurs.These diversified multimedia services certainly require higher bandwidth configuration in contrast to traditional network traffic, such as web and email.Generally, excellent rate control mechanisms are design for video stream in dynamic heterogeneous network scenarios.It yields well-known variable bit rate (VBR) traffic.In this case, resource management should possess potential robustness and efficiency, aiming at guaranteeing high bandwidth utilization, so that any QoS requirement can be protected [18]- [20].Conventional fixed allocation of network resource suggests the reservation of a great many bandwidth for a specific QoS guarantee.Nevertheless, dynamic allocation under the framework of video traffic prediction is an alternative technique to solve this intractable issue [21].
From data characteristics, VBR traffic shows the properties of strong nonlinearity, variability, and slow-decaying autocorrelations between samples.These implicit traits determine the long range dependent (LRD) of video streaming.Fixed allocation mode has to abandon the available bandwidth.To this problem, dynamic bandwidth allocation appears to prevent probable network congestion.In this dynamic mode, frame size prediction is regarded as a crucial supplementary mean.However, the one-step ahead prediction models on LRD video traffic can not respond to dynamic bandwidth allocation in high-latency links.Hence, long-term video traffic prediction should be required for ensuring flexible network control strategies [22].
A great deal of machine learning research has been focused on multiple-steps ahead video traffic prediction, mainly including statistical learning methods and neural networks (NNs).
The key issue is how to improve nonlinear approximation capacity as much as possible [23]- [25].It is demonstrated that classic AR, ARMA and ARIMA can only seizure both linearity and short range dependencies (SRD) hidden between video data, but are powerless to LRD, which leads to weak performance in video traffic prediction [26].In [27], a logistic smooth transition autoregressive was developed for VBR video traffic prediction, where adaptive least mean square and its extensions were considered for the determination of model parameters.It could achieve a nonlinear approximation up to 300-step ahead prediction.In [28], a seasonal autoregressive method was used for high definition video stream modeling.In [29], a Markovian model was presented for the specific Bframe prediction, aiming at reducing bandwidth requirements and smoothing the video stream.In [30], a hybrid deep computing model, consisting of stacked restricted Boltzmann machine and minimum complex reservoir, was proposed for one-step video frame size prediction.In [31], six NN models were compared according to the capacities in predicting each frame type of MPEG-4, i.e., I, P and B.Moreover, this work was further extended, where more NNs were introduced for whole MPEG-4 video stream processing [32].In spite of these positive results, the accurate multiple-steps ahead prediction of video traffic in autonomous network management remains an challenging task and the technology is far from actual deployment.This is mainly due to the hight-variability in frame size of video traffic, i.e., burstiness.This paper focuses on exploring more effective multi-steps ahead prediction method of video traffic supporting intelligent resource management towards B5G networks.
Support vector machine (SVM) is deemed as a powerful machine learning tool for prediction, whose success lies in Vapniks pioneering study in statistical learning theory [33], including robustness, restorability to over-fitting, optimal solution, and modularity of kernel mapping.Concretely, SVM has remarkable characteristics of impressive generalization capability, no local minimum and sparse representation of solution [34].It has been used widely for multi-steps ahead prediction in numerous scenarios, such as inbound tourist arrivals, physiological abnormal signals and consumer's heat load [35]- [37].Besides, it has been demonstrated that data smoothing is beneficial for nonlinear approximation in time series prediction [38].This preprocessing method could handle significant fluctuations and outliers by adjusting the built-in sliding window.Inspired by these, we consider the combination of SVM and smoothing for the multi-steps ahead prediction of VBR video traffic.
In this paper, we propose a smoothing-aided SVM (SSVM) model for VBR video traffic prediction method in order to meet the requirement of dynamic resource allocation.Structurally, it can be viewed as a uniform and successive system with functional modules of smoothing and regression.The employed smoother is capable of reducing the size of leptokurtic frame, thereby alleviating drastic fluctuation of video stream.This way can facilitate obtaining a well-behaved SVM for multi-steps ahead prediction.The effectiveness of our proposal is illustrated on different video traffic prediction tasks.
The rest of this paper is organized as follows.For the dynamic resource allocation demands of B5G networks, Section II provides a detailed description on the smoothed-aided SVM model for nonstationary video traffic prediction.Section III gives some experiments to verify the performance of our proposal, including smoothing analysis, prediction accuracy and statistical validation.Finally, Section IV concludes the paper and points out the future research directions.

II. MODEL DESCRIPTION AND OUR PROPOSED METHOD
Tackling the challenges of resource management in B5G network environments, a promising machine learning model, combining with smoothing methods and SVM, is applied for VBR video traffic prediction, called SSVM.In fact, it is derived from the promising finding in [38].In structure, our video traffic prediction model consists with functional devices of smoother and predictor.The prioritized smoothing aims at effectively eliminating the tricky abrupt fluctuations of video traffic, i.e., short-term burstiness.It is conducive to providing more suitable initial points for nonlinear approximation.The well-behaved SVM serves as a predictor, characterised by a high-dimensional feature transformation based on kernel projecting.Combining these two components, our model is expected to achieve a strong prediction performance.For a better insight, we provide a detailed description on the functionalities of both.
Generally known, significant burstiness often appears in compressed video over numerous time scales.This is largely due to the special frame structures determined by different encoding schemes and high variations related to scenes.Moreover, burstier VBR traffic can be formed from aggregations of multiple video streams attaching orchestrated text, audio and images.In our scenario, smoothing is used to decrease bit rate variability appropriately, so that it can produce a relative smooth video stream in priority for acceptable nonstationary VBR video traffic prediction.The following four smoothing methods are considered [38], given by Moving average (MA) is viewed as a simplest version of smoothing methods.Its readout can be generated from the average of its corresponding neighboring points in a customizable window, expressed as follows where z denotes a sample of this smoother input, 2n + 1 is the size of moving window, and j offers an index of the current readout value.Gaussian smoothing (GS) is on the basis of the wellknown normal distribution, defined by the following probability density function where x is the sample value obeying Gaussian distribution, σ, σ 2 and µ denotes the standard deviation, the variance and the mean of samples, respectively.
Robust locally weighted smoothing (RLWS) is achieved by a bisquare function w i , given by where r i denotes the i-th residual value generated form the current smoothing, and κ is the sixfold median absolute deviation (M AD) of these residuals, given by Savitzky-Golay smoothing (SG) is actually regarded as a low-pass filter that has the ability of local least-square polynomial approximation.Considering a window i ∈ (M, N − M ), the smoothing has the following form where N denotes the sample size, n denotes the polynomial degree, and a denotes the vector of coefficients, calculated by Once the smoothing on VBR traffic streams is completed, the LS-SVM based predictor works for the subsequent nonlinear approximation.For a given video dataset , the regression output is measured by a nonlinear mapping function ϕ(•), defined as follows where w denotes a weight vector, and b is the corresponding bias.Given this, a separating hyperplane emerges for the fitting of o and s.
Through the rule of structure risk minimization, the regression related to LS-SVM can be converted to the following optimization issue, given by obeying where e i denotes a randomly generated error, and C > 0 denotes a regularization measure.It can be solved by the Lagrange multiplier method, formulated as follows where α i is a Lagrange multiplier.Furthermore, we have the following partial differentials Through eliminating the common parameters w and e i , we have 0 where and Ω ij = ϕ(s i ) T ϕ(s j ) = K(s i , s j ), for i, j = 1, 2, 3, • • •, l with K a Mercer kernel function.In our case, the linear kernel, the polynomial kernel, the radial basis function kernel (RBF) and multi-layer perceptron kernel (MLP) are considered, given by Solving the linear equation ( 12), α and b can be determined.It implies the completion of the LS-SVM training.Our optimal model is given by Subsequently, the trained model can be used for real-world VBR video traffic prediction.

III. EXPERIMENT RESULTS AND DISCUSSIONS
In this section, an exhaustive simulation is provided for the performance assessment of our SSVM paradigm, considering real-world nonstationary VBR video traces.During evaluations, we consider different types of smoothers under user-defined sliding window, such as MA, GS, RLWS and SG, as well   the grid search method.All results are given over an average of the experiments repeated 10 times.

A. Data and Assessment Criteria
Here, NBC news dataset from the video trace library is used in experiments, found at http://trace.kom.aau.dk/h264/index.html.With regard to its characteristics, there exist frequent transitions among scenes.Two types of NBC news video stream (QP=10 and QP=34) are considered, and the number of frame is 49521.These video frames encoded by H.264/AVC have a structure of G16B3 GOP, in which 16 frames are introduced between adjacent I and B frames.The prediction performance of our model is measured by normalized mean squared error (NMSE), given by where σ 2 x is the variance of video traffic series x over L points, x(n) denotes the predicted value of x(n).

B. Smoothing Analysis
To investigate the effect of the smoothing on the series characteristics, we plot the trends of the 21 video traffic series as a function of the time window in Fig. 1, where the time window is 0, meaning the original series, while its other settings correspond to the 20 smoothed series.From   this figure, it can be seen that the original video series shows a dramatic fluctuation, since there exit a great deal of bursty sources.The specific data characteristic enables accurate multi-step ahead prediction of video traffic to be a quite challenging work.In fact, smoothing effectively reduces significant oscillations in our scenario.Moreover, as the time window increases, this smoothed series become relatively stationary for the considered smoothing methods, which is highly beneficial for prediction.However, this increase is not arbitrary, because video streams may have fundamental changes in internal and external characteristics, such as selfsimilarity, bursty and trend.Hence, the reasonable choice of the time window in smoothing is critical for given video traffic prediction tasks.It requires an effective tradeoff between prediction accuracy and data characteristics, but this is beyond the scope of this paper.Our study aims at exploring the feasibility in smoothing assisted video traffic prediction for dynamic resource management.

C. Prediction Performance
Fig. 2 plots comparison curves of the actual signal and the predicted output yielded by our SSVM model over frame numbers, where multiple-steps ahead prediction, such as 1, 50 and 200, is considered for NBC News video traffic traces (QP=10 and QP=34), as well as the most suitable smoothing method and its time window are suggested for each prediction mode.It can be observed from this figure that our model readout is consistent with the actual series, especially for the one-step ahead prediction.Nevertheless, this trend fitting results become relative poor as the prediction step increases, as shown in Fig. 2(c) and Fig. 2(f).The superior performance of our model over LSTAR is further listed in Table I.It is worth noting that compared with LSTAR, SSVM has NMSE decreases of 32.35% and 30.94% for one-step ahead prediction, respectively.In fact, its significant performance advantages are mainly ascribed to the prepositive smoothing mechanism controlled by time window, which can effectively alleviate the high variability of video frames, i.e, burstiness.Taking an example of 50-step ahead prediction, the relation between time window and prediction performance is shown in Fig 3 .Obviously, our model is relatively sensitive to the choice of time window.The bigger time window can lead to the better prediction performance.However, the time window is not infinitely increased, because it can make the smoothed video streams lose the essential characteristics.

D. Statistical Validation
In the following part, from the perspective of statistical analysis, we verify the nonlinear approximation capacity of our SSVM in the video traffic prediction tasks.The methods, such as histogram fitting, QQ plot and box-plot, are used in the case.
Fig. 4 provides graphical representations of histogram plots on the shapes and distributions of the actual and predicted   outputs for SSVM in our video traffic prediction scenario.It is seen that for one-step ahead prediction, our model can achieve good fittings of vertical bars and data distribution for the two actual video trace.Whereas, as the prediction step is up to 200, this fitting gets worse, meaning the performance degradation of SSVM.Fig. 5 shows the QQ plots of the actual and predicted video traffic, yielded by plotting the quantiles of video data distribution versus the ones of normal distribution.Obviously, the well-fittings in Fig. 5(a) and Fig. 5(d) indicate that SSVM behaves well in the scenario of onestep ahead prediction.However, in Fig. 5(b)-(c) and Fig. 5(e)-(f), the divergences between distributions mean relatively poor multiple-steps ahead prediction performance.This case can be further demonstrated in Fig. 6 based on the popular box-plot visualization, where we gives an insight on the superiority of our model through these comparisons of the upper and lower quartiles, the upper and lower bound, the median and the outliers.Besides, we can also observe that the smoothed video streams have few outlier, meaning the decrease of bursty points.Above all, these results correspond to the findings in Fig. 2. It demonstrates the efficacy of the SSVM model in our nonstationary VBR video traffic prediction tasks.

IV. CONCLUSION
In this paper, we explore the application of smoothed support vector machines for nonstationary video traffic prediction in the B5G network environment.It is actually a hybrid data processing model, where multiple smoothing methods are considered for video stream preprocessing.The preceding smoother can effectively alleviate burstiness in video data, providing excellent initialization points for the following SVM-based approximation.Experimental results verify the utility of our SSVM model in terms of prediction accuracy and statistical comparison.Further research will concentrate on the adaptive determinations of smoothing methods for given VBR video stream, as well as the selection of the corresponding time windows.

Actual output
Predicted output 1.4

Fig. 1 :
Fig. 1: Comparisons of the original NBC News video traffic series (QP=10 and QP=34) and the smoothed ones via time windows for the considered smoothing methods.The time window is set to the values from 0 to 200 (from left to right), where the time step is 10.

Fig. 2 :Fig. 3 :
Fig. 2: Output comparisons of our SSVM model with optimal smoothing on the prediction tasks of NBC News video traffic traces (QP=10 and QP=34) over different prediction steps.

Fig. 4 :
Fig. 4: Histogram fitting of our SSVM model with optimal smoothing on the prediction tasks of NBC News video traffic traces (QP=10 and QP=34) over different prediction steps.

Fig. 5 :
Fig.5: QQ plots of our proposed SSVM model with optimal smoothing on the prediction tasks of NBC News video traffic traces (QP=10 and QP=34) over different prediction steps.

Fig. 6 :
Fig.6: Box-plots of our SSVM model with optimal smoothing on the prediction tasks of NBC News video traffic traces (QP=10 and QP=34) over different prediction steps.

TABLE I :
Model performance on the prediction tasks of NBC News video traffic traces (QP=10 and QP=34) over different prediction steps.