Two-Layer Distributed Content Caching for Infotainment Applications in VANETs

For vehicular ad hoc networks (VANETs), edge caching has attracted considerable research attention to maximize the efficiency and reliability of infotainment applications. In this article, we propose a two-layer distributed content caching scheme for VANETs by jointly exploiting the cache at both vehicles and roadside units (RSUs). Specifically, we formulate the content caching problem to minimize the overall transmission delay and cost as a nonlinear integer programming (NLIP) problem and propose an alternate dynamic programming search (ADPS)-based algorithm to solve it. In ADPS, we divide the original problem into three subproblems and then we use the dynamic programming (DP) method to solve each subproblem separately. To reduce the complexity, we further propose a cooperation-based greedy (CBG) algorithm to solve the large-scale original problem. Both numerical simulation results and experiments in the testbed show that the proposed caching scheme outperforms existed caching schemes, and the transmission delay and cost can be reduced by 10% and 24%, respectively, while the hit ratio can be increased by 30% in a practical environment, as compared to the popularity-based caching scheme.


I. INTRODUCTION
H IGH-QUALITY vehicular infotainment applications in VANETs, such as in-car entertainment or mobile advertising, have gained considerable attention in recent years. As an example, using interactive screens installed on vehicles, on-board passengers can browse the list and request interested media contents. This is an attractive value-added service to transportation operators such as taxi companies or peer-topeer vehicle sharing companies like Uber. For these type of applications, the primary concern is to meet the stringent requirements of quality of service (QoS). Unlike other applications, such as secure message dissemination or adaptive traffic management, infotainment application can significantly Copyright (c) 2021 IEEE. Personal use of this material is permitted. However, permission to use this material for any other purposes must be obtained from the IEEE by sending a request to pubs-permissions@ieee.org.
impact the amount of data exchanged in VANETs. The high volume of exchanged data will degrade QoS, such as delivery delay. Moreover, from the perspective of operator, the high randomness and high concurrency feature of infotainment application raise challenges for some performance metric of the network, such as content hit ratio. And this may also result in QoS degradation.
Besides, for infotainment applications, the cost of data transmission is also very considerable due to the excessive amount of data to be transferred. Compared with investment into the construction of VANETs, content storage cost, which corresponds to the cost of updating the contents, can be negligible. Hence for operators, it is expected that the cost of wireless mobile communications will become a major source of operational expenses of infotainment application service. The operators need to maximize the revenue and reduce the expense. Hence, taking a commercial perspective, how to reduce the communication cost for infotainment applications has become a crucial problem. Clearly, the cost per unit data transmission varies when the vehicle fetch contents from different nodes in VANETs. If a vehicle always need to fetch content from a remote server, then the total cost of the operator's service will be high.
To address the two problems above, VANET edge caching is proposed to facilitate content delivery. That is, each RSU can selectively store media contents and distribute them to vehicles [1]. However, the caching resource of RSU is constrained. Meanwhile, when vehicles drive through different RSUs, the vehicular connections are always intermittent due to the limited coverage area of RSU. Hence, compared with sole RSU caching, using vehicles for cooperative caching is helpful and reasonable. This cooperation among RSUs and vehicles can provide seamless connections within the interlaced coverage of RSUs. Moreover, the contents for infotainment applications always have different request probability. Hence, a more precise content caching scheme is required, i.e., determining whether to cache a specific content on a specific network node (RSU or vehicle). With this caching scheme, users can access their required contents directly from a much closer RSU or vehicle, which can remarkably decrease the content retrieval delay and transmission cost. The limited storage capacity of vehicles and RSUs can be fully utilized to improve network performance in terms of average content delivery delay, average content delivery cost, and cache hit ratio.
There has been a number of researches focusing on caching in device-to-device (D2D) mobile networks [2]- [13]. Sun et al. [2] propose mobility-aware caching strategies in D2D networks to minimize the network delay. Wang et al. [5] take advantage of the user mobility pattern by the inter-contact times between different users, and propose a mobility-aware caching placement strategy to maximize data offloading ratio. Some of these works study edge caching for conventional cellular networks. However, they have different emphases from the works for VANETs. Firstly, in conventional cellular networks, the network topology is relatively stable. So that server switching is not very frequent. Secondly, the power and storage of a mobile device in a conventional cellular network are limited, so it is inappropriate to choose mobile devices as caching servers or relays. Thirdly, the works always use intercontact model to characterize the mobility pattern, but this model is insufficient in dealing with practical vehicle mobility.
In vehicular networks, most researches focus on caching at RSU layer. Ding et al. [14] propose a centralized RSU caching algorithms to minimize the average time that an onboard unit (OBU) downloads a file. Hu et al. [15] study the scenario of RSU caching, where multiple content providers try to improve the data dissemination efficiency of their own contents by utilizing the storage of RSUs. Su et al. [16] propose a model to determine whether and where to obtain the replica of content upon the request of a vehicle. They propose a crossentropy based dynamic content caching scheme. Qin et al. [17] propose a hierarchical end-edge framework with the aid of computation offloading and content caching to minimize network overheads. AlNagar et al. [18] propose a cooperative proactive caching scheme between RSUs to minimize the communication latency and enhance QoS in VANETs. Ma et al. [19] propose a caching placement policy based on the cloud-based vehicular ad-hoc network (VANET) architecture to minimize the average latency of content retrieve. Hou et al. [20] propose a Q-learning-based proactive RSU caching strategy under the support of long short-term memory (LSTM) neural network, aiming at enhancing the QoS for non-safety related services in vehicular networks.
Caching at vehicular layer via vehicle-to-vehicle (V2V) communications reduces transmission delay and bandwidth competition, as compared to RSU caching. Hu et al. [21] propose an in-vehicle caching framework based on an innovative integration of distributed storage with cached content relay facilitated by one-hop V2V links. They design a dynamic distributed storage relay (D 2 SR) mechanism, which ensures the survival of cached contents with a high probability within the time duration of interest. Yao et al. [22] propose a cooperative caching scheme based on social attributes and mobility prediction for vehicular content centric network with higher cache hit ratio and lower content access delay compared to other state-of-the-art schemes. Deng et al. [23] propose an optimal retention-aware caching scheme to minimize network cost. Zhang et al. [24] model the interactions between caching vehicles and mobile users as a two-dimensional Markov process and propose an online vehicular caching scheme by optimizing network energy efficiency. Han et al. [25] develop a dynamic pricing-based incentive mechanism for content sharing in cellular vehicle-to-everything (C-V2X) based vehicular network to improve the successful delivery ratio and energy efficiency. Yang et al. [26] propose a secure caching placement and delivery algorithm to minimize the maximum vulnerability among all vehicular users.
A few schemes also studied cross-layer cooperative caching [27], [28] (i.e., Cooperative caching in RSUs and/or vehicles and/or base stations (BSs)). Chen et al. [27] propose a cooperative edge caching scheme based on the heterogeneous vehicular networks (H-VNets) with multi-tier edge caching servers (i.e. at BS and RSUs). Qiao et al. [28] design a novel edge caching framework based on the cooperation among base station, RSUs, and vehicles. Then, the joint vehicle scheduling and bandwidth allocation scheme is designed to minimize the content access cost while satisfying the constraint on content delivery latency.
Existing literature is mostly based on the VANETs with simple one-layer structure (e.g., vehicular layer or RSU layer). Their edge caching schemes usually consider vehicular downloading applications with a single QoS metric. At the same time, few research addressed the commercial perspective and investigated the data transmission cost issue. In this paper, we consider joint caching of vehicles and RSUs. The paper aims to fulfill two main objectives. Firstly, full utilization of limited caching capacity of vehicles and RSUs to satisfy as many content requests as possible. Secondly, determining appropriate caching scheme to minimize the overall transmission delay and cost. We form the two-layer caching scheme design as a NLIP problem. Because the problem is neither linear nor convex, we propose an ADPS based algorithm to solve the NLIP problem. In ADPS, we divide the original problem into three sub-problems, then we use DP method to solve each sub-problem separately. Furthermore, we propose a lowcomplexity CBG algorithm to solve the large scale original problem. The results of numerical simulation and test bed verification show our proposed caching scheme outperforms existed caching schemes in transmission delay, cost and hit ratio. The main contributions of this paper are listed as follows.
• We propose a novel two-layer cloud-based VANET edge caching model, consisting of RSU cloud (RC) layer and vehicular cloud (VC) layer. In this model, the networks nodes (e.g., vehicles, RSUs) have different communication interfaces as well as heterogeneous computation, communication and storage capacities. Vehicles may submit multimedia content request, which are offloaded to different network nodes based on a certain scheduling algorithm. • Considering vehicle mobility and content retrieve process in our model, the average transmission delay, cost and hit ratio are analytically derived. Then, considering the limited caching capacity of different nodes, the joint vehicle and RSU optimized caching problem is formulated as a NLIP problem. • We propose an ADPS algorithm to solve the formulated NLIP problem. This algorithm divides the original problem into three sub-problems. The sub-problem is reformulated as a multi-stage decision problem and be solved stage-by-stage recursively. The result of this algorithm is called ADPS caching scheme. • Inspired by the result of ADPS, a CBG algorithm with linear complexity is proposed. This algorithm can give a good enough sub-optimal solution, at the same time, reduce the complexity of computing significantly. So it is more perfect to very large scale network model. The result of this algorithm is called CBG caching scheme. • In addition to numerical simulation, the system model is fully implemented in a prototype VANET using real OBU and RSU devices and real cloud computing service. Both simulation results and test bed evaluation show that our proposed caching schemes have the advantage over other caching schemes. The rest of the paper is organized as follows. Section II gives the system model and problem formulation. Section III and IV respectively propose the ADPS and CBG caching scheme. The simulation results are provided in Section V. In Section VI, test bed evaluation is shown. Finally, Section VII concludes this study.
N otation: Bold lower case letters and bold upper case letters are used to denote column vectors and matrices, respectively.

II. SYSTEM MODEL AND PROBLEM FORMULATION
In this section, we introduce the network model, communication model, and content distribution model, respectively. We analytically derive the average transmission delay, cost and hit ratio, and then formulate the NLIP problem. This problem is key for caching scheme decision. The primary notations used in this section are summarized in Table I.

A. Network Model
As shown in Fig. 1, we consider a bidirectional road scenario, including vehicle, RSU and base station (BS). The whole VANETs can be divided into two layers: RSU cloud (RC) layer and vehicular cloud (VC) layer. In RC, the RSUs are connected via backbone network to form a computing cloud and all RSUs are termed as RC members. VC is formed by moving vehicles within a specific area. Each VC has a VC controller (VCC) which manages caching resource and other vehicles are termed as VC members. VCC can be selected through appropriate algorithms for a VC. In the network, BS and content provider (CP) connect to the Internet through the backhaul link. CP is generating a different set of contents. Based on user interest, if the requested content has been cached at VC and/or RC, it can be delivered to the vehicle by VC and/or RC, depending on the trajectory of the vehicle and the locations of cached contents.
Without loss of generality, we focus on the coverage area of one BS, within which N RC RSUs are deployed along the road using dedicated short-range communication (DSRC) access technology to communicate with vehicles. The coverage areas of different RSUs are assumed to be overlapping and the communication range of RSU is denoted by r. RSUs and vehicles are equipped with caching capacity, with the storage capacity denoted by S R and S V , respectively. A network management controller is deployed at BS, which collects information from vehicles and RSUs and makes decisions on content caching [29].

Entrance
Number of vehicles entering entrance j V C j (t) Number of V C j entering entrance j λ j Average arrival rate of entrance j µ j The expectation of V C j (t) Sq Data size of content l L Number of contents in the networks p(l) Request probability of content l The coverage range of RSU v The average velocity of vehicles R V V Data transmission rate for delivering the requested content between vehicles R RV Data transmission rate for delivering the requested content from nearest RSU R RCV Data transmission rate for delivering the requested content from RC members R P V Data transmission rate for delivering the requested content from CP through remote server P V V Price for delivering unit data between vehicles P RV Price for delivering unit data from nearest RSU P RCV Price for delivering unit data from RC members P P V Price for delivering unit data from CP through remote server Analytical Symbols τ j,l The download delay for entire V C J receiving the whole requesting content l τav The average delay incurred by an arbitrary content request Delay to obtain content l for vehicle The cumulative download of the content l when the vehicle V i in V C j leaves the cover area of RSUm.

Costav
The average cost incurred by an arbitrary content request Cost i,j,l Cost to obtain content l for vehicle V i in V C j Cost i,m,l,j Cost for vehicle V i in V C j to download unit data from content l at RSUm C V C j Binary cache placement matrix for V C j and the dimension is N V C j × L C RC Binary cache placement matrix for RC and the dimension is N RC × L

B. Vehicle Mobility Model
In our model, vehicles take platoon-based driving pattern [30], which is a cooperative driving pattern for a group of vehicles with common interests. The speed of the platoon leader VCC is within [v min , v max ] [31]. Other vehicles adjust their speed with the goal of tracking the speed of VCC and keeping a constant inter-vehicular space gap [32]. The average velocity of vehicles is v = (v min +v max )/2. We assume there is one entrance at each end of the road. The number of vehicles entering entrance j (j = 1, 2), denoted as V j (t), is timevarying and follows a Poisson distribution with the parameter λ j , which denotes the average arrival rate [14], [21], [33]. Then we have: The

C. Content Distribution and Retrieve Process Model
Contents = {1, 2, · · · , L} denotes the set of content piece, with the size S q of each content, which is constantly updated by adding new contents. Considering that some content may be requested more frequently than others, we assume the popularity of content follows a Zipf distribution [34]. The probability that content l is requested is given by: where γ is the parameter of the Zipf distribution.
in RC, we denote binary cache placement matrices at vehicles and RSUs by C V Cj and C RC respectively, where m,l denote the discrete cache placement indicator for content l in vehicle i and RSU m respectively.
otherwise. (5) The content request process is shown in Fig. 2. It should be noticed that each RSU can cooperate with other RC members to fulfill the content request. That means, if a piece of content is cached in one RC member, then all RC members can use it to respond a vehicle's request. We say a hit occurs when the requested content l from moving vehicle V i in V C j can be fulfilled by its cache, V C j or RC. If that still fails, the request can only be fulfilled by CP through the remote server and no hit occurs in this case. So, hit occurrence depends on whether the content requested by the vehicle matches the content cached in VC and RC. When the content request is satisfied by the VC members, the serving RSU, the RC members, and the CP, the data transmission rates are respectively denoted by R V V , R RV , R RCV , R P V . In terms of data transmission cost, the corresponding prices caused by transmitting unit data are denoted by P V V , P RV , P RCV , P P V . It is reasonable to assume that R V V > R RV > R RCV > R P V , P V V < P RV < P RCV < P P V . The transmission delay and cost are 0 if a requesting vehicle obtains the content from its own cache.

D. Average Content Delivery Delay
When V C j members enter entrance j, vehicles in V C j will start to download the requested content l. We define τ i,j,l as the delay for vehicle V i in V C j to completely download the requested content l. The download delay for entire V C j receiving the whole requesting content l is τ j,l . Considering that each content has a different request probability, the expectation of τ j,l is: The average delay incurred by an arbitrary content request, denoted by τ av , can be calculated as: When a vehicle crosses RSU m , we need to consider the downloading data rate R i,m,l,j of vehicle V i in V C j , and calculate the total downloading data rate for V C j and RC. We use R V Cj i,m,l to denote the transmission rate satisfying users request within V C j , then: In (8), indicates the event that the desired content is not cached in V i , but the content can be found in other V C j members.
Similarly, if the request is not satisfied within V C j and is forwarded to RC, then the transmission rate R RC i,m,l is: (10) Considering (8) and (10), the total download transmission rate is: indicates that the request fails within V C j .
Next, we use δ V Cj i,m,l to indicate whether vehicle V i in V C j has completely obtained the content l from RSU m , where: if the vehicle V i in V C j has completely obtained the requested content l from RSU m , 0, otherwise. (12) When the vehicle V i in V C j leaves the cover area of RSU m , we let D V Cj i,m,l to denote the cumulative download of the content l. We have: Based on above analysis, we have: In (14) and (15), C1 means the vehicle can get the content in local cache, and the delay is 0. C2 means the vehicle complete the download from the first RSU it passed. In C3 situation, the vehicle cannot complete the download from the first RSU, but it can complete the download before the last RSU. C4 means when the vehicle passed the last RSU, the download still can not be completed and the unfinished content is obtained through the remote server. The analysis process of τ i,2,l is similar to that of τ i,1,l .

E. Average Content Delivery Cost
The total average delivery cost can be calculated as: when the vehicle V i in V C j passes RSU m , the unit data transmission cost Cost i,m,l,j is: The transmission cost sum Cost sum i,m,l,j can be calculated as: To completely download the content q l , the total cost of the vehicle V i in V C 1 is: where (21) The analysis process of Cost i,2,l is similar to that of Cost i,1,l .

F. Average Content Delivery Hit Ratio
When the vehicle V i in V C j has completely downloaded the content l, we can get the number of hits and the total number of RSUs that V i has passed. Let hitratio i,l,j denote their ratio, then the average hit ratio can be calculated as:

G. Problem Formulation
In infotainment applications for VANETs, delay is a critical indicator of user experience to obtain desired content. Another key point is the cost. The caching scheme will affect delay and cost significantly. In this paper, we aim to make full use of the limited storage capacity of vehicles and RSUs to reduce the overall transmission delay and cost. Then the joint vehicle and RSU caching optimization problem can be formulated as: where α and β are the weight parameters which can be adjusted according to user's preference. τ max and Cost max are the maximum tolerable delay and download cost when the content is completely downloaded. In the worst case, the requested content can only be found in a remote server, and then τ max = S q /R P V , Cost max = S q P P V . The constraint in (23) indicates that the content in each vehicle and RSU cache cannot exceed its capacity. In the objective function of (24), we use τav τmax and Costav Costmax for normalization.
Noting that problem P C is a 0-1 NLIP problem, which is usually NP-hard and difficult to solve. The objective function of problem P C (24) is very complicated, which makes problem P C impossible to be handled by any convex relaxation techniques. 1 In the following sections, we will propose effective algorithms to solve it. The problem of finding the global optimal C V C1 , C V C1 and C RC can be treated as a future extension of this paper.

III. ADPS JOINT VEHICLE AND RSU CACHING SCHEME
In this section, inspired by alternate convex search (ACS) algorithm [35] [36], an alternate dynamic programming search (ADPS) algorithm is proposed to solve problem P C . At first we divide the original problem into three sub-problems. Then we solve each sub-problem using DP method. At the separation step, each time we choose two matrices and fix them, then a sub-problem is created. This strategy will generate three simplified sub-problems for problem P C : Problem P C1 , P C2 , P C3 are still NLIP problem and have similar structure. Next, an optimal DP algorithm will be proposed to solve these three sub-problems separately.

A. Dynamic Programming Algorithm
DP is an effective algorithm to handle integer variables by breaking the original integer problem down into a multiplestage decision problem, and solves the problem stage by stage. In particular, each stage is associated with multiple states, and the optimal decision in each stage is made based on the optimal decision at previous stage according to a recursive relationship. Thus, with the optimal decision at the first stage and the recursive relationship, the optimal decision can be made stageby-stage and the optimal solution of original problem can be constructed. In what follows, we solve P C1 to illustrate the key steps of the proposed DP algorithm. These steps include reformulating problem P C1 as a multi-stage decision problem and finding the corresponding recursive relationship between adjacent stages. The DP algorithm is described in Algorithm 1.
...  2 2) Explanation of 4 paths from stage 2 to stage 3 The optimal value is calculated according to (32 ), which equals the gain of the RSUs caching content 3 plus the optimal value at stage 2. The optimal value is calculated according to (32 ), which equals the gain of the RSUs caching content 3 plus the optimal value at stage 2. Ul in (29) ... end for 15: end for 16: return the optimal cache placement matrix C RC * .
caching scheme for the first l contents at the l-th stage. s l = [s m,l ] L×1 (0 ≤ s m,l ≤ S R ) denotes the state at the lth stage, which represents the remaining caching capacities of all RSUs for storing the first l contents (In P C1 , we only need to consider caching the contents in RSUs).
For state s l at the l-th stage, the cache placement problem for the first l contents is expressed as: (29) For the last stage decision problem with l = L and s n,L = S R , (30) It is easy to observe that problem P c1 L is equivalent to the original problem P C1 . Next we will develop a recursive relationship between adjacent stage problems, the optimal decision of problem P C1 l can be achieved stage-by-stage. Finally, the original problem P C1 is optimally solved.
2) Recursive Relationship : At the first stage, the optimal decision of RSU m is to cache the first content if s m,1 ≥ 1 . Mathematically, s 1 = min{1, s 1 }. Thus, we have: Then, the optimal decision at the l-th stage (2 ≤ l ≤ L) is made based on the decision at l − 1-th stage by using the following recursive relationship, and an illustration is shown in Fig. 3.
where s l−1 = s l −c RC l is the state transition equation between adjacent stages.
Specifically, for state s l at the l-th stage, the optimal decision of c RC l can be achieved by exhaustively searching and finding the optimal one with the minimum value of U l (c RC l ) + J l (s l−1 ), where J l (s l−1 ) has been obtained at the l − 1-th stage. Therefore, with J 1 (s 1 ) and the recursive relationship (32), problem P C1 can be optimally solved.
3) Complexity Analysis : According to Algorithm 1, the computational complexity of stage 1 (i.e., line2-line4) is O((S R + 1) N RC ). For stage l (1 < l ≤ L), each RSU has S R + 1 states, and each state needs to decide whether to cache the content l except for the 0 state, the computational complexity of an RSU is O(2S R +1). So the computational complexity of N RC RSUs (i.e., line6-line14) is O((2S R +1) N RC ). Hence, the whole complexity is O(

B. ADPS Joint Vehicle and RSU Caching Scheme
Problem P C1 , P C2 , P C3 can be solved using DP algorithm separately. Then for problem P C , an ADPS caching scheme is proposed in this paper. The detail of the algorithm to get ADPS scheme is shown in Algorithm 2.

Algorithm 2 ADPS Joint Vehicle and RSU Caching Scheme
8: i = i + 1 9: end while 10: I max = i; // Total number of iterations; 11: j = arg min n=1,2,···i−1 The procedure of ADPS algorithm is to alternately solve problem P C by fixing two caching matrix variables among all the three variables. The iteration process continues until the objective function converges. It should be noted that different initial points might lead to different convergence points. We set C V C1 0 , C V C2 0 and C RC 0 as zero matrices and let them be the initial point, which means that no content is cached. The algorithm ends when the performance of U becomes stable after I max iterations. It should be noted that, given all the parameters, the algorithm is quite effective that U becomes stable within I max = 2 iterations.
Complexity Analysis : According to Algorithm 2, since the computational complexity of the DP algorithm has been analyzed, the computation complexity using DP algorithm in , which increases exponentially with the number of vehicles or RSUs.

IV. LOW-COMPLEXITY COOPERATION-BASED GREEDY CACHING SCHEME
The computational complexity of ADPS algorithm for P c increases exponentially with the number of vehicles or RSUs. For large scale problem, in this section, we design a more practical cooperation-based greedy algorithm (CBG) as a complement to ADPS.

A. Cooperation-Based Greedy Caching Scheme
As shown in Fig. 2, a specific vehicle request can be fulfilled by either a specific VC or RC member. If we let the VC members and RC members cooperate with each other, then a specific content only need to be cached in one of VC or RC members, and this content can be fetched by a requesting vehicle through the relay of some VC and RC members. In this situation, the limited storage space could be fruitfully exploited to improve the network hit ratio.
In Table II, we show the results of the caching scheme obtained by using the ADPS algorithm in a small-scale scenario. We set L = 25, S V = 1, S R = 3, S q = 150, γ = 0.6, N V C1 = 5 and N V C2 = 6, other parameter settings are the same as in Section V. From this result, we can observe that, with the increase of the number of RSUs N RC , VC and RC start caching from the most popular content, and each member caches different content. This caching scheme makes full use of the cache space, so that the requesting vehicle can obtain more content on the edge server. Inspired by this observation and the above analysis, and in order to reduce complexity, we propose the CBG caching scheme to use the idea of mutual cooperation among VC and RC members. Here greedy means the decision of which member the content is cached depends on whether can obtain the lowest objective function value. The algorithm to get CBG scheme is described in Algorithm 3. At first, we start caching the most popular content. For content l, to decide which member of VC or RC to cache it, we consider the cooperative caching behavior of all nodes, calculate an objective value in line 3-5, and select the node whose situation has the optimal value until the vehicle or RSU's cache is exhausted. For vehicles in the VC, we do not need to consider the vehicle number sequence. The vehicle requesting the content can cooperate with the VC member anywhere, which has no effect on the objective value. For RSU, which RSU caches the same content has an impact on the value of the object. We use Fig. 4 to illustrate the concept of line 3-5 in Algorithm 3.

B. Complexity Analysis
In Algorithm 3, the computation complexity is is the number of loops. It is linear with respect to the number of nodes.

V. NUMERICAL SIMULATION RESULTS
In this section, we evaluate the performance of the proposed caching scheme through numerical simulation. Note that both TABLE II ADPS CACHING SCHEME WITH L = 25, S V = 1, S R = 3, γ = 0.85, Sq = 150, N V C 1 = 5 AND N V C 2 = 6.

Caching matrix
The rows of all matrices in this table represent content IDs, and 0 represents zero matrix. As N RC increases, the content cached in VC remains almost unchanged. Extra contents are cached by the newly added RSU.
the proposed two caching schemes in this paper are implemented in a centralized way by BS.  [1,4,6,10]. In addition, without loss of generality, we set equal weights of delay and cost in the objective function, i.e. α = β = 1.
In addition to ADPS and CBG caching schemes, five other caching schemes are also considered for comparison.
• P opularity-Based Caching Scheme (P oBCS): Each caching node caches the most popular contents until their storage spaces run out. No coordination is considered. • P robability-Based Caching Scheme (P rBCS): Each caching node caches a particular content with a certain Algorithm 3 Cooperation-Based Greedy Caching Scheme The maximum caching capacity of all vehicles and RSUs is l max = S V * max{N V C1 , N V C2 } + S R * N RC ; 2: for l = 1 to l max do 3: Select vehicles i, j from V C 1 , V C 2 and RSU k with extra storage space; 4: In order to cooperate with each other, if the vehicle in the V C caches content l, there is no need to cache content l in the RSU k, and if none of the vehicles in the V C cache content l, then the content l needs to be cached in the RSU k;

5:
Considering all the cooperating cache situations, solve (24) and get the optimal C V C1 * l−1 , C V C2 * l−1 , C RC * l−1 ; 6: end for probability, and this probability is related to the popularity of the content. The higher probability, the more likely the content is to be stored by vehicles and RSUs. • Random-Based Caching Scheme (RBCS): Each caching node determines its cached contents randomly, without considering popularity or other parameters. • N o Caching : Each caching node does not consider caching any content. • Low Latency Caching P lacement (LLCP ): This caching scheme is proposed in [19] to minimize the average latency for cloud-based VANETs while satisfying the QoS requirements of vehicles, and the problem is solved effectively via simulated anneal (SA) algorithm. The SA algorithm is also applied to our scheme for comparison. The performance of SA algorithm is related to the initial value.

A. The Effectiveness of Proposed CBG Caching Scheme
The effectiveness of proposed CBG caching scheme is demonstrated in this subsection. Both the performance in terms of delay, cost, hit ratio and the computational time of all caching schemes are compared. Because the computational  complexity of ADPS algorithm is high, we use a small network scenario. We set L = 25, S V = 1, S R = 3, γ = 0.85, N V C1 = 5 and N V C2 = 6. Other parameters are the same as before. Fig. 5 shows how the number of RSUs affects U in (24) for different caching schemes. U keeps decreasing when the number of RSUs increases, except for RBCS and PoBCS. The reason is that, when the number of RSUs increases, the total caching capacity in the network becomes larger and vehicles have more chances to acquire interested contents. For PoBCS, although the total storage space has increased, each RSU caches the same popular content, resulting in non-cooperation between RSUs. When a vehicle requests other content, it can only be obtained from the remote server and cannot be obtained from the RSUs. For RBCS, each RSU determines its cached contents randomly, without considering popularity or other parameters. Due to randomness, the performance of RBCS is quite unstable. Also, both of ADPS and CBG caching schemes outperform other caching schemes, owing to exploiting of mutual cooperation between members in VC and RC. Most importantly, the proposed CBG caching scheme can achieve almost the same performance with ADPS caching scheme.    6 shows the runtime of all caching schemes on a desktop computer with a 3.20 GHz Intel Core i7 processor and 16 GB installed memory. It is straightforward that the runtime of ADPS caching scheme increases exponentially with the increase of N RC . The CBG caching scheme and LLCP increase almost linearly with respect to N RC , which matches the computation complexity analysis in Section IV. Moreover, with the almost same delay and cost performance, the runtime of CBG caching scheme is much lower than the ADPS caching scheme. For example, when N RC = 6, the runtime of the CBG caching scheme is 0.25s, while that of ADPS caching scheme' is 5945s being prohibitively long. This phenomenon shows the effectiveness of the proposed CBG caching scheme. Therefore, in what follows, only the proposed well-salable CBG caching scheme is adopted for the performance comparison in large scale scenarios. Fig. 7 shows how the caching capacity of vehicle and RSU affects U of different caching schemes. U decreases when the caching capacity S R and S V increase. The reason is that, with enlarged capacity, more and more contents will be stored by VC and RC members and then successfully delivered with faster transmission rate and lower price, resulting in lower delay and cost. We use U τ and U Cost to express the first part α τav τmax and the second part β Costav Costmax in (24), their changes with caching capacity S R are shown in Fig. 8. It can be clearly seen that the average delay and average cost continue to decrease as the caching capacity of RSUs increases, resulting in an overall decrease in U . In addition, the proposed CBG caching scheme outperforms other caching schemes, the average content delivery delay and cost can be reduced by 30% and 25% respectively, and the overall reduction is 29% with S R = 10, as compared to LLCP scheme. This is due to the fact that the CBG caching scheme makes full use of partnership, storage space and content popularity information.

B. The Impact of Network Parameters on Caching Schemes
The impact of caching capacity on average hit ratio is investigated in Fig. 9. As expected, the average hit ratio becomes higher with larger caching capacity. Moreover, the proposed CBG caching scheme achieves higher hit ratio than other caching schemes. The reason is that, VC members and RC members cooperate with each other to avoid caching the same contents between them, and more contents in the limited cache space can be cached. An interesting phenomenon that needs to be noted about PoBCS is in Fig. 7 and Fig. 9. It is that when the vehicle caching capacity S V is less than the RSU caching capacity S R , U and the average hit ratio remain unchanged. This is because that PoBCS caching the same popular content which leads to non-cooperation between VC members and RC members. Only after the caching capacity of RSU exceeds the caching capacity of vehicle, after caching other popular content, these contents can be obtained from RSU instead of the remote server. Therefore, our proposed CBG caching scheme has significant performance advantage with limited caching capacity, which is the common case in practice. Fig. 10 shows how the parameter of Zipf distribution, γ affects U of different caching schemes. In Fig. 10, U is decreasing with the increase of γ except for RBCS and No Caching scheme. The reason is that the percentage of requests for popular content goes up when γ increases. As a result, the cache of vehicle and RSU may provide content and satisfy more requests when the percentage of requests for popular content increases. No Caching scheme does not consider caching technique, and all the contents are fetched from the CPs with maximum delay and cost. In RBCS, every vehicle and RSU determines its cached contents randomly, without considering popularity or other parameters, thus its performance is independent of γ. Also, due to randomness, the performance of RBCS is quite unstable, but always outperforms No Caching scheme, which partially indicates the gain achieved by applying caching. In addition, the proposed CBG caching scheme outperforms other caching schemes when γ is changed from 0 to 1.6. However, PoBCS and PrBCS perform better than CBG caching scheme, when γ >= 1.8. We can find the reason in Fig. 11. The probability of request for the top 5 most popular content can be seen in Fig. 11. We can see that the most popular content request's probability is above 0.5 and that of others are all below 0.2, when γ = 1.8 and γ = 2. The reason is that the most popular content has the highest request probability compared to other contents when γ is changed from 1.8 to 2. Therefore, it is worth caching the most popular content, just like the PoBCS and PrBCS, for high performance gains.
Existing studies [16] [34] show that γ may change from 0.65 to 0.85 according to different situations of networks. Fig. 12 shows how γ (0.65 ≤ γ ≤ 0.85) affects the average content delivery delay and cost of different caching schemes in a more realistic situation. With γ increasing, a large proportion of requests arises for the popular content. With the proposed CBG caching scheme, the content can be kept in cache of vehicle and RSU with the consideration of change in content popularity. As the requesting vehicle does not need to fetch the most popular content from other places with high delay and cost, the average content delivery delay and cost can be reduced, resulting in an overall decrease in U . From Fig. 12, our proposed CBG caching scheme also outperforms PrBCS, PoBCS and LLCP in terms of U . It can be observed that, when the proposed CBG caching scheme is adopted, the average delay and cost can be reduced by 24% and 19% respectively, and the overall reduction is 21% with γ = 0.85, as compared to LLCP scheme. Fig. 13 shows the impact of γ and the number of vehicles in VC on average hit ratio for different caching schemes. The result demonstrates that CBG caching scheme can obtain the highest average hit ratio. Besides, the average hit ratio keeps increasing when γ increases in the aforementioned schemes. When the size of platoon (i.e., N V C1 ) increases, the performance of PoBCS when N V C1 = 5 and N V C1 = 10 totally coincide, and this indicates PoBCS is non-cooperative   in nature. For the PrBCS and LLCP when N V C1 = 5 and N V C1 = 10, the increase in average hit ratio is not obvious, The proposed CBG caching scheme can make full use of the partnership, storage capacity, and content popularity information to increase the average hit ratio when γ is changed from 0 to 1. The average hit ratio is mainly affected by request probability rather than the size of platoon when γ is changed from 1 to 2, which is related to the definition of hit ratio in (22). It can be observed that, when the proposed CBG caching scheme is adopted, the average hit ratio can reach over 85% with γ = 0.65 ∼ 0.75.

VI. TEST BED EVALUATION
In this section, we design and implement an infotainment content transmission (ICT) system based on the proposed twolayer architecture.

A. System Prototype Implementation
As shown in Fig. 14, we built a small scale VANETs model using commercial OBU and RSU devices. We use Aliyun cloud server as the content provider, which is configured  with 2.5 GHz CPU and 4G memory. Node vehicle 1 (V1) consists of a Cohda Wireless MK5 OBU [37] and laptop called Laptop1. Laptop1 has 1.6 GHz CPU and 4G memory. Similarly, V2 has 1.6 GHz CPU and 8G memory. V3 has 2.5 GHz CPU and 8G memory. Meanwhile, Laptop4 is connected with two RSUs, with the configuration of 2.2 GHz CPU and 16G memory. V1, V2 and V3 can communicate with RSUs and cloud server through DSRC interface and 4G hotspot respectively. In addition, we made a specific database containing a video data-set [38]. This data-set consists 50 video clips from 50 Hollywood movies with size of 8968-9790 KB. We program each node so that they can perform the content requesting and caching function. As shown in Fig. 15, the node can be programmed as server or client in terms of requirements. Each laptop in Fig. 14 is able to communicate with RSUs and OBUs via User Datagram Protocol (UDP) [39].
B. Performance of Transmission Rate in Test Bed.
In our experiment setting, we test the real content transmission rate R V V , R RV , R RCV , R P V based on the video data-set. Fig. 16 shows different transmission rates in real vehicular communication environments. The average transmission rate of R V V is very close to R RV , this is caused by the same internal structure of Cohda Wireless MK5 RSU and OBU. In addition, other average transmission rates conform to the rate assumption of the two-layer structure. The actual test data obtained by implementing the system prototype can verify the effectiveness of different caching schemes.

C. Performance of Caching Schemes in Real-World Data Set
We verify the effectiveness of our proposed caching scheme through real-world data set (i.e., transmission rate, video size) recorded by the test bed. The residence time of each vehicle with each real RSU is set to 10s to imitate the real movement of the vehicle. The performance of different caching schemes using real data set is shown in Fig. 17 and Fig. 18. It is straightforward that the average content delivery delay and cost are decreasing. Average hit ratio increases as γ increases. When the proposed CBG caching scheme is adopted, the average delay and cost can be reduced by 10% and 24% respectively, and the average hit ratio increases 30% with γ = 0.85, as compared with PoBCS. As expected, cooperation between different nodes plays a key role in the caching scheme design, the proposed CBG caching scheme extremely outperforms other caching schemes in the actual environment, which effectively shows the advantages of our proposed CBG caching scheme.

VII. CONCLUSION
In this paper, we expand the single-layer caching scheme to two-layers. The average content delivery delay, cost and hit ratio have been analytically obtained for a two-layer cloud based VANET architecture in content caching scheme. Then, a joint vehicle and RSU caching optimization problem has been formulated to reduce both the transmission delay and cost considering limited caching capacity. By solving this optimization problem, we propose ADPS caching scheme and low complexity CBG caching scheme. Numerical simulation together with real test bed verification have shown the advantage of our proposed caching scheme. Specifically, the proposed CBG caching scheme consumes remarkably shorter runtime than ADPS caching scheme, test bed evaluation results show that CBG caching scheme reduces delay and cost by 10% and 24% respectively, and increases hit ratio by 30% as compared to PoBCS.
The analytical framework established in this paper can be extended to study general cases of other QoS metrics. For the future work, the prototype system will be further evolved from the current hardware testing to small scale realistic internet of vehicle environments.