Using Interest and Transition Models to Predict Visitor Locations in Museums

Museums o ﬀ er vast amounts of information, but a visitor’s receptivity and time are typically limited, providing the visitor with the challenge of selecting the (subjectively) interesting exhibits to view within the available time. Mobile, electronic handheld guides o ﬀ er the opportunity to improve a visitor’s experience by recommending exhibits of interest, and adapting the delivered content. The ﬁrst step in this per-sonalisation process is the prediction of a visitor’s activities and interests. In this paper we study non-intrusive, adaptive user modelling techniques that take into account the physical constraints of the exhibition layout. We present two collaborative models for predicting a visitor’s next locations in a museum, and an ensemble model that combines the predictions of these models. The three models were trained and tested on a small dataset of museum visits. Our results are encouraging, with the ensemble model yielding the best performance overall.


Introduction
Museums offer vast amounts of information, but a visitor's receptivity and time are typically limited.The possibility of information overload is evident, as the visitor is confronted with the challenge of selecting the personally interesting exhibits to view within the available time.However, appropriate advance information about the exhibits in different areas of the museum is often not readily available, and visitors may change their mind about what they want to see after viewing some exhibits they thought they would be interested in.A personal human guide, aware of a visitor's interests and time limitations, could solve these problems, but the provision of personal guides is outside the resource limitations of most museums.
Advances in mobile, context-aware computing and user modelling point towards an alternative solution: personalised electronic handheld guides.Electronic guides have the potential to (1) make recommendations about items of interest, and (2) personalise the content delivered for these items, based on tracked visitor behaviour.The Kubadji project (http://www.kubadji.org) is developing user modelling and language technologies to support the creation of such guides.A first step is to infer a visitor's interests and activities non-intrusively from his/her behaviour within the museum, and to store the aquired information in models of the user.The physicality of the domain poses practical challenges for such user modelling [13].For example, the spatial layout of the environment influences the curator's decisions about the positioning of the exhibits, and both influence a visitor's decisions about which exhibits to view and in which order.Hence, the spatial arrangement of items is an input that should improve the accuracy of predictions of a visitor's behaviour.To our knowledge, this factor has not been considered to date.
In this paper, we describe a first step in the recommendation and personalisation process, i. e., the prediction of a visitor's interests and locations in a museum on the basis of observed behaviour.Specifically, we consider two collaborative predictive models, Interest and Transition, and an ensemble model that combines their predictions.The Interest Model predicts exhibits to be viewed by a visitor on the basis of his/her observed viewing times in the context of the viewing times of other museum visitors.The Transition Model predicts the next exhibit (location) based on the transitions between exhibits previously recorded for other visitors.These models are employed to predict the next K exhibits (K = 1 and K = 3) using two prediction approaches: set, which predicts a set of exhibits, and sequence, which predicts an ordered sequence.
We trained and evaluated our models on a small dataset of museum visits collected at the Marine Life Exhibition in Melbourne Museum.Our results show that the Transition Model outperforms the Interest Model, indicating that the layout of a physical space with homogeneous exhibits (e. g., marine theme) is a significant factor influencing visitor behaviour.However, the ensemble model yielded the best performance overall, demonstrating the importance of considering also a visitor's interests.
The remainder of this paper is organised as follows.In Section 2, we outline related work.Our approaches for predicting visitor locations using collaborative user models are described in Sections 3 and 4. In Section 5, we summarise preliminary findings of our evaluation, followed by a discussion of future work in Section 6.

Related Research
Our work lies at the intersection of statistical user modelling and personalised guide systems for physical museum spaces.
Personalised guide systems in physical domains have often employed adaptable user models, which require visitors to explicitly state their interests in some form.For example, the GUIDE project [8] developed a handheld tourist guide for visitors to the city of Lancaster, UK.It employed a user model obtained from explicit user input to generate a dynamic and useradapted city tour, where the order of the visited items could be varied.In the museum domain, the CHIP project [2] investigates how Semantic Web techniques can be used to provide personalised access to digital museum collections both online and in the physical museum, based on explicitly initialised user models.
Less attention has been paid to predicting preferences from non-intrusive observations, and to utilising adaptive user models that do not require explicit user input.In the museum domain, adaptive user models have usually been updated from the user's interactions with the system, with a focus on adapting content presentation, rather than predicting or recommending exhibits to be viewed.For example, HyperAudio [17] dynamically adapted the presented content and hyperlinks to stereotypical assumptions about the user, and to what the user has already accessed and seems interested in.The augmented audio reality system for museums ec(h)o [10] treated user interests in a dynamic manner, and adapted its user models on the basis of the users' interactions with the system.The collected user modelling data were used to deliver personalised information associated with exhibits via audio display.The PEACH project [19] developed a multimedia handheld guide, which adapted its user models both from explicit visitor feedback and implicit observations of visitor interactions with the device, and used the information stored in these user models to generate personalised multimedia presentations.
These systems, like most systems in the museum domain, primarily rely on knowledge-based user models, which require an explicit, a-priori engineered representation of the domain knowledge.In contrast, this work investigates non-intrusive statistical user modelling techniques that do not require an explicit representation of the domain knowledge, and takes into account spatial constraints -a factor that has not been considered to date.As far as we are aware, the only instance of the application of a statistical technique [1,20] for predicting a visitor's behaviour in a museum is described in [9].

Using Collaborative Models based on Spatio-Temporal Information to Predict Location Probabilities
We consider two collaborative models for estimating the probability of a visitor viewing a particular exhibit given his/her previous visit trajectory: interestbased (Section 3.1) and transitional (Section 3.2).The interest-based approach predicts a visitor's next location on the basis of his/her interest in unseen exhibits, which in turn is estimated from the time the visitor spent at the exhibits s/he has seen.The transitional approach predicts a visitor's next location from the trajectories of other visitors.In Section 3.3 we propose an ensemble approach to combine the predictions generated by these models [14,18].The utilisation of the estimated location probabilities to predict a set or sequence of next items is described in Section 4.
Recent developments in the area of positioning technology have made possible the non-intrusive indoor tracking of visitors equipped with a positioning device.The availability of such technology to infer a visitor's high-level activities from sensing data, e. g., [15], is crucial to this work, i. e., to perform non-intrusive, adaptive user modelling.In this research, we assume access to a visitor's pathway in the form of a timeannotated sequence of visited items.That is, for each visitor u, we have an ordered sequence of viewing durations t ui 1 , t ui 2 , . . .for items i 1 , i 2 , . . .respectively.This information was obtained manually for this study, by tracking 44 visitors to the Marine Life Exhibition in Melbourne Museum, but is of the same type as information inferable from sensing data in a real-world setting [15]. 1 In total, our dataset comprises 317 data points (Section 5).The small size of this dataset is due to the difficulties associated with collecting and processing data in physical settings (Section 6).

Interest Model
In an information-seeking context, users are expected to spend more time on relevant than on irrelevant information, as viewing time correlates positively with preference and interest [16].That is, the time spent at a given exhibit can be used as a measure of interest.However, viewing time is also positively correlated with item complexity.Additionally, viewing times vary over different visitors depending on their available time. 2 Hence, in order to infer the interests of visitors in different items, the observed viewing times cannot be used directly, but must be transformed into a measure that takes these factors into account.To this effect we have devised the relative interest measure, which reflects the interest of a visitor in an exhibit in the context of the time s/he has spent on previously seen exhibits, and the time spent by other visitors on this exhibit.This measure implicitly takes into account item complexity, as complex items are likely to be viewed for a longer time than simpler items.

Definition 1 (Relative Interest (RI))
The relative interest of visitor u in a seen exhibit i is defined as follows: where t ui is the time visitor u spent at exhibit i, t u• is the average viewing time of visitor u, n •i is the number of visitors that viewed exhibit i, U is the set of visitors, and n vi = 1 if visitor v viewed exhibit i and 0 otherwise.
Algorithm 1 Estimating the relative interests of the active visitor in unseen exhibits 1: Estimate from the observed viewing times the relative interests of all visitors -including the active visitor a -in the exhibits viewed during their visit (Equation 1).2: for all i such that i is an unvisited exhibit do 3: Find a set of item mentors, who have viewed item i, and have the highest similarity with the active visitor.To calculate a visitor-to-mentor similarity, use Pearson's correlation coefficient on the vectors of their relative interests.

4:
Estimate the active visitor's relative interest in item i as the weighted mean of the relative interests of his/her item mentors in i, where the weights are the visitor-to-mentor similarities.

5: end for
The first term in Equation 1 reflects visitor u's viewing time of item i relative to his/her average viewing time, and the second term represents the average relative viewing time spent at item i (over all the visitors that viewed this item).Hence, RI ui measures whether visitor u is (relative to his/her average viewing time) more or less interested in item i than the average interest in item i. 3The collaborative Interest Model (IM) is built by calculating RI ui , the relative interest of visitor u in exhibit i, for all visitors u = 1, . . ., |U| and all items i = 1, . . ., |I|, where |U| is the number of visitors and |I| is the number of exhibits.This yields a relative interest matrix RI of size |U| × |I|, which contains defined values for all combinations of visitors u and items i that occurred, i. e., combinations referring to an item i viewed by a visitor u.These values, which may be viewed as implicit ratings given by visitors to exhibits, do not take into account the order in which the exhibits were visited.In Section 6, we consider the incorporation of spatial information into our Interest Model.
Following a collaborative approach as described in [11], we use Algorithm 1 to predict the missing relative interest values of the active visitor a from the values in RI. 4 These values are mapped into the [0,1] range to estimate the probability of visiting an unseen exhibit.Formally, given a visit where a visitor a has viewed k items so far, the probability of the (k + 1)-th item being item i is represented by the expression Pr(X , where v k a is the user's visit history so far.Thus, approximating this expression by a probability estimated using our Interest Model yields the following formula: (the Interest Model depends on viewing times, rather than transitions between locations). 5

Transition Model
Our Interest Model considers only a visitor's relative interests, and does not take into account the order in which the exhibits were visited.Here we describe an alternative model, denoted Transition Model (TM), which considers the visit order.
The Transition Model is represented by a stationary 1-stage Markov model, where the transition matrix TM approximates the probabilities of moving between exhibits.Specifically, the element TM(i, j) approximates the probability of a visitor going from exhibit i to exhibit j, where i, j = 1, . . ., |I|, and |I| is the number of exhibits.This probability is estimated on the basis of the frequency of transitions between i and j.In order to overcome the data sparseness problem (which is exacerbated by our small dataset) and to smooth out outliers, we added a flattening constant ε (= 1/|I|) to each frequency count.
When we employ the Transition Model to approximate the probability that the (k + 1)-th exhibit viewed by the active visitor a is item i, we obtain the formula , where I k a are the exhibits visited by the active visitor.Since our Transition Model is a 1-stage Markov model, the probability of the next exhibit being item i is further approximated by where i k is the current item.As mentioned before, our observations indicate that visitors rarely return to previously viewed exhibits.Hence, prior to calculating these probabilities, we set to 0 the entries of TM that correspond to the visited items (items in I k a ), and appropriately renormalise the rows.
The Transition Model implicitly captures the physical layout of the museum space, i. e., the physical proximity of items, on the basis of the assumption that transitions to spatially close items occur more frequently than movements to items that are further away.However, in the future, we will also experiment with spatial models that represent more directly the distance between exhibits (Section 6).

Combining Interest Model and Transition Model
As indicated above, the probabilities computed by the Interest Model are based on temporal information, while the predictions made by the Transition Model implicitly capture spatial information.Additionally, while the Interest Model adapts to the behaviour of a visitor, the Transition Model is not personalised.In this section, we propose a Hybrid Model (HM) that combines the predictions made by these models [14,18], thereby jointly taking into account transitional and temporal information.
Formally, we use the probability Pr HM (X k+1 = i | v k a ) generated by our ensemble model to approximate Pr(X k+1 = i | v k a ): This probability in turn is calculated by means of a weighted average of the predictions generated by our Interest Model and Transition Model, i. e., , where 0 ≤ ω ≤ 1.We experimented with different values for ω, with the assignment ω = β/ (α + β) yielding the best performance, 6 where and I\I k a is the set of exhibits not yet visited.This choice of ω assigns more weight to the model with the lower minimum prediction, which may be viewed as the more discriminating model.

Building Models to Predict a Visitor's Next K Locations
In this section, we describe two approaches for using the probabilities estimated in Section 3 to predict the next K exhibits to be viewed by a visitor: TopK, which predicts the next K items as a set and ranks them in descending order of estimated probability, and SequenceK/N, which predicts the next K items as the initial portion of a sequence of N items.

TopK Prediction
The TopK approach assumes that the current visit history suffices to predict a visitor's future behaviour, and that it is unnecessary to consider the impact of future transitions on the visitor's subsequent behaviour.Hence, in order to predict the next K items that will be visited (having visited k items), we find the set of K unvisited items i k+1 , . . ., i k+K that maximises the product of their visit probabilities.This is done by solving This approach is equivalent to computing the probabilities Pr(X k+1 = i | v k a ) for all unvisited exhibits i ∈ I\I k a (pretending that each of these exhibits is the next one -hence the subscript k + 1), then sorting the exhibits in descending order of estimated visit probabilities, and selecting the top K items.

SequenceK/N Prediction
In contrast to the TopK approach, the SequenceK/N approach assumes that future transitions influence a visitor's subsequent behaviour.Hence, in order to predict the next K items that will be visited (having visited k items), we first find the maximum-probability sequence of N unvisited items i k+1 , . . ., i k+N by solving and then select the first K items i k+1 , . . ., i k+K within this sequence.
Assuming that X k+m depends only on the past, this probability is decomposed as follows: Due to this decomposition, the joint probability in Equation 2 can be maximised by recursively spanning a search tree of depth N −1, and performing an exhaus-tive search for a maximising path from its root to one of the leaves.
The probability Pr( ) in Equation 2 depends on the active visitor's visit history up to exhibit i k+m−1 , but in practice this history is available only up to item i k .Future exhibits are incorporated into a "potential history" for the Transition Model by iteratively adding predicted unseen exhibits to construct different potential sequences.In order to incorporate such a potential history into the Interest Model (and hence the Hybrid Model), we require viewing time estimates.The calculation of predicted viewing times is similar to that performed for the estimation of relative interests, and is described in [6].

Preliminary Findings
The data used to obtain the results reported in this paper was collected at the Marine Life Exhibition of Melbourne Museum in 2006.This exhibition consists of 56 exhibits in four sections, comprising a rather homogeneous selection of marine-related topics.With the help of curators, we transformed the original set of 56 exhibits into a set of 22 grouped exhibits by unifying logically related exhibits.To collect the dataset, 44 visitors were manually tracked at the exhibition, which allowed us to construct time-annotated visitor pathways.In summary, we obtained 317 observations, with an average visitor path length of 7.20 grouped exhibits, and the shortest and longest visits comprising 3 and 16 exhibits respectively.
In our experiments, we evaluated the performance of our two approaches for predicting a visitor's next K exhibits, TopK and SequenceK/N, for two values of K (K = 1 and K = 3), and a fixed value for N (N = 3), yielding the four variants Top1, Sequence1/3, Top3 and Sequence3/3.For every combination of prediction mode (set and sequence) and value of K (K = 1 and K = 3), we considered the three prediction models defined in Section 3: Interest Model, Transition Model and Hybrid Model.We evaluated the performance of these 12 variants by measuring their Classification Accuracy and Ranking Accuracy [12].Here we focus on the results obtained for Classification Accuracy (the results obtained for Ranking Accuracy are detailed in [5,6]).Classification Accuracy was calculated using Precision, i. e., Pre = |K ∩ M|/|K|, where K (with cardinality K) is the set of predicted exhibits, and M is the set of actually viewed exhibits.We considered two alternatives for M: (1) the set of K exhibits viewed next (|M| = K), and (2) the set of ex-hibits viewed during the remainder of the visit (which may be of different size for each visitor).These alternatives for M correspond to evaluating the accuracy of our predictions in the immediate future and in the eventual future respectively.
Our results indicate that, as expected, our predictions are less accurate when evaluated for the immediate future than when evaluated for the eventual future (on average, immediate accuracy is lower than eventual accuracy by 20%).Additionally, the spatial structure of the museum space dominates in our constrained, homogeneous setup, i. e., the Transition Model significantly outperforms the Interest Model.For the eventual evaluation setting, we obtained the following Classification Accuracy results for the Hybrid Model (analogous insights apply to the immediate setting).When predicting the next exhibit (K = 1) to be viewed by a visitor, (1) the accuracy of Top1 predictions is comparable to the accuracy of the more complex Sequence1/3 predictions (66% vs. 68%), hence there is no need to apply a complex sequence prediction mechanism; and (2) hybridisation yields only a slight improvement in predictive accuracy over the Transition Model.In contrast, when predicting a sequence of K = 3 exhibits, (1) Sequence3/3 is superior to simple Top3 (59% vs. 49%), meaning that sequence information aids prediction; and (2) the Hybrid Model significantly outperforms the individual models.Consequently, hybridising the Interest Model and the Transition Model is generally beneficial.

Discussion and Future Work
Our results raise several issues with respect to modelling users in physical spaces in general, and museums in particular.These issues pertain to exhibit diversity, amount and quality of data, user modelling strategies, and recommendations.
Exhibit diversity.Our experiments were conducted in the rather homogeneous Marine Life Exhibition.This means that the visitors who decided to enter the exhibition were already interested in marine life.Thus, as indicated by our results, a key factor influencing visitor behaviour in a homogeneous exhibition is the physical layout of the space.However, this conclusion may not be valid in a space of heterogeneous exhibits, such as the entire Melbourne Museum, which has exhibits relating to flora, fauna, Australian history and modern life.These observations motivate future experiments at different levels of granularity, e. g., inter-exhibition versus intra-exhibition, while considering the link between granularity and topic diversity.We expect these experiments to shed light on the influence of exhibition size and exhibit diversity on the applicability of our models.
Amount and quality of data.Our dataset comprises visit trajectories of 44 museum visitors.This is a rather small dataset.However, in contrast to web-based data collections, the collection of visit traces in a physical space is an expensive and time-consuming process, when done by either human trackers or electronic devices.Additionally, cost-efficient tracking technology is still relatively inaccurate, which may affect the accuracy of the derived user models and the quality of the personalisation provided to the visitor [7].This problem should be addressed prior to deploying such devices.At the same time, this problem obfuscates basic user modelling issues, and should be avoided during initial model development.In contrast, human tracking is precise, and hence ideal for initial model development, but clearly cannot be used during model deployment.Using human trackers, we have just completed the collection of additional visit traces at the level of the entire Melbourne Museum.However, the amount of data we collected remained relatively small.User modelling strategies.Our current approach for combining user models belongs to the ensemble category, where the predictions made are combined in a weighted manner [18].However, the models themselves are built separately; the Transition Model from trajectory information, and the Interest Model from temporal information.In the future, we propose to combine these information sources and conduct hybridisation at the model acquisition stage.For example, this can be done by considering the distance from a current exhibit when computing a visitor's Interest Model.That is, the farther a newly visited exhibit is from the last visited exhibit, the higher the interest in the new exhibit.We also plan to investigate an ensemble combination of collaborative user models with content-based models, and intend to take into account the cold-start problem [11,20] by applying machine learning techniques to determine the point in a visit at which personalised models can be deployed.These techniques will also be applied to find the optimal weight of the individual models in ensemble models.
Recommendations.In a physical domain, the transition from interest and location prediction to recommending interesting items (locations; or in this particular work, exhibits) is not trivial.We suggest the fol-lowing approach.First, generate the set of unvisited items, and rank them using predictions from an interest model.Second, use a transition (or hybrid) model to predict the locations that a visitor is most likely to visit next.Implement a strategy for merging these lists, e. g., for whether locations that the visitor is likely to visit anyway should be included (to help build trust in the system) or excluded (to avoid over-communication).Finally, deliver the list of recommended locations to the visitor allowing the visitor to act on the recommendations as they see fit.The modality of the presentation, e. g., visualised on a site map, or provided more explicitly in textual or audio form, should be taken into account in constructing the list.
In the future, we plan to investigate utility-based recommendation generation strategies which balance factors such as those above, e. g., Markov Decision Processes, which were recently proposed for decisiontheoretic and user-adaptable planning in the shopping guide domain [3].
where t k a is the time component of the visit history v k a