Reliability and relative validity of a food frequency questionnaire for Italian adults living in Sicily, Southern Italy.

Abstract The aim of this study was to develop and test the reliability and relative validity of a food frequency questionnaire (FFQ) specifically developed for individuals living in Sicily, southern Italy. This study was conducted on a convenient sample of 178 adult volunteers aged 18–80 years recruited in the urban population of Catania. Dietary intake estimated by 2 FFQs was compared with six 24-h recalls covering a period of 10 months. A total of 110 food items were included in the FFQ. Person’s coefficients between the first FFQ and mean of the six 24-h recalls showed high correlations for coffee, tea, pasta and dairy products, alcohol, total fats and carbohydrates (in women). The test–retest analysis showed high reproducibility of the FFQ. We showed that our FFQ provided a useful estimate of both food and nutrient intake in a healthy adult population.


Introduction
Nutritional epidemiological studies have risen during the last 50 years as an important instrument to investigate the effect of nutrition on human health. The ongoing trends in increasing burden of chronic diseases such as cardiovascular and cancer have been attributed, at least in part, to a gradual shift from more traditional dietary pattern to a ''Westernized'' diet rich in carbohydrates and saturated fats (Grosso & Galvano 2016). Several methods have been developed to determine dietary habits of individuals, but there is no perfect tool that represents the completeness with regard to simplicity, effectiveness and economicity. Food frequency questionnaires (FFQs) have been suggested to be suitable tools to collect dietary information in largescale prospective cohort, with more contained costs and less-affected respondents' culture and motivation than 24-h dietary recalls (Kipnis et al. 2002). FFQs permit to explore the frequency consumption of a variety of foods corresponding to one's usual diet. Compared with 24-h recalls, FFQs allow counting a wider range of foods, including those consumed rarely, and can be administered once whereas to describe usual dietary habits and dietary adequacy with a reasonable validity is required administration of several 24-h recalls. Among the main limitations, FFQs may include only a limited number of foods, which influence the overall dietary intakes of one person (generally, longer food lists may overestimate intakes). Another consequence is that a predefined food list generally includes foods supposed to be traditionally consumed by the population of interest, limiting the possibility to apply the same instrument across different populations.
To date, only few FFQs have been developed for the Italian population (Pisani et al. 1997;Turconi et al. 2010;Buscemi et al. 2015). The most of them have been tested with individuals living in the northern Italy and they are not suitable to be used in southern populations (Pisani et al. 1997;Turconi et al. 2010). The one conducted in South Italy included 36 items, which is unlikely to comprehensively characterize the dietary peculiarities of the Sicilian Mediterranean dietary habits (Buscemi et al. 2015). Moreover, it has been demonstrated to be a useful tool for investigating the association between diet, metabolic and cardiovascular diseases, but its application to other fields remains limited . Thus, the aim of this study was to develop and test the reliability and relative validity of a FFQ specifically developed for individuals living in Sicily, southern Italy. We validated the instrument measurement method by comparing results with a reference method (24-h recall) and tested its reproducibility.

Study population
This study was conducted on a convenient sample of adult healthy volunteers aged 18-80 years recruited in the urban population based in Catania, a metropolitan city of 500,000 inhabitants on the eastern cost of Sicily, southern Italy. Between October 2014 and August 2015, 200 potential participants were invited to complete twice the FFQ (at the beginning and the end of the study) and a bi-monthly 24-h recall not referring to special days of food consumption due to travels, holidays or illness. The inclusion criteria were nativity and current residence in Sicily and no history of diseases that altered the usual nutritional habits. Participants were asked before joining the study, if they had a serious illness or any reason that modified their diet in the past six months, otherwise they were excluded. Out of the 200 invited participants, 194 agreed to participate in this study. Among them, participants with missing 24-h dietary recalls (n ¼ 16) were excluded from the analysis, resulting in a final sample of 178 subjects (65% females), who were included for the analysis. The study period covered 10 months in order to ensure potential influence of seasonal variations. This study was approved by the Ethic Committee of the University of Catania, and informed written consent was obtained from each subject.

Food frequency questionnaire
The choice of the foods to be included in the questionnaire was made after a review of the literature and previous FFQs developed for Italian population (Pisani et al. 1997;Turconi et al. 2010). From an initial pool of 80 foods retrieved by the other FFQs, a number of typical foods and highly consumed by the Sicilian inhabitants were lacking. Among them, we included some fruits that typically grow in the Sicilian area and demonstrated important health benefits, such as red oranges (Grosso et al. 2013), prickly pears (Feugang et al. 2006), pomegranates (Sreekumar et al. 2014), artichoke (Rondanelli et al. 2013) and some characteristic foods that may have peculiar nutritional profile [i.e. ''ricotta'' whey cheese have less fats than more common cheeses (Madalozzo et al. 2015)]. We also included several varieties of fisheries and nuts, which may be highly characterized in the Sicilian area and have been demonstrated to exert peculiar effects on health (Gil & Gil 2015;Grosso & Estruch 2016), and distinguished between extra virgin olive oil and other oils (Sacchi et al. 2014). Overall, the final FFQ included a pool of 110 food items consumed during the last 6 months. For a full list of food items included, please consult the Supplemental data.
Trained dietary interviewers administered both the FFQ at the beginning and the end of the study. Participants indicated their food consumption frequencies with nine response options ranging from ''never'' to ''4-5 times per day''. For each food item of the FFQ, a picture of a medium serving size was shown using an instruction manual that included photographs of each food item. Whether the portion size commonly consumed by the participant was not corresponding to the picture, frequency of consumption was modified accordingly. The interviewer read out the food items on the FFQ while showing the response options to the participants and recorded their frequency consumption. Food items supposed to not be available the whole year were indicated as ''seasonal foods'' in order to take into account their average consumption over their seasonal period. The interview session took about 30-45 min. The interviewer for both FFQs was the same for each participant.

Reference method (24-h recall)
The choice of reference method relied on the hypothesis that food habits may vary over seasons, thus we preferred to cover a longer period of time and administered six 24-h recalls over a period of 10 months (bimonthly) rather than a 3-day recall (one of the most common reference method) (Cade et al. 2004). For each 24-h dietary recall, participants were asked by telephone by a trained interviewer on randomly drawn days about their food consumption during the previous day. The 24-h dietary recalls were equally distributed over all days of the week and the weekend. Participants were accordingly asked to report in detail the type of food consumed (i.e. brand name for prepackaged foods or constituents of mixed dishes) as well as the quantity or volume using common household or other measures (i.e. glasses, spoons or fractions of a plate). Mixed dishes were converted into their ingredients according to the subjects' report on the amount of the food item consumed, taking into account variations in meal preparation recipes. The interview session took about 30 min. The interviewer for all recalls was the same for each participant.

Dietary data
The average food consumption was calculated (in g or ml) by following the standard portion sizes used in the study and then converted in 24-h intake. An advanced search was carried out in the Italian nutrition facts tables of the Research Centre for Food and Nutrition (CREA) to retrieve mean content values for most important nutrient contained in the foods (http://nut. entecra.it/646/tabelle_di_composizione_degli_alimenti. html, accessed June 2015). Items not present in the Italian database were searched for nutrient content in the USDA National Nutrient Database for Standard Reference (U.S. Department of Agriculture 2011). The food items on the FFQ were grouped according to their nutrient contents in eight food groups, as follows: (i) Meat and fish products; (ii) Sweets, nuts and snacks; (iii) Vegetables; (iv) Fruits; (v) Drinks; (vi) Cereals and starchy foods; (vii) Milk and dairy products; and (viii) Oils and seasonings. The recalled food items were assigned to the food groups fitting with the food groups defined by the FFQ.

Statistical analysis
Continuous variables are presented as mean and standard deviation (SD). Using the residual method, energyadjusted food group intakes were calculated to exclude the possibility of variation due to differences in age and energy intake (Willett & Lenart 1998). Food groups and nutrients distribution was controlled by Kolmogorov-Smirnov test, and it roughly followed a normal distribution, partially skewed towards high values. Thus, energy and nutrient values were log e (x þ 0.001)-transformed and normality was improved for all variables after transformation. Within-(intra-) and between-(inter-) individual coefficient of variations (CV) of each food group were estimated from the 24-h recalls in order to evaluate possible variation by season (i.e. summer and winter) or between participants. The within-(CV w ¼ S w /mean of each nutrient) and between-individual (CV b ¼ S b /mean of each nutrient) CVs were calculated using a formula provided by Beaton, where S w and S b are the square roots of the estimated intra-and inter-individual variance, respectively (Beaton et al. 1979). Mean intakes derived from both methods and differences of intakes of food groups between both methods were calculated. The validity of the FFQ was evaluated by means of the unadjusted Pearson correlation coefficient between intake of each food group computed from the first questionnaire and the corresponding average value derived from the six 24-h recalls. Pearson correlation coefficients of each food group were calculated between the two FFQs to assess the temporal stability and reproducibility of the instrument. Pearson correlation coefficients between intake of mean nutrient intake derived by the first FFQ and the corresponding average value derived from the six 24-h recalls were calculated. The Bland-Altman statistical method (Bland & Altman 1986) was used to evaluate the agreement between the two dietary intake assessment methods regarding the mean intakes of macro-and micro-nutrients. All analyses were performed using SPSS software v. 17.0 (Chicago, IL).

Results
The mean age of the sample was 56.2 years (SD 16.8). Gender-stratified intra-individual and inter-individual correlation coefficients of each food group between six 24-h dietary recalls, adjusted for energy intake, are presented in Table 1. Inter-individual variation in dietary intake of all food groups was greater than intra-individual variation (ratio CV b :CV w >1) for almost all food groups. Examination of within-and between-individual CV (CV w and CV b , respectively) showed that high ratios resulted from low CV w for some food groups such as coffee, milk & dairy products, eggs, pasta & rice and bread and from high CV b for tea, alcoholic beverages, soft drinks, coffee and nuts. Some foods, such as coffee, alcoholic beverages and dairy products, had both high CV b and low CV w resulting in the highest ratios (Table 1). An analysis of each food revealed that some specific plant foods, for instance watermelon, peach, grapes and strawberries had a high intra-individual variation or have been reported only in some recalls (i.e. those performed during the summer), Table 1. Within (CV w ) and between (CV b ) variation of the main food groups between FFQ and mean of 24-h recalls by gender. mostly reflecting the seasonality of these fruits (data not shown). The mean intake of 16 food groups measured by the FFQ1 and the mean of the six 24-h dietary recalls, and the percentage differences are shown in Table 2 for both genders. The mean values of the FFQ1 were significantly higher than the 24-h for fruits, fish, milk and dairy products in both men and women, and for eggs only in women. In contrast, a wide underestimation was observed for bread, cheese, nuts and soft drinks in both men and women.
Pearson's correlation coefficients of each food group between the FFQs and mean of the six 24-h recalls are shown in Table 3. Correlations between the two FFQs varied from 0.54 for bread to 0.97 for coffee in men and from 0.57 for soft drinks to 0.96 for coffee in women. Correlations between FFQ1 and mean of six 24-h recall ranged from 0.30 for soft drinks to 0.96 for coffee in men, and from 0.27 for soft drinks to 0.96 for coffee in women. Table 4 shows the mean intakes of the 10 macronutrients expressed as grams per day, estimated by the FFQ1 and by the mean of the six 24-h dietary recalls. The mean daily energy intake reported by FFQ was 2359 kcal in men (-3.5% of the 24-h recalls estimation) and 2126 kcal in women (-4.3%). An underestimation was found for all the macronutrients except for protein (7.9 in men and 6.9 in women) and dietary fibre (7.2 in men and 6.8 in women), with similar results in the two genders. The intake of carbohydrate was underestimated by some 25-26%, followed by monounsaturated fat by some 6-10%.
The correlation coefficient between FFQs and six 24-h dietary recalls was calculated for macronutrients ( Table 5). The highest values of correlation between FFQ1 and FFQ2 were for alcohol (0.83 men and 0.88 women) and energy (0.68 men and 0.73 women). Ageand energy-adjusted Pearson correlations between FFQ1 and 24-h recalls ranged from 0.31 for fibre and cholesterol to 0.65 for alcohol in men, and from 0.32 for polyunsaturated fatty acids (PUFA) to 0.74 for alcohol in women. Correlations greater than 0.4 were observed for alcohol, total fat and monounsaturated fat in men and for all food groups except dietary fibre, total fat, monounsaturated fatty acids (MUFA) and PUFA in women.
The agreement between the two methods is illustrated by the Bland-Altman plot for daily energy, protein, fat, carbohydrate, cholesterol and dietary fibre (Figure 1), where the 95% limits of agreement are shown. Evaluation by the Bland-Altman method  showed that there was good agreement between the FFQs and 24-h recalls for each nutrient estimate.

Discussion
In this study, we explored the reliability and relative validity of a FFQ specifically developed for individuals living in Sicily, southern Italy. We showed that our FFQ provided a useful estimate of both food and nutrient intake in a healthy adult population. A number of food items showed high within-individual variability.
Most of the items included some seasonal fruits (i.e. watermelons, peach, grapes, strawberries) and vegetables (i.e. cauliflower, artichokes, asparagus). However, also other items, such legumes and tea, which are available the whole year, but their intake was probably affected by the high spring-summer temperature, which discourage consumption of ''warm'' foods. On average, men reported lower estimates in FFQ compared with 24-h recalls than women. There was a difference between pasta and bread consumption assessed through the FFQ and the 24-h recalls in both genders. Pasta consumption showed no substantial differences between methods, probably due to the fact that it is habitually consumed within a dish course, while bread consumption was underestimated in the FFQ.
This underestimation was translated into an underestimation of carbohydrates. Also cheese intake was underestimated in the FFQ probably because they may be consumed as topping of pasta or contained in several typical Sicilian dishes but not well recognized and recorded through the FFQ. Previous FFQ validation reported underestimation of unhealthy foods, such as sweetened soft drinks and high difference between FFQ and 24-h recall (Pisani et al. 1997). We did not find significant differences between the two methods probably because the mean age of our sample was relatively high and consumption of such beverages is not popular among older individuals living in Sicily. Overall, alcohol intake showed the best correlation between methods while all other nutrients showed slightly weaker, yet significant and similar each other results. The most limiting factors to be taken into account to plan large epidemiological studies are feasibility, time and costs. To describe the usual dietary habits of a population in a wide range of time, a series of 24-h recalls or a FFQ are the best methods suitable to accomplish this aim, but the former require a high motivation and collaboration by the participants and high cost to be administered frequently and to manage the data retrieved. FFQs consist of a predetermined list of foods that if not well designed and specifically   targeting the population may result in missing important food items. Regarding our FFQ, none of the single foods retrieved from the six 24-h recalls were missing in the list of food items included in the FFQ. The overestimation of one's dietary intake is a typical limitation of FFQs, which generally result to record higher dietary intakes with the increase of number of food items (Kipnis et al. 2002). The number of items in the food list often depends on the objective of the study (i.e. studies aimed to assess the intake of a single nutrient), food availability and variability of food consumption in the population under study. The selection of foods to build the questionnaire should take into consideration the hypothetic frequency of consumption of every single food. In our case, to explore with high accuracy the dietary habits of population living in southern Italy, the inclusion of specific foods (i.e. red oranges, pomegranates), variety of foods (fisheries and nuts), complex recipes (i.e. high-caloric street foods) and traditional foods (i.e. whey cheese ricotta) would be needed to better identify dietary habits of inhabitants of southern Italian islands. The attention paid to such peculiarities of the traditional Sicilian food availability and cuisine was translated with a high validity in macronutrient retrieved by the FFQ. The Mediterranean-Sicilian diet includes a variety of fisheries with very different PUFA content. Thus, a differentiation in several food items was needed and only foods with comparable nutrient content could be grouped in a single food item. For instance, despite tuna is characterized by high content in PUFA, the main sources of PUFA in Sicilian inhabitants are anchovies and mackerels. Similarly, the overall energy intake would be greatly affected by the lack of specificity in high-caloric street foods, which were included in the starchy food group. A factor that could have contributed to the high accuracy is the high number of items that would most likely cover most of the foods consumed. However, as previously argued, too many food items may affect accuracy due to the longer time needed to be filled, lead to potential biases when administered to people with low levels of education, finally resulting in lower quality data (Cade et al. 2004). The high accuracy obtained by our FFQ may depend on the fact that was administered, through interview, by trained interviewers, which were able to clarify any doubts concerning the tool and avoid missing items. However, compared with previous tools validated on similar population (Buscemi et al. 2015), our FFQ needed 30-45 min to be administered (versus 10-15 min), thus it is not likely to be used in clinical settings or in the context of regular visits, rather used for specific investigations for which the participant should be already advised.
This study had some other limitations. Exact validity of a FFQ can be obtained only by comparison with direct test of biomarkers (Cade et al. 2004). Moreover, the small sample size of the present study did not allow categorizing subjects according to their BMI or physical activity, which might affect food consumption. Nevertheless, our study had several strengths: the high number of 24-h recalls administered covering several seasons helped to avert the random errors resulting from intra-individual variations and allowed to identify possible seasonal variation of food consumption; the use of pictures of dishes most likely increased the ability of interviewed to recognize their usual consumed portion sizes; finally, the stratifications of results by gender, which is supposed to have a high impact on food consumption.
In conclusion, the FFQ presented in this study is an acceptable and suitable method to be used in largescale epidemiological studies for identifying both dietary pattern and adequate nutrient intakes.