"What is the utility, feasibility, propriety and accuracy of the health promotion evaluation model compared with the realist evaluation model within healthcare?"

2017-03-06T05:42:51Z (GMT) by Boyd, Leanne Michelle
There are three broad themes integrated throughout this thesis. The first relates to the development and implementation of a multimedia healthcare program called `Ambulances for Asthma’. The second relates to program evaluation within healthcare and recounts the methodological challenges related to the evaluation of this program. The third portrays the self reflective evaluation journey of the author informed by auto-ethnography. Six linked projects are presented, culminating in a metaevaluation of the health promotion [process, impact and outcome (PIO)] and realist evaluation models from healthcare practitioner and policy maker perspectives. This thesis aims to determine which approach has greater utility, feasibility, propriety and accuracy within healthcare. The first project undertook a needs analysis to identify barriers to optimal emergency management of asthma in the pre-hospital setting in rural Australia. The major barrier identified was a decision to use self transportation to hospital during an acute asthma event. The second project described the development, implementation and base evaluation of `Ambulances for Asthma’. A willingness to change behaviour by calling `000’ for asthma was reported by 64% of participants. The third project details the PIO evaluation of the program. The focus of this approach is to determine if a program `works’. The primary outcome was a statistically significant increase in ambulance utilisation for asthma within the project regions indicating the program achieved its stated goals. The fourth project applied a realist evaluation model to the same evaluation dataset. This framework aims to generate transferable program theory of `what works, for whom, in what circumstances and in what context’ rather than the generalisable `does it work question’ framed within the PIO approach. The findings revealed that the program was effective for older people who received 1:1 education. The program was not shown to be effective for young males. The fifth project provides an auto-ethnographical comparison of the two evaluation approaches. The key differences identified in this thesis are the elucidation and testing of the underpinning program theory, the way in which the research questions are posed and the method of hypothesis testing. The sixth project performed a metaevaluation of the PIO and realist evaluations using the criteria of utility, feasibility, propriety and accuracy as developed by the Joint Committee on Standards for Educational Evaluation. Both PIO and realist approaches rated highly in regards to propriety and accuracy. The areas of contention lie within the utility and feasibility of each approach. Healthcare practitioners found the realist approach to have a higher level of utility than the PIO approach but had concerns regarding its feasibility. Policy makers found the PIO approach had the highest level of both utility and feasibility. Further work needs to be done to enhance the feasibility of realist evaluation and the utility of PIO evaluation. Both evaluation approaches have much to offer healthcare however the need to continually evolve and develop is highlighted in this thesis.