How do IVF patients interpret claims about fertility treatments? A randomised survey experiment

Abstract Trials evaluating the efficacy of IVF and various treatment options often focus on upstream outcome measures, improvements which may not translate into clinical outcome improvements. A cross-sectional online survey was distributed globally among IVF patients. Respondents were randomised to view one of 16 statements about a hypothetical IVF treatment option called ‘FertiSure’, stated to improve one of four upstream outcomes. Statements varied in whether they contained information stating that FertiSure was not proven to improve live-birth rates and about potential risks. Many patients inferred that improvements in upstream outcomes would result in improvements in the probability of live-birth. Nearly 80% of respondents were willing to use FertiSure. Respondents told that FertiSure was not proven to improve live-birth rates and were less willing to use FertiSure. More respondents agreed that FertiSure may pose a risk to patients when they were told this was the case. However, this did not affect their willingness to use FertiSure. Interestingly, 34% of respondents believed FertiSure would not improve the probability of live-birth but were still willing to use it. These results have implications for IVF clinic websites and information about treatment options which may not routinely contain statements about the limited evidence-base and possible risks.


Introduction
IVF clinic websites dominate as a source of information for patients when choosing clinics and treatment options (Competition and Markets Authority, 2020). However, these websites may not always be an equitable source of information (Goodman et al., 2020;S. Lensen, Chen, et al., 2021). In settings where treatment is privately funded, clinic websites commonly feature advertising for add-on treatments, often involving claims that they are beneficial. However, most of these claims lack information about the possible risks of various treatment options, the safety of which is often not established due to the lack of robust research in this area (S. Lensen, Chen, et al., 2021;van de Wiel et al., 2020;Wilkinson et al., 2017). Moreover, many claims relate to benefits to upstream outcomes and rarely state whether there is a benefit to live-birth (S. Lensen, Chen, et al., 2021;van de Wiel et al., 2020).
While it may appear intuitive that improvements in upstream outcomes will translate into improvements in downstream outcomes such as live-birth, this is not necessarily the case. It is possible for treatments to affect the clinical outcome via unforeseen and unrecognized pathways that operate independently of the upstream outcome (Fleming & DeMets, 1996). A hypothetical example is an ovarian stimulation regimen which increases the numbers of oocytes and embryos but adversely affects the uterine environment, reducing the probability of implantation and live-birth. Another possibility is that the improvement in the upstream outcome is simply insufficient to affect a corresponding improvement in the clinical outcome.
Thus, examples exist where interventions that improve upstream outcomes have later been found to confer no benefit to patients, or worseharm (Barnhart, 2014;Echt et al., 1991;Svensson et al., 2013). To illustrate, cycles using an enhanced 'G5' embryo culture medium experienced a lower rate of fertilisation but higher rates of live-birth (Kleijkers et al., 2016). Similarly, in a trial of intracytoplasmic sperm injection (ICSI) versus IVF in cases of non-male factor infertility, the incidence of fertilisation failure was drastically reduced with ICSI, but pregnancy rates were also lower (Bhattacharya et al., 2001).
The plurality of outcomes presented on IVF clinic websites may be confusing to patients, representing a potential barrier to informed decision-making. It is unclear how IVF patients interpret claims relating to upstream outcomes, and whether they recognise that benefits to these outcomes may not translate into an improved chance of having a baby. The aim of this study was to determine if IVF patients interpret claims of improvements in upstream outcomes to imply improvements in subsequent outcomes. This was done using statements about a hypothetical IVF treatment option called 'FertiSure'.

Eligibility and recruitment
This study utilised an online survey of previous and future IVF patients (and their partners). Eligible participants were required to have had at least one IVF cycle in the previous five years or were planning to have an IVF cycle in the following 12 months. Exclusion criteria were the use of IVF for purposes other than having their own child, non-fluency in English, and age under 18 years. A global Facebook advertisement, targeted to users with interests related to fertility, was launched in three stages and was active between May and July of 2021. To augment recruitment, an email was sent to a panel of 437 Australian IVF patients who had previously agreed to be contacted regarding research participation opportunities (University of Melbourne, 2021).

Survey design
The survey was hosted by Qualtrics XM (Version May-August 2021) and used a branched format with 23-29 items (5-10 minutes completion time) (Qualtrics, 2005) (Supplementary File 1). The survey was piloted on three IVF patients, resulting in minor edits to improve comprehension. Upon initiation, respondents read a statement about the purpose and eligibility for the survey with a link to a detailed participant information leaflet. Following this, eligibility was established. Eligible participants proceeded to items capturing demographic data (e.g., country of residence) and infertility and IVF history (e.g. previous experiences of IVF success or failure) (Supplementary File 1).
Participants were then randomly allocated to view one of 16 possible statements about a hypothetical treatment option called 'FertiSure' (Figure 1). The statements varied based on three conditions, allocated in a factorial design: (1) Proximity of outcome to livebirth: while each statement claimed that FertiSure conferred some benefit to an IVF cycle, benefits to four different upstream outcomes were possible: number of oocytes collected, fertilisation rate, number of good quality embryos, or implantation rate. (2) Presence or absence of a disclaimer relating to livebirth: 'However, no studies have been conducted to evaluate whether FertiSure increases the chance of live-birth.' (3) Presence or absence of a statement about possible risks: e.g. 'As with many laboratory interventions, there is always the potential for adverse reactions or side-effects which may affect the embryo or a resulting pregnancy'. Simple randomisation was used, and there was an equal probability of being randomly allocated to each of the 16 statements. Statements were modelled on those used to advertise add-on treatments on IVF clinic websites, based on the findings of a recent study (S. Lensen, Chen, et al., 2021).
All respondents answered the same set of nine questions relating to the benefits of FertiSure. Questions asked whether participants agreed that FertiSure was likely to improve the number of oocytes collected, fertilisation, implantation/pregnancy, and live-birth rates, and whether they would be willing to use FertiSure. Subsequently, participants were asked for the maximum price they would be willing to pay for FertiSure (in their local currency) and then given the opportunity to share additional information as free-text.

Statistical analysis
Data analysis was undertaken using R statistical computing program (R Core Team, 2019). Proportional odds logistic regression was used to examine the effect of the manipulations on three ordinal outcomes: an agreement that FertiSure improves live-birth rate, willingness to use FertiSure, and agreement FertiSure may be associated with risks (Supplementary File 1). Sensitivity analyses were performed where those answering 'not sure' were excluded. In the analysis of the first question (agreement FertiSure improves livebirth rate), proximity and presence of the live-birth disclaimer were included as categorical covariates. This was extended by adding the interaction between proximity and the presence of the live-birth disclaimer. In the analysis of the second and third questions, all three manipulations (proximity, disclaimer, and risks statement) were included as categorical covariates. Pvalues were obtained using Likelihood Ratio tests.
Additionally, exploratory analyses were performed to determine whether participants with previous IVF experience interpreted statements differently. These were performed by fitting proportional odds models with proximity and experience (yes or no), together with their interaction. The experience variables considered were implantation failure, pregnancy loss, embryo development, poor response to ovarian stimulation, and poor fertilisation. Other analyses were descriptive. During analysis, it became clear that data pertaining to the price respondents would pay for FertiSure was too heterogenous to analyse due to variations in currencies and apparent widespread misunderstandings in responses.

Sample size
A sample of 1000 participants was selected to allow the proportion of participants who interpreted claims relating to upstream outcomes as indicating improvements in live-birth to be estimated within a 3-percentage point margin of error. Dividing into two groups of 500 (being shown or not shown the live-birth disclaimer and being shown or not shown the risks disclaimer) would allow a difference of 10 percentage points to be detected at a 5% significance level and 90% power.

Ethical approval
The present study received ethics approval from the University of Melbourne Human Research Ethics Committee on 07/04/2021 (reference number: 2021-21231-16146-3). All research was performed in adherence to the National Statement on Ethical Conduct in Human Research (2007)

Results
A total of 1683 respondents answered the survey, of which 1,607 provided consent. Of those, 1037 were excluded due to ineligibility (non-fluent in English, n ¼ 793; no recent IVF history/intent, n ¼ 231; undertaking IVF for reasons other than having a baby, n ¼ 13) or incomplete questions/reporting they didn't understand questions (n ¼ 64), leaving 506 eligible responses. Most respondents were aged between 31-40 years old (69%), female (91%), and tertiary educated (86%) (Supplementary Table 1). Respondents' locations reflected recruitment techniques with the most common continent of residence being Oceania. Most respondents had undergone an IVF cycle within the previous five years (62%); a similar proportion was planning on undertaking a cycle within the subsequent 12 months (73%). Of those who had IVF, 74% had undergone 1-3 cycles within the previous five years and 67% had experienced 1-3 embryo transfers in that time. The majority had experienced a cycle with a disappointing outcome (81%); implantation failure being the most common occurrence (59%).

Effect of proximity of surrogate outcome to livebirth
Survey participants were informed that FertiSure improves an upstream outcome: number of oocytes retrieved, fertilisation rate, number of good quality embryos, or implantation rate. Generally, respondents agreed that FertiSure would confer benefit to the outcome it was stated to improve (Supplementary Table  2). Overall, 44% believed FertiSure would improve the live-birth rate. As FertiSure was stated to improve outcomes more proximal to live-birth, a higher proportion of respondents agreed that the probability of live-birth would increase; 56% of respondents told that FertiSure improves implantation rate agreed that it improves the chance of live-birth, compared to 37% for statements about the number of oocytes collected (OR 2.05, 95% CI 1.29-3.28, p ¼ 0.02, p ¼ 0.03 in sensitivity analysis) (Figure 2, Supplementary Table 3). Interestingly, a significant proportion of respondents agreed that FertiSure would confer benefit to outcomes occurring before the stated benefit; 38% of participants told that FertiSure improves the number of good quality embryos agreed that it improves the number of oocytes collected (Supplementary Table 2). Respondents who had never had an IVF cycle prior to completing the survey were more likely to answer in this way (Supplementary Table 4).
Overall, 78% of participants were willing to use FertiSure (Supplementary Table 2). An overall effect of proximity on willingness to use FertiSure was not demonstrated (p ¼ 0.08, p ¼ 0.37 in sensitivity analysis) (Supplementary Table 3).

Effect of information about lack of evidence for live-birth
When respondents were informed that there was no evidence that FertiSure improves the live-birth rate, a smaller proportion believed that it did (39% vs 44%) (OR 0.73, 95% CI 0.53-1.01, p ¼ 0.06, p ¼ 0.03 in sensitivity analysis), (Table 1). There was no evidence to suggest that this effect differed with the proximity of the upstream outcome to live-birth (test of interaction, p ¼ 0.12). Those who were informed about the lack of evidence supporting FertiSure's effect on live-birth were also far less likely to use FertiSure (73% vs 82%, OR 0.62, 95% CI 0.44-0.86, p ¼ 0.005).

Effect of inclusion of information about risks
When respondents were informed that there was potential for FertiSure to pose a risk to themselves or resulting pregnancies, a greater proportion of respondents agreed that this was the case (60% vs 33%, OR 3.23, 95% CI 2.27-4.62, p < 0.001, sensitivity analysis: p < 0.001) ( Table 2). Comparing participants who were and were not informed about potential risks, a near equal proportion of respondents agreed that FertiSure would improve the live-birth rate and were willing to use it; 77% of those told there were possible risks associated with the use of FertiSure were willing to use it, compared to 78% of those who were not made aware (OR 0.85, 95% CI 0.61-1.20, p ¼ 0.36, sensitivity analysis: p ¼ 0.38).

Influence of IVF history and experience of implantation failure
The results suggested that participants who were told FertiSure improved implantation rates were more likely to believe FertiSure would increase live-birth rates and be willing to use it than those shown statements about other upstream outcomes. However, of respondents who had previously had an IVF cycle, 59% had experienced implantation failure (Supplementary Table 1). Therefore, it was not clear whether the results reflected an effect of proximity of implantation to live birth, or were due to the personal experience of participants. Therefore, post-hoc analyses were conducted to test the assumption that willingness to use FertiSure was influenced by experimental manipulations irrespective of personal experiences; in total, the proportion of respondents willing to use FertiSure was similar among those who had (69%) and had never had IVF before (78%) (Supplementary Table 5). Additionally, experiences of disappointment (p ¼ 0.24), pregnancy loss (p ¼ 0.07), poor embryo development (p ¼ 0.42), poor stimulation response (p ¼ 0.52), and poor fertilisation (p ¼ 0.73) did not demonstrate an overall change on the effect of proximity on willingness to use FertiSure (Supplementary Table 6). However, an upwards trend in willingness to use FertiSure was observed in those who had a greater number of previous IVF cycles (Supplementary Table 5).
The secondary analysis determined that the proximity of FertiSure's benefit to live-birth tends to impact respondents' willingness to use FertiSure among those who had and had not experienced implantation failure differently (test of interaction, p ¼ 0.02) (Supplementary Table 6). Those who had experienced implantation failure were less willing to use FertiSure when it was stated to benefit fertilisation rate, and more willing to use it when it improved embryo availability (Supplementary Figure 1). When FertiSure was stated to improve implantation rate, those who had and had not experienced implantation failure (including those who have never had IVF) reported a similar willingness to use FertiSure.

Discussion
The present study found that a substantial proportion of IVF patients infer that IVF treatment options improve live-birth rates when they are exclusively stated to improve upstream outcomes. Consistent  with similar work by Havrilesky et al. (2014), which found that ovarian cancer patients ranked progression-free survival (surrogate outcome) as more important than overall survival (clinical outcome), these results suggest that IVF patients consider improvements in upstream outcomes to be linked with clinical outcomes. The large proportion of respondents willing to use FertiSure in this cohort (78%) may reflect this inferential link. In a real-life setting, this perceived link may drive patient decisions to use fertility providers that advertise success rates based on upstream outcomes or to use add-ons when they are stated to improve upstream outcomes. Given the lack of evidence for translation between improvements in these outcomes and improvements in live-birth rates, it is possible for this to mislead patients. Several Cochrane reviews report that treatment options which improve upstream outcomes show a lack of good-quality evidence for the benefits of live-birth (Boomsma et al., 2022;Cornelisse et al., 2020;Edi-Osagie et al., 2003). For example, while increasing FSH dose in predicted poor responders increases the number of oocytes retrieved, it does not appear to increase the probability of ongoing pregnancy or live-birth rates (both cumulative and per cycle initiated) (S. F. Lensen et al., 2018). Furthermore, some treatment options have even been shown to impact live-birth in the opposite direction to which they impact the upstream outcome; IVF cycles using enhanced 'G5' culture media have shown a lower rate of fertilisation but higher rates of live-birth (Kleijkers et al., 2016).
When patients in this study were told there was no evidence for FertiSure's effect on live-birth, their interpretation of the benefits of FertiSure and their willingness to use it decreased significantly (willingness decreased from 82% to 73%). IVF clinic websites do not always report success rates in terms of live-birth (Goodman et al., 2020), and only 16% of claims of benefit for advertised treatment options are accompanied by caveats outlining a lack of evidence for the outcome of live-birth (S. Lensen, Chen, et al., 2021;van de Wiel et al., 2020). While regulators may encourage IVF clinics to use live-birth when reporting success rates, this is often not mandatory (Human Fertilisation and Embryology Authority, 2021) and there appears to be little guidance or regulation relating to the use of upstream outcomes, either as measures of clinic success rates or in advertising of treatment options such as IVF add-ons. It should also be noted that consumer protection legislation in many jurisdictions prohibits the provision of information that is inaccurate, incomplete or creates a false impression. The results presented here suggest that patients amend their treatment decisions when informed about a lack of live-birth data, and the omission of such a statement could therefore be considered misleading. This may be particularly important for IVF clinics, as these websites are trusted by patients and dominate as an information source shaping decisions (Competition and Markets Authority, 2020).
When respondents in the current study were informed that it was possible for FertiSure to pose a risk to patients or resulting pregnancies, more respondents agreed that this was the case (60% vs 33% in those not informed about the risk). However, the cohort was just as likely to use FertiSure whether they were told it was risky or not. This may seem surprising, given that previous research indicates that IVF patients consider the evidence for the safety of treatment options to be highly important (S.  A possible explanation is that while patients may value information about safety in order to be prepared for possible adverse events, they are ultimately willing to opt for treatments which may present an increased chance of success, despite the potential risk. It is also possible that IVF patients assume that treatment options offered to them at IVF clinics are safe; trusting that their clinician or IVF centre would not provide or recommend a treatment that could harm them. A recent survey of IVF patients notes that over half of IVF patients assumed that addons were known to be safe (S. . Blanket statements like the one used in the current study may be too general and therefore not effective in risk communication. It is also important to note that neither the risk statement nor the live-birth 'lack-of-evidence' statement used in the current study raised the possibility that FertiSure could reduce the live-birth rate, as has been observed with previous treatment options approved on the basis of their effect on upstream outcomes (Cornelisse et al., 2020). Where safety data for add-ons are available, perhaps a more specific framing of risks would be appropriate to improve risk communication.
Interestingly, while only 44% of this cohort believed FertiSure would increase the live-birth rate, 78% were willing to use it i.e. 34% didn't believe FertiSure would improve the chance of live-birth but still reported that they would use it. The reason for this is not clear but it is possible that achieving progression to the next part of the IVF cycle may be a motivating factor. For example, achieving a positive pregnancy test may make some patients feel as if they had given themselves the best chance, or experienced some measure of success, even if they never have a child.
Moreover, it is not clear whether clinicians recognise the possibility that upstream outcomes may not reflect clinical benefit. If clinicians overestimate the importance of upstream outcomes in IVF or fail to understand that these may not translate to clinical benefits to their patients, they cannot be expected to create accurate patient resources or appropriately counsel patients. Thus, the interpretation of upstream outcome improvements in IVF clinicians should be explored in future research. Finally, it remains possible that some upstream outcomes are indeed valid 'surrogates' of live-birth under some circumstances. If valid surrogates could be identified, this would allow treatments to be evaluated more efficiently. Methods to investigate the validity of surrogate outcomes have been described, and research is needed to establish whether valid surrogates in IVF can be identified (Ciani et al., 2017).

Strengths and limitations
This study is one of very few to explore patient interpretations of upstream outcomes. Critically, it is the first study to explore any upstream outcome interpretations in IVF patients (Caverly et al., 2016;Havrilesky et al., 2014). The factorial nature of the survey design allowed for several complementary research questions to be answered simultaneously. As a result, the implications drawn from this survey are wide-ranging. The international reach of the survey enabled responses to be broadly comparable to the global IVF patient population. This study design benefited from the involvement of an IVF clinician, statisticians, and extensive piloting with patient representatives. While the obtained sample size was only 50% of the projected value, it was still sufficient to observe several interesting trends.
However, the study has several limitations; the survey was based on a hypothetical treatment option and questions asked patients for their theoretical willingness to use this option, therefore, results may not reflect values and decision-making in a real-life setting. In addition, a large proportion of the study cohort was both tertiary educated and had experienced an unsuccessful IVF cycle. Thus, these respondents may be more likely to opt for innovative treatments. The nature of online survey recruitment is prone to selection bias, as IVF patients who opt-in to research may not be generalisable of the IVF population. Furthermore, the method of online recruitment precludes any verification that answers provided were in fact truthful and authentic; several responses contained anomalies indicating the possibility of errors or misunderstandings.

Conclusions
IVF patients infer that improvements in upstream outcomes will lead to improvements in live-birth, and they are willing to use treatments based on improvements in upstream outcomes. This suggests that claims made about IVF add-ons are likely to be misinterpreted. However, the proximity of the upstream outcome to live-birth does not clearly influence patients' willingness to use treatment options. Patients take note of potential risks and caveats about the lack of evidence for effects on live-birth and are less willing to use treatments not confirmed to improve live-birth rate. Information focussing on improvements in upstream outcomes only may be misinterpreted by patients. Thus, patient-facing material circulated by infertility treatment providers should contain caveats about both the safety profile and evidence base for treatments, at a minimum. For instance, success rates and possible benefits of treatment options should be expressed in terms of live-birth where possible, and if the treatment option has not been proven to improve live-birth, this should be stated.

Data availability statement
The data underlying this article will be shared on reasonable request to the corresponding author.