Knowledge Acquisition for Clinical-Trial Selection

,


I. INTRODUCTION
Cancer causes 550.000 deaths in the United States every year, and the treatment of canccr is an actiw research area.Medical experts explore new treatment methods, such as drugs, surgery tcchniques, and radiation thcrapics.An experiment with a new treatment procedure is called a climcal trial.Whcn researchers conduct a trial, they rccmit patients with an apprcpriate cancer typc and medical history.The selection of patients has traditionally been a manual procedm, and srudics havc s h w n that clinicians can miss up to 60% of the cligiblepaticnts [12, 22, 301.
A reccnt project at the Univcrsity of South Florida has been aimed at automatic selection of patients for clinical trials.We havc developed an intelligent agent that prompts a clinician for a patient's data and identifies all matching trials 11, 111.It includes a knowledge base with information about available clinical trials, criteria for selecting patients, and related medical tests.
We r c p a t the work on a w e b b m d interface that enablcs a clinician to enter ncw trials without the help of a programmer.Wc havc uscd the interface to build a knowledge base for clinical trials at the Moffitt Cancer Center, located at the University of South Florida We review the previous work on medical expert systems (Scction II), explain the knowledge representation in the developed agent (Section III), and dcscribc the interface for adding ncw knowledge (Section 1").

PREVIOUS WORK
Researchers began to work on medical applications of artificial intelligcnce in the early sewnties.Shortliffe and his colleagues developed the MYCIN system, which diagnosed bacterial diseases 15, 25, Early expert systems did not have knowledgeacquisition tools, and programmers hand-coded the related rules.To simplify knowledge entry, researchers implemented specialize3 tmls for some systems [13, 151.Eriksson pointed out the need for tools that would allow efficient knowledge acquisition, and dmrihed a s y s tem for building such twls

KNOWLEDGE BASE
Physicians at the Moffitt Cancer Center have about 150 clinical trials available for cancer patients.They havc identified criteria that determine a paticnt's eligihility for each trial, and they use these criteria to select trials for eligible patients.Traditionally, physicians have selected trials by a manual analysis of patients' d a t a The review of resulting selections has shown that they usually do not check all clinical trials and occasiondly misr an appropriatc trial.
To addrcss this problem, we have built an intelligent agent that helps to select trials for each patient.It prompts a clinician to enter the results of medical tests, and uses thcm to identify appropriate trials.
In Figurc l(a), we givc asimplified exampleof eligibility criteria for a clinical trial.This trial is for young and (a) Eligibility criteria 1.The patient is female.We encode the eligibility for a clinical trial by a logical expression, which may include variables that r e p resent the available medical data, a well as equalities, inequalities, "set-element" relations, conjunctions, and disjunctions.For example, we encode the criteria in the first part is for adding information about medical tests (Figure 2), and the secand is for eligibility criteria (Figure 3).The interface includes ten screens; two of them are "start sueens," which can be reached from any other wren.We give an example of entering eligibility criteria, describc the two parts of the interface, and present experiments on its effectiveness.

She
Example: Suppose that a user needs to enter the criteria shown in Figure 1.First, she utilizes the "Adding tests" screen to enter the three tests (Figure 4).Then, she adds the related questions; to enter questions for a specific test, she selects the twt and clicks Wmli7y" (Figure 4), and the agcnt displays the "Modifying a test" screen (Figure 5).To add a question, she cli& the appropriate button at the bottom (Figure 5 ) and then types the question (Figure 6).
After adding the questions for all tests, the user goes to the *Adding c l i n i d t d " Screcn and initializes a new trial (Figure 7).She gets the "Selecting tests" screen and chooses the tests related to thc current trial (Figure 8).Then, she marks relevant questiom and the answers that make a patient eligible (Figure 9).If the eligibility criteriaindude disjunctions, she has t o w the screen for composing logical exprssions (Figure IO).
Tests and questions: The interface for adding tests and qucstions includes six screens (Figure 2).The start screen is for viewing the available tests and dcfining new ones, whereas the other screens are for modifying tests and adding questions We show the start screen in Figure 4; its 1efGhand side allows viewing quostions and going to a modification screen.If the wcr selects a test and di& "View," the agent shows the questions related to this test.If thc wer clicks "Mod& it displays the "Modifying D test"scrcen (Figure 5).The right-hand side of thestart screenallows adding a new test by spccifylng its name and cast.
The "Modifying a test" screen shows the information about a specific test, which includes the test namc, cost, and related questions.Thc user can changc the test name and cost; the four bottom buttons allow moving to thc screens for adding and deleting questions.Eligibility conditions: The rncchanism for entering eligibility criteria consists of four screens (Figure 3).The start screcn allows the user to initialize a new clinical trial and view the criteria for old trials.If the displays the t e s t s e l e c t a n screen (Figure 8).The user then chooses related tests and question types, and clicks *Continue" to get the question list.
The next screen (Figure 9) allows the user to select specific questions and mark thc amwcrs that makc a patierit eligihlc.For a multiple-choice question, thc user may specify %vera1 eligibility options: for examplc, a patient may bc eligible if her Cancer stage is Il or For a numeric question, user has tO spedfy a range of values; for instance, a may be e,igiblc if her age is between 0 and 45 years.If thc usm clicks "Simuser n m b to modify a clinical trial, tho agent first the agent generates a conjnr,ction the

Entry time:
We have run cxperiments with sixteen novice users, who had no prior expcrisnce with the interface.First, every user has cntcrcd four sets 01 medical tcsts; cach sct has includcd thrm tcsts and ten q u e tions.Then, cach user has added eligibility expressions for ten clinical trials used at the Moffitt Cancer Center; the number of questions in an eligibility expression has varied from ten to thirty-fivc.
W e have mcasured the entry time for each test set and each eligibility exprczsion.In Figure 11, we show the mean time for every test sct and thc time per question for the same sets.All uscrs have entered the test sets in the Same order, from 1 to 4; since they had no prior expcriencc, their pcrformance has improved during the cxperiment.In Figurc 12, we give similar graphs for the entry of eligibility expressions.
The experiments have shown that novices can efficiently use the interface; they quickly lcarn its full functionality, and their learning c u m flattcns aftcr about an hour.The avcragc time pcr question is 31 sceonds for thc entry of medical tests and 37 seconds for elmbility criteria, which means that a user c m enter all 150 cancer trials used at Maffitt in about two weeks.

V. CONCLUDING REMARKS
We have developed knowledge-acquisition tmls for an agent that automatically assigns cancer patients to clinical trials.We have described the rcprescntation ofeligibility criteria and a webbased interim for adding now trials.The experiments have shown that a user can enter a new trial in fifteen to thirty minutes.Novices can use the interface without prior instructions, and they reach their full speed after about an hour.Although cancer researdl at Moffitt has provided the motivation for this work, the agent is not limited to cancer, and we can use it for trials related to other diseases. [6].
Figure I(a) by the expression in F i b r e l(c).The agent collectsdata until it can determine whether the eligibility expression is TRUE or FALSE.For instance, if a patient's sex is MALE, then the expression in Figure I(c) is FALSE, and the agent immediately rejects this trial.If the sex is FEMALE, the agent has to ask more questions.If the knowledge base includes many clinical trials, the agent checks a patient's eligibility for each of them.It first asks for the tests related t o multiple trials, and then requests additional tests for specific trials IV.ENTERING ELIGIBILITY CRJTERIA We have desigued a webbased interface for adding new clinical trials [IS], which consists of two main parts;

Fig. 9 .
Fig. 9. Selecting questions and answers.Thc user chocks the questions for the current clinical trial and marks the answers that satisfy thc eiigibility criteria.

Fig. 11 .Fig. 12 .
Fig. 11.Entry time for test seta (left) and the mean time per question for each set (right).W e plot the average time (dashed lines) and the time of the fastest and slowet users (vertical bars).100,

5, 141, such as NEOMYCIN, PUFF, CENTAUR, and VM.
261.Experiments showed the effectiveness of MYCIN, which led to the development of other medical systems (Machado et al. dcveloped the AIDS%Y stem, which also assigned AIDS patients to clinical trials [19].Bouaud et nl.created a cancer expert system, called ONCODOC, that suggated alternative trials for each patient and allowed a physician to choase among them [3,4].S&oused ONCODOC to select participants for clinical trials at two hospitals, which helped to increase the number of selected patients by a factor of three 123, 241.
Musen et al. built a rule-based system, called EON, that selected AIDS patents for clinical trials 1171.Ohno-

5) Fig. 1. Example of eligibility criteria. tests. and qucstiom.
is at most forty-five y e a old.3. Her cancer stage is II or 111.
is the cancer stage?Does the patient have inwive cancer?Biopsy, Cost is $300 What is the cancer stage?How many lymph nodes have tumor cells?What is the greatest tumor diameter?Electmniiogmm, Cost is $200 Does the patient have cardiac arrhythmias?(c)Eligibility expression.se2= FEMALE and age 5 45 and cancer-stage E {II: 111) and Invasive-cancer = NO and lymph-nodes 5 3 and (arrhythmias = NO or tumor-diameter 5 2.middlc-agcd women with a noninwivc cancer at stage 11 or 111.Whcn testing a paticnt's eligibility, a clinician has to order three medical tcsts (Figure Ih).The agent first prompts thc clinician to enter the patient's sex and age.If the patient satisfies thc corresponding conditions, the agent asks for the mammogram rsults and veri& Conditions 3 and 4 then, it reqnsts the biopsy and elntrccardiogram data.Thc agent's knowldgc basc includcs questions, tests, and logical expressions that represent eligibility for each trial.We give an cxamplc of t s t s and questions in Figure l(b), and a logical expression in Figure I(c).it allows the answering of two questions.Different tests may answer the same question; for example, both mammogram and biopsy show the canccr stage.