These are transcripts of semi-structured interviews of 11 Primary Care Providers (PCPs) in the group that used VisualDx, a clinical evidence technology, in a clinical trial. They are part of a mixed methods study that included survey data entitled Use_Skin_CET_survey_dataset.csv. Data were collected in the context of follow-up research on providers who participated in a previous cluster-randomized controlled clinical trial that evaluated the impact or effect of a clinical evidence technology,VisualDx, on the outcomes of patients seen in primary care for a skin problem. The follow-up research included a quantitative survey of primary care providers PCPs) who participated in the original trial and semi-structured interviews with those PCPs in the intervention group that used the clinical evidence technology (CET). Interviews took place 2018-02-01 to 2018-03-31 at the University of Vermont Medical Center. Data from interviews were digitally recorded. VoiceEacdata were transcribed and saved as Word files. Transcripts were de-identified, appended, and saved as a text file (.txt) for this deposit. Each interview is separated by a solid line. This data is analyzed and results reported in "Barriers and Facilitators to Use of a Clinical Evidence Technology in the Management of Skin Problems in Primary Care: Insights from Mixed Methods" Jour Med Lib Assoc July 2020 ________________________________________________________________________________________________________________________________________ Name: Files\\int_id\\PCP 01-45A Interview #1 PCP 01-45A Transcript Interview Date 2_2_2018 Start 8:05 a.m. Interview Code PCP 01-45A Internal Medicine Resident Active Group MB: Could you just say your name and where you work: PCP 01-45A: [Name] Internal Medicine. 3rd year resident MB: What do you remember about being in the Information for Skin Problems study? PCP 01-45A: I remember that when I had a patient that I saw in the clinic that had a skin complaint I was supposed to open VisualDx and like use it to come up with a diagnosis or treatment or what not MB; And did you do that? PCP 01-45A: I did. For a few months, I tried to be pretty diligent about it. Maybe not for every skin complaint but I certainly did use it. MB does something stand out in your mind? One time that you used it or not? PCP 01-45A: About what I liked or not? One time when I found it useful - I’m a resident and so I haven’t seen a ton of this stuff- was when a patient came in and he was pretty convinced he had poison ivy and I was pretty convinced that he didn’t. I honestly haven’t seen a lot of poison ivy, and I ended up using it on him, and I came to the conclusion it really was poison ivy, and he was right. I would have treated it differently had I not opened it up and used some of the pictures to help. And I like how -and I think I’m remembering this correctly - when you open it up you pick which part of the body, and that’s one of the first things you can use with it - the “diagnosis builder “[mb differential builder] and you select which area and I think that’s helpful, that you can organize it like that. MB: Did you find it hard, easy, or complicated? PCP 01-45A: I think I found it was easier to use when I kind of had an idea already what I thought it might be. Just because it was then nice to see that, and see pictures, and feel like, yes, this is just eczema or something like that, sort of treatment. I found it clunky with some more unusual things just because it’s hard to --- We learn the text-book descriptions of things but they never really look like that. If everything looked textbook we wouldn’t need tools like this I guess. Sometimes I found it a little bit burdensome. You put in something like a description in the differential builder and you just come up with so many things and it’s hard to know which of those are more likely or which are really zebras that you really shouldn’t even be thinking about necessarily. So sometimes I found it burdensome but overall, I think it was helpful. MB: So, you mentioned that you changed one diagnosis. Can you think of any other difference that VDx made? PCP 01-45A: They put in there treatment recommendations, right? I definitely used it for those. Maybe not for confirming more common diagnoses but if people came in and I had an idea of what was going on, and it could be a couple of different things and I used some pictures to help me decide, And if I had tried the usual things you try in the primary office to treat that, it was helpful to see some of the other treatment options they had available on there. Specifically, in terms of diagnosing, I can’t really think of anything in particular other than the one. MB: How about referrals or return visits? Did it ever prompt a return visit or avoid one? PCP 01-45A: I had a lady who came in with something strange on her eyes and we couldn’t figure out what it was. Based on using VisualDx I came up with something I hadn’t considered on my differential that was treated quite differently than how we were currently treating her. That did prompt a referral to dermatology. MB: Was there ever a time when you hoped you would find an answer but were disappointed? PCP 01-45A: No, don’t think so. MB: Do you use VisualDx now? PCP 01-45A: Every once in a while, I’ll still open it. I’m not so diligent about using it, but I will open it. Before this I never would have considered opening it. I didn’t know it existed. Now every once in a while, if I’m unsure about something I’ll take a look. Or if I have no clue what’s going on I’ll use it.. MB: What other information resources did you use during the trial for skin? PCP 01-45A: I’m definitely an UpToDater. MB: For skin problems too? PCP 01-45A: Yes. UpToDate is sufficiently obvious that you have to know what you are looking for if using UpToDate, so it’s hard to be as broad, and be like and go like, “There is something in someone’s armpit. What do I think it is?” UpToDate pictures can be helpful, not any more helpful than VisualDx. Sometime I think there is a little more information [in UpToDate] but again VisualDx is better if you have no clue what the diagnosis is. MB: So, with VisualDx you can start from the ground level, whereas with UpToDate you need an idea. PCP 01-45A: Right. If I had no idea what something was I would go to VisualDx first, but if I knew I was treating eczema or skin cancer I would look at UTD probably. MB: You used VDX and UTD then during the trial for skin, anything else now for skin problems? PCP 01-45A: I don’t really. UpToDate and VisualDx are my two go-tos. Every once in a while, I’ll do a Google Image. If someone comes in and they say “I think I have X” and I never heard of it, or they’ve heard of it from a family member, I’ll use Google Images just to take a look. So those three things. MB: Google is used by a lot of doctors, studies show. PCP 01-45A: Yes, it’s an important tool. M Anything else you’d like to tell me about using clinical information sources for skin problems? What about access? What device or platform did you use for VisualDx? PCP 01-45A: I always went to it a desktop through EPIC. Never accessed it through the hospital intranet. Never used the mobile. When I’m in clinic I don’t really use my phone. Hardly ever. And we’re so tied in. We’re always on the computer, writing the notes, or looking at vitals, looking at meds… MB Anything else you’d like to share about using VisualDx? One other thing is the way it’s more exciting to look at than UpToDate. There’s more pictures and the information is a little more succinct. And that makes it more helpful to share with patients when you are sitting there in the office. You can say “Oh this is what this looks like” and “This is how we treat it”. I don’t know that I did that that often, but maybe a couple, once or twice. I think its very user/ patient friendly. Not patient –friendly really. It’s written for providers. But the pictures are helpful to show folks. MB: Is that an important aspect of care, to have that communication? PCP 01-45A: Right. Sometimes in the primary care office, people sometimes think you are blowing their complaints off a little bit, so if you are able to say “This is really just eczema and this is what eczema looks like on other people, and it’s just like yours”, and it gives you a kind of rapport with them. PCP 01-45A How long was the study? MB: for you, Feb 16 through July. Turned off recorder. _____________________________________________________________________________________________________________________________________ Name: Files\\int_id\\PCP 02-44A PCP 02-44A Interview #2 Feb 2, 2018 Start time: 10 AM Family Med, Attending, Active group MB compared to written notes and re-listened to recording 5_8_2016 MB: So, we can get started, but we can stop at any time or I can turn this off any time you want. So, could you just say your name and what your role is here at the university?  PCP 02-44A: I'm [Name] and my role is I practice at [Name of Practice]. - MB: You were in the study, from… for about 6 months, if you want to recall this, from February 2016 until about August 2016 when we ended it. So, it was almost like 2 years ago exactly that you were in it, the date that I have when we finished, your date of enrollment was like February 2nd.  PCP 02-02-44A: Okay.  MB: So, let's see and get started. So, what do you remember about it, if you can?  PCP 02-02-44A: So, I remember the idea was that we saw patients with skin conditions we would use the VisualDx to enhance our treatment of the patients. And I remember multiple times you know going to the reference and using it to help me. Most of the time, I would say, well I wouldn't say most, about half of the time, I would use it for helping to expand my differential or confirm what I thought was a diagnosis related to a skin problem. And then probably half or a little bit more was to look up the most recent recommendations for treatment of the condition if I thought I knew what it was. But I would say it was about half and half using for those different reasons.  MB: So about half helping to expand the differential, and about half to look up a treatment, a recent treatment?  PCP 02-02-44A: Right, or maybe not expand the differential, but come up with a differential.  MB: Or come up with. Does anything stand out?  PCP 02-02-44A: I thought it was - you know, obviously - Sometimes there's barriers like you just don’t have time or if it's a simple thing that you kind of deal with a lot and you feel like you know what it is and how to treat it then you obviously wouldn’t use the resource in that situation. So, I think that, I would say most of the time that I felt like I wanted to use it, it was pretty easy to use, in terms of getting to the website. You know it’s right on the EMR so it was pretty easy to access. It's even easier now actually, now that they've kind of upgraded the navigation and places, where you can save it as one of you favorites now.  MB: Oh!  PCP 02-02-44A: If you were able to do that in the past I didn't know how to do it.  MB: I didn't know you could do that!  PCP 02-02-44A: There's a new version. And so, I remember it being fairly easy to access and you know fairly helpful. I liked the- in terms of the first part of developing the differential - I liked the part that you could just put in type of rash and location and it would give you a differential based on that. Easier than having to have all the information based on what you knew, so I thought that was pretty good. And the treatment recommendations were fairly simple and seemed to be well organized. A little easier than maybe going to one of the standard secondary literature sources like UpToDate or Dynamed or something like that, just because it was specific to the skin.  MB: Okay, you answered this question a little bit, but.. How did you get to VisualDx? For example, what device? PCP 02-02-44A: Yeah, usually it was a desktop. PCP 02-02-44A: There was a link within our EHR to VisualDx, so I would usually use that, sometimes, if I was at another computer and wasn't actually in the Electronic Health Record, then I would just usually go to the Dana Library site or Google it and find it that way.  MB: Okay, when you Googled it did you have to put in another password? PCP 02-02-44A: That's a good question, I don't remember. I think if you're in the network then it, if you go to it you're okay as long as you don't go to an outside computer. If I was going to use it at home or something then I would use the Gateway so I would be inside. MB: Did you find that cumbersome to get to?  PCP 02-02-44A: No, no. Since it was one that I used a fair amount, I knew where it was. Like I said they have made it easier in, with the new latest upgrades of the EMR. You can save it, it's kind of like a bookmark so you can put across the top with the places that you tend to go. MB: Oh, so that bookmarking thing, is in the EMR? It's not in VisualDx? PCP 02-02-44A: Yeah. It used to be you had to kind of scroll through a bunch of menus to find it but now once you find it you can right-click on it and put it up in your menu, so it makes it easier. MB: Oh, that's easier. So, thinking back a little bit, can you think of what difference VisualDx might have made in any aspect of patient care? PCP 02-02-44A: I don't know as I can recall a specific case. But I can say, I can definitely say it helped me feel a little more confident about a diagnosis. And also, more confident that I was using the most up to date treatment. There were definitely times that I would review the treatment and would say "Oh, I didn’t realize that that was now what is recommended." So, I was using something that was a little out of date in terms of the treatment, so it would help that. So, I would say that was definitely helpful.  MB: Do you think it ever, - what about, avoiding or prompting a referral?  PCP 02-02-44A: Yeah, that's a good question... I guess I felt a little more confident. I don’t recall a situation that it prompted me to make a referral but I definitely would say there probably were situations where it would have made it a little less likely to refer because I had more information.  MB: What about patient communication, did it ever have any effect on that?  PCP 02-02-44A: I think that, I think it actually maybe has patient handouts and stuff that I didn't use. I didn't use those specifically. I mean sometimes just reading about the condition on there or - I wouldn't usually show the pictures to the patients because the cases that they show in there usually are a lot more severe.  MB: That's true, the worst looking cases. PCP 02-02-44A: Yeah and you don't want to show them. But in terms of just understanding like the pathophysiology - you know, a lot of skin conditions, we don't really know what causes them- just reviewing that before I talk to the patient is helpful. Or saying that half the time we don't find a particular cause for this but this is the recommended treatment. It was kind of helpful to be a little more confident about recommending, describing the condition and recommending a treatment.  MB: I know you said you couldn't remember one, but could you think a little harder, maybe you'll remember a little bit, a time when it either made a difference or you were hoping it would make a difference but you were disappointed. PCP 02-44A: Yeah... I think one case was an onychomycosis, which is a fungal infection on the nails. There's a couple of different treatment options that I wasn't aware of. There was a new topical treatment that was an option that had recently been FDA approved and I hadn’t used it before. So, I remember that. What other situations... Sort of helping, to just in terms of choosing a steroid, I can remember, there was some good information about the different potencies, you know, different potencies of the different steroids and how much of them you should use in a particular timeframe. That was helpful. Trying to think, … there may have been situations where the software gave an expected duration of treatment to follow up, and that may have been helpful in terms of, you know, coming up with a follow-up plan.  MB: That's good. But were you disappointed? What disappointed you?  PCP 02-44A: You know, it is time consuming and if you're in, you know, you're busy. It’s easier to get at treatment than it is to get a differential because it just takes longer. And I was less likely to do that part. So I think those were, you know, sometimes things are just undifferentiated and it doesn't help, you know.  MB: Yeah, just sometimes it just didn't work…had no effect.  PCP 02-44A: And that's probably -maybe the condition wasn't defined enough yet or something, you know, an undifferentiated condition.  MB: So, do you use VisualDx now? Sometimes?  PCP 02-44A: Yes. Yeah, I do.  MB: And what prompts you [to use VisualDx now]?  PCP 02-44A: Usually it's the same situations either something that's so unusual that I'm not sure what it is or I wanted to double check what I think it might be to see if it comes up with any other, anything else on the differential. And the other thing is, you know, looking up a current treatment because it's pretty good for that.  MB: So, you're still using it, that's good.  PCP 02-44A: Yeah.  MB: What other information resources did you use then, during the study, or now, for skin related problems?  PCP 02-44A: Right, yeah, I don't, it's kind of my go to when it comes to skin, I do use DynaMed and UpToDate but they don't usually have too much in the way of pictures and differentials, and stuff like that so, for the things that VisualDx does, I think it's better.  MB: Okay, and would you like to tell me anything else about using information resources relevant to dermatology or skin problems? PCP 02-44A: Right. I wish they were more integrated into the record, there's no reason that the literature shouldn't be connected directly into the patient's information so that as you're taking the history -- you know, the ideal situation would be, as you're taking the history the differential is being calculated based on probabilities.  MB: ..that would be nice.. PCP 02-44A: And the treatment recommendations are being... And I think that will happen in the next 5 years as the kind of medical literature gets integrated with the electronic health record. I think that would be kind of the ideal state when those two things come together in the record is actually informing the literature and the literature's informing the record. MB: Yes, that would be wonderful.  PCP 02-44A: That would be true point of care reference. MB: Point of care, and decision support, true decision support, we don't really have that.  PCP 02-44A: Right, right, exactly.  MB: So, well this is very, very helpful and so, here's my little brief questionnaire, they are kind of similar to these questions but they're more short answer. PCP 02-44A: Sure, alright, want me to just do it here? MB: Yeah just run through it. [starts to fill out] It's double sided.  PCP 02-44A: How many times did you use VisualDx, that's a good question. I did it for 6 months.  MB: So, if you did it for 6 months... PCP 02-44A: I can backtrack on that... I would say 20, 25 times, I'd say once a week. End of interview at 17:12 min. __________________________________________________________________________________________________________________________________ Name: Files\\int_id\\PCP 03-48A PCP 03-48A Interview #3 Interview 2/2/2018 4:00pm PCP 03-48A Attending/Internal Medicine/Active MB: So would you say your name and where you are practicing now? PCP 03-48A: [Name] and I'm practicing at [Name of Practice].  MB: Thank you. Just to help with memory, you were in the information for skin problems study from mid-January 2016 until the end of July 2016, so that was about 6 and a half months. And, it kind of started two years ago, in fact. PCP 03-48A: Yeah. MB: What do you remember about being in the study?  PCP 03-48A: Well, I remember the design of the study was to compare people who had access and didn't have access to the VisualDx, which is baked into the workflow in the medical record and easy access. And I was in the intervention arm and they were just going to recruit until we had a certain number of people accrued and that there was going to be follow up of the patients after their visit until they either got better or their problem resolved or came to resolution because they needed to be seen by dermatologist or some other specialist. And that you were tracking also- did people have return for care? MB: So, you remember it pretty well.  PCP 03-48A: Better than most. MB: Better than most. And how would you describe your experiences? Does anything stand out in your mind about it?  PCP 03-48A: Well the thing that stands out is - before having participated, I knew of VisualDx, I had used it not so much to solve a particular problem but more just to browse and see what it's all about kind of thing. The first time I did that was probably years ago, and I never really tried to incorporate it into my day to day practice. What I would more typically use, I think, before this study was a textbook. I have a couple of derm atlases that sit on my shelf and I might turn to those. So, as a result of this study, I think I more systematically used it, actually you know, tried to incorporate it into the daily workflow. And I think to me the thing that stands out is it made me think a lot about how I make a dermatologic diagnosis, and how much of dermatology is a diagnosis question versus a management issue. So if you take for example, Zoster, Shingles, it's usually completely obvious what the diagnosis is. There is no ambiguity or uncertainty about it. So I would never typically have to look that up to make the diagnosis. But I might have to look up what is the actual dose of medication to use- 1000 3x a day or 500 2x a day or whatever. And then, if you think about the other common things I see, - you know my practice is older,- so I see a lot of age related skin problems, so things that are suspicious for skin cancer or actinic keratosis that might need to be treated with liquid nitrogen or something. So again, for many of those there may be some diagnostic uncertainty. So - is this a little ulcerated papule that's benign or is this an ulcerated papule that's basil cell carcinoma? There may be some uncertainty, but it’s not necessarily going to be answered by looking it up on VisualDx; it just needs to be biopsied. So I think, again, in that scenario, the precise diagnosis may not be as important as what's the next step? And VisualDx doesn't necessarily answer that question for you. In the end you need to biopsy it, either then and there, or in follow up, or by referring. One of those.  MB: So, what was it useful for? Or was it useful for anything?  PCP 03-48A: So I'm trying to remember, so talking out loud. Was there a specific case when it was helpful?  MB: That was actually the next question. PCP 03-48A: Okay, so I can't actually remember off the top of my head, there isn't one particular case that stood out to me as being super helpful. I think I do remember looking things up, and being I guess maybe frustrated with the interface being... You know, I tried to use it as the workflow implied, you know, this sort of step wise, is it big or small? Ulcerated or? This or that you know?  MB: The differential thing? PCP 03-48A: Yeah the differential builder. I found that it didn't add a lot of great input or information for me and it actually seemed like it had a lot of false positives maybe. Like I would go through this algorithm and then it would offer me things that clearly it wasn't, it was almost too broad, or too sensitive, giving you more options rather than really narrow. So, at least for my level of experience it didn't offer me a whole lot in terms of narrowing my differential diagnosis. It was a lot more about broadening it, and I'm not sure if that was helpful to me. MB: Yeah, okay. So, did it, maybe thinking about some of these things a little bit more specifically, did it ever make a difference in either a diagnosis or treatment? Or even did it prompt a referral or stop one? PCP 03-48A: I would have to say no, that it didn't really offer me a different path forward. So a different decision, whether a treatment decision or a referral decision.  MB: Okay. Did it ever help with patient communication in any way?  PCP 03-48A: You know, I didn't ever use it in the room. I think my workflow was to go into my office, next door and use it there. And I think my rationale for doing that, was that, well I didn't feel like I was an expert skilled user of it, and for things that I really know what I’m getting into and I know what's going to pop up, it's fine to include the patient in that. But for something that you haven’t used a lot before and you're not really sure and there might be some pretty graphic pictures that pop up, it’s not the type of thing that probably lends itself well to browsing through with a patient looking over your shoulder, it could be shocking or nauseating or something for squeamish people.  MB: Dermatology can be kind of shocking. How hard or easy to use?  PCP 03-48A: I think it was relatively easy to use, I didn't have any barriers.  MB: And did you find the content, at least reliable? Or, true?  PCP 03-48A: Yeah I had a lot of confidence that the material was accurate and properly edited or authenticated by experts in the field. I didn't have any concerns about that.  MB: What about the access technology? What device did you tend to use? PCP 03-48A: I would use it on my desktop computer. MB: At the clinic? PCP 03-48A: In the clinic, yup.  MB: Was that through the intranet, or prism? Or? PCP 03-48A: I think directly through the EMR. MB: The EMR. Did you ever use the mobile? PCP 03-48A: You know, I tried to use it, not in real time, just, that was again more or less a curiosity, i didn't use it in real time with the patient right there. Just to see - did it work, did it turn on, could you browse through it. My preference is always, and this is probably based on just preference for bigger screens, if I have an option I'm going to use a big screen so I can see better rather than fumbling with a small device. And in my work, I'm almost always, if not always, two feet from a computer where I can pull it up pretty easily. So it didn't offer an advantage. MB: Yeah, desktop is... PCP 03-48A: In a different setting, that may be different. But that’s the advantage of, if you're in the ER, it's there.  MB: Do you use VisualDx now?  PCP 03-48A: I don't think I've used it since the study ended.  MB: Yeah, and what other information resources might you have used then or do you use now, for skin problems. PCP 03-48A: I think the two, I'll use Epocrates for looking up doses of meds, and I do that once or twice a week maybe. And then, I'll use UpToDate, if I'm trying to get a broader overview of a topic. MB: Is there anything else you'd like to tell me about using clinical information sources relevant to dermatology or patient skin problems?  PCP 03-48A: You know I think I still have the textbooks, the atlases on my shelf, and I might be inclined to pull those out, maybe more for teaching purposes. So if I have a medical student with me, I might reach over and review some of the features of common skin cancers might be. So, that has the advantage of I kind of know exactly where to find it, I know roughly where it is in the book, it’s a familiarity thing.  MB: Do you know what atlas or textbook you have?  PCP 03-48A: I don't even know the name, I know it’s yellow and blue.  That's all I know. MB: Okay, the yellow and blue one.  PCP 03-48A: The yellow and blue skin atlas. But you know, that is an interesting thing I'm thinking about it now, the difference between an information resource that has infinite ways of coming into it or navigating it so you don't know exactly where you're going to be. It has the advantage of maybe taking you on an efficient path to get to where you need to be, but it lacks the "Oh, I know exactly where this is, it's on page 30 of this book" and it's right there. For some things, I'll bookmark a website and I'll go straight to it because I know exactly what I want. Other times I might browse, because I don't know what I want. So it's like the difference between a static textbook and one that you meander through. And you know the way UpToDate is set up is clearly in very tiny bite-sized pieces that are pretty similar for each topic. You know for whatever topic, there will be signs and symptoms of that topic, or diagnosis of that topic or treatment of that topic. There's a certain similarity to that or symmetry to it. You know there's going to be a chapter about, even if you've never looked at it, you know probably what the form is going to be. And so, I guess, with VisualDx one of the issues might be, you can arrive at a diagnosis of skin cancer coming at it from “Oh these are things that occur on your face” versus “Oh these are the things that show up as an ulcerated papule,” or “These are things that are bleeding.” There's so many different ways of coming at it. It's a strength and a weakness that there isn’t a static way of looking at things. MB: There's that, you can come at it a little more broadly, maybe?  PCP 03-48A: Yeah, definitely more broadly. And you can come at it with a sense of uncertainty.  PCP 03-48A: So again when you think about some information sources, if you know someone's got pneumonia, you can look up pneumonia. If someone comes in with a fever and cough, UpToDate will have a chapter on evaluation of fever and cough. So you can kind of come at it from unknown diagnostic side or you can come at it from a “this is pneumonia” side. And if I was, and to make that analogy to skin conditions, if I'm pretty sure something is an ulcerated papule, I can probably look up ulcerated papule because I know the broad diagnosis, maybe not the specific diagnosis.  MB: VisualDx does have the differential builder and you can put in a symptom or a diagnosis.   PCP 03-48A: And maybe I didn't use it that way enough, I might not have looked at it from the point of view of -- Although I do remember, for example, basal cell carcinoma, if you put basal cell carcinoma into VisualDx, it has the positive attribute, I suppose, it’s a thousand pictures of every possible way a basal cell carcinoma can show up - which can be misleading, because it's not showing you what the typical ones are. So if 95% of them look like this, and very few look some other way, they are showing you all of those. So in a way it's almost too much information.  MB: So that’s, yeah, this is very helpful. I am taking your advice about just kind of throwing in these short answer, short questions... PCP 03-48A: Oh, like yes or no questions?  MB: Yeah, and I could turn off the recorder now, if that's ok.  _____________________________________________________________________________________________________________________________________ Name: Files\\int_id\\PCP 04-02A PCP 04-02A Interview 3/2/2018 Interview #4 (Recorded on the mb's first recorder; sessions, 5&6) Attending/Internal Medicine/Active group MB: So, looks like this is recording. could you just say your name just for starters so it's on the recording? PCP 04-02A: [Name] MB: And where is your practice?  PCP 04-02A: [Clinic Name and location] MB: Thank you. So, could we talk a little bit about your being in information for skin problems for study? Do you remember it?  PCP 04-02A: Yes, I was in the group to use VisualDx when I had patients with skin conditions, rashes or other conditions.  MB: Right. And just to help you remember, you were on the of the first people to volunteer for it, so you were actually in it for ten months, between October of 2015 and the end of July or beginning of August 2016 MB: And, I know this is kind of a memory quiz here! What do you remember about it? Either what you were supposed to do or anything like that? PCP 04-02A: I was, in that time period I was supposed to use VisualDx, which is embedded in the electronic health record when I saw a patient who had a skin condition.  MB: And can you describe how either you did that or didn't do that or your experience?  PCP 04-02A: Yeah you would pull up the program and you would try to type in characteristics that matched the rash or skin condition. MB: And how did that feel to you? PCP 04-02A: It took more time and was a little cumbersome because it didn't always match what I was looking for. MB: Did you like VisualDx? Or what was your opinion on the whole? PCP 04-02A: It was fun to look at and browse but at an actual patient level I didn't find it too helpful. And I would consider myself a novice user of it so if I was more of an expert maybe I would have found it more helpful. MB: Okay... But did you use it? During the study? PCP 04-02A: I did use it during the study.  MB: Could you estimate maybe the number of times per month? PCP 04-02A: Maybe 10 or 15 times per month  MB: Wow! So, how useful was the content? PCP 04-02A: I guess I would say I only found the content mildly useful. And in terms of how easy or hard it was to use, I didn't find it too hard to use but I didn’t know if I was using it to the maximum ability that I could. MB: Okay. How did you usually access it? Like what device or?  PCP 04-02A: Yeah, my... on the desktop either in my office or in the patient exam room.  PCP 04-02A: So, hospital PCs.  MB: Was the access method okay? PCP 04-02A: Yeah. MB: Like, did the images download? Some people said they were very slow to download. PCP 04-02A: Oh, I didn't find it slow, no.  MB: Did VisualDx ever make a difference in an aspect of patient care?  PCP 04-02A: Not in the particular cases that I used it for. MB: Okay. So, did it ever, so for example, did it.. I know you're thinking of this but, - change a diagnosis, avoid or prompt a referral, patient communication? PCP 04-02A: I mean I think patient's like the fact that you could use it as a tool to help figure out what they have, but if I was going to refer to dermatology, I was going to refer to dermatology. That wouldn't change my mind. MB: Did you use it with patients?  PCP 04-02A: A couple of times. MB: Did you ever show them the picture?  PCP 04-02A: Yeah, a couple of times. Especially if they had something that went away; then they could say "Oh it did look like that", kind of self-report.  MB: Oh yeah. So, it sounds like generally it didn't make a difference except maybe that patient communication. PCP 04-02A: Patient communication and education. Absolutely.  MB: Made a little bit of difference. PCP 04-02A: Right. MB: Could you just tell a time that you were, you hoped it would make a difference and you were really looking for something but it didn't?  PCP 04-02A: Yeah, I think there were a few instances that I couldn't find what I was looking for, so maybe I was not using the tool to its fullest extent.  MB: Did you use the differential builder? PCP 04-02A: I used that, yup. Symptoms. But sometimes I just couldn't get what I wanted.  MB: Okay. Do you use it now? PCP 04-02A: No.  MB: What other information resources did you use then or do you use now? PCP 04-02A: I actually use dermatology books. MB: Okay. Do you know which ones? Fitzgerald??  PCP 04-02A: Yeah, I use that one... and there's two other small colored atlases, they're paperbacks that I've used for years... I forget the authors. MB: Okay. What else are you thinking of or you would like to tell me about using information resources relative to dermatology or skin problems?  PCP 04-02A: Well I do see the residents I work with use it [VisualDx]. So maybe some of the younger generations use it more if they're more tech savvy. I see them using it for rashes since that tends to be a difficult subject in internal medicine; so I do see residents using it.  MB: Okay. And what do you think of information resources or the relevance to evidence-based medicine? Is there any connection? Maybe not just this resource but just... PCP 04-02A: Other tools? I suppose some can be helpful and some not, I guess it depends on how they good they are, but then there's also, that it takes the time away from the physician with the patient. So there's that. Or if it's used in a collaborative way I guess it's good. It depends on what it is. MB: Yeah. Okay. Well, see I told you this was going to be quick. These are.... So these are just a few, you've almost, these are almost like recapping some of these questions.  PCP 04-02A: Okay.  MB: I have a pen here.  PCP 04-02A: I've got one. Is blue okay? MB: It's perfect. This is the low-tech survey. PCP 04-02A: Got it.  MB: And yeah, some of the questions, like if you wanted to figure out, well you already did, like approximately how many times a month and you were in for 10 months so... PCP 04-02A: So this is during the study?  MB: This page is during the study. And these are recent, yeah. Recording ended 12:50. Transcription start: 16:36 End: 17:00 _______________________________________________________________________________________________________________________________________ Name: Files\\int_id\\PCP 05-34A PCP 05-34A There is no recording of this interview owing to a recorder malfunction. Transcript is from Interviewer notes Date of Session___3_2_2018___ Time begin___1:30 pm -Time end. 140 pm Interview #5 PCP 05-34A Provider Dates of Participation in CRCT: _December 6__-July 2016_ Family Medicine/Attending/Active [PCP Name and Clinic Name] MB. What do you remember about being in the “Information for Skin Problems Study?” You were in it for 5 months between March and August 2016. PCP 05-34A: I don’t remember very much. MB Please describe your experience of being in the study? What stands out in your mind? PCP 05-34A: I remember accessing the tool 2 or 3 times. It was cumbersome. MB: What was your experience using VisualDx? Did you use it during the study? About how many times in 5 months? PCP 05-34A: I used it 3 times in the first month. MB: How useful was the content? PCP 05-34A: Moderately useful in the beginning but then couldn’t access it. I asked for help don’t remember who (Gary?) and maybe I got a response. But maybe I didn’t try to fix or follow the answer, and didn’t use it again. MB: How hard or easy to use was it? PCP 05-34A: (shrug) MB: How did you usually access VisualDx, how did you get there? PCP 05-34A: Clinic, at desk, once in the exam room. MB: How hard or easy to find or get to and use the access method? PCP 05-34A: Hard, poor method. Don’t remember the method. MB: What difference did VisualDx make in an aspect of patient care? (Such as change diagnosis, avoid or prompt a referral, patient communication?) PCP 05-34A: I used it once with patient present; patient like the images and information. MB: Could you describe a time (or 2) when it did make a difference? What was the difference? PCP 05-34A: No MB: Could you describe a time when you hoped it would make a difference, but did not? PCP 05-34A: No MB: Do you use VisualDx now? If yes, what prompts you to use it? Or if no, why not? PCP 05-34A: No MB: What other information resources did you use then or do you use now for evidence for skin problem? PCP 05-34A: I used textbooks available at the clinic. Fitzpatrick’s maybe? MB: What else you would like to tell me about using clinical information resources relevant to dermatology or skin problem or relevance to evidence-based medicine? PCP 05-34A: Relevant to evidence-based medicine, I use UpToDate except for derm, for which I use textbooks, but the textbook is now missing from the clinic. MB: Thank you so much. This is very helpful. Please answer on the paper these structured questions. [Unfortunately, the recorder did not record, but I was taking notes. MB] Time_end__1:40 _____Length of session 10 min, _________________________________________________________________________________________________________________________________ Name: Files\\int_id\\PCP 06-15A PCP 06-15A Interview #6 3/7/2018 Time: 4:50 pm to 5:05 Location: DML Med Hist. Room Interview #6 PCP 06-15A Resident Internal Medicine, Active Group MB: So, could you just say your name and where you are currently working? PCP 06-15A: So, my name is [Name], cardiology fellow here at University of Vermont Medical Center currently. MB: Great, thank you. And, so, not sure if you remember but just to, kind of, refresh your memory, you were in the information for skin problems study for about 8 months between November 2015 and August 2016. So that's just about two years ago, or more than two years ago. And you were in the arm that used VisualDx for when you saw a patient with skin problems. And then we called, we identified your patients and called them to ask them questions about their experience. So, do you remember your experience of participating in the study? What do you remember about it?  PCP 06-15A: I remember it somewhat. I basically didn't have a lot of clinic, I was the chief resident that year, which meant that I worked in clinic for one session per week in primary care. Before the study, I think my standard practice had been for rashes that were not immediately obvious to me, I already used VisualDx as my go to resource. So, this really wasn't a major change of practice for me. I would say that I don’t think I saw a lot of people with rashes, which means I feel like I went through this maybe a handful of times, definitely less than 10 during the whole year.  MB: But you did use it sometimes during the study? PCP 06-15A: Yes, yes, I did. MB: So, about how many times during those months?  PCP 06-15A: I think it was some number less than 10, but I don't remember. MB: So, how useful was the content of VisualDx for you? PCP 06-15A: I think it's very useful for me, because I'm not someone who already is very familiar with rashes; it was an area I was weak in and I feel like VisualDx was an important tool to make sure I had things on my differential that should be on my differential. And I'm not currently doing primary care, but if I were I feel like I would want to use VisualDx or something like it currently.  MB: Can you remember anything that stands out in your memory? Any time you used it, any case or? PCP 06-15A: To be honest with you, it's been a long time, I don't really remember MB: Okay. You might remember one as we go on. If you do just, you know, let me know. How did you usually access VisualDx? PCP 06-15A: So, I would open it up on the computer in the patient’s room. Oftentimes in the patient's room and go through it with them. Make sure I had gone through all of the data and sometimes I would also do it outside the room later. MB: So, it was always on a desktop computer? PCP 06-15A: Yeah, I didn't use it on a phone or something like that. MB: Or mobile? PCP 06-15A: ....just because there's desktop computers all over the place in the clinic so.  MB: So, I guess this is sort of like two things, there's like the content of VisualDx, like how hard or easy to use was that? And then there's the technology, but... Do you want to think about the hardness or easiness to use of the content? PCP 06-15A: I felt like I had already been somewhat exposed to how to make the selections and how to come up with the differential, how to pick where's the rash, is it macular is it papular things like that. And I felt like once I knew what I was doing it wasn't hard to use. If that's the question.... MB: Yeah and then the getting to it? Like how did you get to it? PCP 06-15A: I think I usually opened it up through an UpToDate link. ...to VisualDx. And to be honest I use UpToDate a lot. MB: Do you think that VisualDx ever made a difference in any aspect of patient care like a diagnosis or a treatment?  PCP 06-15A: I think during the study maybe once or twice but prior to the study, I don't know if that's fair game... MB: Sure, it's fair game.  PCP 06-15A: It has helped me before to make sure I have things on my differential. I do think it helps, I do think it.... Especially for someone who's not a dermatologist, it helps with diagnoses. MB: So, was it more likely a diagnosis that it helped with or a newer treatment? PCP 06-15A: I generally used it for diagnosis. I don’t think I used it to look up therapy of different conditions or anything like that.  MB: Okay. Did you ever use it for patient communication? You mentioned in front of the patient you often did it. PCP 06-15A: I think maybe a handful of times I.... You’d show them a picture of this and say this could be this or that but.... in that way maybe, but… MB: So, they weren't freaked out by the pictures or? Sometimes....  PCP 06-15A: No, no not really.  MB: You found it, I don't want to put words in your mouth, but it was at least useful to show the patient something?  PCP 06-15A: I think that was a useful part of it.  MB: So, could you describe, kind of trying again for an example, a particular diagnosis or was it more likely to be a rash or? PCP 06-15A: I'm sorry, it's just it’s been more than a year.  MB: Ancient now. And did you - was there a time that it disappointed you? Was there anything about it that just did not help? PCP 06-15A: I just remember that sometimes I would play with it and change the setting or you'd say oh-this person was itchy or not itchy, and you'd get somewhat of a different differential because you know you’d have 4 or 5 or 3 of this [a bit garbled]. And I did think that, I had to be kind of careful with how I made my selection. But I, just given that I don't do a lot of derm, I just don't remember times when it completely missed anything or something like that. MB: So, was it in general factual? As far as you were concerned?  PCP 06-15A: Yeah, in general it was good.  MB: But sometimes just didn’t apply to the patient or to the? PCP 06-15A: Well I mean for people who I used it on, who had dermatologic conditions, it was useful.  MB: What other information sources did you use then or would you use now if you were looking for evidence for skin problems? I guess it would be then, you know.  PCP 06-15A: Well back then, just if I thought of something I'd look it up on UpToDate and see if the pictures, descriptions matched and, the other important thing is what someone else in the office would think. That's very frequent that I would say, “Can you look at this?” Well not, given I didn't do it a lot but often if I didn’t know what I was doing on something I would do. I do think as a relatively new PCP at the time, I had kind of a low threshold to biopsy things. Just in case I was wrong, like tissue makes the diagnosis on a biopsy. So I guess that's an alternative way to make sure you know what you saw, just biopsy it.  MB: Good. So, UpToDate, VisualDx and a biopsy.  PCP 06-15A: And colleagues.  MB: Colleagues, yes. What thoughts do you have about using clinical information resources relevant to dermatology or any other conditions? Do you ever look anything else up? PCP 06-15A: I mean a ton. It's very important in every medical field to have something to reference. Especially in areas you're a little weaker on.  MB: Would you call that like practicing evidence-based medicine? Is that relevant to the idea of evidence-based medicine? PCP 06-15A: Well I think it's relevant when you don't know all the evidence well that it's useful to find someone else who, or some other source that, has reviewed and created a useful tool for you that can summarize the evidence for you. MB: Anything else you'd like to add or?  PCP 06-15A: Oh, you know what, this was after the study but I remember a case- there was a lady on the cardiology service who basically kept complaining, what she was basically complaining was that there was something on her tongue. And they talked to the intern and they talked to the resident and finally as a fellow someone asked me "Can you go look at it?" And I looked at it and I thought I knew what it was and I put all of her stuff into VisualDx and then I made a diagnosis. And people didn't quite trust me and the next day they called derm. And the dermatologist took me aside and said you know you already and he talked to the team and said you should have trusted him. You got it right yesterday. And it was this person had geographic tongue. So, that helped.  MB: What was it? PCP 06-15A: Something called geographic tongue. It's this benign condition that some people have and it goes away... MB: Geographic tongue? PCP 06-15A: Geographic tongue.  MB: Like t o u PCP 06-15A: Yeah MB: Tongue. And it's a benign condition? Well....  PCP 06-15A: So, we answered that question for this lady and, not cardiac but....  MB: Yeah, good for you. That's an example! PCP 06-15A: Yeah PCP 06-15A: I mean unfortunately they still asked dermatologists to look at it but.... VisualDx helped us make the answer, or get the answer.  MB: So, it.... She didn't get all the way to a dermatology appointment?  PCP 06-15A: No, they just asked the dermatologist to kind of look at it  MB: So, there was no actual referral? But...  PCP 06-15A: I mean a dermatologist saw this patient as an in-patient. I mean the resident did. I don't know if that like qualifies as a referral. MB: Was a consult. But you were vindicated. That’s great. Well, those are basically my questions. This little short answer questionnaire, which basically asks a lot of these same questions but just in the short answer. And especially since you aren't doing it now, it's even shorter.  __End of semi-structured interview______ Transcription start time: 18:01 Transcription end time: 18:49 Recording length: 14:36 _______________________________________________________________________________________________________________________________________ Name: Files\\int_id\\PCP 07-21A PCP 07-21A Interview Interview #7 Date 3/9/2018 Time: 9:00 am – 9:20 am Location Dana Library Medical History Room PCP 07-21A Resident, Internal Medicine, Active group Recording length: 17:03 min. MB: So, again, thank you and could you just say your name and what department you're in? PCP 07-21A: Yes, [Name], Internal Medicine and I participated in this study probably while I was a resident.  MB: Yes, so you're not a resident anymore? PCP 07-21A: No longer, I'm a chief resident now.  MB: Chief resident? Congratulations. PCP 07-21A: Thank you. MB: So, you were in the study, just to remind you, - it was two years ago - for seven months, between January 2016 and August 2016. So, it was about seven months. And do you remember what your role was?  PCP 07-21A: So, I would have been the end of a second year, and the beginning of third year of being a resident.  MB: And so, you were in the [Name of Practice] primary care? PCP 07-21A: Yes, doing continuity clinic every five weeks.  MB: Right, right. So, you were in the arm that was supposed to use VisualDx. PCP 07-21A: Yes. MB : And so, can you describe your experience just briefly being in the study? And what you recall of what your role was.  PCP 07-21A: Yeah, I guess dermatology is not- we don't get a ton of training on dermatology -, so I was kind of using VisualDx now and again prior to the study, so it really didn't, - I was glad that I was in the intervention arm so I - didn’t have to change what I was doing because I use that as an easy resource.  MB: Yeah. And just to remember, we identified your patients in the electronic health record during the period that you saw [them] for a skin problem. Then we called them up, and if they consented, we asked them questions about the care, how the rash had turned out, or whatever it was. And so, we did consent about 50% of the people that we called. Just for your information. So, what was your experience using VisualDx, did you like it? Did you have opinions of it?  PCP 07-21A: Yeah, I guess the format works, it's always hard to plug in symptoms and signs that really match exactly what you see on a patient, and sometimes like I would go through the VisualDx just as an exercise to see if it could come up with something common like molluscum contagiosum or something like that. And I would always find that I would always have a differential that was different on VisualDx. Like it wasn't complete. Like I liked to look at it because it would get me thinking about a broader range of differential diagnoses but I would usually have something that was separate on my differential than what came up on VisualDx. You know you put the symptoms in, the signs in, location, and then all the pictures you would get would look totally different than what you would see on the patient.  MB: I noticed that sometimes, too.  PCP 07-21A: And usually they'd be more impressive rashes on VisualDx. Whereas the thing you were looking at is actually quite subtle. You know, and you don't know what to call it or how to label it but it's a subtle finding compared to everything on there.  MB: Well so did you use it during that study period?  PCP 07-21A: I did but I don’t know that I had that many rash patients, or skin lesion problems.  MB: But you did use it? Can you think of like months, number, like approximate? PCP 07-21A: Like how many times in a month?  MB: Yeah and then do a little multiplication over the ... PCP 07-21A: Maybe like three, two or three? MB: So three times, in seven months...  PCP 07-21A: 20 times, yeah or something like that. MB: And so, did you find it hard or easy to use? PCP 07-21A: I thought it was pretty easy to use, sometimes there were times that I would get down the path and it wouldn’t come up and I realized I missed a step early on or something. Because you know you have to put in, like, I don’t remember what the first prompt is, like...  MB: Well if you go to the differential, yeah, you start putting in symptoms basically. Did you usually choose the differential builder?  PCP 07-21: That's what I would use most frequently.  MB: Because you can also just put in a diagnosis. PCP 07-21A: And sometimes I would do that, because, if I had a good idea of what it was. But I use UpToDate probably more frequently to be honest for, like, management.  MB: So, was the content that you got, how useful?  PCP 07-21A: I didn’t use it that much to be honest. Like I don't think it ever changed my management or diagnosis. I mean that's my sense, I mean I don’t know… Maybe it did. Maybe by being exposed to that information it did.  MB: Well we can think about that a little bit more. And just quick, did you usually use a device through handheld...? PCP 07-21A: No through Epic and the bookmarked tool on there I think. MB: And getting to it through Epic, was that, hard, easy? PCP 07-21A: Easy. Question 5 MB: Coming back to something you alluded to. So what difference did VisualDx make in any aspect of patient care? PCP 07-21A: Again I don't think it changed any diagnosis that I made, but it helped me think more broadly about what it could be.  MB: In the sense of expanded your differential? PCP 07-21A: Yeah, my differential, things I wouldn't have thought of. MB: So, did it help, let's see, change a diagnosis?  PCP 07-21A: I don't think so. MB: Avoid a referral?  PCP 07-21A: I don't think so. MB: Prompt a referral? PCP 07-21A: I don't think so, but I think part of that might have been driven by the fact that we don't see that many skin problems. You know the things, I rely a lot on colleague expertise and just like, you know we don't do that much primary care, a lot of our residency is based on in-patient stuff so if I, if there was something I was worried about or I didn't know I'd look in VisualDx but then I'd really rely on like "Hey do you mind coming to look at this?" and letting me know what they think.  MB: With a colleague?  PCP 07-21A: With a colleague, with one of the attendings.  MB: Did you ever use VisualDx with a patient present in the room? PCP 07-21A: Not usually, because I'd do my differential builder and there'd be so many different things, and like I said usually none of them looked exactly like what they had.  MB: Yeah, just my opinion, sometimes they look worse,  PCP 07-21A: Right, yeah.  MB: So, can maybe be scary... PCP 07-21A: Yeah, right, exactly. Question 6 MB: So just thinking, can you describe a time or just remember a time when you looked up VisualDx, you used it? PCP 07-21A: This was probably outside of the study period.  MB: It's alright. PCP 07-21A: Just a pediatric rash, like more a family member had a child with a rash and I'm not, pediatrics is totally out of my wheelhouse and dermatology is out of my wheelhouse so I used it then. And I think it probably helped me confirm what I thought it was. But again, I went to a friend, took a picture and texted a dermatology friend.  MB: Can you think of a time that when were disappointed or you were kind of looking for something and thought? PCP 07-21A: Like with a specific diagnosis you mean?  MB: Yeah or a patient or... PCP 07-21A: It's hard to think of a specific patient... I'm trying to think of a specific diagnosis but again, my sense was it didn't really change my opinion, it didn’t sway my opinion in any way, it was more like these [images] don’t look like this rash at all even though I followed the differential builder. So,  MB: So, you had to find something else?  PCP 07-21A: Yeah. MB: So, I don't know, not to put words in your mouth, but that sounds disappointing actually.  PCP 07-21A: Yeah. MB: Do you use VisualDx now? PCP 07-21A: Yeah occasionally, like that example was after the experiment.  MB: So, what would prompt you to use it? PCP 07-21A: Just any rash that I don't understand.  MB: What other information resources did you use then or do you use now for evidence about skin problems? PCP 07-21A: Yeah so, dermatology books, actually. We have a few of them in our clinic. And then I think I rely most heavily on colleagues. Primary care physicians who have more experience in like out-patient care dermatology as well as colleagues who have gone on to be dermatologists.  MB: Great, and are they, how satisfying are they? PCP 07-21A: Very. MB: So, if you have any other thoughts about using information sources relevant to dermatology or skin problems, are they, is there any difference using them or trying to look up something for skin problems, compared to another condition. Like is dermatology kind of, or skin problems, harder, easier? PCP 07-21A: Way harder. Well, way harder because we just don’t have the exposure, also so much of it is how it looks rather than like a description of symptoms. So, I think something like VisualDx is totally necessary, you need the pictures and distribution and things like that. There a few categories on VisualDx, like I don't think they have a distribution category, where it's like scattered or few, or like clustered or something. It's always like legs, trunk, and it's not always totally symmetric like that so, I'd always have to fudge my way through. I’d be like, I guess this is the best answer for this question.  MB: Does it, is it a question of fitting, like how does it fit the patient? PCP 07-21A: Right, exactly and most of the time it wouldn't fit, you know, perfectly. And the results that I would get in that situation I didn't think were consistent with what I was seeing.  MB: So where does a resource like this, or maybe even other resources, fit if you're trying to do evidence-based medicine in practice? PCP 07-21A: Where does VisualDx fit?  MB: VisualDx or other information... PCP 07-21A: Dermatology?  MB: Yeah, thinking more in dermatology, but... You mentioned the colleagues, the UpToDate, the books... Are you, do you think, it's, I don’t know, what is your opinion on this type of resources for evidence based, practicing evidenced-based medicine.  PCP 07-21A: I definitely think it has a place, I just wonder how broad the library is of their diagnoses. Because there are plenty of benign sort of things that are hard to put your finger on in the outpatient setting and they can look, they can present in so many different ways. I guess one thing - yeah, I think it's a great tool for people to have as a starting point if they really have no clue where they're going to build their differential from, but I just found it hard to rely on it for my diagnosis. So, I think it's a good starting point. I think one of the features that I really liked was like, once you had a diagnosis you could click on it and see a bunch of different examples. So you see like a more broad spectrum of the presentation, but they all still seemed kind of extreme and not so subtle.   MB: Well this is really, very helpful, and so you couldn’t really describe a time that it made a difference. PCP 07-21A: No, not that I can recall.  MB: If you think of one, ever. PCP 07-21A: You mean like after the study if it happens again?  MB: Well yeah, but even like when you're walking down the corridor as you leave here and you say "Oh I just remembered about this time". If you could send me a text of what the topic was.  PCP 07-21A: You don't, did you keep track of the encounters? Because that could jog my memory.  MB: Well, in a way we weren't allowed to, so as far as patient information goes, we could look in the health record and match a clinical encounter up to you because you were in the study, that kind of fit. The only, so we could only use it for recruitment of the patients. Once we talked to the patient, it's like we had to forget everything that we saw before. But I can tell you I think we recruited probably five or six of your patients.  PCP 07-21A: Okay, good. Yeah I will keep it on my mind.  MB: Alright, that is it. Thank you so much! PCP 07-21A: You're welcome.  MB: Have a chocolate or two.  PCP 07-21A: Thank you _______________________________________________________________________________________________________________________________________ Name: Files\\int_id\\PCP 08-43A PCP 08-43A Interview # 8 Resident/Internal Medicine/Active /Not in primary care currently Date: 3/15 start: 2:30 pm End 2:55 Location: This interview was conducted by phone and recorded using Snowball microphone and Sony recorder and saved as mp3 on Audacity software. Note: Phone call dropped twice MB reviewed and corrected transcription gaps and confirmed pityriasis rosea diagnosis with PCP because it was inaudible in the recording. 5/25/18 MB: I can just summarize it really quick. So, we're asking you to take part in this interview to help better understand your experience using VisualDx or other clinical information sources in the course of caring for patients with skin disease, especially during that skin problem study that was in 2016 that you were in. PCP 08-43A: Okay. MB: Okay, and your participation is voluntary and, so, I'm recording the information that you share with us. So, do you agree to participate? PCP 08-43A: I do. MB: Oh, thank you so much So could you just say your name and where you are currently working, or your current role? PCP 08-43A: My name is [Name] I'm currently a hospitalist at [Name of Hospital]. MB: Right and were you a resident during the time of that study in 2016? PCP 08-43A: I was, I was a third-year resident at UVM, in Medicine. MB: Right so what do you remember about being in the information for skin problems study? PCP 08-43A: So, I remember using VisualDx in clinic primarily when I saw patients with any kind of rash or skin condition I would, you know, use that, use VisualDx as a search engine and get information for those patients. MB: That's great, that you recall that. And just to refresh, you were in the project for seven months, you started participating or at least consented, in January 2016 and the project was over in July, so it was about seven months that you were in it, theoretically. So, what was your experience using VisualDx, what was your opinion of it? PCP 08-43A: The thing that I found it most helpful for was to broaden my differential to give me ideas. I would often look at a skin lesion or rash and have an idea of what it might be about or a couple of ideas and then I would put, put information into the VisualDx search engine and it would broaden my differential and sometimes completely change my initial opinion. Primarily what I used it for, I did, on a few occasions, have no idea what I was looking at in the patient and used it to like to try to figure it out. So I think mostly I was using it for broadening of the differential. I recall, you know, finding the pictures, not as helpful as the information that was there, because the pictures were generally of really severe conditions and generally not what we see. We're seeing a more minor version of whatever it is they're showing us in the search engine. MB: Yeah, do you, so it sounds like you did use it during the study. Do you kind of have any idea how often, like how many times in a month or something like that? PCP 08-43A: Yeah, I mean I used it, so we're in clinic for five days out of every month, like one clinic week out of every month out of every five weeks, and I used it every time. I'm not sure that I used it every day but I probably used it close to everyday that I was in clinic, so like five times a month or something. MB: Okay, so if you were in the study for six or seven months, maybe it was 30 times? Maybe something like that? PCP 08-43A: Right. MB: So, what prompted you to use it? Or not? Because probably you didn't look up something every time you saw a skin problem. PCP08-43A: I guess... MB: Or maybe you did. PCP 08-43A: I mean, I think I probably used it close to every time I saw skin problem, I mean unless it was a super obvious, something that I, you know, was absolutely sure I knew what it was. But even then I would look it up and use it to get ideas for treatment recommendations, like workup and treatment. So I guess, I don't know, I knew I was in the study, I thought it might be helpful to use it, it wasn't onerous at all to use it. MB: That was my next question, how hard or easy was VisualDx to use? PCP 08-43A: I mean, in general I would say it's fairly easy. Like sometimes the way that it - I haven't used it in a while so I'm just remembering - it took- . There was a small learning curve to like figure out the best way to use it. So that I wasn't getting - I mostly got a lot of extraneous information and things that I wasn’t - weren't appropriate for what I was looking for, but I just had to sort of - so some of that time using it was wasted -- like wading through a bunch of baloney. You know, I think in general it was fairly easy to use. MB: And was the content itself, you know, reasonably accurate or useful? PCP 08-43A: Yeah, I mean, I think I found it accurate enough for my purposes. I think it was my major use that I got out of it was giving my more ideas. It was really broadening my differential and making me think of things I hadn't previously. And then, you know, it would give me some ideas for treatment options. MB: And how did you access it usually? Was it mobile device, or on the network? PCP 08-43A: I always accessed it via Prism in the room… Or in my work room, one or the other. MB: Yeah, and was that access fast or slow? You know, any difficulty downloading the images? PCP 08-43A: I don't recall, I don't recall anything like that in particular. MB: Okay. Yeah, I mean other people have said that, as far as that went, the technology worked pretty well. So, did, what difference, if any, VisualDx make in an aspect of patient care, like you know, changing a diagnosis or avoiding a referral or anything like that? .... [dropped telephone line, MB called her back, PCP08-43A answered, and the interview was resumed.] MB: I don't either! I’m sorry. We're almost done anyway. So yeah so, I was asking you, did VisualDx ever make a difference in an aspect of patient care like changing a diagnosis or avoiding or prompting a referral or in patient communication? PCP 08-43A: Yes, I would say frequently. MB: So, which, could you describe a time it made a difference and what the difference was? PCP 08-43A: I would say, again the most frequent difference was giving my more ideas so that I could, sort of think through. Like I would look at something, and think, “I don't know, that looks sort of vaguely fungal” and then I would look at VisualDx and it would give me four additional ideas with additional sort of history associated with those ideas. So, then I would talk to the patient more, come up with a diagnosis that I thought was likely enough and to act on. So, I think it changed my diagnoses. I think it changed, probably my rate of dermatology referrals because I willing to, you know, diagnose skin conditions with somewhat better - I don't know - confidence, more confidence to act on those diagnoses. MB: Can you, if you can remember an example... PCP 08-43A: Oh boy… MB: I know it was a while ago. PCP 08-43A: I'm trying to think of a specific example. I feel like I had this young woman with these, this rash on her back, I had thought - I looked at it and thought maybe psoriasis, and then you know, used VisualDx and it looked, based on what I found on VisualDx, it looked more like pityriasis rosea [10:50m. on recording] so I, you know, went that way instead. I have a memory of doing that. MB: Do you think that it might, like, helped the patient, or I don’t know, be relieved of their symptoms faster or better. So, in other words if you changed a treatment... PCP 08-43A: Yeah, sure, if I was diagnosing people more accurately and thus treating them more quickly, than certainly. Question 5 MB: Can you think of a time when you hoped that it would make a difference but it didn't? You were just disappointed in the thing? PCP 08-43A: In VisualDx? MB: Yeah. PCP 08-43A: Oh yeah, I'd say like, just as frequently as I found that it was helpful, I found that it was not helpful at all. I couldn’t find, you know anything new, it didn't help me with new ideas. I, you know, or it wasn’t, the format wasn't helping me out. My memory of that is mostly like, I would be looking for examples of the rash that I, I'd be describing the rash I was seeing like If I didn’t have any idea what it was, and then finding a bunch of stuff that was you know, that I, with my experience, looked at and said, “No it's not that, it's not that, it's not that, it's not that.... and it wasn't helpful in that way. I'd say just as often unhelpful and helpful. MB: So about half the time. That's something. PCP 08-43A: Yeah Question 7 MB: Do you use VisualDx now? Ever? PCP 08-43A: I... boy... very rarely. I'm a hospitalist and, I don't see a lot of rashes in the hospital. And occasionally, I suppose I've used it more with learners like in my per diem work at UVM than I have here. Still really rare. (Not in Primary Care) MB: Okay. Do you have, what resources, just as, curiosity, at [Hospital Name], do you have access to information resources? PCP 08-43A: I don't even know if we have VisualDx. MB: I think you might have it. And you have UpToDate there, of course. PCP 08-43A: Yeah, we have, oh yeah. it is on here, Micromedex, UpToDate, VisualDx and the cardio, the EKG. PCP 08-43A: Yeah, so it's here but I haven't ever used it here. MB: What other information resources did you use during the study? PCP 08-43A: I used UpToDate quite frequently. And I use Micromedex quite frequently. MB: And did you use, you know, would you have used those resources for skin problems, or some aspect of skin disease? PCP 08-43A: Yes, and I don’t think my use of VisualDx changed my rates of use of those other resources. MB: Okay, yeah. What do you think about your education or preparation in dermatology or skin disease? PCP 08-43A: Poor but average for Internal Medicine. MB: And what would you say your level of confidence in diagnosis was maybe during the study? PCP 08-43A: Poor... but average for internal medicine. MB: Exactly. And what about the relevance of VisualDx or other resources like it, like UpToDate, Micromedex and others for the practice of evidence-based medicine? Do they help, hurt, are they irrelevant? PCP 08-43A: I think that they helped, I think that UpToDate and Micromedex helped significantly, I don’t, I guess I didn’t develop an opinion about VisualDx because I wasn't looking at evidence, that the information. I guess I hoped that the evidence, or the information included there, that I was accessing was evidence based. MB: So, you mean like in UpToDate for example, they will often tell you the grade of evidence? PCP 08-43A: Right and they have references there. If there are references in VisualDx I don't recall seeing them, and I never pursued them. I didn't ever like click on them and go that way. I was always using it in a very quick - Like almost always in the room with the patient, super-fast, and not like spending a lot of time reading. Like I said I was using it to broaden my differential primarily, it was sort of a quick review type thing. MB: Okay, well that's very helpful. These are very insightful thoughts that you're having, so I appreciate them a lot. Is there anything else you'd like to say about the trial or using clinical information sources relevant to skin problems? PCP 08-43A: No, I guess I mean I had the thought multiple times over the seven months that I was glad I was in that, the intervention arm of the study, because I was glad to be using it. And so, you know, I, it was sort of motivation for me to use it more and see how it changed my practice and so I appreciated it. ---------------------end of interview ---------------------------------------------------------------------- Recording length: 22:18 _______________________________________________________________________________________________________________________________________ Name: Files\\int_id\\PCP 09-46A PCP 09-46A Interview transcription Interview date: March 20, 2018 10:15 a.m. to 11:00 Dana Library Med History room; recorded Interview # 9 PCP 09-46A Family Medicine/Attending/Active MB re-listened to recording and edited 5_23_2018 MB reviewed information summary with interviewer and received verbal consent. MB: So could you just say again your name and where you are working or your current role? PCP 09-46A: My name's [Name] and I'm working at [Name of Practice], Family Medicine, I'm a primary care doctor.  MB: and you were in the information for skin problems study, and it turns out you were in it for about 6 months between February and the end of July in 2016.  PCP 09-46A: Oh, okay! Yes! MB: Which is about two years ago. PCP 09-46A: Okay, yup. MB: And you were in the intervention group, what do you recall about the study? What was your role or just in general what do you recall?  PCP 09-46A: My recollection is that I was in the group, more the intervention group or the group who could use VisualDx as a tool to help my clinical decision-making regarding skin diseases.  MB: Right, and does anything about it stand out in your mind? PCP 09-46A: About the study? MB: Yeah, or just an overall impression. Question 1 PCP 09-46A: I thought it was neat that we're doing a study on utilization of tools for primary care providers to help . I think that there's a lot of potential for technology to aid in health care decision-making. And I like the visual element of a tool that helps with skin diseases because it's one of the trickiest parts of diagnosis for medicine is when you have a rash and you're not sure what it is, "Where do I go next?" MB: Yeah. So, what was your experience using VisualDx itself? And did you use it during the study?  Question 2 PCP 09-46A: I think I, it's hard for me to know because it was a long time ago now, but I've used VisualDx a few times, you know, certainly a handful of times over the course of the past couple of years, so hopefully I used it during the study.  MB: Well, see if you think back, in those maybe six months, did you make a conscious effort to use it when you had a concern about a skin disease?  PCP 09-46A: Yeah, I think it was a little more in my consciousness in terms of another tool. I think my first tool during those first, because I graduated in 2015 and started September 2015, so....  MB: You graduated as a resident? So, you finished residency?  PCP 09-46A: Into being an attending.... and I was working with a person who was very, - next to a person who was a very skilled, older practitioner. So often times my first recourse would be going to him, describing it to him, maybe asking him to take a look. So that may have decreased my use. I use it a little more now that he's retired and I'm on my own. And in addition, I have a few books and sometimes I look there, but that's probably my last resource.  MB: So, was the content in VisualDx useful to you during the study?  Question 2b. Useful PCP 09-46A: Hit or miss, sometimes I found what I was looking for and sometimes it seemed like either I didn’t have enough. Seems like a lot comes out - so things that are very rare as well as things that are sort of more of what I’m looking for. So, it's hard to sift through and find what you're looking for.  MB: So, you just used the word hard, would you say content was hard or easy?  Question 2.C Hard/Easy PCP 09-46A: I mean it's an easy interface to use but the output is not always coming up with the most common causes... Or the things I'm most likely to think it is. Sometimes it comes out with very rare diseases that I'm pretty sure are not going on, or very red flag or concerning diseases that I don't really feel are on my differential. So, I'm not sure if I’m just not putting in enough or....  … If I just don't know enough about those rare diseases that.... maybe I should have on my differential...  Question 3 MB: So how did you usually access VisualDx?  PCP 09-46A: Well that was really easy, it's really, I don’t know if I have somehow preferred it, but it’s on my dropdown from Epic, [electronic health record} very easy to access. MB: Yeah. Did you ever use the mobile access or use it at home? PCP 09-46A: I didn't. No, I don't carry my phone with me when I'm at [Clinic Name], moving from room to room, so that limits it. I don't think I've ever used it from home, but I try to minimize -. When I’m making a diagnosis, I’m making it right in the room, or in the office, not after, because it's too far removed. MB: So, when you're in the room with the patient, you’d use that computer? And then in your office. PCP 09-46A: Yes, and in my office after if I'm struggling with still trying to figure out what's going on, or I want to include a couple more things to think about for contingency plan so,  sometimes I would access it there. MB: So, can you think in any difference, a time when VisualDx made a difference in an aspect of patient care? In other words, maybe, did you come up with a different diagnosis or you know, did it prompt you to refer to dermatology or not. Did it prevent a referral?  PCP 09-46A: I can't think of a particular instance where it clinched it for me or made a clinical decision distinction or difference. It was more of a tool that I used to augment, you know whatever I was looking into, otherwise. Or I had a... sometimes I have an idea of what probably is going on but I want to make sure I expand to see whether there's anything else that might be going on. And sometimes I have no idea what, what this is, and I use it as a way to see if anything comes up that might be thought-provoking or might lead me in a certain direction. But it's hard for me to tell whether it changed anything. If I don’t know what's going on, I have to find more resources or refer to dermatology or do a biopsy and if I do, I usually was - So it's a tool but I don’t know if it was a fulcrum from which I made a difference, or a change.  MB: Well, if you think of a time, if you just kind of, while were talking, if you come up with a time, because you know… You might also remember a time when it disappointed you. It sounds like you can remember those too, when you might have thought you might get something and you didn't. PCP 09-46A: It's true.... it seemed a little more that way [disappointing] where if I have no idea what’s going on and I type in rash on the shins and.... papular, purple, you know, in a middle-aged man, you know, I'm kind of hoping it will just come out with what it is....some magic, perfect match, and often times, it's like oh it doesn’t look like that or it doesn’t - I'm trying to think of a particular instance...... [Pause] [Positive change example] Well, now that I'm thinking about it, there may be one instance where I had a patient who had a rash that somebody had diagnosed as cellulitis and then somebody else had diagnosed as something different. And then he went to urgent care and the doctor said "you know, I think I know what this is, erythema nodosum, I believe it was called? And it comes as part of sarcoidosis, or it can come as an autoimmune disease. And it had these big purple nodules that almost look like Lyme - Oh, one person had diagnosed it as Lyme disease and he [the patient] had undergone treatment for that - and she [ Urgent care doc], said "you really need prednisone and you need to be evaluated for, you know, consider sarcoidosis in this diagnosis." The prednisone [had] really helped and so he came to us: it was a resident and myself, so I was supervising. So, we already had that in our mind, like oh well, didn't improve with doxycycline, looked like it didn’t improve with an antibiotic, now prednisone has improved it. And I looked on VisualDx to be able to compare this rash, he had in the flesh to what they were coming up with, and that was rewarding. It looked like it was pretty consistent. And it opened my eyes to something that I didn't really know about clinically. So that was rewarding.  MB: So, it could have, well I'm sorry, I interrupted you...  PCP 09-46A: Yeah well it helped me feel reassured because I was the supervisor, so you know ideally, I want to make sure we're still continuing on with the plan, that whatever was outlined by the plan, our preconceived, what was stated before seemed like the right path for this person. And then we did an x-ray and the person did end up having sarcoidosis. So, other clues were helpful and that was a really interesting case. PCP 09-46A: And then one time I had a patient with some sort of folliculitis I think. But she had little nodules and I was, little pink papules scattered across her abdomen and I looked in VisualDx and I think in that case it was, you know, I was just hoping it would say "It is folliculitis! Or it isn't" and something like that, I'm not sure if it was that case or some other, but I couldn’t really find what I was looking for there, so, I guess two instances.  MB: Do you think, so, maybe in these two cases or another case, would have made that more, I guess, precise, or better diagnosis, do you think it could have led to the patient getting better faster? Or at least having some relief of symptoms?  PCP 09-46A: For VisualDx? MB: Well in other words, so you looked at VisualDx, maybe you modified a diagnosis or confirmed a diagnosis and might that, I know you don't know, might have relieved the symptoms of the patients, might the patient have said, "Oh yeah I'm better than the last time I saw “Dr. so and so" or something like that. PCP 09-46A: Oh, I don't know, it's hard to tell, I mean, possibly, yes... because I think we all draw from....  [Digression: description of the cluster randomized controlled trial methods ] MB: So, do you use VisualDx now? Sounds like? PCP 09-46A: Occasionally, yeah. MB: And what prompts you to use it? What, when would you say to yourself, "Oh, I think I'll look"? PCP 09-46A: If I have an uncertainty about what is going on with a person, certainly. If I'm in the room, it's easier to use it there versus to go out and get a book or talk to a colleague and ask them to come in, so the ease is kind of nice.  MB: Oh I kind of forgot to ask you, do you use it in patient communication? PCP 09-46A: Encounters? Oh yeah, I have. That's one of the most helpful parts of it I think. If you can find the - if you know what it is - you can find pictures easily, sometimes I just use Google Images, but I find that is much less helpful because it sometimes comes up with the wrong type of problem.  MB: So, what other information resources.... PCP 09-46A: Or UpToDate.  MB: Yeah, did you use then or now? PCP 09-46A: I use UpToDate a lot... It’s a little less helpful for skin stuff because there's only a few pictures in there.  MB: Yeah. And Google Images you said?  PCP 09-46A: [And Google Images] - Just if I know what it is and I want to try to explain it to a patients. So that's more for a patient communication tool rather than for my own knowledge. MB: And any other resources?  PCP 09-46A: I have two books that I use pretty rarely. I'm not sure their names, so there's "Skin Diseases" and.... MB: Is it print or electronic?  PCP 09-46A: Print. I bought it with my resident funds for CME. Again, it's just hard, when you don't know what something is. VisualDx is more helpful than a book because you're not going to just leaf through a bunch of pages.  PCP 09-46A: So at least if you're typing in, you know, the part of the body, you're coming up with something that's thought provoking, more so, it's more interactive than a book is. MB: So what would, just switching topics a little bit, what do you think about your education or preparation in skin disease? PCP 09-46A: So we had one attending in my residency who was, more interested in skin issues, and she was a good resource. I went to, she's done a grand rounds and we had some skin biopsy techniques and suturing lecture, you know, an interactive... We - I went to a course, just a general course for primary care doctors, "Dermatology for the Non-dermatologist", and that was helpful! MB: Since you were a resident?  PCP 09-46A: Since I was a doctor, attending.  PCP 09-46A: That was very helpful. So, I don't think I've - I still don't feel… It's an area that I feel I continue to work towards improvement versus some other areas where I feel very prepared and skilled to embrace the uncertainty. This is an area where it’s a little bit more "Well if I don't know what it is, we're going to have to try something, or we'll biopsy it, or we'll send you to dermatology" So I mean, it's still like we've got a good plan but sometimes when I look at it, I feel like I should have an answer for a patient of what this is and I don't always have that, so that's a little harder.... MB: Harder, yeah. What is the relevance of resources like this for your practice of evidence-based medicine?  PCP 09-46A: The relevance of it?  PCP 09-46A: I mean I think that it's helpful, yeah. Because it helps you pull some of the evidence in, in a visual way to kind of compare and see, rather just pulling from your prior knowledge, you can kind of.... MB: Does it help.... Yeah, dig in? PCP 09-46A: Move to a bigger knowledge base.  MB: What about applying with patient values? You know, that's a big one like.... PCP 09-46A: Push right now.  MB: Does it really relate to the patient at hand? Does it help you find an answer for that patient? PCP 09-46A: Gosh, I mean if it helps me find an answer than I would say yes that aligns with their values because they want an answer as well as to know what to do to make it better. I think that's a really interesting distinction actually in some patient encounters, doctors often feel like we want to make sure that nothing’s going to kill a patient or hurt a patient and that that's enough. Sometimes I found, either with family members or patients that you can kind of get an undercurrent of the fact that they are not satisfied with that. They want to know what this is and what caused it, am I making it worse? And so sometimes I feel that people continue to seek out other resources because they haven't found that answer yet. I don't know that doctors really are that tuned into the fact that they feel like, you know, reassured, "I'm not worried that you have bullous pemphigoid or something you know, scalded skin syndrome or and so, or you know, some skin manifestation of a cancer. So good, alright well, just use some steroid cream, moisturize, moisturize, moisturize see if it gets better" and the patient's still saying "well, I am super itchy and this is really affecting my quality of life. And what is this, am I allergic to eggs? Am I allergic to...." So, it's kind of interesting, I find that that’s when you see people going to a natural provider or a someone who's going to provide a little bit more of a service and.... Get to that psychologic anxiety or concern that might be underpinning it. So, I mean in that way, VisualDx sometimes if you can use a visual to show somebody and say "Oh this looks like really what you have" they gain a little bit more confidence, Okay. Because I think people come and they're worried for their health, they feel like "Is something wrong with me?" and I feel like sometimes we do a good job addressing it and sometimes we're able to entertain more uncertainty because it's not our bodies.... I don't know that's probably....  MB: Yeah... Yeah.... And so I guess, it, it helps with the patient feels better sometimes?  PCP 09-46A: Sometimes and you're able to show them visually something. MB: And sometimes. PCP 09-46A: Yeah, if you can't find it then....  MB: Yeah. Well this is, is there anything else you'd like to add?  PCP 09-46A: I don't think so, you know it is hard, the one thing I'm thinking of is, I'm trying to spend a little less time in front of the computer screen in front of patients. So, you know, I want to take a stock at the fact that the more time I spend staring at the computer screen while they're waiting for me, that might lead to some discontent so I think the only one things to do, if I’m using it, often times turning the screen a little bit, I don’t want them to think that I don’t know what I'm doing, but if I'm in the point that I can communicate to a patient with a picture, then I should turn the screen so then I feel like I’m engaging with them and that I’m not just, you know, routing them through Newark kind of thing while they're just standing there with their baggage, like “oh boy, when is this over?” MB: Save me...... So, this, these were my semi-structured questions. PCP 09-46A: Okay. MB: So, this is just a quick, short answer survey... PCP 09-46A: Okay. MB: That, so you see on here this page is what you can remember during the study and this these two short pages here are since the study. PCP 09-46A: Okay, should I answer it now or?  MB: Yeah, I think you will answer it in one minute.  PCP 09-46A: Okay. Recording length: 23:57 _________________________________________________________________________________________________________________________________________ Name: Files\\int_id\\PCP 10-01A PCP 10-01A Interview Interview #10 Mar. 22, 2018 Location: PCP office 3:00 p.m. -3:40 p.m. Recording length: 34:30  Attending/Internal medicine/Active Staring at 2:19 of the recording. MB: So, could you just say your name again, and where you're currently working or what your role is now?  PCP 10-01A: [Name] I work down at, I think it changed its name since the start of the study, and there’s been so many name changes. So it's now officially [Name of Clinic]. [Moving recorder for better sound] MB: Good. And, so, could you describe your experience of being in the study. PCP 10-01A: Just very generally?  MB: Just generally, although I was going to remind you, I was distracted, that you were actually in the study for 9.5 months, in other words between October 2015 through July of 2016.  You were in fact our index provider participant. Number one. PCP 10-01A: Wow, do I get extra chocolate?  MB: You might, although I'd have to go over and get it. But... PCP 10-01A: Yeah, it's hard to believe it myself. Okay.  MB: So, it was quite a while and it was kind of two years ago, so, what do you remember about being in the study? PCP 10-01A: Just generally remember?  MB: Yeah. PCP 10-01A: So, I think the VisualDx is something that I maybe had used once or twice before then. And so, certainly tried to make a more concerted effort to use it more often. It was great in the study to occasionally get those reminder emails to remind me to use it. And... Yeah, I can't remember exactly how many times I used it, I think I used it a little bit. You know, I don't see lots of rashes all the time, for a variety of reasons, but I was most typically to use it when I was supervising the residents because we see a lot of acute care visits at that time. So, we would occasionally pull that out. I even pulled it out two weeks ago, which had been the first time I'd used it in a while. For, believe it or not, one of my friends sent me pictures of her son's face, and said "What is this rash?" I thought I knew what it was, but I was like "I'm not a pediatrician", so I went into the VisualDx and started punching in all the information and it agreed with me, and in the end that's what he had. But it was nice to know, since a friendship was on the line, that I had the right diagnosis.  MB: Yeah, what did he have?  PCP 10-01A: Fifths Disease. Fifth, like after fourth.  Yeah, fifths disease. It's more like, yeah, after fourth, so fifth. Or you could say parvovirus B19 infection. MB: Okay, parvovirus. Afterword I, I actually had a very good experience using it too, to diagnose a friend’s difficulty, so, anyway… PCP 10-01A: Cool. MB: So, let's see, you made a little bit more of a concerted effort to use it, you appreciated the emails, which I appreciate that you recall that, and so you were in the study for about 9 months, can you think about how many times you would have used it during a month?  Question: 2A PCP 10-01A: Well, I would say, probably at the most, once a month, at the most. It was probably less than that, would be my guess. And it's probably mostly because of the number of rashes I see... Again, I don’t see that many rashes, and then I'm going to be more likely to use it when I don't know what's going on, so then it's going to be a rash where I'm really not sure. And then I would remember to use it, which I wasn't perfect with, but the emails certainly helped. So, when you, all those things come into alignment, if I were to make a real guess, I probably used it once every other month, would be an approximation.  MB: So maybe 5 times?  PCP 10-01A: Maybe, I bet you guess probably know though, you probably tracked it.  MB: We can't know, we didn’t. We can't track it.  PCP 10-01A: Oh well then, I used it like thirty times a day!!  MB: No, we want you tell the real thing here, there are no right answers. PCP 10-01A: Okay.  Question 2B MB: So, when you did use it, what prompted you to use it?  PCP 10-01A: Certainly, a rash and a rash I didn’t know, those would be the big ones. MB: And what, how was the content of VisualDx, was it useful or not in those times? PCP 10-01A: It's useful for generating a differential diagnosis, for sure. The challenge I think that I experienced with it, is it, and this is fairly fresh in my mind cause I just used it, [Mar.2018] was it was a little hard to get, figure out how to get the right data into it, I remember staring at it saying, you know when I opened it, when I was trying to figure out what this rash was, -is "Where do I put the information in?." So, it wasn't as user friendly for data input, and then on the output, it definitely casts kind of a broad net, there are certainly things that show up on the differential diagnosis that aren’t really on the differential diagnosis. You probably want a program that errs on the side of too much rather than too little so, yeah. But if the data entry was easier I think it would be a better tool. MB: Actually, have you noticed a difference in data entry between when the study was, and now?  PCP 10-01A: I think it might be harder, yeah, because I don’t remember it being a problem during the study. MB: Okay. How did you usually access VisualDx, during the study? PCP 10-01A: Through, - I think there's a link off of Epic, or what we used to call Prism, everything's changed its name.  MB: Yeah, well it was Epic then, but it had a local name which was Prism.  PCP 10-01A: Now that's gone. MB: Well that's good that makes it easier. Did you find that that in Epic was hard to get to, easy to get to? PCP 10-01A: If I remember correctly, I had to do something to set it up so it was right there but once it's there, it's easy to find.  MB: Did you ever use the mobile or the Dana library link or any of those things?  PCP 10-01A: No, I don’t think so. MB: So, what difference did VisualDx make or ever make, in an aspect of patient care such as a diagnosis, or prompting a referral or avoiding one or... PCP 10-01A I think when I work with a patient, it feels like it fills two niches, when I don’t have a clue, if something is really weird and I’m trying to figure out, what are the possibilities here, it certainly helps generate a list of possible conditions. And then the second would be like that example I just gave you, again it wasn't during the study period, but when you think you know what it is but you're not-- you know, it's something you might not have seen in years, or maybe you have never seen it, you’ve just heard about it, and you think that's what it is, but you just want to make sure you haven’t forgotten something. And, I have another favorite dermatology book that I use like I would use VisualDx. Fitzpatrick’s is the name of it. So, I really, I like having that, what else should you be thinking of?  MB: Yeah, you're the first one to actually remember Fitzgerald's. Do you use Fitzgerald online or in the print volume?  PCP 10-01A: I use mostly the print. When I teach I go online, I think, I want to think it might be Habif though. I own both, for the Habif, I think if you, you can go online and download all the pictures that you want for the chapters if I remember correctly, so. MB: You can do that with Fitzpatrick's too. PCP 10-01A: Yeah, I don't think I've used it that way.  MB: It's in Clinical Key. [MB note: it’s in Access Medicine, Habif is in Clinical Key] PCP 10-01A: Yeah, anyway those are my two main go-to books. MB: Can you describe a time, just any example, of when it made a difference? Maybe you got the diagnosis faster or… PCP 10-01A: Well the one that I, I don’t know if it's appropriate because it wasn't during the study period, but the one that's fresh in my mind is the Fifths disease, is that a good example?  MB: Well, I will take all examples. PCP 10-01A: Any? So, in that case, someone sent me pictures, which is becoming progressively more common, is for people to email you pictures. And it looked like Fifths disease to me, however, this is an interesting twist. It looked a little different, the rash. So, Fifths disease, or parvovirus B19 in children presents with a rash that's called a slapped cheek rash, so the cheeks get pink; but he had this very interesting thing where he had a very discrete border all the way around the rash, so that it looked almost like a target, such clear borders. And so, I was like, well I've never seen Fifths disease look like that in all the pictures that I’ve ever seen before. Being an internist, I've only seen adults with it and adults don't get that rash, adults usually get something else. So, I went onto VisualDx and started looking at the pictures, and I also used Google, and just googled a bunch of images, and sure enough that's one of the ways that it can look. So, I was able to get a confirmation of what these other variants are that you might not have seen before. And then also consider what are some of the other potential explanations for the rash too. MB: Could you, do you think like an example like that, or another example, finding - getting a better diagnosis, might help the patient to get better faster or avoid going to the dermatologist or anything like that? PCP 10-01A: Right, well, so in terms of the actual impact, one is, some rashes are contagious so in that case, you know, it was about being able to supply information to the family. "Look you may be getting these symptoms", and interestingly, that person, they’re not my patient, but they went to see their physician. The physician didn't recognize it as Fifths disease and just said I think it's a virus.” So by me looking it up and being able to confirm the diagnosis, I could say “oh this is the virus and these are the things for you to look for”; and sure enough, like three days later the kid developed a whole new rash which is the other rash you get with Fifths disease, and I was able to say “This is completely expected”, and also say, you know, these are other symptoms you can look for. And if anybody in the family has any of these things, because fifths disease can cause other problems, they need to, you know, see their physician. So, I think, better education comes from a better diagnosis.  MB: Better patient education? PCP 10-01A: Better patient education. I'm, forgive my fuzzy memory, but my recollection of other cases during the study period, I mean, -- a better diagnosis is going to result in more appropriate therapy. If you are more confident in the diagnosis for a rash, you can use a narrower prescribed therapy. Like if you can't figure out whether it's a contact reaction or a fungus, sometimes you end up prescribing both the fungal treatment and a steroid, but if you’re more confident in your diagnosis, then you might just prescribe the steroid, or just the antifungal.  MB: So that kind of avoids... PCP 10-01A: Unnecessary therapy. Theoretically - it didn’t happen in these cases, but - if I saw a rash that I hadn’t considered that would require a dermatology referral, theoretically, it could have prompted a referral, when I would have otherwise taken care of the patient, but I don't remember that happening. MB: Did you find - what about, therapy, did you ever find you know, a therapy, or treatment that was better or that you hadn't thought of in VisualDx? PCP 10-01A: I didn't look at the therapeutic information in there, so the only way it could influence me would be a better diagnosis and a more appropriate therapy.  MB: How about patient communication, did you show patients the pictures? Did it help with that? PCP 10-01A: No, not that I remember. Usually when I am stumped on a rash, I go back into my office, and I'm looking up information there, and if I'm working with a resident, we're both looking at it together, it's not at all unusual in this day and age for residents to take a photo of the rash in the room, bring it to me in my office and we may actually work out a differential diagnosis before I actually even lay eyes on the patient, that's something that's definitely been changing. So, a lot of this is happening outside of the patient room. I do definitely remember in the last year, bringing a picture of a rash to show a patient. But I think that might have been outside of the study period and I don’t think it was VisualDx because what would have to happen, is I would have to log out of VisualDx, or I'd have to close out of it in my room, go back in the patient room, log back in, it would be extra steps versus if I have a book open, all I have to do is bring the book in.  MB: Yup.  PCP 10-01A: What we need is that thing that people have in their houses when they're watching the movie down in the basement and it allows them to walk upstairs and the movie follows them around. MB: I've seen that but I don't have that.  PCP 10-01A: Yeah, I don't have it either. MB: So did you ever look it up, look up something, when you hoped VisualDx would provide the answer and it did not, it just did not make a difference or it just did not apply?  PCP 10-01A: The only thing I remember is getting more hits back than I was expecting, like a lot more diagnoses than I was expecting some of which didn't even look close to what I described; so that was the part was unhelpful that I remember. MB: Yup, okay. So, well you kind of answered the next question already, do you use VisualDx now? And what prompts you to use it?  PCP 10-01A: Yup, I use it now, I’d have to say I use it a little less than I did during the study period. And.... MB: What prompts you? Same reasons as it did before? PCP 10-01A: I think it's the same reasons, yeah.  MB: You have a question, uncertainty...  PCP 10-01A: Yeah, I'm trying to remember -- that you wanting to meet with me for this reminded me that the tool existed, because I most recently looked it up after we exchanged some emails. So, I could have been influenced by that, and said boy I should double check and see if this is really Fifth disease. So, something about reminders is a good thing.  MB: Well this is the qualitative part of the study, so it's okay for the researcher to kind of influence the participants in some way, or at least acknowledge it. PCP 10-01A: Well, there's so much technologically that's available now, and it’s complicated and if you don't use something every day it's hard to remember. We just had a meeting the other day where we were talking about documenting chronic pain visits and somebody reminded me of an Epic shortcut that I had completely forgotten it existed, and so, you know, reminders are a good thing.  MB: Okay, I'll make a note of that. You kind of already answered this too, what information resources, what other information resources, did you use then or do you use now for evidence for skin problems? PCP 10-01A: Yeah, did I mention UpToDate?  MB: You didn't actually, but UpToDate. PCP 10-01A: Yup, UpToDate, I probably use UpToDate the most for skin therapeutic choices, because you don’t have to leaf through a book to find it, and you just type in the disease and you get some reasonable options. I'll occasionally use UpToDate diagnostically, but I don’t think it's as useful for diagnosis, versus Fitzgerald and VisualDx are I think are both very helpful for diagnosis.  MB: And you mentioned Google images sometimes for...  PCP 10-01A: Yeah Google images -- you got to know what to search for. So, if I'm searching Google Images I already know what diagnosis I'm thinking of, and I’m plugging that in, and I’m just looking at the variants that pop up. The problem with Google Images is, you know, anybody that has a computer can upload a picture and tag it with a diagnosis, so you don't necessarily know what you're getting. Versus at least VisualDx has been validated by somebody that these pictures are the real deal.  MB: So that's a little bit of the next question, what is the relevance of these sources to evidence-based medicine and your practice?  PCP 10-01A: Yup. Well there's layers to that question, I think I answered the first layer, which is at least VisualDx has been vetted by somebody, as any of the books that I mentioned probably have been and UpToDate has been vetted by somebody. But just because it's been vetted by somebody doesn't mean it's evidence-based, so.... MB: Well there are the levels of evidence. PCP 10-01A: Yeah, yeah, so you know, at this moment we're dealing with basically expert opinion...  MB: Right. PCP 10-01A: And I would say, in dermatology, to be honest, I don't think I necessarily learned a lot of evidence-based medicine that has to do with diagnosis. Certainly, with therapy, I will try to read about therapies and how one is compared to another, and response rates and things like that, I think that's of interest. VisualDx, I haven't used at all for that. Theoretically, I mean there is a whole field of diagnostic evidence-based medicine, and you know, what that would look like would be this: I'll give you a for instance. If somebody has an abnormal mole and you're looking at the diameter and you're looking at the asymmetry and the roughness of the borders. You know, true evidence-based medicine would be someone has come up with a decision rule that says if you meet at least three out of the five of the ABCDE criteria that you have an X percent chance this is a melanoma. That’s the kind of information that I think of when I think of evidence-based medicine and none of the tools that I've described are useful that way that I've ever seen. MB: What about, and the last aspect of EBM is really how does it apply to this patient? Did you, or was information that you found applicable to the patient. PCP 10-01A: Yeah, I would definitely say it's useful that way.  MB: So, and kind of last question, what do you think about your education or preparation in skin disease as a whole and also since you're a resident educator, what about the education or preparation of residents in medicine and primary care? PCP 10-01A: Yeah, I would say in, as a general rule, in my field, a huge proportion of the dermatology that we learn is through self-education. Residents don't spend much time in dermatology. MB: How much time do they spend?  PCP 10-01A: Depends on the resident, but they might spend like a week or two in three years of residency. So, they're learning a lot of dermatology from non-dermatologists, and when you learn about things where you shoot from the hip and it's on the fly, that kind of education can be fairly unstructured and hard to assimilate. So when I teach, I must confess I haven't used VisualDx yet to teach, again I've used Habif, and to some extend Fitzgerald. And I may use UpToDate for the teaching part as well. I'm trying to think, I just did a session for the residents on HSV1 and HSV2 and so I'm trying to remember everything I used for that. Oh yeah, I also used the Yale curriculum, I don't know if you've ever, are you familiar with that?  MB: No. PCP 10-01A: It's a primary care, three year, rotating primary care curriculum put out by Yale for internal medicine. So, so we subscribe to it, and they come out with these modules that are in print, there's no pictures that come with them, but there's resources to look up. So, I had that and UpToDate and Habif and Fitzgerald and Google and I used those to try to create a really nice experience for the residents. And I was able to, from this background of think about a differential diagnosis, I gave them a little quiz in the middle of “Here's a slide, do you think it's HSV and if it's not HSV what else could it be?” To get them thinking that way, because I think it's really easy for people to look at rashes and just say "Oh I think that's this" or "Oh I think that's that". And VisualDx gets you out of that thinking. Fitzpatrick’s gets you out of that thinking. And I think that's the right approach if you're going to be a good diagnostician. But I don’t think they're exposed to it as much as they could.  MB: Are the other resident instructors as diligent as you are? In that kind of teaching, or... PCP 10-01A: It may depend on which field we're talking about, I think there are certain areas that get - that approach can be applied to everything, but there are certain areas where internists in particular, don't maybe have as much training and we tend to fall into more less rigorous ways approaching a diagnosis. I think dermatology can be one of those; sports medicine can be one of those, where somebody comes in with shoulder pain, and people are having trouble grasping all the possibilities and how to test for them. And there, physical diagnosis is very important, too. Psychiatry is one of those areas where people, someone comes in with anxiety and just somebody’s thinking “Oh anxiety well here take an SSRI”, when they're not taking a step back and saying oh anxiety, well anxiety could be a manifestation of depression or substance use disorder or hyperthyroidism or ADHD. So, I think VisualDx is kind of a nice support that way, cause when it throws a bunch of stuff back at you, instead of one diagnosis, you have to run through the list of options. And I think all doctors are trained initially to think that way, again, it's just easy in certain situations to slip outside of that mode.  MB: I've heard some people say that they know that if they do to the differential it’s going to take time, and so that differential builder, or the diagnosis, then that's a barrier, it's like...(ugh)  PCP 10-01: Yeah, it's, there's two sides to that kind of comments. One of them is (A) I feel your pain. You know, when you're already 45 minutes behind schedule and you got patients waiting for you and someone comes in with a goofy looking rash, it's very easy to say "Well, I think it's this, let's try it, see how it goes, if it doesn’t work call me back". You know one of those things, very easy to do that. (B) On the other hand, is that, you know, if your mother, your father, your child was seeing somebody, is that how you'd want the doctor to take care of them? Is, you know, “I'm 60% sure that this is what they have” ... or would you want someone to thoroughly consider the options? Or if, at the very least, send them to somebody who can review the options with them. And so, there's a certain amount of courage and perhaps even a little narcissism, that's required to sit down and say I'm the captain of my ship, I'm here to give this patient my very best, and sometimes that means people are going to have to wait, and I might be home late for dinner. And I realize if that happens to much that that might be particularly painful, but it would be harder for me to live with myself and sleep well otherwise, as well, so. It's easier said than done. MB: Yup... And that’s an inspiring answer. I'll definitely put that in the article.  PCP 10-01A: Yeah, sure.  MB: I have just very quickly - This was a little tiny REDCap survey and I just took it out and put it in paper form and it's, you've kind of, you are prepared to answer these questions because you have gone through them. So this has to do with like during the study and these two are just more recent.  PCP 10-01A: Recent… alright.  MB: That would be if you're not... End of recording. ______________________________________________________________________________________________________________________________________ Name: Files\\int_id\\PCP 12-17A PCP 12-17A Interview #12 4/18/2018 8:45 a.m. at Dana Medical Library Medical History Room. Attending/Family Medicine PCP 12-17A: I'm [Name] and I work at [Name of Clinic].    MB: So, what do you remember about being in the study? Could you describe your experience of participating in it?  PCP 12-17A: Well so, I'm pretty sure that I was aware that VisualDx was available before the study. I don't recall using it much, if at all, prior to the study. I think I probably used it a little bit more during the study than I did prior to that, but even that was relatively infrequently.  MB: Okay. So, what would prompt you to use VisualDx?  PCP 12-17A: So, I think if I saw a patient for just a skin complaint, if I wasn’t sure about what the problem was and I thought that using VisualDx might help me with that, that's usually how I decided to use it. There were certainly a lot of patients where I felt comfortable with what I thought the problem was, or even if I didn’t, if I had a plan to say, do a biopsy on my own to confirm that diagnosis, I might not use VisualDx to do that. Or there might have been situations where I felt it was important to see a dermatologist and made the referral without using it, or maybe the patient might request it.  MB: So, you were pretty sure?  PCP 12-17A: Right, so the majority of the skin complaints would fall in those categories and only a few would be where I'd be scratching my head going "Hmm, maybe I need to look at something". That's how, that's pretty much how I used it. MB: So, when you did use it, how did you like it? Or what was your opinion of it? PCP 12-17A: I would say the majority of the time that I used it, it didn't help me to confirm the diagnosis just with that reference. I thought it was useful to have available, but I think there are some challenges that maybe limit how effective it might be. One is that if I see something and I don't know what it is - dermatology is a very visual thing, right - so right off the bat, it's different than the more cognitive knowledge things that we might use, say, UpToDate for, which I use frequently.  MB: Which you do use frequently?  PCP 12-17A: Yeah, absolutely. Every day, pretty much. So that's maybe good and bad in some ways. But for VisualDx, if I had seen something, going to a reference to have something to cross- reference with that, is tough. MB: Did you say cross reference?  PCP 12-17A: Yeah, I can't just take a picture of what I’m looking at and put it into something that will help me to decide what it actually is. I thought it [VisualDx] was useful to give me some things to think about that I might not have thought about otherwise.  MB: Okay.  PCP 12-17A: But I felt like there was a lot of noise, for lack of a better word, with the information it would come up with.  MB: So, just in general then, you kind of said this, how useful? PCP 12-17A: I think it was useful as a tool that's unique and different than say UpToDate, and I think it's helpful to have available. I don't know that it's something that I use as often as some other things. I'll just give you another example, there's more along the lines of a tool for coming up with a diagnosis for unexplained symptoms, there is something out there for that...  MB: Do you know what the name of that is?  PCP 12-17A: Yeah, I'm trying to remember. I think I might have it in...[Looks in smart phone] It's not specific to dermatology, but it's a similar concept.  MB: Yeah.  PCP 12-17A: And I think they did a great job with trying to come up with something that would work for that. Isabelle!  MB: Oh, Isabelle, I've heard of that. We haven't subscribed to it in the library for everybody but I guess it’s something you can put on your phone. PCP 12-17A: Yeah, I just have the basic version, but I think you have to spend a lot of money to get the whole nine yards.  MB: Yeah. Was VisualDx hard or easy to use?  PCP 12-17A: I think in terms of the user interface it's relatively easy.  MB: Did you usually access it, well, how did you usually access it when you did? Did you go through the network or on your iPhone?  PCP 12-17A: I think I just used it on my own with the standard computer and a browser or on my phone but it's limited on there. I was just trying it out more than anything. MB: In your clinical office?  PCP 12-17A: Right, and I would say I used that even less than VisualDx.  MB: Okay. You used the mobile less than? With VisualDx?  PCP 12-17A: I would say that I used, Isabelle, overall, less than VisualDx.  MB: Oh, okay. PCP 12-17A: And VisualDx a lot less than UpToDate.  MB: Right, okay. PCP 12-17A: Another example is if I knew what the problem was for a skin complaint, but wasn't sure how to manage it, I might be more likely to use UpToDate than anything else. MB: So VisualDx more for the diagnosis, UpToDate more for the therapy or... PCP 12-17A: Sure, that's a fair way to put it.  MB: Did you go, but when you found VisualDx say on, did you go through Prism or the internet? PCP 12-17A: Always Prism, I don't know that there's another way. I suppose I could have used it in other ways. But for the most part if I'm seeing patients I'm already in Prism and its available there, I already know where it is, it's easy to find, 99% of the time that's what I would do. MB: The drop down... So, when you did use it, did it ever make a difference in an aspect of patient care such as change a diagnosis or prompt a referral or not or?  PCP 12-17A: That's a hard question to answer, I can't, I don't know that I can give you any Ah-Ha moments that it completely changed or made me come up with a diagnosis that really made a difference. I can say that in general in the few times that I did use it, it did give me some additional things to think about that I might not have thought about otherwise, but I don’t know that it actually changed what the diagnosis was or how I managed it.  MB: Can you remember a time that you used it, an example?  PCP 12-17A: I haven't used it recently, so it's hard to remember, but that's what I’m trying to say, it was relatively infrequent. MB: Okay, if you think of one as we're going along, sometimes I've had people kind of be like "Oh yeah now I remember a time!"  So, and then another, the other side to that, can you remember a time that you hoped it would make a difference but it really disappointed you? So, either, any example.  PCP 12-17A: Yeah so for example, I think I do remember a time when I, you know, maybe had a patient come in, I can't remember specifically but the patient might have come in with a rash on the arm or the face and I might have put in, you know, things that are on the face, more regional differential diagnosis, and the things that came up initially were so broad and so numerous that it was a bit challenging to sort of sift all through that to kind of - to get to things that would have been useful.  MB: Yeah, so how long did it take you to sift through that? Was that an issue? PCP 12-17A: It's hard, again, often these appointments are 15 minutes long, I've already burnt through 10 of them just getting the story and sorting out the issue, so there's not a lot of time really to sift through it, and it would be rare that I would go back afterwards cause often I'd have to make a decision and move on to the next thing so...  MB: So, do you use VisualDx now? You kind of said... PCP 12-17A: I would say if something came up, you know, today, and I see a patient and I'm looking at something that I wasn't sure, sure I might consider using it. Would I use it every time? Probably not. MB: So, the same things that prompted you during the study maybe would prompt you now, but maybe not as often.  PCP 12-17A: Yeah.  MB: And so what other information resources did you use during the study or do you use now for skin problems?  PCP 12-17A: So, I mentioned UpToDate, which isn't really… If I'm confident about what I think the diagnosis is, that would be what I would most, more likely to use it for. I do use a textbook fairly often, that I've had around, you probably know it, Habif.  MB: Habif, yup. PCP 12-17A: Actually, my copy now is old... MB: You know it's online? PCP 12-17A: Oh, it is?  MB: It's in the, yeah, if you go into, you could bookmark it, but if you go to the Dana Library Website and just type in on the big thing, the big blank thing, just like a google search, Habif Dermatology it would come up to the textbook and you could bookmark it.  PCP 12-17A: That's great.  MB: But anyway, you have your print copy. Other people have mentioned Habif also. PCP 12-17A: Great.  MB: So, what do you think would be the relevance of, first VisualDx, and maybe other resources to practicing evidence-based medicine?  PCP 12-17A: I think it's relevant, I think the more tools we have available at the site of patient care the better. And I think integration of that with the electronic medical record is important. It's difficult, I think, for clinicians that are in a fast-paced clinical environment, to be able to stop and reference things if you don't have it immediately at your fingertips. We do have browsers that are on the computers in the room, but still that's an extra few steps to get out of the EHR, bring up a browser, research for what you want, click on, you know, you have to do several things to find what you want. Even if you know what you’re looking for it can still take some time to do that. So, having it imbedded and available in Prism is really helpful.  MB: But yet it's not really linked to the actual patient record, in other words, it's not a decision support in that way, where...  PCP 12-17A: No, but I don't see that as a downside as long as it's relatively easy to get to, it's fine. Right now, the format for how we use our patient encounters in the record has changed recently, so it's now a widescreen view which is different than we used to have. And just getting used to that is hard enough. So, I think what a lot of people struggle with is as things get updated and changed they are constantly being [changed] - we're constantly being forced to adjust to that and the more you have to do that the more fatigue is associated with that so having something that's easy to find and easy to use is really key.  MB:  Did you ever use VisualDx for - how do you think it worked in terms of - patient communication? PCP 12-17A: Not so well, but I wonder if I haven't really used it to its full capability. I think if I was confident about what I thought the problem was but needed to show the patient some other examples so they could say "Oh yeah this is what it looks like, similar to mine",.. I might. I might be more likely to just go to a web browser and search and show them, you know, give them information that way. MB: So like Google Images? PCP 12-17A: Yeah, essentially right.  MB: Okay, so I put that down as another source of information. Have you done that, used [it with patients]?  PCP 12-17A: Yeah, and again, not often, but I do recall doing that.  MB: So, would, but does that kind of help with the patients’ understanding, or believing that you really got it.  PCP 12-17A: Yeah, it does. MB: So maybe they'd be more likely to follow the treatment, whatever it is.  PCP 12-17A: I might do a similar thing with UpToDate. So, if I say, someone has a seborrheic keratosis which is a benign lesion, the patient is worried about cancer. I might go to UpToDate find summary seborrheic keratosis; and within that section there might be links to pictures and so that I might have used them to show people, "Oh this is what it looks like, this is what we know about it, it's benign". They might read some of the text information that's on that same page. So, I've done that. To be honest with you, I'm not sure how well VisualDx does with that, I suppose I could have made more of an effort to use VisualDx in the same way, it probably would work okay.  MB: And what do you think about your education or preparation in skin disease and dermatology? PCP 12-17A: I think relative to my colleagues in the same specialty, I would say I'm at the stronger end of it. I frequently have talked with colleagues who feel like they really don't feel all that confident and maybe aren't as interested in pursuing it. But, aside from that, the training that I had, again, was limited to course work in medical school and rotations during residency and periodically CME courses after I've been in practice. So, I have attended some dermatology conferences that were focused on that for primary care, not for dermatologists. I would say I've probably gone to two or three of those.  MB: Yeah... So, you've been in practice and out of medical school for... PCP 12-17A: 20 years.  MB: 20 years, yeah. What else would you like to tell me about either being in the study or using information resources relative to skin problems, or anything?  PCP 12-17A: Well, so, really, I guess I didn't really think about the fact that I was in the study, I was pretty much doing what I would do, you know, every day.  MB: Did you get reminders from us though? We sent you quite a few emails and letters.  PCP 12-17A: If I did I don't remember. Maybe once or twice. MB: Okay.  PCP 12-17A: Maybe once or twice, I can't remember. So, but I do think that electronic references are where it's at for patient care. I still have a few textbooks on my shelves, I would say there's probably less than five that I've looked at in the last six months. There's a couple that I wish I had, that I haven't looked at because I didn't have them around; so, Sanford’s Antibiotic Guide is one, for example. It's a little paperback, you've probably seen it. Residents use it all the time and there's been a few times that I wish I had one. But in terms of everyday patient care I use UpToDate I would say pretty much every day.  And I worry about it a little bit, because it's a great reference but the text is written by, you know, single providers or groups of providers and in some ways some of it is their opinion.  MB: Expert. And VisualDx does not tell you either, what level of evidence it is either, so it's kind of in a way similar to UpToDate, its expert opinion for the most part. Okay well, basically that's it. You’re off the hook. So, even, so even, so I'm working on my PhD in Clinical and Translational Science so that's why I'm doing this whole thing. Although I really like to do this kind of research and so hopefully it will, I will get the degree, and both articles will be published. The article about the study itself and then this one. What I'm trying to do is find out why did VisualDx fail to make a difference in either the referrals or the relief of symptoms of patients.  PCP 12-17A: Yeah, it's a challenge to get something like that to [work]. I think it's a lot easier to just have a broad base sort of encyclopedia format which is really what UpToDate is. I mean all you have to do is write stuff and put it on there. To have a tool that would be able to, that has to respond to the information that you input and come up with something that's useful - is a lot more challenging. I could see on the development side how that could be a lot harder.  MB: Yeah, it’s harder. Well, thank you very much.  PCP 12-17A: Thanks for the chocolate. We’ve been so busy. Interview start: 1:04 minutes into recording End: at 20: 48 minutes into recording Length: 19:44