Evaluating Nursing Collections With the ICIRN's Essential Nursing Resources List

This study examines libraries’ nursing collections using the Interagency Council on Information Resources in Nursing's Essential Nursing Resources’ (ENR) 26th edition. An inventory of the online collections of 235 libraries was assembled and compared to free, government, or National Library of Medicine resources and licensed resources from the ENR. The top five resources listed on library websites in descending order were MEDLINE, CINAHL, ERIC, PsycINFO, and Google Scholar. The availability of specialized resources varied, based on factors such as the level of nursing degree at each institution or the libraries’ National Network of Libraries of Medicine membership statuses.


INTRODUCTION
The Essential Nursing Resources List (ENR) has been a standard resource for both nurses and librarians since 1966. 1 The ENR has been utilized by librarians for collection development and by nurses for current awareness of resources for their professional education or for career advancement. ''The premier product'' of the Interagency Council on Information Resources in Nursing (ICIRN) has been typically published biennially. 2 Organized in 1960, the ICIRN comprises librarians and nurses from an agency or organization that is concerned ''with providing library and informational resources for nursing and improving access to library services for all nurses.'' 3 The librarians who have partnered with nurses on the resource selection for the ENR have been listed as co-editors. The ENR has evolved from a primary focus on print resources to include both freely available and licensed electronic resources and, most recently, mobile resources.
The ENR was first published in the journal Nursing Outlook from 1966 until 1990; the 17th edition was published in the American Journal of Nursing in 1992. 4 From 1994 to 2009, the ENR was published in the National League for Nursing's journal, which underwent a series of name changes: Nursing & Health Care: Official Publication of the National League for Nursing in 1994, 5 N & HC Perspectives on Community in 1996, 6 Nursing and Health Care Perspectives in 1998 and 2000, [7][8] and Nursing Education Perspectives between 2002 and 2009. In 2012, the 26th edition was published on the ICIRN website in an online format due to its length. This study used the 26th edition of the ENR as a basis for evaluating libraries' online nursing collections. 9 While the ENR has been listed as a recommended resource for a nursing virtual reference shelf or as a selection tool, [10][11] the ENR's content does not appear to have been used as the basis for a research study since its initial publication almost 50 years ago. This current study used selected content from the ENR to determine which licensed resources were available to nursing students at their institutions.
A 2012 study from Stankus and Parker examined 50 nursing schools' content on their LibGuides, or resource guides. 12 Their sample came from both US News & World Report top-rated and unrated nursing schools. In addition to looking at the LibGuides' contact information and structure, the frequency of databases and point-of-care tools was also recorded. This current study used the ENR as a basis for the selection of resources, drew from a larger sample size, and examined nursing guides created within any online format.

LITERATURE REVIEW
Librarians have used several tools to evaluate nursing collections. Since 1969, the American Journal of Nursing (AJN) has published an annual list of mostly books and electronic media judged to be the best in nursing, the AJN Books of the Year. 13 In addition to covering specific areas of nursing such as Advanced Practice Nursing, this list also included Consumer Health and Public Interest and Creative Works. The Brandon=Hill Selected Lists were considered the standard for print books and journal selection in health sciences libraries for almost 40 years since 1965. 14 A nursing-related version of the Brandon= Hill Selected Lists, The Brandon=Hill Selected List of Print Nursing Books and Journals, was first published in 1979 15 and last published in 2002. 16 The discontinuation of updating the original Brandon=Hill Selected Lists was announced in 2004. 17 Nevertheless, in the following decade, these Brandon=Hill branded lists were still being included as authoritative standards in research studies' methodology sections for years beyond their discontinuation, [18][19][20][21][22][23] including the Nursing and Allied Health Resources Section's (NAHRS) ''Mapping the Literature of Nursing'' studies. 24 Doody's Core Titles (DCT) succeeded the Brandon=Hill Selected Lists in 2004 as an authoritative selection standard. 25 Published by Doody's Enterprises, this annual list was available through subscription only. Covering 121 specialties, including nursing, content specialists review and list the titles for the annual list, while librarians score the titles selected for the annual list. In 2011, the Medical Library Association's (MLA), The Medical Library Association's Master Guide to Authoritative Information Resources in the Health Sciences, was published as a potential replacement for the Brandon=Hill Selected Lists. 26 Unlike DCT, each subject area was limited to ten of the most important books and ten of the best serials. This book had two major nursing sections: the general nursing section included nursing research and nursing theory and the nursing specialties section had recommended resources in 21 specialties. The past five years have also witnessed the addition of other nursing selection tools. The 2011 ALA Guide to Medical and Health Sciences Reference included a nursing section with Internet resources in addition to print books. 27 A 2013 article focusing on recommended titles for hospital libraries included a section on nursing. 28 This study used the ENR to identify the current state of library resources for nursing programs, the exposure of nursing students to free high-quality resources, and beneficial recommendations for both nursing libraries and professional selector guides.

METHODS
A spreadsheet of 586 accredited institutions with Baccalaureate (BSN), Master's (MSN), or Doctor of Nursing Practice (DNP) programs was developed from an online directory available on the Commission on Collegiate Nursing Education (CCNE) website in February 2013. 29 A random number generator was used to select 235 institutions from this spreadsheet. The sample size was determined by consulting a table of recommended sample sizes from a given population for surveys. 30 In cases where an institutional consortium of nursing programs had been selected, all of the programs' libraries were examined. In the CCNE, institutions that were a member of a consortium or offered a joint degree were listed both individually and under the consortia name.
Since the ENR's 26th edition was 36 pages long, categories from the ENR were selected if the majority of resources included both freely available and licensed electronic resources. Freely available resources included government resources, such as National Library of Medicine (NLM) resources. The five selected categories were: Complementary=Alternative Medicine (CAM), Consumer Health=Patient Education, Databases and Indexes, Drugs, and Evidence-Based Nursing. Appendix 1 has all of the databases covered in this study by category. Non-governmental websites, whether hosted by nonprofit and for-profit organizations, were excluded because the intent of this study was to focus on .gov resources. Print resources were also excluded. If a resource was under multiple categories in the ENR (e.g., AHRQ, MedlinePlus), it was assigned to the first category in which it appeared.
The library websites of the selected institutions were examined between April and September 2013. Data were recorded in Qualtrics, an online survey tool. The list of questions is available in Appendix 2. The following information was recorded: . Institution's name and state; . The highest level of nursing degree from the CCNE website; 29 . Institutional data such as public versus private status, the total number of enrolled students, and the classification level of the institution from the Carnegie Classification of Institutions of Higher Education; 31 . The library's member status in the National Network of Libraries of Medicine (NN=LM), according to the NN=LM directory; 32 . The library's name, main library website URL, and the URL of the nursing or other relevant guide; . If an ENR resource was listed on the library's website; . Resources listed on library websites but not included in the ENR. These resources were mostly licensed resources that would be familiar to most health sciences librarians.
Nursing subject guides were a primary source of information in most cases. If a library did not have a nursing guide, health sciences subject guides were examined. The Databases A-Z lists were also examined because a number of databases (e.g., Google Scholar) were not reliably listed on the nursing guides. Whenever feasible, searches were conducted on library websites or guides for resources such as NLM resources to determine if they were listed elsewhere on the websites. These resources may have been listed as a website on one library's website or as a database on another's.

RESULTS
A total of 225 library websites were examined. Ten institutions were excluded because they were not located within the United States, their resources were not listed on the library website, or the resources were inaccessible from public view due to the libraries' proxy servers. Two institutions' libraries referred to other libraries' websites for their electronic resources.
One hundred thirteen (50.2%) of the institutions had both BSN and MSN level nursing programs. Figure 1 shows the levels of nursing degrees at each institution. Six institutions (2.7%) had only nursing programs and 38 (16.9%) also had medical schools. The student population breakdown consisted of 74 (33.2%) institutions with a total student population of 1,001-5,000, followed by 50 (22.4%) with 5,001-10,000 and 25 (11.2%) with 10,001-15,000. Eleven (4.9%) had a total student population of less than 1,000, while 22 (9.9%) had a total student population exceeding 30,000. Figure 2 shows the total number of Carnegie Classifications per institution. Two institutions (0.9%) that were not included in the Carnegie Classification only had nursing programs. The only institution that did not appear in any of these categories was classified as a four-year primarily associate's degree institution. One hundred eight (48.4%) of the schools were private, not-for-profit institutions, 111 (49.8%) were public institutions, and 4 (1.8%) were private for-profit.
One hundred twenty-four (55.1%) libraries were NN=LM members and 101 (44.9%) libraries were not members. Among BSN-only programs, 40 (17.7%) institutions did not have libraries with NN=LM membership. As for institutions that offered both BSN and higher level nursing degrees, 70 (31.1%) institutions offering both BSN and MSN level nursing programs had NN=LM membership, while 26 (11.5%) institutions with BSN, MSN, and DNP programs had NN=LM membership.

154
R. Raszewski Figure 5 shows the total number of evidence-based nursing resources licensed by libraries. The resource that was listed the most on library websites was the Cochrane Library (n ¼ 170, 75.5%). Interestingly, libraries had more licenses in the Evidence-Based Nursing category for resources such as DynaMed (n ¼ 39, 17.3%) and UpToDate (n ¼ 47, 20.8%) than nursing-specific point-of-care-tools such as Mosby's Nursing Consult (n ¼ 15, 6.6%) and Nursing Reference Center (n ¼ 17, 7.5%); the Discussion section offers a possible explanation for this. NN=LM libraries listed NLM or government resources more frequently on their websites compared with non-NN=LM libraries. Figure 6 shows 17 NLM or government resources from the ENR categories selected for this study. In this study, 16 of the 17 resources appeared more frequently on NN=LM libraries' websites compared with non-NN=LM libraries. For example, MedlinePlus, was listed on 113 (50.2%) NN=LM members' websites compared to 75 (39.9%) non-NN=LM libraries. The only resource that non-NN= LM libraries listed more frequently was health.gov.

Evaluating Nursing Collections
Some categories had NLM or government resources listed more frequently than licensed resources. In the Consumer Health=Patient Education category, MedlinePlus (n ¼ 188, 83.6%) and AHRQ (n ¼ 133, 59.1%) were listed more frequently than subscription-based resources such as Health Reference Center Academic (n ¼ 66, 29.3%) and Health and Wellness Resource Center (n ¼ 51, 22.7%). In the Drugs category, Drug Information Portal (n ¼ 62, 27.6%) and the Dietary Supplement Labels Database (n ¼ 50, 9.8%) were listed more frequently than subscription-based resources such as MicroMedex (n ¼ 46, 20.4%) and Medical Letter on Drugs and Therapeutics (n ¼ 25, 11.1%). Yet, in the CAM category, the licensed resource Alt HealthWatch (n ¼ 100, 44.4%) was the most frequently listed resource.

DISCUSSION
The libraries examined were inconsistent in where databases were linked throughout their websites. Although the majority listed MEDLINE via   35 Neuhaus, Neuhaus, and Asher's 2008 study on the presence of Google Scholar on both university and library websites also showed the acceptance of Google Scholar. 36 Out of a sample size of 948 institutions, the average number of libraries from research institutions which had Google Scholar on their websites was 41.72% compared with libraries from Master's and Baccalaureate institutions, which was an average of 2.28% and 2.03%, respectively. For this study, the majority of libraries that listed Google Scholar were from institutions with Master's programs.

R. Raszewski
Of 225 libraries from this study, 101 (44.8%) were not members of NN= LM, which is a free membership. If these libraries did become NN=LM members, they may fall into the Affiliate Member category, which means that a library does not have to meet all of the requirements for being a full member such as participating in DOCLINE, NLM's automated interlibrary loan system. 37,38 Regardless of libraries' election for NN=LM membership, the field would benefit from raising awareness about the importance of alerting future nursing graduates to the availability and benefits of NLM resources in the event they work somewhere without access to a health sciences library or abundant licensed resources. Librarians would render a significant service to the nursing and health care professions by teaching classes or providing information on resources accessible after graduation. In addition, health sciences libraries may partner with public libraries in providing expertise on low-cost or free resources in the current cost-cutting climate.
Although point-of-care tools, such as DynaMed and UpToDate, were accessible through more libraries than nursing-specific point-of-care tools, such as Nursing Reference Center, this may be due to the existence of a medical school program at the institutions. For example, out of the 39 libraries that licensed DynaMed, 25 libraries had both medical and nursing school programs while 14 did not have a medical school. Yet UpToDate was subscribed to by 25 libraries without medical schools compared with 24 with medical schools. This was due to there being a dentistry or pharmacy program at these institutions, or a hospital affiliation. Resources such as Clinical Pharmacology, Embase, Health and Psychosocial Instruments (HaPI), International Pharmaceutical Abstracts, and Natural Standard were subscribed to more by institutions with both medical and nursing programs than institutions that did not have a medical school. In Starkus and Parker's study, fewer libraries licensed Nursing Reference Center compared to this current study. More libraries in this study also licensed DynaMed and UpToDate compared to Starkus and Parker's study, with almost 15 libraries having a license to each resource. 12 Although nurses have been included in studies on health care professionals' usage of library resources that cover point-of-care tools, 39,40 little research has been done exclusively on nurses' use of point-of-care tools. A 2011 study focusing on nurses' usage of Nursing Reference Center at one particular institution was an exception to this. 41 Further research should be conducted for nurses' preferences regarding point-of-care tools for all levels of nursing education.
Libraries' subscriptions to more specialized resources differed based on the level of nursing degree offered at their institution. This observed difference associated with higher level nursing degree-granting institutions also carried over into the tendency to list government or NN=LM resources on library websites. Government resources such as CDC Wonder were found on more libraries' websites from BSN=MSN (n ¼ 35, 62.5%) and BSN=MSN=DNP (n ¼ 11, 19.6%) programs compared to BSN-only (n ¼ 6, 10.71%) from 56 libraries. NLM resources such as LactMed were found on more libraries' websites from BSN=MSN (n ¼ 18, 66.7%) and BSN=MSN=DNP (n ¼ 6, 22.2%) programs compared with BSN-only (n ¼ 3, 11.1%) from 27 libraries. Physician Data Query (PDQ) and Health Services=Technology Assessment Texts (HSTAT) were not located on libraries' websites from BSN-only programs. Yet there were a few interesting exceptions to the trend of fewer government or NLM resources on BSN-only library websites, including linking MedlinePlus (n ¼ 46, 24.5%) on BSN-only websites more frequently compared with BSN=MSN=DNP programs (n ¼ 38, 20.2%) from 188 libraries. This was also reflected with the Carnegie Classification. Libraries with institutions that were classified as Research Institutions had access to more licensed CAM and drug resources compared to other institutions. Otherwise, the majority of resources were usually available in the most frequent category in this study, Master's L: Master's Colleges and Universities (larger programs).
Resources, which were not on the ENR but were available more than once on libraries' websites were also tracked, and should be considered for the ENR's next update. Figure 7 shows the ten most frequently listed resources, which included Health Source Nursing Academic Edition, ProQuest Nursing and Allied Health Source, PubMed Central, and Scopus. In addition to Web of Science, Scopus should also be considered for inclusion under the Related=Specialized Databases category. Scopus had  R. Raszewski the nursing category from the Journal Citation Report and serves as an additional resource for nurses trying to measure their own impact. 42,43 Pro-Quest Nursing and Allied Health Source, PsycARTICLES, and ScienceDirect may not have been included in the ENR because they may be considered more for their full-text holdings than citation coverage. CQ Researcher may be considered for libraries with BSN programs, which may have students writing opinion papers on various health care issues. The only free resource that was not on the ENR but was frequently listed on libraries' websites was PubMed Central. Since PubMed Central has free, full-text articles available, this resource may fill a gap for nurses or institutions with limited access to electronic resources or have difficulties paying for full-text article access. PubMed Central will help nurses obtain freely available full-text articles.
The latest release of CINAHL from February 2013, 44 CINAHL Complete, should also be added to the ENR. CINAHL Complete, which has almost 600 full-text journals and indexes almost 200 more journals than CINAHL Plus with Full Text, 45 was subscribed to by 23 (10.2%) libraries at the time of this study. The number of subscriptions to CINAHL Complete will continue to grow over time.

LIMITATIONS
Multiple, diversified websites were evaluated and some content may not have been recorded because the location of resources on the websites impeded detection. For example, some websites listed ACP Journal Club on their Databases A-Z lists. On other websites, ACP Journal Club was found through an e-journals list or through the library catalog. Although multiple places were checked in consideration of these variations, some content may have been missed due to resources' location or their descriptions. For example, the majority of libraries did not list which components were included with their MicroMedex license. Libraries' electronic resource licenses may have also changed since the study was conducted. While the selected resources will give librarians an idea of what resources are being licensed or listed on their websites, the entire ENR list was not searched. The selected resources may not be entirely representative of a library's nursing collection.

CONCLUSION
The majority of libraries from institutions with nursing programs do provide access to core databases such as CINAHL and multiple versions of MEDLINE. The majority of these libraries also provide access to specialized databases such as ERIC, PsycINFO, and Web of Science. Libraries with NN=LM membership include NLM resources such as MedlinePlus and LactMed on their subject guides or database lists at greater rates compared to non-NN=LM libraries. More outreach may need to be done with non-NN=LM libraries for both membership recruitment and education regarding the availability of NLM resources. Additional resources, such as PubMed Central and Scopus, should be considered for the ENR's next update.
Librarians who are responsible for collections should compare their library's collection with the data from this study to identify resources that their library does not license or list on their website. Whether librarians are facing shrinking collection budgets, or are at an institution that is adding new or additional nursing programs, they should turn to the ENR for ideas. Librarians would do well to become familiar with the resources on the ENR to see if their library has access to these resources, especially free or NLM resources. Librarians should consider adding point-of-care resources or specialized databases. The ICIRN does a great service to the profession and community by continuing to make the ENR freely available, and it is hoped they will continue on this path.