How and Why Libraries Are Changing: What We Know and What We Need to Know

In November 2000, the Digital Library Federation (DLF) and the Council on Library and Information Resources (CLIR) commissioned a white paper to initiate discussion of how and why libraries are changing. Eight academic library directors met with representatives from the DLF and CLIR in March 2001 to discuss the issues. The outcome of this meeting was a proposal to conduct research that will begin to fill significant gaps in our understanding and to better position libraries to meet the needs and expectations of academic users and university and college administrators. The white paper and the forthcoming research should be of interest to anyone trying to cope with or make sense of transformational change in academic libraries. To reach a broader audience, the white paper has been revised and updated for publication in portal: Libraries and the Academy. The article examines the importance of understanding how and why libraries are changing, analyzes the limitations and difficulties of traditional library performance measures, and explores environmental factors that may help account for why library use is changing. It concludes with an overview of research designed to develop an understanding of how user behavior and preferences affect demand for and use of library collections, services, and facilities, and a call to contribute conscientiously to the legacy of academic libraries and librarianship.

I n November 2000, the Digital Library Federation (DLF) and the Council on Library and Information Resources (CLIR) commissioned a white paper to initiate discussion of how and why libraries are changing. Eight academic library directors met with representatives from the DLF and CLIR in March 2001 to discuss the issues. The outcome of this meeting was a proposal to conduct research that will begin to fill significant gaps in our understanding and to better position libraries to meet the needs and expectations of academic users and university and college administrators. The white paper and the forthcoming research should be of interest to anyone trying to cope with or make sense of transformational change in academic libraries. To reach a broader audience, the white paper has been revised and updated for publication in portal: Libraries and the Academy. The article examines the importance of understanding how and why libraries are changing, analyzes the limitations and difficulties of traditional library performance measures, and explores environmental factors that may help account for why library use is changing. It concludes with an overview of research designed to develop an understanding of how user behavior and preferences affect demand for and use of library collections, services, and facilities, and a call to contribute conscientiously to the legacy of academic libraries and librarianship.

The Problem
Academic libraries are changing in response to changes in the learning and research environment and changes in the behavior of library users. The changes are evolutionary. Libraries are adding new, digital resources and services while maintaining most of the old, traditional resources and services. Finding and funding the appropriate balance of digital and traditional initiatives challenges strategic and financial planners. Library administrators feel pressured to respond to the transforming needs and expectations of users, and in some cases, are pressured by university and college administrators to account for their expenditures and demonstrate the outcomes they achieve. Data are required to respond to these pressures, find the appropriate balance, and plan for the future. Libraries need data to justify their existence, secure resources, advocate their initiatives, and know what's happening. We need data that illuminate not only how, but also why libraries are changing if we are to explain shifting patterns of library use, envision the trajectory of our evolutionary path, and win or bolster support for the changing directions of academic libraries.
What do we know about how and why libraries are changing? We know something about how libraries are changing, but with close scrutiny must admit that even this knowledge may be incomplete and uncertain. We know almost nothing about why libraries are changing because our traditional data collection practices tend to be myopic, counting selected activities within our purview and relying on anecdotal evidence about the larger context in which we operate as a basis for interpreting our data. If our discipline is library science, we've overlooked some significant independent variables that influence the dependent variables of library use. Let's examine what we know about how libraries are changing. Traditional measures quantify a library's raw materials or potential to meet user needs (inputs), and the actual use of library collections and services (outputs). Input and output statistics reveal changes in what libraries do over time; for example, they provide a longitudinal look at the number of books purchased and circulated per year. Traditional approaches to measuring inputs and outputs focus on physical library resources. Libraries are struggling with what to measure and how to measure inputs and outputs in the digital environment. We currently have no standard, comparable data to assess digital library trends within or across academic libraries. Similarly, usage data from commercial vendors of electronic resources cannot be compared easily because they measure or define the data differently. Even input and output data on traditional library resources reflect these problems. The data that libraries gather may not be consistent within or across institutions, so interpretation is difficult and the value of time-series trends and peer comparisons is dubious.
In the parlance of traditional library performance measures, the purpose of all inputs and outputs is to achieve outcomes. Outcomes are measures of the impact or effect that using library collections and services has on users. Good outcome measures are tied to specific library objectives and indicate whether these objectives have been achieved. 1 Outcomes assessments may indicate how well user needs are being met, the quality of library collections and services, the benefits or effectiveness of library expenditures, or We know almost nothing about why libraries are changing because our traditional data collection practices tend to be myopic, counting selected activities within our purview and relying on anecdotal evidence about the larger context in which we operate as a basis for interpreting our data.
whether the library is accomplishing its mission within the larger institution. Outcomes assessments can be difficult and expensive to conduct. For example, how do you articulate and assess a measure of the library's impact on student learning and faculty research? Libraries have no standard definitions or instruments with which to make such assessments and no source of aggregate or contextual data to facilitate comparing and interpreting their performance. If university and college administrators do not require outcomes assessments, libraries may not pursue them. What we know about how and why library outcomes are changing is miniscule and speculative.
Traditional library performance measures do not cover the full scope of how libraries are changing or explain why these changes are occurring. For example, trend data indicate but do not explain why library use varies in relation to library size. Explanations of why libraries are changing require contextual information and interpretive techniques that we currently do not have. The tasks of interpreting and comparing data are confounded by different institutional goals and local library policies. Differences in institutional mission affect not only support for and (therefore) use of the campus library, but the library's commitment to data collection and analysis. Confronted with these difficulties and yet clamoring for some vision of what is happening, the tendency is to aggregate existing heterogeneous data to reveal trends, then interpret the normalized data cautiously because they may be misleading. Even if these complicated problems were solved, traditional library measures would still provide an incomplete picture of the information landscape because they focus strictly on information services provided by libraries, ignoring information services provided by other entities on or off campus. Real library science requires examining activity beyond our walls and websites -activity that affects and helps explain behavior within our walls and websites.
The absence of standard definitions and procedures for gathering and interpreting reliable information that would enable us to document and to explain shifting patterns in library operations and use is adversely affecting strategic planning and the cases that library directors must make to win or bolster support for the library and its changing directions. Academic libraries cannot prepare effectively for the future or position themselves on campus until they understand their changing roles in the current learning and research environment, which is radically different from the environment a decade ago. Understanding and evaluating library usage patterns and developmental paths are prerequisites to formulating a critical and appropriate response to widespread, rapid changes in higher education. Arriving at this understanding absolutely requires a thorough examination of library assumptions and practices, and an exploration of changes in libraries and in the larger context in which libraries operate.
Academic libraries may be gathering data because they are easy to gather or because they have always been gathered, rather than gathering data that inform clearly articulated purposes or important decisions to be made. We need to gather meaningful, useful data. We do not have the resources or mandate to do otherwise. Yet what are we doing? Though ARL explicitly discourages interpreting quantitative rankings as indicative of performance or an assessment of quality, 2 we continue to use traditional measures of library inputs, outputs, and calculated ratios to rank and compare libraries. We do this along lines well entrenched in the profession, but invest little if any effort in surfacing the assumptions behind these ratios and comparisons or gathering the contextual information that would enable library administrators to interpret the data critically and apply them to their current strategic planning and case-building efforts. What does it mean, for example, to know that X number of books was added to the collection this year, Y materials were circulated, or Z reference questions were answered? Though ratios that relate traditional output measures to the size of the campus community facilitate comparison, what good is it to know the ratio of total volumes or library staff to the student population? Are we assuming that more is always better than less? Is it necessarily bad if market penetration of interlibrary loan or reference service reaches only a small percentage of students and faculty? Is there some magic formula for allocating the appropriate percentage of a library's total budget to materials, staff, and operating expenses that will guarantee library outcomes in line with the university's mission? Ratios and percentages may be interesting indicators of local trends or progress toward local goals, but what do they really mean for the future of libraries and librarianship? Web server statistics are another case in point. The data easily gathered by web servers may be interesting, but their meaning and application are elusive. Is the number of hits on a web page low because the page is unnecessary or because it is difficult to find 4 Traditional library performance measures fail to explain fully what's happening in libraries today because their scope is too narrow to encompass the field of change. For example, traditional measures do not capture sufficiently the readily apparent changes in the definition, preservation, and delivery of library collections. In the past a "collection" was what the library physically owned. Records in the library catalog referred to items in the collection. Libraries now license access to remote electronic collections that they do not own. The library catalog contains records with interactive URLs pointing to these licensed items. If a print subscription is cancelled, the library retains ownership of the (previous) physical volumes. If an electronic subscription is cancelled, the library often does not retain access to the (previous) digital volumes. In the past, multiple purchasing or subscribing libraries in effect archived and preserved print publications. In the digital arena of licensed access, few libraries continue to play this role and they do so only through negotiated agreement with the publisher.

New technologies have rendered traditional measures less effective in explaining what is happening in libraries.
Traditional library performance measures do not reveal these significant changes or the real concerns or serious implications that arise from them.
Commercial publishers and information aggregators play a more active role in collecting, organizing, and preserving information in the digital environment than they did in the print environment. As the work of librarians in this arena declines and new technologies change relationships among libraries, publishers, authors, and artists, the role of libraries is shifting. Libraries are publishers when they digitize collections, host journals that are "born digital," or assemble student or faculty works online. 5 Librarians are politicians when they lobby faculty authors not to sign away copyright to a print publisher, who then requires them or the library to pay for use of their own works. They are teachers when they help users develop information retrieval and evaluation skills in the digital environment and assume greater responsibility for the learning and research outcomes of their institution. They are researchers when they conduct user studies to assess user needs and expectations or the usability of their digital resources and websites. Librarians are expected to employ a greater variety of research methods in the digital environment than they did in the print environment (for example, user protocols, card-sorting studies, cognitive walk throughs, heuristic evaluations, and paper prototyping), and to collaborate with a wider range of people than in the past, including computer scientists, graphic designers, pedagogy experts, archivists, and museum curators. The core competencies required to perform these new tasks are different from those required of librarians in the traditional print environment. Traditional library performance measures do not-because they were not designed to-capture these changes or their implications.
New technologies are also changing the services that libraries provide; for example, online reference and instruction, desktop document delivery, self-checkout, and userinitiated library loan and direct borrowing. Librarians disagree about whether the shift to user-initiated services is a needed simplification of library operations or a cheapening or devaluing of library services analogous to fast food. 6 Usage statistics and cost analyses of these services are not readily available, but even a simple change in service can have significant impact on library operations. For example, the shift to providing email notices of overdue books and enabling online renewals resulted in a significant drop in revenue from fines in Carnegie Mellon University Libraries. According to a survey conducted by the Digital Library Federation, several libraries experienced a significant decline in revenue from fines and photocopying when electronic reserves were implemented. 7 Again, traditional measures do not-because they were not designed to-capture these changes or their implications.
Whether the fundamental mission of libraries has changed may be a matter of interpretation or local policy, but the environment and circumstances in which libraries pursue their mission is dramatically different from the environment and circumstances of the past. In contrast with library philosophy and practice centuries ago, libraries today often acquire materials and offer services "just in time," rather than "just in case." The cost of access to information appears to be more affordable than the cost of ownership. The escalating cost and volume of publications over time, widespread adoption of technology, and reduced barriers to Internet access may account for the trends of licensing access, increased use of interlibrary loan, and speculation that the digital divide is disappearing. 8 If the access model continues to offer more information at less cost to an increasing number of people, the worst-case scenario would reserve the ownership model for only high-cost, low-use materials. 9 Though this worst-case is unlikely to happen (at least in our lifetime), the access approach to acquiring library materials is risky business. If no one purchases or preserves the materials, interlibrary loan is not viable. If no one archives and moves digital collections to the inevitable new formats and platforms that the future will bring, access to them will be lost when the hardware and software become obsolete. 10 Libraries may have new measures to capture new inputs (for example, number of licensed databases, full text e-journal or e-book titles), but the measures do not capture the implications of the shift in library practice from purchased ownership to licensed access.
Traditional library measures indicate some differences among libraries and changes over time within libraries, but in the absence of additional standardized library measures, the consideration of contextual factors, and clearly articulated assumptions, these measures offer nothing that will help us recognize which differences or changes are significant in terms of fulfilling our mission and serving our constituencies in higher education. The situation is critical, but not necessarily dire. With additional knowledge, we can explain to university and college administrators how and why libraries are changing, demonstrate that our efforts contribute substantially and cost-effectively to the mission of the institution, and engage them in planning support for our future position on campus. To do this we need to discuss the issues, to examine our assumptions, to set strategic goals, to conduct research that will provide the information necessary to identify and to explain significant changes in library operations and use. Library directors must understand how and why libraries and library use are changing, must articulate and assess proficiencies for users and staff, and should plot a course into the future that is flexible enough to cope with the speed of change precipitated by information technologies and the Internet.
According to Raymond Kurzweil, the rate of technological progress currently doubles every decade, so a hundred years of progress at the current rate will happen in twenty-five years. Change itself will reach an exponential rate of growth by 2015. 11 We know that we cannot plan effectively for the future by projecting increased access to current technologies, but we dare not be discouraged. We must not allow the speed of change to inhibit or to paralyze our attempts to make sense of what's happening in libraries and intervene to serve our constituencies. Though admittedly change sometimes occurs so quickly that by the time a research project is completed, the results are no longer applicable we must begin now to understand how and why libraries and library use are changing if we want to position libraries effectively in higher education a decade from now. 12

Trends in Traditional Library Performance Measures
What do we know about how and why libraries are changing? All that we appear to know with confidence is what trends have emerged in traditional library inputs and outputs. The trend data are indicative but not explanatory of change. They are difficult to interpret because they lack context and are rife with hidden assumptions. Nevertheless, reasonable speculation abounds to account for the trends. The trends discussed in this paper are based on an examination of aggregate ARL data, reports from selected Oberlin Group libraries, and the recent survey of usage and usability practices at DLF member institutions. 13 The points address trends over the past five years and do not necessarily reflect every library's experience. Library trends and the contexts for interpreting them are significantly different across institutions.

Trends in Traditional Input Measures
Budgets. With rare exception, over the past few years, libraries appear to have experienced slight increases in both their materials budget and educational and general operating (E&GO) budget. Some libraries are given additional funding when their institution creates new degree programs or research centers. Other libraries are expected simply to stretch their existing materials budget to cover the new information needs. What's happening with staff salary lines is unclear, but even when salary lines are increased, they seldom keep pace with inflation. Current budgets appear to be insufficient to simultaneously: • Keep pace with the skyrocketing cost of materials • Maintain traditional resources and services • Mount new digital resources and services • Purchase and replace an increasing array of technologies that rapidly become obsolete • Recruit or retrain staff with the requisite skills to use and maintain the technologies • Accommodate the shifting need for space and supplies precipitated by increased automation and use of digital resources According to the 1986 Standards for College Libraries published by the Association of College and Research Libraries (ACRL), "to sustain the range of library programs required by the institution" library budgets "shall be six percent of the total institutional budget for education and general purposes," excluding capital costs and the costs of physical maintenance. The appropriation should be more than 6 percent if the library is responsible for acquiring, processing, and servicing audiovisual materials and microcomputer resources. 14 Another rule of thumb for a successful library operation is that the percentage of increase in library budgets each year should equal or exceed the percentage increase of tuition and fees. 15 Regardless of these guidelines, many libraries receive budget increases smaller than the suggested percentages, and there is no evidence that they are not providing the resources and services their institutions expect. Perhaps this is why arguments that the library is entitled to such budget increases fail with university and college administrators and why the January 2000 revision of the ACRL Standards does not make these claims. 16 Endowments, monetary gifts, and different campus funding formula complicate efforts to understand the financial situation of academic libraries and the implications of their budgets. Aside from these considerations, library expenditures do not provide administrators with a complete picture of the cost of library resources and services in a networked environment. The cost of the campus-computing infrastructure must also be taken into account. Traditional library performance measures and unsubstantiated claims about the cost of successful library operations are inadequate to convince university and college administrators to increase the library's budgets. Libraries need to define and determine how to measure what constitutes a successful library, and to figure out what success costs in a particular campus context and what percentage of that cost is to be provided by the institution, endowments, gifts, grants, etc.
Collections. To keep pace with increasing user demands for more desktop delivery of materials, libraries have been spending a growing percentage of their materials budget on licensing access to electronic resources. Over the past five years, some libraries have increased the number of serial and monograph titles purchased. Many libraries, however, have canceled serial subscriptions and purchased fewer monographs during this same period. Obviously collection growth is constrained by the financial situation of the library. The increased cost of materials, a more effective distribution of materials through collaborative purchasing (via consortia, organizational networks and inter-institutional agreements), or a shift in user expectations may explain the decline in acquisitions. "Just in time" information delivery may be becoming an acceptable replacement in some cases for the traditional "just in case" archival imperative. Some libraries may be operating with fewer physical volumes per student than in the past, but there is no contextual information in which to determine if this is good or bad for these library users. The belief is that users have access to more information overall. Martha Kyrillidou describes an "increasing and widening gap between the 'access' model libraries and the 'ownership' model libraries," which may simply be a way for libraries to focus on what they do best "rather than trying to be everything to everybody." 17 Staff.
Overall staff size appears to have been declining slowly over the past five years, but many library systems departments are hiring more people to maintain the equipment and software. Libraries are also hiring user interface designers to help develop their websites and researchers to conduct user studies, assess organizational effectiveness, and manage library statistics. Clerical positions are being eliminated throughout the library and positions are being created or reclassified at higher levels (with higher salaries) because more sophisticated technical or managerial skills are needed now than a decade ago. Typically, new or upgraded positions are accommodated by combining open positions because salary lines are insufficient to do otherwise. The need to retrain staff to keep pace with technological change adds the burden of finding increased funding for travel and training in an already strained budget. 18 Some libraries may be operating with fewer staff per student than in the past, but again there is no context in which to determine if this is good or bad. It may be fine in some library environments, but not others.

Libraries need to define and determine how to measure what constitutes a successful library, and to figure out what success costs in a particular campus context and what percentage of that cost is to be provided by the institution, endowments, gifts, grants, etc.
Equipment. Most libraries have replaced the card catalog with an automated system that runs on a powerful server. Microform equipment and photocopiers have been augmented with other computer hardware, software, systems, and peripherals, for example, to view and print full-text journal or newspaper articles. Many libraries have capital budgets insufficient to replace all of this equipment before it becomes obsolete. Equipment purchased with one-time funding from grants adds to the burden of replacement costs. Tracking all of the equipment and planning its replacement is a tedious task. Aside from these concerns, traditional measures do not help libraries with equipment planning. They provide no contextual information that would facilitate calculating how many computers the library should provide for public use. Relevant data to inform this decision would include the number of students who own a desktop or laptop computer, the number of computers available in public clusters or laboratories on campus, the number of electronic resources provided by the libraries, the purpose and configuration of public computers in the library, and the use of the library by the general "walk-in" public. Lacking this information, difficult and time-consuming queuing studies may be required to determine whether the number of public computers in the library is appropriate for the user population.
Space. Many libraries have been reducing or eliminating reader and staff spaces for years to accommodate growing physical collections. Many use or are considering offsite storage to solve their space problems (and wondering how to fund offsite storage from an already strained budget). Current library standards address different types or purposes of space and the square footage to be allocated to them. The standards provide guidelines for the dimensions of a designated reader, staff, or collection space, depending on its purpose, and guidelines for how to calculate the need for each type of space based on the type of library. For example, guidelines describe how to calculate the number of lounge chairs, individual study carrels, and group study tables a residential campus should have based on the campus population. The standards do take into account the space occupied by technology (computers, printers, scanners, fax machines, etc.), and the classroom space required for library instruction. 19 Often library spaces are smaller or fewer than the standards or guidelines propose. Whether this is good or bad depends on the local context. For example, at an institution where 95 percent of the students own their own laptop, 75 percent of the use of electronic resources is remote, and public computers in the library are seldom used by people not affiliated with the institution, it's probably fine for the library to provide fewer readers spaces and public computers than the guidelines suggest. However, on a residential campus where few students own computers, few computers are available for student use elsewhere on campus, and many people unaffiliated with the institution use the computers in the library, it is probably not good for the library to have fewer reader spaces and public computers than the guidelines propose.
Traditional measures or standards for library space allocation have been ineffective if not irrelevant in efforts to convince university and college administrators that the Internet and digitization are not a near-term method of accommodating collection growth or solving a space shortage in the library. Rough calculations performed at Carnegie Mellon indicate that licensing and creating digital content will not begin to alleviate overcrowding in the libraries until 2006, with meager space savings of 16 percent.
Libraries must digitize 100-125 volumes at a cost of (at least) $2500 to free ten square feet of floor space. 20 We need to understand how the licensing and creation of digital content affect use of library facilities, the need for collection, reader and staff spaces, and the appropriate allocation of library space for the campus.

Trends in Traditional and Emerging Output Measures
Materials circulated. Use of video and other media appears to be increasing. Depending on the local context, circulation and in-house use of print resources may be increasing or decreasing. Aggregate ARL data indicate an overall decline in circulation in large libraries. However, Oberlin Group libraries report substantial increases in circulation, which suggests that library size may be a factor. We lack sufficient contextual information and data on the use of digital resources to interpret and to understand trends in the use of physical library materials. If users can find an electronic book or journal or something comparable or good enough for their purposes on the Web, when and why would they come to the library to check out a printed book or use a printed journal? In the absence of composite measures of traditional and digital library resource use and any data about student and faculty use of information resources provided by entities other than the library, what does a decline in circulation really mean in terms of supporting education and research? Traditional library performance measures cannot explain the trends they reveal.
Reserve items circulated. The circulation of print reserves appears to be declining rapidly, even in institutions that do not offer electronic reserves. Data on use of electronic reserves are not readily available. In some institutions faculty are putting fewer materials on reserve. The availability of full-text resources on the Web-provided by the library or by other entities-may account for the decline in traditional reserve items and usage. Faculty may be providing pre-printed course packs or mounting full-text materials in course management software like BlackBoard, thereby eliminating use of the library but accomplishing the same purpose. 21 The decline in reserve use could also simply reflect the behavior of current students. 22 In the absence of usage data on other information sources, what does a decline in the use of reserves mean for the quality of education and research? Again, traditional library performance measures cannot explain the trends they reveal.
Electronic reserves are popular with many students and faculty, but the added value of desktop delivery is accompanied by added costs and the loss of revenue. The cost of providing electronic reserves entails equipment, staff training, and staff time to scan, store, link, and track use of the materials. At some institutions, additional costs are incurred in seeking copyright permission to digitize the materials and to support free printing of e-reserves items in the library. Revenue declines from photocopying traditional reserve items and fines for overdue materials. Concerns about user acceptance, technical problems that could interfere with access, and the loss of revenue in some cases are sufficient to warrant running a dual system of both traditional and electronic reserves.
Reference questions answered. Use of reference service has been fluctuating for several years. It appears to have dropped significantly in the past year, but one year of data does not constitute a trend. Furthermore, gathering reference statistics is fraught with difficulties and disparities. Traditional face-to-face reference service with a librarian is being transformed by information technologies deployed to reach an increasingly remote audience, for example, electronic mail, web-based forms, and "chat" or "see you, see me" videoconferencing software. Usage statistics from these various venues may not be compiled and reported. Reference statistics may reflect only questions asked at the reference desk. Whatever venues are included in reference statistics, the annual statistical report may be projected based on sample data gathered periodically. The result can be an incomplete and misleading picture of reference service use.
Why is reference service fluctuating and what does it mean? Traditional library performance measures cannot answer this question. Is reference service being challenged by reference-like services provided by entities outside of the library? Why, for example, would users ask a reference librarian when they can Ask Jeeves, Allexperts.com, or one of the many other Ask-A services proliferating on the Web? In the absence of usage data and quality assessments of these other services, what does fluctuation or decline in the number of reference questions answered in or by the library mean for the quality of education and research? Libraries appear to have incomplete data about the reference service we provide and insufficient contextual information to interpret the data we do have.
Interlibrary loan transactions. Use of interlibrary loan (ILL) is increasing, in many cases dramatically. We have substantial information about the use, costs, and quality of ILL. 23 We know, for example, that new technologies are transforming ILL, blurring the lines between ILL and document delivery services, shifting costs, and increasing user self-reliance. Initial increases in costs for staff training, hardware, and software (like ILLiad and ARIEL) exceed decreases in staff costs associated with photocopying and mailing. However, after the start-up costs are absorbed, staff productivity improves and the cost of patron-initiated ILL is significantly less than the cost of traditional ILL. 24 The added value to users of initiating their own ILL requests, having materials delivered to the desktop, and the ability to track their requests online no doubt increase user satisfaction. What we don't know with certainty is why ILL is increasing. What is the relationship between the use of ILL and cancelled journal subscriptions, purchasing fewer monographs, the provision of online library catalogs and citation databases that index materials not owned by the library, improved service quality, and increased user self-reliance? Furthermore, how do the nature and mission of the institution and the character of the users it serves influence this relationship?
Library instruction classes. The number of traditional library instruction sessions and participants was increasing until recently, but now appears to be on the decline, perhaps because distance-learning technologies are being deployed to deliver library instruction. Libraries are beginning to gather statistics on use of online instructional materials, quizzes, and tutorials. Other environmental factors that may be affecting library instruction are the increasing technological savvy of users who can transfer skills from one vendor's databases to another, or student and faculty use of information resources and services not provided by the library, in which case library instruction is not in step with user needs or behavior. Again, we lack the necessary contextual information to interpret what a decline in library instruction sessions or participants really means or why it is happening.
Gate counts. Gate counts in many libraries are declining. Why go to the library if you can find the information you need using your personal computer, create an Internet chat room to discuss your group project, or use a similar discussion facility provided in course management software like Blackboard that integrates your class syllabus, assignments, readings, quizzes, and grades? Yet gate counts seem to increase following renovation of library spaces, which suggests that the library is still valued as a physical place. What is the relationship between gate counts and the attractiveness and suitability of library space for student and faculty work or recreation? What is the relationship between gate counts, the volume of electronic resources provided by the library, the penetration of computer ownership, and the quality of the network infrastructure on campus? We can speculate all we want, but traditional library performance measures do not provide the contextual information we need to understand trends in gate counts or the impact of gate counts on the quality of life, student learning, or faculty research.
Electronic resource use. The demand for desktop delivery of materials appears to be increasing everywhere. Use of electronic resources is growing, often more rapidly than expected. Use of older journal volumes in electronic format is increasing even more rapidly than use of electronic resources overall. 25 Users want access to more full-text electronic resources and many, including some faculty, appear to believe that everything is available in electronic format and that access to the information is free. Libraries are trying to educate their constituencies about the costs of electronic resources, and collaborating in consortia to negotiate with vendors for better prices and more meaningful usage statistics. The problems associated with vendor statistics are well known and many organizations, including the Council on Library and Information Resources (CLIR) and the International Coalition of Library Consortia (ICOLC), are working with vendors to create standard definitions, methods of measurement, and delivery formats to facilitate comparative analyses.
Printing and photocopying. The trend in many libraries is that the volume of printing is increasing and the volume of photocopying is decreasing. Librarians speculate that this is because of the increased availability and printing of full-text e-journals and ereserves. However, as with all of the measures described in this article, additional contextual information is needed to understand what is actually happening in any given institution. In cases where the library recovers costs for printing, but printing elsewhere on campus is free, printing may drastically decline, while photocopying remains constant. 26 At some institutions where printing is free campus wide, the volume of printing in the library has leveled off or slightly declined over the past year or two, but photocopying continues to decline dramatically. For example, at Carnegie Mellon Why go to the library if you can find the information you need using your personal computer, create an Internet chat room to discuss your group project, or use a similar discussion facility provided in course management software like Blackboard that integrates your class syllabus, assignments, readings, quizzes, and grades?
University Libraries, photocopying has declined by approximately 50 percent over the past four years. Printing was increasing rapidly until 1998-1999, but has been slowly declining since then. At the University of Washington, the decrease in photocopying does not equal the increase in printing; fewer total document reproductions are being made in the library. We do not understand the relationship between photocopying and printing, or the relationship between printing and the availability of electronic resources or user readiness to read these materials online. We do know that regardless of the volume of printing, more expensive printers and supplies are required now than in the past to accommodate printing color and new file formats like PDF, TIFF, GIF, and JPEG. Assuming that libraries do not offer free photocopying, if they can recover costs for printing, the new income may eventually offset the decrease in revenue from photocopying. The equipment that enables cost recovery for networked printing is expensive. "Cost recovery" in this context probably means strictly the cost of paper and toner. The revenue from printing will probably not fund the replacement cycle for the cost-recovery equipment or the printers themselves.

Trends in Outcomes Assessment
Efforts to measure the impact or influence effect of library collections and services are hampered somewhat because they rely on output measures that are difficult to interpret and on institutional performance objectives that frequently are only vaguely defined and perhaps even contested on campus. Outcomes assessments rely on output measures in the sense that they attempt to measure the achievement of some objective based on the use of library resources-though there's no way to control or monitor mitigating factors like the use of resources not provided by the library that may influence the objective being measured. Clearly defined performance objectives and methods for measuring their achievement are essential to demonstrate that libraries are accomplishing their mission and contributing to the mission of their institution. Some academic libraries are being pressured to demonstrate the impact of their efforts on student learning and faculty research. Others are not. If libraries are serious about outcomes assessment, librarians need to engage administrators, faculty, and students in a discussion of the outcomes they expect libraries to achieve, then take the lead in articulating measurable objectives and proficiencies to be derived from use of library collections and services given the nature and mission of the college or university.
Libraries are beginning to measure the effect of their collections and services on users. In the absence of pressure or methods to measure clearly articulated objectives or proficiencies, the effort to date has focused on assessments of user satisfaction and service quality. The easiest and most popular measurement is a survey of user satisfaction, but this assessment alone is a "facile outcome" because it provides little if any insight into what contributes to user dissatisfaction. 27 Nevertheless, the trend in these assessments indicates that user satisfaction is a function of the individual user's perception of the quality of the library's resources, the competence and demeanor of library staff, and the physical appearance of library facilities. 28 In contrast, service quality focuses on reducing the gap between users' expectations of excellent service and their perception of the service delivered. Studies of service quality assess the collective experience of many users and suggest that reliability is the most important characteristic of service quality. 29 Though these approaches to outcomes assessment are sufficiently interesting and informative to warrant continued use and development, they may be doomed to failure in terms of winning or bolstering support for the library because the outcomes they assess are not based on objectives or proficiencies aligned with the nature and mission of the institution or integrated with the campus's overall assessment efforts. Assessments of user satisfaction and service quality are not viable substitutes for assessments of learning and research outcomes. Conducting meaningful and convincing outcomes assessment is the shared responsibility of librarians, faculty, and university and college administrators.
Other dimensions of outcomes assessments relate to costs. Libraries want to analyze both the effectiveness and the benefits of their financial investments in collections, services, staff, and facilities. By identifying areas where expenditures are not efficient or sufficiently beneficial, such analyses can prompt libraries to explore alternatives, streamline internal workflow, and improve the users' experience. Such analyses also help library directors keep university and college admnistrators aware of the cost of quality library operations.
Assessments of cost-effectiveness measure the ability of the library to deliver outputs and outcomes with economic efficiency. Understanding the costs associated with library resources and operations is extremely difficult and time consuming. The difficulty of allocating the costs of a particular collection or service to content, staffing, facilities, hardware, and overhead is compounded by the difficulty of distinguishing between start-up costs and ongoing expenditures in an era of rapidly changing technologies, prices, and workflows. A thorough cost analysis of an electronic product, for example, must include purchasing or licensing costs and the operational costs of selecting, ordering, cataloging, networking, printing, associated reference questions, instruction, technical support, and maintaining related web pages and links. Furthermore, such analyses must be repeated to keep abreast of changes. For example, a 1993 functional cost analysis of interlibrary loan indicated that staff costs accounted for 77 percent of the total cost of the service. 30 Given the technological changes and accompanying cost shifts in providing patron-initiated ILL, this analysis needs to be repeated.
Assessments of cost-benefits measure how users subjectively value library resources. Cost-benefit analyses are also difficult and time consuming to conduct. They require a clear definition or understanding of what users value, a method to assess how a particular resource, service, or product feature provides what users value, and some meaningful way to relate this qualitative indicator to a quantitative indication of costs. Service quality assessments can uncover some of what users value, but what is the appropriate role of academic librarians when what users want (value) is not what they need? For example, undergraduate students appear to value convenience and delivery speed more than the quality of the information delivered. Purchasing electronic resources or digitizing information add value, but often increase costs. In an environment where faculty want electronic access but are reluctant to dispense with print, the additional expense of acquiring and maintaining both print and electronic collections is difficult to determine but predicted to be exorbitant and unsustainable. 31 What values can or should libraries support and at what cost?
A study of costs associated with electronic journals conducted at Drexel University illustrates the complexity of doing cost analyses. 32 The Drexel study reveals that providing electronic journals creates significant shifts in staffing and operational costs. While the purchasing power of an electronic journal dollar is greater than that of a print journal dollar because of bundling and backfiles, providing the requisite infrastructure of hardware, software, and systems staff significantly increases operating costs. The cost of staffing increases with electronic journals, partly because library administrators must be involved in negotiating licenses, joining consortia to get better pricing, developing collection strategies, and managing change (restructuring workflow, reorganizing staff positions, and building staff with appropriate skills, including the skills needed to track usage statistics and produce meaningful reports). The cost per unit of processing electronic journals is less than print, but requires staff with computer skills and the ability to adjust to continuous change in procedures. Even if the physical journals are no longer maintained, the increased cost of system maintenance, license negotiation, printing, and reference is greater than the decreased cost in physical collection acquisition and maintenance (including check-in, claiming, circulation, reshelving, weeding, binding, converting to microform or offsite storage). The question remains, is the overall increase in costs incurred by providing electronic journals offset by the added value of desktop delivery to users? We evidently believe that it is, but how do we effectively make that case to university and college administrators to secure the funding required to purchase needed electronic journals?
Difficult tasks take a significant amount of time-which means they are expensiveto accomplish. Librarians and staff are already overburdened with increasing and changing responsibilities. Even if the desired objectives or proficiencies have been articulated and the library can allocate human and financial resources to conduct outcomes assessments, many libraries do not have people with the requisite skills to conduct the research or to present the results effectively in the limited time typically allotted for such presentations to university and college administrators.
In the absence of clear guidelines and supportive models, libraries appear to be doing what is simple. For example, libraries perform simple calculations of the costper-search or cost-per-session of electronic resource use, based on the license cost alone, to determine whether to maintain a subscription, regardless of the fact that the calculation is misleading because it ignores the many associated costs and the user-centered qualitative dimension that ought to be the focus of any outcomes assessment. Usage is not synonymous with value. Libraries need to explore alternatives and conduct more cost analyses like the 1998 journal study at Wellesley College to compare the costeffectiveness of subscribing to a title with the cost of purchasing articles from that journal on an as-needed basis. 33 Such comparative analyses would ideally be accompanied by assessments of user satisfaction or service quality with the different alternatives. Fully informed decisions interpret costs in light of what users value and what libraries are willing to pay to provide that value.
What value do users place on library collections and services? The commercial enterprise called Questia may provide a measure of the library's dollar value to students. Questia's success depends on students being willing to pay $20-22 a month for access to 50,000 digital volumes and software tools that facilitate writing their papers. If Questia succeeds, will it help or hinder academic library efforts to illustrate their value and to win support for digital library development? We don't know the answer to this question in large part because we do not really know what the library is expected to contribute to the institution's mission. Though libraries have traditionally operated and received budget increases without having clearly articulated objectives and performance measures, we have to wonder whether we will continue to win support for our initiatives in the current strained economy and networked environment of freely accessible webbased information without conducting meaningful and convincing outcomes assessments.

Environmental Factors
In the absence of contextual data, traditional library performance measures are difficult to interpret and explanations of why library use is changing are destined to be speculative. Library "science" demands that we examine the larger context surrounding us to identify independent variables in the environment that may be influencing the changes occurring in libraries and confounding interpretation of library trend data. The list of factors explored below is not comprehensive. It is presented to stimulate reflection and discussion. Exploration of additional factors is invited and encouraged.

Changes in Literate Habits
Anecdotal evidence suggests that a rapidly growing percentage of the use of electronic library resources occurs outside of the library. Though many libraries don't gather data on remote use of digital library collections and services, those that do indicate that a significant percentage of such use is remote. For example, when Carnegie Mellon was preparing its Briefing Book for the Library Advisory Board visit in 1999, Lehigh University reported that 46 percent of its electronic resource use is remote. Johns Hopkins reported that 60 percent of its electronic resource use is remote. Approximately 75 percent of Carnegie Mellon electronic resource use was remote in the early 1990s, when electronic resources were loaded and usage was tracked locally. We believe the percentage of remote use is higher now, given that 95 percent of our incoming freshman arrive owning a computer or planning to purchase a computer their freshman year. Remote access to library resources means that users of unrestricted electronic collections and services may not be affiliated with the institution.
Where once students and faculty turned to libraries, they now turn to their personal computers when they need to find information. Faculty members appreciate the convenience of the Web, but know that often the best resource is still only available in print. They know how to determine whether an information source is authoritative and In the absence of contextual data, traditional library performance measures are difficult to interpret and explanations of why library use is changing are destined to be speculative.
timely, and generally have months or years to complete a project, so interlibrary loan and document delivery services are viable options for their research. Students, in contrast, are unable to distinguish appropriate from inappropriate resources for their assignments, have little time to complete their projects (in part because they procrastinate) and are enamored of the Web. In many cases, if the information is not available on the Web, it does not exist for them. In response, librarians are developing web-based training materials to teach critical thinking skills and search techniques to remote or webenthralled users.
Focus group and survey research conducted at Carnegie Mellon indicates that undergraduate students typically turn to popular web search engines when they need to find information. These search engines index only the "surface Web," where less than 7 percent of the information is appropriate for educational or scholarly purposes. No single web search engine indexes more than 16 percent of the surface Web, yet we have no evidence that students use more than one search engine when they look for information. 34 According to BrightPlanet, the "deep Web" is 500 times larger and growing faster than the surface Web. The deep Web provides information in all disciplines, for all constituencies; that is, 1,000-2,000 times better in quality than the surface Web. Approximately 95 percent of deep web content is publicly accessible without fees or subscriptions, but deep web content, like scholarly commercial resources licensed by the library, is not indexed and therefore not accessible using popular web search engines. 35 The growing concern among academic librarians interviewed by the author in the DLF usage and usability survey is that many undergraduate students may be searching only 0.03 percent of the Web to complete their assignments, ignoring entirely the books, journals, databases, full-text digital resources and other scholarly materials provided by the library. The consensus appears to be that undergraduates are using library collections and services less than in the past because access to the surface Web is easy and convenient. Librarians and faculty are concerned that the quality of information and tools on the surface Web imperils the quality of student learning. For this reason, some faculty members do not allow their students to use web resources in class projects.
Even if undergraduate students turn to the scholarly electronic resources licensed by libraries, their search skills are poor. They seldom if ever use advanced search features, do not understand that result sets are not necessarily organized by relevance to their query, and look only at the first couple of web pages of ten to twenty items retrieved. Carnegie Mellon students who use library-vetted electronic resources want a way to restrict their queries to retrieve just full-text electronic resources, regardless of whether the best material for their assignments is available only in print. (What is the role of the academic librarian here?) If undergraduate students come into the library, they seldom consult a reference librarian. These are desperate times for outreach to students.

Changes in Students and the Curriculum
Even if the Web accounts in part for changing patterns of library use, other factors must be considered if we want a complete picture of the environment in which libraries operate and the constituencies we serve. Do current students read less or have less intellectual curiosity than former students? Are they just too enamored of the surface Web or too busy to explore or unmotivated to learn how to explore what libraries offer? Years ago, Carnegie Mellon students had difficulty using the online catalog, but could find the books they needed on the library shelf using the Dewey Decimal System. Today, they have no difficulty using the web-based catalog, but they cannot find the books they want on the shelf because they do not understand the classification system. They ignore the numbers to the right of the decimal point and appear to be oblivious of the alphanumeric Cutter number. 36 What library skills are high schools teaching?
Students today want 24/7 access to digital library collections and services, as evidenced by a study of the online habits of 2,000 American college students conducted by netLibrary.
• 82 percent of the students surveyed own a computer and "virtually all of them use the Internet." • 93 percent claimed that finding information online makes more sense than going to the library. • 83 percent said they were frequently unable to get the materials they need from the library because it is too late or too early to go to the library. • 75 percent said they do not have enough time.
• 75 percent liked the convenience and 71 percent liked the time saved by finding information online any hour of the day. 37 Library directors can only wonder what impact Questia will have on student use of the library. Perhaps students have (or will have) little if any need to use the library.
Technology is also affecting faculty teaching and research, which no doubt has an impact on library use. Faculty may be assigning students fewer projects that require use of library resources. Like some librarians, some faculty members are being held accountable for the educational effect of their efforts. Does the preparation of coursepacks or the use of courseware like BlackBoard that bundles all the materials students need for a course simplify faculty outcomes assessments by eliminating the independent variable of student skill in using the library? 38 Do faculty in certificate and graduate degree programs that are designed to move students quickly through with minimal burden on their time pre-package materials to eliminate the need to spend time using the library? What influence does the growing interest in online courses and distance education have on library use? Access to most licensed electronic resources is restricted by IP address. Libraries provide proxy servers for campus users without campus IP addresses to access restricted resources, but this technology is problematic, high maintenance, and may be circumvented by pre-packaging course materials in printed course packs or course management software.

Changes in the Technological Infrastructure
As long as the space provided is attractive and suitable, students will probably continue to come into the library for quiet or group study or to socialize with friends. But as computers become more affordable and more and more students purchase their own, how does this affect library use? Students clearly prefer desktop delivery of information and if they have a personal networked computer, they may think they have no need to come to the library-hence the decline in gate counts and reduced circulation of traditional library materials. Do differences in student ownership of computers, the bandwidth of the campus-computing infrastructure, and the volume of electronic resources provided by the library account for the different trends in circulation and gate counts among large and small libraries? Is the percentage of remote use of electronic resources and services influenced by the penetration of computing and network bandwidth of the campus? What impact do wireless access and the growing number of student-owned laptops have on library use? At what point will student ownership of laptops reduce the number of public computers that the library needs to provide?
Perhaps equipment configurations and replacement cycles are also a factor in library use. Why, for example, would students come to the library to retrieve electronic resources using obsolete equipment when their own computers are faster, better equipped to handle multimedia, and loaded with all of the software they need to complete their assignments? Libraries may restrict their public workstations to information retrieval tasks only, preferring students to go elsewhere to do e-mail, word processing, programming, etc.

Information Resources and Services Provided by Entities Outside of the Library
What influence do Ask-A services, Questia, and websites like LibrarySpot have on student use of their local library? We need systematic quantitative and qualitative studies of these information resources and services to understand their effect on library use and the constituencies that libraries aim to serve. If the goal is to provide quality service to users, does it matter whether libraries provide the service or someone else does? If students are using these services and their quality is poor or inconsistent, how do librarians direct them to better services and teach them how to critique the information they retrieve? Perhaps most importantly, how do libraries factor in the impact of these services in their efforts to assess the educational outcomes of the collections and services that they provide?

Proposed Research
Librarians must work with university or college administrators, faculty, and students to articulate clearly what objectives they expect the library to achieve given the nature and mission of their institution. Meanwhile, librarians must continue to develop strategic plans for the future and endeavor to win or bolster support for the library and its changing directions. To do these tasks effectively, we must understand how and why libraries and library use are changing. Traditional library measures do not capture the scope of the changes or provide a context in which to interpret them. Substantial research is required to fill the gaps in our understanding. Existing trend data, anecdotes, and speculation, however reasonable, are insufficient evidence for planning and case building with university and college administrators. We need a complete picture of library activity and the world in which we operate, which means meaningful traditional and digital library performance measures, and measures of significant factors in the environment that affect library use. Furthermore, we need comparable data, so that the cases we present to administrators illustrate how we stack up to our peers and provide them with a context for interpreting the data.
Performance measurement is admittedly a political activity. Electronic resource vendors are hesitant to share certain kinds of usage data for fear libraries will cancel subscriptions. 39 Similarly, libraries are hesitant to share certain kinds of usage data for fear their institutions will interpret the data to mean that the value or importance of libraries is declining and consequently cut their budgets. However, non-disclosure of data inhibits understanding and jeopardizes effective planning. 40 The challenge is twofold. First, stakeholders must understand how to interpret the data and the context in which it is meaningful. Second, they must trust one another. To understand how and why library use is changing, library directors must be bold but cautious, strong yet sensitive. The importance of comparative data and trust must outweigh concerns about sharing confidential information that may reveal significant declines in library use and the striking impact of competitors in the information marketplace. The bottom line is that to understand how and why libraries and library use are changing, we must change what we're doing. We need to broaden the scope of our vision and research to include not only traditional and digital library measures, but measures of activities beyond our walls and websites. We must revitalize the science of librarianship by examining the independent variables that constrain use of library collections and services.
Management literature offers encouraging guidance for coping with and cultivating change. For example, the eight-stage process for creating change outlined by John P. Kotter begins with • Establishing a sense of urgency by examining competitive realities and identifying opportunities • Creating a guiding coalition to articulate the vision, plan the strategy, and lead the change 41 The usage, usability, and user support initiative of the Digital Library Federation has embraced this role. The white paper on which this article is based served to establish the sense of urgency and motivate selected library directors from small liberal arts colleges, mid-size and large universities to attend a meeting convened by the DLF and CLIR in March 2001. This group accepted the role of guiding coalition and the task of designing research to examine the competition and begin to fill the gaps in our understanding of how and why libraries are changing. The range of changes that libraries are experiencing and the many environmental factors that must be explored to explain them are too broad for a single research study to address. To help decide where we should begin, the DLF conducted an informal survey prior to the March meeting to discover what library directors considered high priority areas for research. The results indicated that the top research priority is to better understand student and faculty information seeking and usage behaviors. We need to know more about how academic users find and use information to meet their needs and expectations for information.
To understand how and why library use is changing, library directors must be bold but cautious, strong yet sensitive.
The outcome of the March meeting was an agreement to commission Outsell Inc. to survey academic users. The research will contribute empirical evidence to our currently speculative understanding of the overall information environment, how academic users view the library's role or position in this environment, and how user behaviors and preferences affect demand for and use of library collections, services, and facilities. Plans are for Outsell Inc. to conduct quantitative telephone interviews of faculty, graduate students, and undergraduate students in different disciplines in liberal arts colleges and public and private universities to discover • What information and services they use to support research, teaching, and learning • How the different user groups locate, evaluate, and use these information sources and services • Gaps where information needs are not being met This baseline understanding of user behaviors will help academic libraries and their institutions plan information services focused on the current and emerging needs of their users, and avoid investing in what is not, or is no longer, important to them. The data will facilitate evaluation of the library's current and possible future roles in the information landscape, and provide essential contextual information for interpreting known trends in the use of library resources. The results of the study will also benefit the academic community by helping publishers and other information providers create better information products based on increased knowledge of user needs and preferences.
The DLF and CLIR are currently seeking funding to conduct the study. Contingent on securing financial support, the tentative timetable is to finalize the survey questions by September 2001, conduct the research, and submit the reports by February 2002. The reports will be publicly accessible on the DLF website. The DLF will deposit data gathered during the study with the Inter-University Consortium for Political and Social Research (ICPSR), where they will be accessible for non-commercial use. Following the study, the DLF will collaborate with library directors in the guiding coalition to assess how the research results help explain trends in library use and how the results might or should influence strategic, organizational, financial, and human resource planning for academic libraries. A few brief case studies will be prepared to demonstrate how an enhanced understanding of the academic information landscape and its use can facilitate interpretation of trend data and inform plans for collections, services, and facilities.
For practical reasons, the DLF and CLIR invited only a small number of libraries to participate in the March meeting and guide the resulting research, but all academic libraries are invited to follow the group's work and encouraged to contribute additional research that will deepen our understanding of academic users, the environment in which we work, and library costs, benefits, and use. We need to change the current situation in which we cannot explain the trends we track, and in which we cannot gain support for our initiatives because the cases we make to university and college administrators lack persuasive data or other evidence. Stephen R. Covey says that to change a situation, we first have to change ourselves. 42 We must expand our focus beyond library inputs and outputs, study the behaviors and preferences of our users in the overall information landscape, assess the effectiveness and benefits of our financial investments in light of this knowledge, and be prepared to change what we're doing as a result of what we learn.
Covey says that we must put first things first because where we're headed is more important than how fast we get there. 43 Serving users is the "first thing" in librarianship. We serve them best when we know what they need, what they do, and what they prefer. Acquiring this knowledge will take substantial time and effort. That's fine, but it's time to begin. The future of libraries and librarianship and quite possibly the learning and research outcomes of our institutions of higher education will be determined by what we initiate today. The question is do we have the chutzpah to pioneer the frontier that challenges us? Do we have the fortitude to endure and persevere through the cycle that Daryl R. Connor's research indicates inevitably comes to those open to change? Are we prepared to move from "uninformed optimism" to "informed pessimism" to "hopeful realism" to "informed optimism" and finally to "completion" in the design of new measures for an expanded vision of library tradition? 44 As librarians at the turn of the millennium, what kind of legacy do we want to leave behind us? Our followers will hold us accountable for what we contribute-or fail to contribute-to defining academic librarianship and positioning academic libraries in the digital environment. The choices we make today-to change or not to change-will create a legacy for librarians and library users that defines the meaning and purpose of libraries, and structures how users will perceive and experience libraries in the future. What will our legacy be? What are you prepared to contribute to that legacy? What does your conscience say about that?
The author is a Distinguished Fellow at the Digital Library Federation and an Associate University Librarian at Carnegie Mellon; she may be contacted via e-mail at: troll@andrew.cmu. edu.