Diary

Internal Reports

Completed Papers

Papers in Progress

Conference Presentations

Site Visit and Quarterly Reports

Main DLI Page

Web Client- DeLIver

Observations of Bibliographic Tool Use at The Grainger Engineering Library

Prepared by Robert J. Sandusky for Site Visit, March 1996

Members of the Illinois Digital Library Initiative (DLI) Social Science Team conducted a small set of observations of Grainger Engineering Library users, primarily graduate students, during the summer of 1995. Our objective was to present a picture of the current usage of some of the most commonly used online bibliographic tools available at Grainger. The Testbed Team felt that such data would aid them in the design of the Illinois Digital Library interface and retrieval infrastructure. We observed people using a number of online tools such as: IEEE/IEE Publications Ondisc; INSPEC Database Ondisc; Engineering Index on CD-ROM; WWW browsers; and the local online library catalog.

Our findings were in the main consistent with our expectations. We found, for example, that subject or keyword searching without use of a controlled vocabulary is the most commonly used search strategy; that barriers to use include a mismatch between the users' conception, or mental map, of the system they are using and the system itself; that users find the operation of the interfaces and underlying retrieval mechanisms difficult; that users rely upon information presented to them at the interface for guidance rather than seeking on-line help or librarian assistance; that, in this setting, these bibliographic tools are used primarily to locate journal articles; that people will print what's available, in the fullest format possible (i.e., either full text or citations with abstracts); and that the users' searches often support a wide array of tasks such as writing journal articles or theses or dissertations, surveying primary research fields, exploring unfamiliar cognate fields, and even looking for employment. One surprising finding resulting from the analysis of these observations was that the reliability and availability of some of the bibliographic tools may be low.

The following ten questions were used as organizing categories for this set of observations:

There are four main areas for discussion resulting from this study. First, there are implications for system design. These observations suggest that library users rely most heavily upon the information provided to them on the screen for help, hints, and suggestions as to how to proceed. Rarely do they attempt to use the built-in help systems. The quality of the help systems embedded within these systems varies widely. On some systems, help is available only from a subset of the screens. The contents of the help screens are frequently not in the language of the user, but in the language of the system builder. The design of the Illinois Digital Library could benefit from incorporation of ideas from research into online help, reference, and tutorial systems (see, for example, Barrett, 1989).

Another area of concern is how the population of potential users will be introduced to the system, and once they begin using the system, how they will gain familiarity and confidence with the system. How are users introduced to existing systems? How does the nature of the introduction affect their ability to use and conceptualize the nature of these systems? How effective are the methods in use today? Are these methods appropriate for systems that most users only touch occasionally? The findings of the NASA/DoD Aerospace Knowledge Diffusion Research Project (Pinelli, et al., 1994) suggest that engineering students at the University of Illinois rely more upon personal collections of information than the collections organized by or available via the library. That study also shows that less than one third of the students in engineering and the sciences receives specific training related to the use of bibliographic resources in those disciplines. To compensate for what may be larger systemic issues, another goal for the Illinois Digital Library might be maximization of mutual intelligibility. That is, the design of the system should cue the user as to its purpose, its expectations in its current state, and the actions and responses now available to the user. Also, if possible, the design should allow the system to accept signals from the user in regard to the user's current understanding of the unfolding of the session.

Third are the management issues that emerge when considering use of bibliographic tools. The most obvious issue is based on the reliability and availability of the systems themselves. One of the bibliographic tools observed in use seems to be particularly unreliable. Recurring problems of this magnitude beg the question of how central these systems are to the academic library as an institution. If the digital library we build today is the model of the future, is the academic library as an institution ready or willing to make the adjustments needed to place such systems in a truly central place? If the academic library commits to greater digitization of humanity's knowledge, it must also acknowledge and address the associated responsibilities of reliability, availability, and ease or transparency of use, which are largely moot points in the book oriented repositories of today. Reliability, availability, and ease or transparency of use are not merely properties of technology but also imply changes in the roles and responsibilities of the staff working within the institution.

Finally, the bibliographic tools that our subjects used were with only one exception based upon the classic information retrieval model where a user's query is matched to a representation of a document. World Wide Web browsers, however, seem qualitatively different from the other systems, perhaps for the following reasons. First, a WWW browser is not really a bibliographic tool as instantiated today: searching mechanisms are primitive and the body of information lacks organization. Second, WWW browsers have hypertext capabilities which the bibliographic tools we focused on do not. At this point, there is little functional overlap between the bibliographic and WWW tools. In the future, these functional distinctions should begin to fade and the patterns of usage of newer bibliographic tools with embedded links, etc., can also be expected to merge.

Barrett, Edward, ed. The Society of Text: Hypertext, Hypermedia, and the Social Construction of Information. Cambridge, Massachusetts: MIT Press, 1989.

Bates, Marcia. "The design of browsing and berrypicking techniques for the online search interface", Online Review, 13, 5, 1989, pp. 407-424.

Pinelli, Thomas E., Rebecca O. Barclay, Laura M. Hecht, and John M. Kennedy. The technical communication practices of engineering and science students: Results of the Phase 3 academic surveys. NASA/DoD Aerospace Knowledge Diffusion Research Project: Report Number 27. December, 1994.

NOTE: The full report is available at http://www.grainger.uiuc.edu/dli/observe.htm