Diary

Internal Reports

Completed Papers

Papers in Progress

Conference Presentations

Site Visit and Quarterly Reports

Main DLI Page

Web Client- DeLIver

Software Instrumentation and User Registration

Prepared by Robert J. Sandusky for Site Visit, March 1996

The Illinois Digital Library has two primary mechanisms for automatically collecting usage/user data. First, a number of the software components are instrumented, or programmed, to collect detailed transaction logs of each user session. Second, an automated user registration process collects demographic information about each user and provides a confidential mechanism to link the data in the transaction logs to individual users.

These data are being collected to serve two primary purposes. The first purpose is to provide various sorts of management data. This includes summary data to the project management and the outside world concerning the number of users and their aggregate behavior, as well as providing system performance data to ensure that the Digital Library is operating within acceptable tolerances. The second objective met is the collection of detailed data on individual users and their behavior. Management data are provided both by the data generated by the instrumented software and by the data collected during the user registration process. For example, the registration data can be summarized to provide profiles of the Digital Library user community. The transaction logs can be used to provide summaries of user behavior in the aggregate, answering questions such as how many searches are performed during a given unit of time, what are the most frequently used functions, etc. This type of "how much, how often" information is also useful to Digital Library systems administrators in terms of monitoring performance of the Digital Library and planning for changes in demand and capacity. Information about the behavior of individual users is provided by both the instrumented software and the user registration data. Prior to being allowed access to the Digital Library, a prospective user must successfully complete a simple on-line registration process which collects contact information as well as information about the user's professional background and the extent of the user's familiarity with common computing and communications systems. Information which could be used to identify the users is kept confidential and in a separate location.

The Illinois Digital Library is an aggregation of several independently developed software systems, including database management systems, database search engines, thesauri, a user interface, SGML tools, and SGML and other data format viewers. Instrumentation is being introduced into those systems most under our control to provide details (in the form of textual transaction logs) of user interaction with each system. Essentially, each of the instrumented systems reports details on menu selections, search requests, and other specified actions in order to provide a fairly clear indication of what the user was doing at the system's interface. By logging the user's interactions with each of these systems, we can focus on one of the systems in isolation or multiple systems as components of a Digital Library. A higher level methodological question we hope to address is the extent to which these types of transaction logs can be used effectively to understand the behavior and needs of individual users, instead of using methods such as interviews and field observations.

We plan to ask the following questions of the registration and log data:

We will also use the transaction logs to learn about the context of use of the DL system in the larger networked environment (e.g., is there movement back and forth from the digital library to other networked applications or systems? Are users cutting and pasting material and mailing it on? If so, are there important patterns of communication emerging?)

We will apply a variety of methods when analyzing these data including statistical analysis (frequencies, cross-tabulation, etc.), transaction log analysis (Peters, 1993), and examination of logs in combination with interview and observation data.

Reference

Peters, T. A. The history and development of transaction log analysis. Library Hi Tech, 11, 2, 1993, pp. 41-66.

Transaction Log Format:

        Transaction code                four digits
        Date                                    yyyymmdd
        Time                                            hhmmss
        User_ID                                         a serial number
        IP Client Address
        Transaction Contents                    variable

Some specific transactions (examples):

2001 19960301 151011 00003122  255.255.12.1     Log in to DL system
2109 19960301 151043 00003122  255.255.12.1     Erase all search terms
2110 19960301 151152 00003122  255.255.12.1     AND must also contain
2111 19960301 151217 00003122  255.255.12.1     OR could have the
alternate term
2119 19960301 151221 00003122  255.255.12.1     Execute the search
2120 19960301 151228 00003122  255.255.12.1     Search results:  provide
details on retrieved
                                                        set of documents
2121 19960301 151256 00003122  255.255.12.1     Display results
2122 19960301 151415 00003122  255.255.12.1     Suggestions
2161 19960301 152207 00003122  255.255.12.1     Quit the DL system

NOTE: The Illinois Digital Library User Registration Form can be viewed at: http://anshar.grainger.uiuc.edu/dlisoc/rfpt.html