graham.p65 36 College & Research Libraries January 2004 Subject No-hits Searches in an Academic Library Online Catalog: An Exploration of Two Potential Ameliorations Rumi Y. Graham This paper describes a study that explored ways in which users’ subject- searching problems in a local online catalog might be reduced. On a weekly basis, the author reviewed catalog transaction logs to identify topics of subject searches retrieving no records for which appropriate information resources may actually be represented in the catalog. For topics thus identified, the author explored two potential ameliorations of the no-hits search results through the use of authority record cross-references and “pathfinder” records providing brief instructions on search refinement. This paper describes the study findings, discusses possible concerns regard­ ing the amelioration methods used, outlines additional steps needed to determine whether the potential ameliorations make a difference to users’ searching experiences, and suggests related areas for further research. he landmark series of online catalog studies sponsored by the Council of Library Re­ sources (CLR) in the early 1980s established unequivocally that sub­ ject searching was the most popular type of online catalog search and that catalog users experienced the most difficulty with subject searches.1–2 Spurred by the find­ ings of the CLR studies, discussions and research efforts to improve subject search­ ing in online catalogs flourished through­ out the 1980s and early 1990s. Researchers attempted or suggested ways to improve subject searching by, for example, enriching the subject content of the catalog database; enhancing browse displays and other aspects of the user– system interface; increasing the sophisti­ cation and power of the catalog search engine; and exploring user-centered rather than system-centered philosophies of, and approaches to, system design and improvement.3–19 Curiously, despite the momentum built up during this period of concentrated research, the online cata­ log sector of the subject access research front became comparatively quiet over the subsequent decade. Web Solutions? Reasons for the loss of momentum in re­ search addressing the improvement of online catalog subject searching over the 1990s remain unclear. However, we may speculate that the slowdown was perhaps indicative of the extremely complex na­ ture of the remaining subject -searching Rumi Y. Graham is Manager, Collections and Database Services, at the University of Lethbridge Library in Lethbridge, Alberta, Canada, and a doctoral student in information studies at the University of Toronto; e-mail: grahry@uleth.ca. 36 mailto:grahry@uleth.ca Subject No-hits Searches in an Academic Library Online Catalog 37 problems. Another possibility is that re­ search efforts were diverted elsewhere toward, for instance, the development of Web interfaces to online catalogs and other information retrieval systems or the creation of new Web-based information retrieval tools and resources. The mid-1990s emergence of Web in­ terfaces to online catalogs heralded the commencement of significant expansions to the accessibility and content of library catalogs. For all their added reach and content breadth, however, it is disap­ pointing that the subject-searching capa­ bilities of Web catalogs appear to be much the same as those of pre-Web, second-gen­ eration systems.20 In a discussion of new approaches to online subject access at the end of the 1990s, Sandy Roe listed five common sub­ ject-searching problems encountered “first in the OPAC, often still in Web- based library catalogs, and certainly in general Web searching today:” The first was “no items retrieved,” and the second was “too many items retrieved.”21 A year later, Chris Evin Long undertook a criti­ cal evaluation of subject searching in sixty Web catalogs and concluded that of the “Web-based OPACs currently in opera­ tion … many deficiencies present in ear­ lier generations of online catalogs have been passed down to the next genera­ tion.”22 Subject-searching Problems in a Local Catalog The proposition that the problem of “no items retrieved” (hereafter referred to as no-hits searches) persists in today’s Web catalogs is supported by use statistics for the University of Lethbridge library online catalog, whose existence has spanned the pre-Web and Web interface eras. This catalog had a character-based interface from its installation in 1990 up to 1997, when the library began to offer an additional Web-based interface. From 1999, the Web interface became the pri­ mary catalog interface, although the origi­ nal character-based interface remained available. The year 1999 also marked the imple­ mentation of an agreement between the University of Lethbridge and Medicine Hat College, located 200 kilometres apart in southern Alberta, to share the university’s Innopac automated library system, which included the online catalog.23 From 1990 to 2002, the highest percentage of all searches in this catalog most often occurred in the subject index, and the subject index consis­ tently accounted for the highest or second- highest percentage of all no-hits searches.24 Thus, conversion to a Web interface alone did not result in marked changes in the fre­ quency of subject no-hits search results in the University of Lethbridge catalog, which remained significantly high. Suppositions and Goals of the Study The impetus to investigate subject-search­ ing problems in the University of Lethbridge catalog arose from three sup­ positions: • Some subject no-hits searches rep­ resent topics covered in cataloged library materials that may not have been discov­ ered by the user. • Some subject no-hits searches may recur periodically (e.g., from semester to semester or year to year). • Some subject no-hits search results may be preventable if different catalog responses can be engineered, leading to the retrieval of some records that future users may judge to be relevant or worth pursuing. These suppositions gradually coa­ lesced out of many years of gleaning in­ dications of subject-searching problems from experiences of assisting students with their research assignments at the reference desk and from the examination of two types of transaction log data: de­ scriptive statistics on catalog use, and the text of users’ subject no-hits search terms (which the author scanned periodically).25 On the one hand, we know that transac­ tion log data alone provide no informa­ tion about users’ intentions or needs. That is, some searches resulting in no retriev­ als may be judged by the user to be suc­ cessful because the no-hits results were http:periodically).25 http:searches.24 http:catalog.23 http:systems.20 38 College & Research Libraries January 2004 TABLE 1 Studies of Subject No-hits Searches in Online Catalogs # Subject No-hits Subject Searches Data Collection Searches Resulting in Authors Period Analyzed/Logged No-hits Lestem (1989) 120 days**, 1989 1,524 60.1% Tolle (1983) 17 days, 1982 (SULIRS) 9,235 52.8% Kemn-Simimenko (1983) 1 day, 1982 (System A) 204 46.0% Wilkes & Nelson (1995) 14 days*, 1991 351 40.6% Kemn-Simimenko (1983) 1 day, 1982 (System C) 69 39.0% Kemn-Simimenko (1983) 1 day, 1982 (System B) 231 35.0% Bammett & Maticka (1990) 5 days, 1989? 3,215 32.0% Data collection period durations originally reported in weeks* or months** were converted to days for ease of comparison. expected and, conversely, some searches retrieving at least one record may not be successful because the user judges all records to be irrelevant or an overwhelm­ ing number of records are retrieved. On the other hand, we tend to think of no-hits search results as representing un­ satisfactory results most of the time. Be­ cause users’ exact no-hits search terms can be captured in transaction logs, analysis of these data may assist us in taking initial steps toward achieving a better under­ standing of the characteristics of subject no-hits searches and toward developing methods to ameliorate them effectively. An interest in discovering what could be done locally to improve subject searching led the author to embark upon an exploration of the third supposition using transaction log data.26 The goals of the present study were to examine users’ subject no-hits search terms systematically in order to determine what types of searches are logged as sub­ ject no-hits searches in the University of Lethbridge catalog and to explore possible methods of enhancing the database that could potentially reduce the occurrence of subject no-hits searches. Previous Research Defined as “the study of electronically recorded interactions between online in­ formation retrieval systems and the per­ sons who search for the information found in those systems,” transaction log analysis is a popular means of studying system performance and use of online catalogs and other information retrieval systems.27,28 A number of studies have used transaction log analysis to investi­ gate subject no-hits search results in online catalogs, a sampling of which is presented in table 1.29–33 As well, a longi­ tudinal transaction log study used analy­ ses of no-hits search results to improve online catalog searching through en­ hancements to the catalog search inter­ face, and one result of this study was a significant decrease in subject no-hits search results.34 Another possible means of reducing subject no-hits search results is the inclu­ sion of actual no-hits search terms as cross- references in authority records, thereby automatically leading the user from no-hits search terms to potentially relevant con­ trolled vocabulary terms such as the Li­ brary of Congress Subject Headings (LCSHs). The potential of new authority record cross-references for reducing the frequency of no-hits searches in online catalogs has been explored in several stud­ ies, but with a focus on author searches or author and title searches, representing known-item searches rather than un­ known-item (subject) searches.35–39 In 1982, a project led by Pauline A. Cochrane established a procedure for staff http:results.34 Subject No-hits Searches in an Academic Library Online Catalog 39 at other libraries to forward suggestions to the Library of Congress (LC) for con­ sideration as new LCSH cross-refer­ ences.40 In an appendix to the project’s fi­ nal report, Cochrane explained that the intent behind the project was to improve LCSH for use in online catalogs. This suc­ cessful project laid the groundwork for a cooperative approach to maintaining the LCSH authority file, now known as the Subject Authority Cooperative Project (SACO) coordinated by LC. No other published studies were located that at­ tempted to improve subject searching in online catalogs through new LCSH cross- references, although, in her assessment of the state of reference structures in online catalogs, Jane Greenberg noted that “there are many examples of librarians manipu­ lating authority control modules and au­ thority data in order to improve reference structure access.”41 Methods For the duration of the study, on the first working day of each week, complete lists of subject no-hits searches from the pre­ vious week were downloaded from trans­ action logs of the University of Lethbridge catalog. The author analyzed each no-hits search in the lists using an exploratory and interpretive process of seeking an­ swers to a series of questions: • Is the search string intelligible (i.e., can plausible meaning be inferred)? • If intelligible, what topic or topics might the search string represent? • Does the catalog contain informa­ tion that might be relevant to the search? • If the catalog potentially contains information, how might the database be enhanced to prevent the recurrence of the no-hits search result? To answer the last two questions, top­ ics potentially represented by the subject no-hits searches were searched again in the Innopac cataloging subsystem where the author was able to view, update, and create MARC authority and bibliographic records, as needed. After some initial ex­ perimentation, the following two pro­ cesses were developed as potential en­ hancements to selected subject no-hits search results when the catalog was found to contain records that might be relevant to an inferred topic of a subject no-hits search: 1. creating or upgrading an authority record to provide a link between terms comprising a no-hits search and a valid LCSH represented in at least one biblio­ graphic record in the catalog; or 2. creating a “pathfinder” record us­ ing the bibliographic record format to provide a brief set of instructions on how to retrieve some potentially relevant records using the catalog’s “LIMIT” fea­ ture.42 Enhancement 1 involved adding cross- references comprising the verbatim text of selected subject no-hits searches to MARC authority records whose autho­ rized headings were interpreted to be possibly the same as, or related to, the new cross-references.43 Verbatim no-hits search terms were used as cross-refer­ ences because these terms were users’ natural language vocabulary and thus represented terms that other catalog us­ ers might use in subject searches. The author assumed that converting subject no-hits search terms to be in compliance with LC policy governing the creation LCSH cross-references may be less help­ ful, a supposition partially corroborated by a study of catalog users’ understand­ ing of LCSH that found that users as­ signed correct meanings to less than half of the LCSHs they examined.44 Most subject no-hits searches poten­ tially representing topics covered by ma­ terials in the library’s cataloged collec­ tions were found to be amenable to en­ hancement 1. Some, however, were dis­ covered to represent combinations of top­ ics (e.g., weight and body image, and eth­ ics in sport psychology) for which no single LCSH could be found. Although it is possible to add a given cross-reference to more than one authority record, the author was doubtful that users would find two or more “see” references to be helpful, if individually, none matched their exact search topic. http:examined.44 http:cross-references.43 http:ences.40 40 College & Research Libraries January 2004 Thus, enhancement 2 was developed as a more direct means of leading users to potentially relevant records when ap­ propriate single LCSHs could not be found. It involved the adaptation of the bibliographic record format to serve as a “pathfinder” record, defined here as a brief record whose purpose is to provide instructions on how to conduct two-step searches on particular topics. The author­ ity record format is unable to perform the function of a pathfinder record because it does not support the direct presentation of search instructions to catalog users when a field containing an authorized heading is not present. For each no-hits search interpreted to be intelligible, the author determined whether application of enhancement 1 or 2 could lead to the retrieval of at least one poten­ tially relevant record, but not too many. However, what constitutes “too many” is a situation-specific and subjective judgment that can be made only by the user and vary­ ing suggestions are found in the research literature on the number of retrievals that users consider to be too many.45 Lacking empirical data on what University of Lethbridge catalog users considered to be too many records retrieved, the author at­ tempted to minimize information overload for users by adding database enhancements resulting in fewer than 100 records re­ trieved, whenever possible.46 The author recognized that, on the one hand, regarding the two enhancements as true improvements to the database is easily challenged because both involved significant deviation from standard sub­ ject cataloging practice. On the other hand, the purpose of exploring nonstand­ ard uses of authority and bibliographic records was to support the main goal of subject cataloging, which is, in the author ’s view, the effective communica­ tion of the main subject matter of docu­ ments to catalog users. When we suspect that a significant proportion of users’ needs may not be adequately met, it is legitimate to consider alternatives to cur­ rent cataloging practices and policies in order to serve catalog users better. Limi­ tations represented by the nonstandard nature of the two database enhancements FIGURE 1 Enhancement 1 Example http:possible.46 Subject No-hits Searches in an Academic Library Online Catalog 41 FIGURE 2 Enhancement 2 Example explored in this study are further consid­ ered in the discussion section. Database Enhancement Examples “Assisted human reproduction” is an ex­ ample of a subject no-hits search result that was modified using enhancement 1. This search was logged in the Medicine Hat College subset of the shared Univer­ sity of Lethbridge catalog on March 20, 2002. Before modifications were made, the catalog response to this search was a “nearest neighbors” browse display, with the highlighted message “Your entry as­ sisted human reproduction would be here – Search as Words.” If the user selected the option to “Search as Words,” the catalog would have automatically reexecuted the search for “assisted human reproduction” in the keyword index, where some records may have been retrieved and perhaps judged by the user to be useful. Hence, it is pos­ sible that the user judged the outcome of this no-hits search to be successful, al­ though transaction log data show that the “Search as Words” option is rarely used. Changing the database using the en­ hancement 1 methods involved adding “assisted human reproduction” as a cross- reference to the subject authority record for “human reproductive technology.” “Human reproductive technology” is an LCSH that was interpreted by the author to be similar in meaning to assisted hu­ man reproduction and was present in at least one catalog record in the database. After this database enhancement was made, searching the subject index for as­ sisted human reproduction invoked a new system response illustrated in figure 1. As long as at least one bibliographic record containing a subject heading be­ ginning with “human reproductive tech­ nology” continues to exist in the catalog, if a user encountering the display in fig­ ure 1 clicks on “Human reproductive technology,” the catalog automatically performs another subject search that re­ trieves all records containing this LCSH. An example of enhancement 2 is found in the “before” and “after” catalog re­ sponses to the subject search for “ethics in sport psychology,” which was logged 42 College & Research Libraries January 2004 as a no-hits search on March 19, 2002. Before enhancement, the catalog pre­ sented the standard “nearest neighbors” browse display containing the user ’s original search terms with the option to re-search the terms in the keyword index. Once again, if the user selected the “Search as Words” option from this dis­ play, it is possible that the search for eth­ ics in sport psychology actually resulted in useful retrievals via an automatically reexecuted search in the keyword index. Because no LCSH incorporating all of the topical elements of “ethics in sport psychology” was identified, the author was unable to create an authority record cross-reference for this no-hits search. The author thus determined that enhance­ ment of the catalog response to this par­ ticular no-hits search was better handled in a pathfinder record containing the fol­ lowing note: “To find some information on ‘ethics in sport psychology’: search the SUBJECT index using ‘sports psychologi­ cal aspects,’ then LIMIT your search re­ sults using ‘moral and ethical aspects’ in WORDS in the SUBJECT.” The enhanced catalog response to the subject search for ethics in sport psychology is illustrated in figure 2. Following the instructions in the new pathfinder record for ethics in sport psy­ chology on October 30, 2002, led to the re­ trieval of one record: The Making of High Performance Athletes: Discipline, Diversity, and Ethics. The two LCSHs in the retrieved record were “Sports—Psychological as­ pects” and “Sports—Moral and ethical aspects.” Arguably, these two headings together cover the topic ethics in sport psy­ chology, whereas, singly, neither one ad­ dresses all three component concepts of sports, psychology, and ethics. Findings The findings of this exploratory study are reported in two sections. A detailed analy­ sis of a one-week sample of subject no- hits searches is presented first, followed by a statistical summary of longitudinal study findings. One-week Sample Analysis The sample analysis is a retrospective look at how the subject no-hits searches for the week of March 18–24, 2002, were reviewed and enhanced during the study. This week was selected because it oc­ curred in the most recent regular aca­ demic term (spring 2002) and was the Author Title Subject Keyword Call # Sep. 2001-Aug. 2002 Mar.18-24, 2002 0% 5% 10% 15% 20% 25% 30% 35% FIGURE 3 Comparison of Sample Week and Full-year Search Frequencies Subject No-hits Searches in an Academic Library Online Catalog 43 FIGURE 4 Comparison of Sample and Full-year No-hits Search Frequencies Author Title Subject Keyword Call # Sep. 2001-Aug. 2002 March 18-24, 2002 0% 10% 20% 30% 40% 50% 60% week in which the highest number of sub­ ject no-hits searches (1,940) was recorded. As rough measures of whether searches in this sample week were representative of the full year ’s searches, data on total searches and no-hits searches for the week of March 18–24, 2002, were compared to those for the complete 2001–2002 school year. Both comparisons reveal that frequen­ cies of all searches (figure 3) as well as no- hits searches (figure 4) across the five types of search indexes were quite similar, thereby suggesting that the sample week was not atypical of searching done over the full year. Of the 1,940 subject no-hits searches, 419 were discarded because they were duplicates, leaving 1,521 unique searches logged during the week of March 18–24, 2002. The author analyzed each unique search by seeking answers to four ques­ tions described in the methods section of this article and found that 515 (approxi­ mately 34%) were amenable to enhance­ ment. In 90 percent of cases, the enhance­ ment was achieved through the addition of new authority record cross-references and 10 percent through the addition of a new pathfinder record. To determine what types of searches were represented in the list of 1,521 no- hits searches, a categorization scheme was created for the retrospective analysis. Pro­ ceeding heuristically, nine categories emerged from the list, which are coded and defined in table 2.47 After categorizing the 1,521 sample no- hits searches according to the scheme in table 2, percentages distributions were calculated for: • the number of sample searches (1,521) in each of the nine no-hits catego­ ries; • the number of enhanced searches (515) in each of the nine no-hits catego­ ries; • the number of sample searches (1,521) that were enhanced in each of the nine no-hits categories. These three sets of percentages are pre­ sented in table 3. By far the greatest proportion (71%) of the sample subject no-hits searches oc­ curred in category 1 (correctly spelled topical search, but not in LCSH format), as did the greatest proportion (86%) of en­ hanced no-hits searches. As well, 41 per­ cent of category 1 no-hits searches were found to be amenable to enhancement, and this percentage is second only to that of category 7 (correctly spelled geo­ 44 College & Research Libraries January 2004 TABLE 2 No-hits Categories Used in One-week Sample Analysis Category Code No-hits Category Definition Examples Topic; correctly spelled but not in LCSH fornat 0 Health risks with older mothers 2 Personal/corporate name; misspelled or in a o Terry Fox nonstandard fornat 3 Topic; incorrectly spelled and not in LCSH o Heirarchy in the church fonnat 4 Topic; inconnectly spelled, but in LCSH fonnat o Lierature and society 5 Title; should have been searched in title or a o Canadian Journal of course reserves index Political Science o History 2290 o APA citation guide 6 Undetenmined meaning o Eduus o 1 love you 7 Geographic place name; correctly spelled, but o The Caribbean not in LCSH fornat 8 Topic; correctly spelled and fornatted LCSH o Dams-Australia 9 Personal/corporate name; correctly spelled o Aga Khan and formatted graphic name, but not in LCSH format) by a very narrow margin. Category 2 (personal or corporate name containing spelling errors or in a nonstandard format) represented the sec­ ond largest percentages of no-hits searches and of enhanced searches, but the percentages values were small (9% and 7%, respectively). It is worth noting that name searches (categories 2 and 9) may have represented searches for infor­ mation about a person or organization or information by a person or organization; in the latter case, these searches might have been more appropriately conducted in the author index. Table 4 provides examples of database enhancements made in response to subject TABLE 3 Distribution of No-hits and Enhanced Searches in One-week Sample Category Abbreviated % of 1,521 % of 515 % of No-hits Code Meaning No-hits Enhanced Searches That Searches Searches Were Enhanced 1 Non-LCSH topic 71% 86% 41% 2 Nonstandard name 9% 7% 25% 3 Misspelled non-LCSH topic 8% 1% 4% 4 Misspelled LCSH 3% 2% 24% 5 Wrong index 3% 2% 24% 6 Undetermined 2% 0% 0% 7 Non-LCSH geographic name 1.5% 2% 42% 8 Valid LCSH 1.5% 0% 0% 9 Valid name 1% 0% 0% Subject No-hits Searches in an Academic Library Online Catalog 45 TABLE 4 Exam[les of Enhanced No-hits Searches Category Original No-hits Search String New Cross-reference New Pathfinder Record* 1 1 canadian scholarships aboriginal parents Scholarships-Canada -Directories 1. Parents 2. Indians of North America 1 industrial globalization 1. Globalization 2. Industries 2 3 charlotte whitton shakespear sonnets Whitton, Charlotte, 1896-1975 Shakespeare, William, 1564-1616. Sonnets 4 capital punishment ethic aspects Capital punishment- Moral and ethical aspects 5 the crc handbook of chemistry and physics Chemistry-Tables 5 APA citation guide Publication manual of the American Psychological Association 7 the caribbean Caribbean Area * Entries in the New "Pathfinder" Record column indicate I. the LCSH to be searched, and 2. the LCSH or subject words to be used to limit the results of searching I, where these two steps are described in the statement: "To find some information on '[no-hits search terms],: search the SUBJECT index using "[I.]" then LIMIT your search results using "[2.]" in WORDS in the SUBJECT." no-hits searches in each no-hits category that received at least one enhancement. In most cases, cross-references were added to subject authority records, but on occasion exceptions were made. For example, the two no-hits examples listed for category 5 in table 4 were interpreted to represent searches for titles, but only the first one was enhanced by the addition of a new cross- reference in a subject authority record. The second example, “APA citation guide,” was interpreted to be a search for the American Psychological Association’s publication manual, a popular reference source used heavily by students. Because no single LCSH could be identified that would re­ trieve catalog records describing only, or primarily, the APA’s publication manual, an authority cross-reference was created to lead users from the subject no-hits search “APA citation guide” to a title search for “Publication manual of the American Psy­ chological Association.” Longitudinal Study Findings Table 5 presents data for subject no-hits search analyses that took place for each calendar year of the study. Subject searches represented, on average, 27 per­ cent of all no-hits searches annually. Over the course of the study, approximately 23 percent of all subject no-hits searches re­ ceived database enhancements, represent­ ing a smaller percentage of searches ame­ 46 College & Research Libraries January 2004 nable to enhancement than was found in the one-week sample (34%), but totaled almost 38,000 enhanced-subject no-hits searches. Discussion This section considers two questions: How do the findings of this study com­ pare with those of previous studies of subject no-hits searches in online cata­ logs? What are the study’s limitations? Comparison with Previous Studies Although no other studies were located that attempted to improve subject search­ ing by systematically enhancing the cata­ log database in direct response to no-hits searches, several online catalog studies were identified from the 1980s and early 1990s that analyzed the frequency of sub­ ject no-hits searches, a sampling of which was presented in table 1. One can see that the present study’s data collection period (44 months, or approximately 1,320 days) and number of no-hits searches analyzed (162,663) far exceed those pertaining to the studies in table 1. Moreover, the average percentage of subject no-hits searches ob­ tained over the course of the present study (27%) is lower than that of the other stud­ ies, which ranged from 32 to 60.1 percent. It is possible that the present study’s lower percentage of subject no-hits searches is attributable, at least in part, to its considerably larger sample size and longer data collection period. Other fac­ tors that may account for some of the ob­ served differences in the results of these studies and perhaps may reduce their comparability are the variety of online catalog systems studied, the use of vari­ ant definitions of subject searches or no- hits searches, and variations in data col­ lection, sampling, or analysis methods.48 At the same time, the database enhance­ ments explored in the present study may have played a role in the slight reduction in the frequency of subject no-hits search results. The 1999 expansion of the Univer­ sity of Lethbridge’s catalog user commu­ nity to include Medicine Hat College us­ ers led to an unsurprising 10 percent in­ crease in the frequency of subject index searches, which remained 6 to 10 percent higher than pre-1999 levels to the end of the 2001–2002 school year. However, a cor­ responding proportional increase in sub­ ject no-hits search frequencies was not found: from 1997–1998 to 1999–2000, the percentage of no-hits searches occurring in the subject index remained constant at about 29 percent and then fell gradually to 26 percent by 2001–2002. Hence, a pos­ sible explanation for the slight decrease in subject no-hits search percentages when we otherwise would expect them to have TABLE 5 Longitudinal Study Findings 1999 2000 2001 2002* Total Subject no-hits searches 42,134 47,044 48,396 25,089 162,663 Subject no-hits searches as % of all no-hits searches 31% 25% 25% 25% 27% No. of enhanced subject no-hits searches 10,487 10,413 10,493 6,594 37,987 Percent of subject no-hits searches enhanced 25% 22% 22% 26% 23% New authority records created 3,454 2,772 2,735 1,559 10,520 New pathfinder records created 298 536 815 663 2,312 Ratio of new authority records to new pathfinder records 11:1 5:1 3:1 2:1 4:1 * Data collection for 2002 covered eight months only (January to August). http:methods.48 Subject No-hits Searches in an Academic Library Online Catalog 47 remained constant may be the enhance­ ments made to the database in the course of conducting the present study, which spanned January 1999 to August 2002. Study Limitations Although the present study deliberately set out to explore new techniques for im­ proving subject searching, there is much to question regarding its enhancement methods. A general concern pertains to their subjective nature. Because transac­ tion log data provide no information about users’ intentions, the process of analyzing the subject no-hits searches was necessarily a fallible, interpretive process, and the author may not have correctly discerned the intended meanings of these searches. Thus, it is possible that from the user ’s viewpoint, the post-enhancement catalog response represented an irrel­ evant or less-helpful response than did the original no-hits response. With regard to enhancement 1, which involved the creation or updating of au­ thority record cross-references, the non­ standard coding of authority records is a limitation. In many cases, the new cross- references comprising verbatim no-hits search strings did not represent headings formatted according to LC subject catalog­ ing policy and practice.49 Some of the no- hits search strings contained spelling and grammatical errors, and therefore the au­ thority record cross-references con­ tained these same errors (e.g., “Malayasia,” and “rythm and blues”). It may be argued that these types of cross-references are not enhancements because they deliberately introduce misspelled cross-references into the database. In addition, they do not bring errors to the attention of the user, thus per­ haps missing an opportunity for learning to take place. If automatic spelling error detection becomes a standard feature of online catalogs, however, the potential usefulness of this type of cross-reference may diminish. As well, new authority records were keyed in USMARC or MARC 21 format, but no attempt was made to code each one fully. The goal of creating these brief records was to generate the desired new cross-references quickly in order to com­ plete the analysis of the current week’s subject no-hits searches before the subse­ quent week’s list was generated. Thus, the utility of these new authority records may be limited to the local catalog, as their rel­ evance for other catalog databases is un­ known. There is equal, or perhaps even more, cause for concern when we consider the limitations of enhancement 2, which in­ volved the creation of pathfinder records. First of all, using bibliographic records solely to present information about search refinement is unorthodox and could be viewed as unwise tampering with a record format that was not designed for this purpose.50 Second, an undesirable outcome of in­ troducing pathfinder records into the da­ tabase is the interfiling of the verbatim text of subject no-hits searches in the controlled vocabulary subject index. This is a require­ ment because, otherwise, pathfinder records have no access point that can be used to retrieve them in the catalog. These non-LCSH headings, therefore, appear in the online catalog subject index where us­ ers may encounter them while browsing the subject index and mistake them for valid LCSHs. In subject index browse dis­ plays in the University of Lethbridge cata­ log, pathfinder record access points (sub­ ject no-hits search terms) are visually in­ distinguishable from other valid headings. Another undesirable feature of path­ finder records is the appearance of ver­ batim no-hits search strings masquerad­ ing as valid subject headings in path­ finder records themselves. In a regular bibliographic record display, the user may click on any of the subject headings in­ cluded in the record, labelled “Subject,” because this action automatically executes a subject index search for the selected heading without requiring the user to type in a new search. In a pathfinder record, however, the subject no-hits search term is labelled and is selectable as if it is a valid LCSH in a bibliographic record. (See the example in figure 2.) If http:purpose.50 http:practice.49 48 College & Research Libraries January 2004 the user clicks on the term displayed in the “Subject” field of a pathfinder record, a potentially confusing loop is formed because the catalog responds with a browse display containing the original no-hits search string, and selecting this “heading” from the browse display put the user back into the pathfinder record. An additional potential limitation of enhancement 2 is the users’ ability to un­ derstand and use pathfinder records ef­ fectively. Users may choose not to read or heed the suggestions for search refine­ ment contained in pathfinder records. If the suggestions are not used, the path­ finder records perhaps pose more of a hazard than a help to catalog users. Even the willing searcher who wishes to act on the suggested steps for reexecuting a search differently may not understand how to carry out the instructions. Al­ though it can be a powerful tool for search refinement, transaction logs show that the catalog LIMIT option is used infrequently. Use of the LIMIT option was present in each pathfinder record’s instructions for search refinement. Table 5 indicates that the ratio of new authority records to new pathfinder records decreased steadily over the course of the study. This may have been due to changes over time in the author’s percep­ tions of how best to handle enhancements. Other possibilities are that users’ search topics became more interdisciplinary, thus rendering their topics less likely to be rep­ resentable by a single LCSH, or perhaps users began to construct online catalog subject searches in a manner similar to that habitually used with Internet search en­ gines. In any case, the creation of path­ finder records likely contravenes more as­ pects of standard cataloging and author­ ity control practice than does adding non­ standard authority record cross-references. If widespread use of pathfinder records is not advisable, alternative means for en­ hancing catalog responses to interdiscipli­ nary or multitopic no-hits searches are needed. Last but not least, the enhancements explored in this study are not easily au­ tomated or scalable to larger databases or user communities. At the busiest times of the academic year, completing the analy­ sis of the current week’s list of subject no- hits searches required the equivalent of one to two full workdays, while other regular work responsibilities continued to receive first priority. However, it would be difficult to automate the time-consum­ ing intellectual analysis portion of the study because of the complex array of sources that informed the author ’s inter­ pretations and subsequent selective en­ hancements of subject no-hits searches. These sources included knowledge of the university’s curriculum, areas of research specialization on campus, current student research assignments, popular research topics, areas of library collection strengths and weaknesses, idiosyncrasies of the Innopac automated system, LC subject cataloging policy, and MARC 21 biblio­ graphic and authority record coding con­ ventions. Further Research Two distinct areas of further research are considered in this section. The first in­ volves next steps required to determine whether, in fact, the potential subject no- hits search ameliorations explored in this study make a difference to users’ subject- searching experiences. The second area of further research comprises additional re­ search questions related to this study’s explorations. Next Steps The present study developed and ex­ plored two potential ameliorations of sub­ ject no-hits search results and found some indication that enhancements 1 and 2 may have reduced the frequency of no-hit sub­ ject searches. However, to determine whether, in fact, enhancements 1 and 2 enhance subject searching in the Univer­ sity of Lethbridge catalog, we also need to discover whether they make a positive difference in users’ subject-searching ex­ periences.51 How might an evaluation of the effectiveness of enhancements 1 and 2 be undertaken? http:periences.51 Subject No-hits Searches in an Academic Library Online Catalog 49 Unobtrusive monitoring of the use of only new or enhanced authority records and pathfinder records resulting from this study is currently not possible in the Innopac system. If obtrusive data collec­ tion methods are used, perhaps quantita­ tive and qualitative data on a cross-sec­ tional sample of individual users’ search­ ing behaviors, intentions, and judgments could be collected in an experimental set­ ting involving study participants search­ ing for topics shown to be frequent sub­ ject no-hits search topics in previous transaction logs. It may be possible to gather transac­ tion log data if the computers or partici­ pants involved in such an experiment could be uniquely identified, thus allow­ ing study participants’ searches to be tracked under a distinct identification code in the transaction logs. Alternatively, perhaps screen-capture software could be used to record the content of Web catalog screens encountered by each participant in the experiment. However, the informa­ tion we might wish to elicit from partici­ pants of such a study begs the question of what aspects of online catalog users’ subject-searching behavior we have the greatest need to understand better. In this regard, it is helpful to consider Christine L. Borgman’s discussion of two sources of problems in searching online systems: differences among individual users, and differences in individual sys­ tem features. She suggested that “on any given system, people will search in dif­ ferent ways, with different levels of suc­ cess and satisfaction. Until we can iden­ tify the factors responsible for low suc­ cess rates, it will be difficult to narrow the performance range, moving users toward the upper end of the success scale and thus removing barriers to access.”52 Borgman further indicated that research has revealed several possible individual user factors that may influence success rates, including frequency of use of a given database, amount of training re­ ceived, and academic major. Controlling for some or all of these user factors may therefore be advisable in a formal evalu­ ation of the enhancements explored in this study. Another evaluation approach, based on a suggestion by Ray R. Larson for evaluating subject searching in online catalogs, is to measure precision (the ex­ tent to which results of a search contain relevant items) and recall (the extent to which all relevant items are retrieved) of enhanced subject no-hits searches.53 Be­ cause the database enhancements result­ ing from the present study can be sup­ pressed and unsuppressed from the pub­ lic catalog, it may be possible to compare search results on selected no-hits topics obtained in two different states of the cata­ log, one containing the database enhance­ ments and the comparison state lacking the database enhancements, to see whether differences in precision and re­ call are found. Others have suggested that precision and recall measures of relevance in auto­ mated information retrieval are indicative of an objective, system-centered view con­ cerned only with the presence or absence of a match between the topic of a subject search and the topic of a document to the exclusion of the user.54 Proponents of an alternative user-centered approach view the judgment of relevance (e.g., the suc­ cess of subject searches) as a multidimen­ sional cognitive process that may be in­ formed not only by the stated or inferred topics of documents, but also by other changeable, subjective, and time- and context-dependent factors.55 An approach to the evaluation of the present study’s database enhancements from the subjective relevance point of view could involve real-world information- seeking needs and contexts of catalog us­ ers, and elicitation of their cognitive pro­ cesses as they interact over time with the catalog, its database enhancements, and the actual documents retrieved and used. Such a longitudinal study could involve periodically interviewing a small group of users as they progress through a complex library research task to elicit information about their experiences and cognitive pro­ cesses. Concurrently, users’ online catalog http:factors.55 http:searches.53 50 College & Research Libraries January 2004 subject searches could be monitored via transaction logs, and no-hits searches could be enhanced, when possible, using the methods described in the present ar­ ticle. Thus, interviews could include dis­ cussion of particular subject no-hits searches as well as users’ views on whether subsequent catalog database enhance­ ments are useful in the context of their particular information-seeking task. Related Research Questions In addition to further evaluative research, other issues have emerged from this ex­ ploratory study that may be worthy of investigation. For example, it has been suggested that in online catalogs, infor­ mation overload (retrieving too much) and search failure (retrieving nothing) can be equally problematic.56 Although cap­ tured in past transaction log reports, data on differences in retrieval set sizes across the University of Lethbridge catalog’s various search indexes have not been ana­ lyzed longitudinally. However, an exami­ nation of the retrieval set sizes for the one- week sample from March 18–24, 2002, showed that although at least 80 percent of searches in all indexes retrieved fewer than a hundred records, the highest fre­ quency of searches retrieving a hundred or more records occurred in the subject index (20%). Determining the extent of information overload could be achieved through a retrospective analysis of re­ trieval set sizes by index over the years. It also may be useful to determine whether information overload and search failure have been correlated in this cata­ log by comparing retrieval set size and no-hits search frequency patterns across the different search indexes. Two other questions about subject searching in online catalogs are worthy of further investigation: • How does an individual user’s sub­ ject-searching behavior and cognitive pro­ cesses change over time as catalog search­ ing experience is acquired? • What types of subject-searching enhancements might be suggested by answers to the preceding question? Larson noted that “a longitudinal study of a set of users from their first in­ troduction to the online catalog through one or two years of use, using transac­ tion monitoring to record search behav­ ior, and questionnaires and interviews to trace changing attitudes and needs” is one way to address questions that are not di­ rectly answerable using transaction log data alone.57 Conclusions Because the University of Lethbridge and its partner, Medicine Hat College, are pri­ marily undergraduate institutions, on a continuing basis, many users of their shared library catalog are novices. We know that novice catalog users most of­ ten search the subject index, that library users in general have little understand­ ing of the LCSHs that comprise the con­ trolled vocabulary used in the catalog’s subject index, and that users experience the most difficulty searching the subject index, which is frequently manifested in high proportions of no-hits searches.58–60 Thus, finding ways to improve the sub­ ject-searching experiences of users of this shared catalog by, for example, reducing the frequency of subject no-hits search results, is of particular concern. Database enhancements 1 and 2 ex­ plored in the present study may have con­ tributed to the slight reduction in the pro­ portion of subject no-hits search results observed from 1999 to 2002, which oc­ curred at a time when the proportion of subject searches logged as no-hits searches would otherwise have been ex­ pected to remain constant after expansion of the catalog’s primary user community in 1999. However, the more significant question of whether these database en­ hancements actually make a difference to users’ subject-searching experiences re­ quires further research. If the present study’s methods of en­ hancing subject searching are shown to be beneficial to users’ subject-searching experiences, a reexamination of aspects of standard cataloging practice may be worthwhile. Current policy governing the http:alone.57 http:problematic.56 Subject No-hits Searches in an Academic Library Online Catalog 51 updating of LCSH cross-references states that additions, changes, and deletions to 4XX cross-references require citation of “sources that support the form of the pro­ posal or references.”61 This requirement, based on literary warrant, presents prob­ lems when applied to the evaluation of LCSH cross-references because it is author centered. That is, terms used by authors (which are selected into the LCSH con­ trolled vocabulary by subject catalogers) are not necessarily the same as those known to users. Use warrant, explored in the present study, is an alternative principle for guid­ ing the creation of new LCSH cross-ref­ erences. In 1990, Elaine Svenonius noted that relying on use warrant, or common usage, for the selection of controlled vo­ cabulary terms posed difficulties in the past, but she also suggested that “the rela­ tively new possibility of analyzing trans­ action logs of users’ searches in online databases offers some promise of progress in the understanding of terms commonly used in information seeking.”62 The present study demonstrated that it is pos­ sible to respond in a timely manner to users’ natural language (no-hits) search terms by selectively expanding the entry vocabulary using two database enhance­ ment techniques. The results of this ex­ ploratory study can be viewed as support for the suggestion that use warrant bears reconsideration as a criterion for evaluat­ ing proposed new LCSH cross-references in addition to literary warrant. Because many of the well-documented problems of subject searching in online catalogs have remained the focus of vari­ ous research efforts for more than two de­ cades, progress in improving subject searching may continue to evolve gradu­ ally on different research fronts and from a variety of research approaches. Thus, small-scale studies such as the one de­ scribed here may be worthwhile for those preferring to explore alleviations to per­ ceived subject-searching problems of lo­ cal online catalog users that might be achievable within a more immediate time frame. Notes 1. Joseph R. Matthews, Gary S. Lawrence, and Douglas K. Ferguson, eds., Using Online Cata­ logs: A Nationwide Survey: A Report of a Study Sponsored by the Council on Library Resources (New York: Neal-Schuman, 1983). 2. Karen Markey, “Users and the Online Catalog: Subject Access Problems,” in The Impact of Online Catalogs, ed. Joseph R. Matthews (New York: Neal-Schuman, 1986). 3. Carol A. Mandel, “Enriching the Library Catalog Record for Subject Access,” Library Re­ sources & Technical Services 29 (Jan./Mar. 1985): 5–15. 4. Markey, “Subject-Searching Experiences and Needs of Online Catalog Users: Implications for Library Classification,” Library Resources & Technical Services 29 (Jan./Mar. 1985): 34–51. 5. Alex Byrne and Mary Micco, “Improving Opac Subject Access: The ADFA Experiment,” College & Research Libraries 49 (Sept. 1988): 432–41. 6. Shirley Anne Cousins, “Enhancing Subject Access to Opacs: Controlled Vocabulary vs Natural Language,” Journal of Documentation 48 (Sept. 1992): 291–309. 7. Mia Massicotte, “Improved Browsable Displays for Online Subject Access,” Information Technology and Libraries 7 (Dec. 1988): 373–80. 8. Jeffrey C. Huestis, “Clustering LC Classification Numbers in an Online Catalog for Im­ proved Browsability,” Information Technology and Libraries 7 (Dec. 1988): 381–93. 9. Dorothy McGarry and Elaine Svenonius, “More on Improved Browsable Displays for Online Subject Access,” Information Technology and Libraries 10 (Sept. 1991): 185–91. 10. Martha M. Yee, “System Design and Cataloging Meet the User: User Interfaces to Online Public Access Catalogs,” Journal of the American Society for Information Science 42 (Mar. 1991): 78–98 11. Mark T. Kinnucan, “Fisheye Views as an Aid to Subject Access in Online Catalogues,” Canadian Journal of Information Science 17 (Jul. 1992): 25–40. 12. Tamas E. Doszkocs, “Cite NLM: Natural Language Searching in an Online Catalog,” Infor­ mation Technology and Libraries 2 (Dec. 1983): 364–80. 13. Stephen Walker, “Improving Subject Access Painlessly: Recent Work on the Okapi Online Catalogue Projects,” Program 22 (Jan. 1988): 21–31. 14. Ray R. Larson, “Classification Clustering, Probabilistic Information Retrieval, and the 52 College & Research Libraries January 2004 Online Catalog,” Library Quarterly 61 ((Apr. 1991): 133–73. 15. Mary Micco, “The Next Generation of Online Public Access Catalogs: A New Look at Sub­ ject Access Using Hypermedia,” Cataloging & Classification Quarterly 13, no. 3/4 (1991): 103–20. 16. Micheline Hancock-Beaulieu and Stephen Walker, “An Evaluation of Automatic Query Expansion in an Online Library Catalogue,” Journal of Documentation 48 (Dec. 1992): 406–21. 17. Marcia J. Bates, “Subject Access in Online Catalogs: A Design Model,” Journal of the Ameri­ can Society for Information Science 37 (Nov. 1986): 357–76. 18. Micheline Hancock, “Subject Searching Behaviour at the Library Catalogue and at the Shelves: Implications for Online Interactive Catalogues,” Journal of Documentation 43 (Dec. 1987): 303–21. 19. Micheline Hancock-Beaulieu, “Evaluating the Impact of an Online Library Catalogue on Subject Searching Behaviour at the Catalogue and at the Shelves,” Journal of Documentation 46 (Dec. 1990): 318–38. 20. The notion of “generations” of online catalogs was introduced by Charles R. Hildreth, “Pursuing the Ideal: Generations of Online Catalogs,” in Online Catalogs, Online Reference: Con­ verging Trends, ed. Brian Aveney and Brett Butler (Chicago: ALA, 1984). 21. Sandy Roe, “Online Subject Access,” Journal of Internet Cataloging 2, no. 1 (1999): 69–78. 22. Chris Evin Long, “Improving Subject Searching in Web-based Opacs: Evaluation of the Problem and Guidelines for Design,” Journal of Internet Cataloging 2, no. 3/4 (2000): 159–86. 23. In this paper, references to the University of Lethbridge catalog covering 1999 or later should be understood to be referring to the shared catalog of the University of Lethbridge and Medicine Hat College. 24. From 1990 to 1995, no-hits search results occurred most frequently in the subject index and second most frequently in the title index, but from 1996 this pattern has been reversed. The pattern reversal is likely due to increased searching in the various online periodical indexes, which became available to University of Lethbridge users in greater numbers from about 1996. To locate desired articles identified in online periodical index search results, users must search the online catalog title index, but in many cases these searches retrieve no records because the university owns a very small percentage of the journals indexed in the online periodical index databases available to its users. 25. Patricia Flaherty, “Transaction Logging Systems: A Descriptive Summary,” Library Hi Tech 11, no. 2 (1993): 67–78. 26. Exploration of the third supposition was possible without the need to gather any addi­ tional data because transaction log data were monitored as a part of normal work responsibili­ ties. It was not possible to pursue the additional resources and time to explore the other two suppositions at the time the study was conceived. 27. Thomas A. Peters, Martin Kurth, Patricia Flaherty, Beth Sandore, and Neal K. Kaske, “Trans­ action Log Analysis,” Library Hi Tech 11, no. 2 (1993): 37. 28. Thomas A. Peters, “The History and Development of Transaction Log Analysis,” Library Hi Tech 11, no. 2 (1993): 41–66. 29. Marilyn Ann Lester, “Coincidence of User Vocabulary and Library of Congress Subject Headings: Experiments to Improve Subject Access in Academic Library Online Catalogs,” (Ph.D. diss., Univ. of Illinois at Urbana-Champaign, 1989). 30. John E. Tolle, Current Utilization of Online Catalogs: Transaction Log Analysis, Final Report to the Council on Library Resources: Volume I of Three Volumes (Dublin, Ohio: OCLC Online Computer Library Center, 1983). 31. Cheryl Kern-Simirenko, “OPAC User Logs: Implications for Bibliographic Instruction,” Library Hi Tech 1 (winter 1983): 27–35. 32. Adeline Wilkes and Antoinette Nelson, “Subject Searching in Two Online Catalogs: Author­ ity Control vs. Non-Authority Control,” Cataloging & Classification Quarterly 20, no. 4 (1995): 57–79. 33. Beverley Barrett and Margaret Maticka, “An Analysis of User Failure in Subject Searching an Online Catalogue,” in Garbage In, Garbage Out: The Need for Quality in the Age of Automation, ed. Alan Bundy and Judith Bundy (Adelaide: Auslib Pr., 1990), 38–49. 34. Deborah D. Blecic, Josephine L. Dorsch, Melissa H. Koenig, and Nirmala S. Bangalore, “A Longitudinal Study of the Effects of OPAC Screen Changes on Searching Behavior and Searcher Success,” College & Research Libraries 60 (Nov. 1999): 515–30. 35. Arlene G. Taylor, “Authority Files in Online Catalogs: An Investigation of Their Value,” Cataloging & Classification Quarterly 4, no. 3 (1984): 1–17. 36. Elisabeth Sinnott, “Fewer Errors Resulting from the Users’ Misconception of the Opac in 1992 Than a Decade Ago: A Comparative Study of No Direct Hits and Zero Hits in Author Searches,” Cataloging &Classification Quarterly 18, no. 1 (1993): 75–101. 37. Nirmala S. Bangalore, “Authority Files in Online Catalogs Revisited,” Cataloging & Classi­ fication Quarterly 20, no. 3 (1995): 75–94. 38. ———, “Mystery Names: Plausible Sounding Names Resulting in Zero Hits in Opacs,” Subject No-hits Searches in an Academic Library Online Catalog 53 Technical Services Quarterly 14, no. 1 (1996): 15–31. 39. Jean Dickson, “An Analysis of User Errors in Searching an Online Catalog,” Cataloging & Classification Quarterly 4, no. 3 (1984): 19–38. 40. Pauline A. Cochrane, LCSH Entry Vocabulary Project: Final Report to the Council on Library Resources and to the Library of Congress (Washington, D.C.: Council on Library Resources, 1983). 41. Jane Greenberg, “Reference Structures: Stagnation, Progress, and Future Challenges,” Information Technology and Libraries 16 (Sep. 1997): 108–19. 42. The Innopac LIMIT feature becomes available only after a user has first conducted a search retrieving more than one record. Clicking on the LIMIT THIS SEARCH button presents a menu of criteria by which the original search may be limited or sorted. The third item in this menu allows limiting by “Words in the Author, Title, Subject, or Publisher.” The default is “Words in the Author,” but users may switch to one of the other choices through a pull-down menu. 43. Coding details for specific types of fields in MARC authority records are found in Marc 21 Format for Authority Data: Including Guidelines for Content Designation (Washington, D.C.: Library of Congress Cataloging Distribution Service, 1999). 44. Karen M. Drabenstott, Schelle Simcox, and Eileen G. Fenton, “End-user Understanding of Subject Headings in Library Catalogs,” Library Resources & Technical Services 43 (Jul. 1999): 140–60. 45. Stephen E. Wiberley, Robert Allen Daugherty, and James A. Danowski, “User Persistence in Displaying Online Catalog Postings: LUIS,” Library Resources & Technical Services 39 (July 1995): 247–64; Stephen E. Wiberley and Robert Allen Daugherty, “Users’ Persistence in Scanning Lists of References,” College & Research Libraries 49 (Mar. 1988): 149–56; Marcia J. Bates, “The Fallacy of the Perfect Thirty-Item Online Search,” RQ 24 (fall 1984): 43–50. 46. In some cases, avoidance of large retrieval sets was not possible. For instance, the subject no-hits search for “America politics and government” was interpreted to be equivalent to the LCSH, “United States—Politics and government,” and was therefore added as a cross-reference to the LCSH authority record. The number of records retrieved by the LCSH, “United States— Politics and government,” however, exceeded a hundred. 47. Other researchers such as Barrett and Maticka, “An Analysis of User Failure” and Peters, “When Smart People Fail” created no-hits categories as part of their analyses of “failed” con­ trolled vocabulary catalogue searches. However, the researcher determined that none of these schemes adequately covered the subject no-hits searches analyzed in the present study. There­ fore, a customized categorization scheme was developed. 48. Allyson Carlyle discussed similar problems of LCSH “matching” studies in “Matching LCSH and User Vocabulary in the Library Catalog,” Cataloging & Classification Quarterly 10, no. 1/2 (1989): 37–63. 49. See sections H195, H373, and H200 9b in Library of Congress, Cataloging Policy and Sup­ port Office, Subject Cataloging Manual: Subject Headings, 5th ed. 4 vols (Washington, D.C.: Cata­ loging Distribution Service, Library of Congress, 1996). 50. Pathfinder records were keyed as non-MARC bibliographic records. MARC and non- MARC bibliographic records can coexist in this catalog database (an Innopac system) with few differences in how they are indexed or displayed. 51. Although a formal evaluation of the effectiveness of the database enhancements was not within the scope of the present article, the library reference staff at the University of Lethbridge as a whole endorsed the idea of experimenting with enhancements 1 and 2 as possible ways to ameliorate catalog users’ subject-searching problems. Over the course of the study, the author received unsolicited positive feedback on the database enhancements from several reference staff members and one problem with pathfinder records was reported, which was described in the study limitations section of this article. 52. Christine L. Borgman, “Why Are Online Catalogs Hard to Use? Lessons Learned from Information-Retrieval Studies,” Journal of the American Society for Information Science 37 (Nov. 1986): 387–400. 53. Larson, “Between Scylla and Charybdis: Subject Searching in the Online Catalog,” Ad­ vances in Librarianship 15 (1991): 175–236. 54. For example, see Carol L. Barry, “User-defined Relevance Criteria: An Exploratory Study,” Journal of the American Society for Information Science 45 (Apr. 1994): 149–59; and Stephen P. Harter, “Psychological Relevance and Information Science,” Journal of the American Society for Informa­ tion Science 43 (Oct. 1992): 602–15. 55. A useful overview of research on the meanings and uses of relevance in the context of information seeking and use is found in Linda Schamber, “Relevance and Information Behav­ ior,” Annual Review of Information Science and Technology 29 (1994): 3–48. 56. Larson,” The Decline of Subject Searching: Long-term Trends and Patterns of Index Use in an Online Catalog,” Journal of the American Society for Information Science 42 (Apr. 1991): 197–215. 57. Ibid. 54 College & Research Libraries January 2004 58. Joseph R. Matthews and Gary S. Lawrence, “Further Analysis of the CLR Online Catalog Project,” Information Technology and Libraries 3 (Dec. 1984): 354–76. 59. Drabenstott, Simcox, and Fenton, “End-user Understanding of Subject Headings in Li­ brary Catalogs.” 60. Markey, “Users and the Online Catalog.” 61. Library of Congress, “Subject Cataloging Manual,” H200 9b. 62. Elaine Svenonius, “Design of Controlled Vocabularies,” in Encyclopedia of Library and In­ formation Science, v. 45, suppl. 10, ed. Allen Kent (New York: Marcel Dekker, 1990), 82–109.