College and Research Libraries CD-ROM Search Techniques of Novice End-Users: Is the English-as-a -Second-Language Student at a Disadvantage? Diane DiMartino, William J. Ferns, and Sharon Swacker This study compared the CD-ROM search techniques of 42 undergraduate native speakers (NS) of English with those of 34 undergraduate English-as-a- Second-Language (ESL) speakers in a controlled experiment. Native speakers were significantly more likely to use plural word forms when searching and to experiment with synonyms and alternative words to search for topics. Both groups of students underutilized basic search techniques, such as Boolean operations and indexing, and searched inefficiently. These findings suggests that bibliographic instruction targeted for ESL students can focus on techniques that circumvent difficulties with plurals and limited vocabulary. General bibliographic instruction in CD-ROM usage can address the other problems that ESL students share with novice users in general. • he introduction of computer- ized bibliographic retrieval systems into academic libraries has brought profound changes affecting library users, staff, service poli- cies, and budgets. Newly empowered end- users can access databases from within ~aditionallibrary settings or from remote sites. This emerging body of diverse users is now accessing a widening variety of elec- tronic databases. With this empowerment come new constraints--each particular da- tabase's interface design and help screens are constant for everyone, regardless of the user's research needs and skills. The explosion of access to computer- ized retrieval coincides with the continu- ing increase of international students (students in the United States on visas) attending American institutions of higher education. While bachelor's de- grees earned by American students in- creased by only 10 percent between 1977 and 1989, students here on visas earned 72 percent more degrees during that same time period. 1 In addition to stu- dents on visas, the 1990 Census statistics for New York City, where this study took place, indicate a continued increase in the city's foreign-born population. In 41 percent of homes in New York City a language other than English is spoken; this represents a 19 percent increase since 1980.2 Baruch College of the City Diane DiMartino is an Assistant Professor and Director of the library's Graduate Business Resource Center and William ]. Ferns is an Assistant Professor of Computer Information Systems at Bernard M. Baruch College, City University of New York 10010. Sharon Swacker, Coordinator of Collection Development, is an Assistant Professor at New York City Technical College's library, City University of New York 11201. 49 50 College & Research Libraries University of New York reflects the in- creasingly diverse population of a mul- ticultural urban public institution of higher education. Within its population of 16,000 students, international stu- dents (on visas) make up 12 percent of the student body, and more than half of the 1991 and 1992 freshman classes come from homes where a language other than English is spoken.M Four-fifths of the in- coming freshmen come from families in which neither parent is a college gradu- ate. 5 These statistics suggest that students are entering college from a multitude of economic, social, cultural, and educational backgrounds, and some may have lim- ited English proficiency (LEP). Some search mechanisms, such as Boolean operations, proximity relation- ships, and word indexing, are basic tools in successful retrieval. Other search techniques rely much more on the user's language skills. These in- clude synonym searching, pluralization, compound words, and variant spellings. The extent of one's vocabulary is a prime component in the process of preparing search strategies in text-based databases. With the increasing reliance on self-service databases, ESL students may be at a disad- vantage in using information resources. By identifying specific problems that ESL students encounter, librarians can focus bibliographic instruction for them in these areas. With the increasing reliance on self-service databases, ESL students may be at a disadvantage in using information resources. This study was a simulation experi- ment comparing the search strategies of 76 novice users divided into native Eng- lish speaking (NS) (N = 42) and English- as-a-Second Language (ESL) (N = 34) groups with particular attention to vo- cabulary-based strategies. The search topic replicated a typical research as- signment given in a required speech or English course. The search results were analyzed for significant differences January 1995 based on whether the user was a native English speaker or an ESL student. These differences in formulating CD-ROM searches could affect bibliographic and technical instruction for these groups of students. LITERATURE REVIEW While the language-diversified stu- dent body at Baruch is not unique to an urban, public campus, few studies exist in the literature that focus on the rela- tionship of ESL to search strategy vo- cabulary, with particular emphasis on novice users. The literature in this broad area of CD-ROMs, English-as-a-Second- Language, and international students generally contains self-reported user surveys, samples of user populations with very diverse levels of skill, and re- ports of international students' use of the academic library primarily focusing on cultural aspects. The research literature investigating CD-ROM users and interface design supports what we know from years of experience in academic libraries. College students generally prefer to use CD- ROMs for speed and ease of use.6 A large percentage of users in self-reporting studies are satisfied with their search results, regardless of precision.7 Self-re- ported satisfaction with results is not an objective measure; user satisfaction does not have a high correlation with com- monly accepted recall and precision evaluation measures nor with the qual- ity of citations.8•9 Human-computer interactions, sys- tem inefficiencies, and syntactic and se- mantic problems continue to exist in information retrieval.10 Additionally, other tasks, such as formulating search strate- gies with correct vocabulary and Boolean connectors, transferring/learning com- mands from one system to another, and understanding keyword capabilities present serious obstacles to users. 11 These studies offer important insights into the overall user population in re- gard to CD-ROM database searching. Nonetheless, we must consider specific campus populations and survey tech- niques before generalizing from the con- elusions of these studies. These studies may not be representative of an urban, multicultural student population, nor do they differentiate the goals of the nov- ice undergraduate searcher from those of the graduate student and academic researcher. Additionally, the survey stud- ies utilize self-reporting instruments that measure only post hoc user attitudes and opinions. These surveys exercise little control over the type of search the user required, and there is typically no con- trol data against which we can compare the results.12 Other studies that focus on novice us- ers typically recruit subjects who areal- ready enrolled in Information Science courses. Because these courses feature instruction and practice with computer- ized databases, the external validity of these studies is weak. The information science student may not be repre- sentative of the novice undergraduate . user with no previous instruction.13 Issues relating to CD-ROM training and the needs of both graduate and un- dergraduate students have been docu- mented. In a study exploring CD-ROM training needs, 89 inexperienced under- graduates attending bibliographic in- struction classes at the University of Illinois' Champaign-Urbana campus rated one-to-one training as the most valuable form of instruction, especially with re- gard to the development of search strat- egy and Boolean logic.14 This is an unsurprising finding, but few institu- tions can afford such an approach. Else- where, the development and use of computer programs that teach the use of CD-ROM tools have been proposed, specifically because individualized in- struction is labor-intensive. 15 Despite vendors' claims of user-friendly systems and the self-reported user satisfaction with CD-ROMs, there is a consensus among information science profession- als that relevance of citations and search speed improve with training.16 International and ESL students in the academic library have been the focus of considerable attention, mostly centered on language and cultural differences ap- parent at the reference desk or in a bibli- CD-ROM Search Techniques 51 ographic instruction setting. Sugges- tions for improving service include greater sensitivity, alertness, empathy, and understanding of these differences by all library staff and faculty, library orientation in native languages, staff development workshops, translations of library procedures, rules, and termi- nology into students' native languages, and bibliographic instruction and orien- tation specifically for international graduate students.17- 21 Nevertheless, lit- tle mention is made of ESL students' use of CD-ROMs, especially with regard to language facility (i.e., pluralization, synonyms, alternative spellings or com- pound words), use of the Boolean opera- tors and and or, and general searching strategies and techniques. These skills are critical in retrieving information suc- cessfully from CD-ROMs. EXPERIMENTAL DESIGN The basic experiment compared the search techniques of a group of English- as-a-Second-Language (ESL) undergradu- ate users (N = 34) to the techniques used by a group of native English-speaking (NS) undergraduate users (N = 42). Search strategies and retrieval results were printed out and analyzed by all three authors using standardized measure- ments devised through pretesting. The Subjects All participants were undergraduate novice users enrolled in four sections of an introductory course in Business Com- puter Information Systems taught by one of the authors. Each student had limited online search retrieval experi- ence using the Boolean and with Baruch's online public access catalog, CUNY+. The average age of the subjects was twenty-two years; 45 (59 percent) were female and 31 (41 percent) male. The participants reported little experience with CD-ROMs, with more than 65 per- cent having no previous experience. The ESL group and NS group were fairly evenly matched in terms of ages, gender, and previous CD-ROM experience (table 1 ). Additionally, the final grades from 52 College & Research Libraries January 1995 TABLEt DEMOGRAPHIC CHARACTERISTICS OF THE SUBJECTS Native Speakers ESL Speakers Pooled Characteristic N Sample size 42 CIS course mean grade CD-ROM experience Mean response 0.55 0 =never 29 1 = 1 or 2 times 7 2 = 3 to 5 time~ 2 3 = 6 or more times 4 ESL student's native language Spanish Chinese/Korean/Japanese European Other Asian Note: All percentages are columnwise. the CIS course were compared to ensure that academic ability was not a con- founding factor. Sixty-three percent were first- and sec- ond-year undergraduates. Sixty-seven percent of the two samples received their high school diplomas in this country; of those 21 percent were ESL students. Of the 34 students who indicated that Eng- lish was not their first language, 13 (38 percent) spoke Chinese, Korean, or Japa- nese, 11 (32 percent) were Spanish- speaking, 5 (15 percent) spoke other European languages, and 5 (15 percent) spoke other Asian (e.g., Urdu, Vietnam- ese) languages. The Search The study took place at Baruch Li- brary's Graduate Business Resource Center, a small electronically based cen- ter where conditions were easily control- led. The subjects were instructed to search the topic "Who takes care of the children when the parent is at work?" in the journal database of SilverPlatter's PsycLit. Written information was pre- sented so that students would consider searching for abstracts including con- cepts such as day cdre or child care, and working mothers or working fathers. The % N % N % . 55.3 34 44.7 76 100.0 86.0 84.7 85.4 0.56 0.55 69.0 21 61.8 50 65.8 16.7 8 23.5 15 19.7 4.8 4 11.8 6 7.9 9.5 1 2.9 5 6.6 11 32.4 13 38.2 5 14.7 5 14.7 experiment's instructions excluded any hints which might lead the student to use a specific search technique with spe- cific topics, such as the use of Boolean operators. Several criteria led to the development of this search problem. The experiment's search topic was representative of a typi- cal essay assignment given to under- graduates. Second, the topic afforded opportunities to use Boolean operations with the intersection of some concepts (employment, child care, parenting) and the union of others (day care or child care). PsycLit provided some implicit intersec- tion and union operations, but the subject could improve search effectiveness by us- ing the Boolean operators and and or. In an attempt to replicate the autono- mous searching experiences one may en- counter in a busy library, individual assistance was not offered. The subjects received the American Psychological As- sociation's PsycLIT Quick Reference Guide of June 1992, which is available to all users.22 Upon completion of the search, participants filled out question- naires soliciting basic demographic infor- mation as well as language background and previous CD-ROM usage experi- ence. This questionnaire was filled out afterward so that if a student realized that the experiment was examining searches based on language back- ground, it would be too late for this re- alization to affect those searches. After data collection, the authors evaluated the search histories and cita- tions, using a carefully designed set of measurements. Some of the major factors analyzed included: (1) language-based techniques such as plurals, synonyms, al- ternative concepts for caretaker such as babysitter or grandparent, keywords, vari- ant spellings, descriptors; (2) procedural techniques such as correct and incorrect employment of Boolean connectors with vocabulary or previous search sets or indexing; and (3) effectiveness and effi- ciency measures, such as the ratio of the relevant citations compared to total number selected and the use of previous search set numbers. The subjects' use of the concepts of employment, parenting, or child care in their searches received particular scrutiny. Selection of the Database The subjects searched SilverPlatter's PsycLIT CD-ROM because this database permitted several observations. It al- lowed the examination of vocabulary us- age in the context of a text-based database. Recorded search histories provided data for examining techniques such as plurali- zation of keywords, Boolean operators, proximity connectors, field searching, truncation, and use of the thesaurus and word index. Since some Asian languages do not offer different words for singular CD-ROM Search Techniques 53 and plural meanings, the database's ab- sence of automatic pluralization proved useful to this study. Statistical Methods Because the sample sizes of both groups were greater than 30, parametric two-sample testing was used in compar- ing quantitative variables; chi-square evaluations were used for categorical data. A significance level of .05 was se- lected. STATGRAPIDCS Version 5.0 was the statistical package used for the data analyses. ANALYSIS AND DISCUSSION An examination of the search steps indicated that many students in both groups encountered similar difficulties in searching effectively and efficiently. Many basic features available in the PsycLIT database were not used by these novice searchers. Whereas some features (permuted thesaurus, field searching) may be too advanced for casual users, other features such as Boolean operators, search modification, and pluralization remain essential elements of computer- ized searching. Search Results Table 2 illustrates measures for the search results. The measures fall into two areas: the number of subsearches (the individually searched terms or phrases) performed and utilized in the overall search statement; and the ratios of the targeted search topics to the num- ber of citations finally utilized. TABLE2 MEASURES OF SEARCH RESULTS English Group ESLGroup Pooled Characteristic Mean sd Mean sd Mean sd p< Number of subsearches 10.14 9.57 7.82 6.69 9.11 8.51 Number of used subsearches 2.45 1.94 1.50 .93 2.03 1.57 .02 Ratios of topics-to-citations · No. of employment citations/ all citations .67 .38 .59 .45 .63 .42 No. of parenting citations/ all citations .75 .35 .70 .43 .72 .39 No. of childcare citations/ all citations .76 .34 .68 .44 .72 .39 54 College & Research Libraries There was no difference between the NS and ESL subjects in the total number of subsearches they performed for their entire searches. Native English-speaking users, however, selected their citations from a greater number of their sub- searches (M = 2.45 subsearches) than did ESL students (M = 1.5 subsearches), t = 2.63, p < .02. It is possible that ESL students selected a lower number of sub- searches because of language difficulties in interpreting the instructions or capa- bilities of the CD-ROM system. The ratios for the citations relevant to specific search topics to the total num- ber of selected citations were calculated. These measured the precision of the sub- jects' selected citations. Although the ESL students had lower ratios, the differences between the groups were not significant. Vocabulary-based Measures There were significant differences in techniques that related most directly to vocabulary skills. The findings are grouped in three areas. Use of Singular versus Plural Word Forms. Native English-speaking users were significantly more likely to use plural word forms, )(l (1, N = 76) = 10.09, p < .002, when searching (table 3). This January 1995 increased the number of hits these sub- jects found for the search topic. Another aspect of this issue that showed a signifi- cant difference was the dangling singular. This term refers to a subject's use of a singular term without using the corre- sponding plural form in another sub- search, thus limiting search results. On the "parenting" search term that the majority of subjects used-mother/mothers-the ESL group was more likely to use only the singular mother without the plural moth- ers when searching, )(l (1, N = 76) = 8.05, p < .005. This limited the number of cita- tions that PsycLIT found. This phenomenon had a significant impact on the searches in several cases. PsycLIT recalls only those citations that use the same word form. Table 4 pro- vides an illustration of this anomaly; if a user searches only for the singular term mother in a search, far fewer citations would be found, limiting the likelihood of finding relevant hits. Even though truncation can easily cir- cumvent the singular /plural anomaly, only one out of 76 students made use of truncation to retrieve word endings. The PsycLIT Quick Reference Guide men- tioned truncation as a retrieval tool, but neglected to specifically suggest or il- TABLE3 SEARCHING TECHNIQUES USED English Group ESLGroup Pooled Characteristic N % N % N % p< Boolean techniques and used at all 23 54.8 19 55.9 42 55.3 and used correctly 23 54.8 18 52.9 41 53.9 and used incorrectly* 3 7.1 2 5.9 5 6.6 or used at all 8 19.0 10 29.4 18 23.7 or used correctly 8 19.0 6 17.6 14 18.4 or used incorrectly* 3 7.1 4 11.8 7 9.2 Other techniques used Thesaurus 2 4.8 3 8.8 5 6.6 Index 7 16.7 5 14.7 12 15.8 Expanded terms 16 38.1 4 11.8 20 26.3 .01 Root (truncation) 1 2.4 0 0.0 1 1.3 Plural forms 40 95.2 23 67.6 63 82.9 .002 • Four subjects, three in the NS group and one in the ESL group, used the Boolean operator and both correctly and incorrectly. Three subjects in the NS group used the Boolean operator or correctly and incorrectly. TABLE4 DIFFERENCE IN NUMBER OF CITATIONS FOUND USING MOTHER (MOTHER [SINGULAR] VERSUS MOTHERS [PLURAL]) No. Search Terms 948 Child care 10 Working mother 70 Working mothers 0 (Working mother) and (child care) 15 (Working mother) and (child care) lustrateits use in searching for both sin- gulars and plurals. Unfortunately, Sil- ver Platter as well as other major producers such as Dialog OnDisc do not provide for automatic pluralization. Synonyms and Expanded Terms. An important aspect of vocabulary usage in electronic databases is synonym search- ing. The NS students were more likely to use expanded terms or synonyms such as parents or grandparents than were their ESL counterparts, X2 (1, N = 76) = 6.72, p < .01. Almost 40 percent of NS students searched on alternative words or con- cepts-for baby-sitters or nannies- where only 12 percent of ESL speakers did so. This provided NS students with more robust searches. This finding is not surprising, as knowledge of vocabulary is strongly bound to the user's language; a native speaker's vocabulary will typi- cally be greater and deeper than that of a non-native speaker, no matter what the language. Underutilization of Other Search Techniques Other basic search techniques were so underutilized in the search strategies of both groups that no significant differ- ences were found in these areas: • Slightly more than half (55 percent) of the students used the Boolean and. • Twenty-four percent used the Boolean or (more than one-third of those stu- dents used the or incorrectly). • Sixteen percent used the index. • Seven percent used the thesaurus. • Only 1 student used truncation. Students virtually ignored advanced techniques, s~ch as field searching, CD-ROM Search Techniques 55 that were exhaustively illustrated in the PsycLIT Quick Reference Guide. These findings are generally consistent with those reported previously with a smaller sample size:23 The lack of significant difference be- tween the groups in terms of Boolean usage was not surprising. Many lan- guages, particularly Indo-European lan- guages, contain words that translate to and and or, and Boolean use is probably a simple translation problem for those stu- dents whose native languages contain these words. Some Asian languages, par- ticularly Chinese, do not contain words for and and or. For this reason, the authors examined Boolean usage between the na- tive English-speaking students and those whose first language was Chinese. There was no significant difference between the two groups, which weakens the transla- tion explanation. Boolean concepts are logical or alge- braic concepts which cross linguistic lines. Evidence suggests that users sepa- rate the natural language usage of and and or from the Boolean use of these words for intersection and union, switching be- tween their respective connotations de- pending upon the context. This is true for both Computer Information Systems majors and computer programmers; both groups of subjects used the words and and or quite differently in natural language contexts as opposed to com- puter programming contexts requiring Boolean operations.24,25 If natural lan- guage usage and Boolean operations are two different bodies of knowledge, the semantic concepts behind Boolean op- erations-union, intersection, and nega- tion-can exist across languages. Word Index Searching from the word index for the terms child care or day care, both of which were retrievable .as single and com- pound words, can be problematic. Since recall is limited to the exact way a term is entered, the system does not retrieve variant spellings or compound words. Unlike the thesaurus, there are no links or suggestions that additional hits may be found by entering the term differently. 56 College & Research Libraries TABLES RETRIEVAL USING ALTERNATE SPELLINGS No. Spelling 264 Childcare 948 Child care 1,057 Childcare or child care TABLE6 INCONSISTENCIES IN LINKING . ALTERNATIVE SPELLINGS IN THE THESAURUS AND INDEX No. Spelling 73 Child-care in de (thesaurus) 366 Child-day-care in de (thesaurus) 137 Child-care (index) 264 Childcare (index) 948 Child care (index) As illustrated in table 5, approximately BOO abstracts could be missed by an in- experienced student searching only for childcare and not for child care. The prob- lem is compounded by the screen dis- play of the index; it is necessary to scroll through several screens to find the word in an alternative spelling form. Although a useful feature, index searching could benefit from an enhancement that pro- vided automated links to alternative (and plural) spellings. Thesaurus Only 5 students (7 percent) used the sophisticated permuted thesaurus which automatically ors the selected descriptor terms, suggests related or narrower terms, and ~xplodes terms. The PsycLIT Quick Reference Guide, however, does not directly relate the use of the broadening or to its thesaurus. By listing the term as child- care, the thesaurus inadvertently reinforces the use as such, although there are more instances of childcare or child care in the index (table 6). CONCLUSIONS This study provides a better under- standing of the problems novice ESL and NS undergraduate end-users encounter when searching a CD-ROM database. January 1995 Because many undergraduates at public urban universities speak English as a second language, the authors were inter- ested in determining whether there were significant differences in searching strategies between ESL and native Eng- · !ish-speaking undergraduates. The main difficulties of ESL students that differed significantly from those of native Eng- lish speakers were concentrated in vocabu- lary issues-utilization of plural forms, and the use of synonyms and expanded concepts for more robust searching. The findings of this experiment showed statistically significant differ- ences between the two groups in two language-based areas. First, NS users were more likely to use plural word · forms when searching (p < .002). The lack of automatic pluralization in some CD-ROM interfaces puts ESL students ~t a diSadvantage. Secondly, NS users were more likely to experiment with syno- nyms and alternative words to search for some topics (p < .01). The main difficulties of ESL students which differed significantly from those of native English speakers were concentrated in vocabulary issues-utilization of plural forms, and the use of synonyms and expanded concepts for more robust searching. Other basic search techniques were so underutilized in the search strategies of both groups that no significant differ- ences were found in these areas; 55 per- cent of the students used the Boolean and, less than a quarter used the Boolean or, 16 percent used the index, and 7 per- cent used the thesaurus. Only one stu- dent used truncation. These findings are generally consistent with findings re- ported previously with a smaller sample size.26 This suggests that CD-ROM biblio- graphic instruction for the general user should include Boolean logic, index and thesaurus usage, and truncation. Bibli- ographic instruction for ESL students should focus on techniques to circum- vent difficulties with plurals and limited vocabulary. Such instruction could focus on the use of truncation and wild-card characters for plural searching, and the- saurus use for alternative vocabulary. Additionally, all database producers should use interface standards that in- clude intelligent automatic pluraliza- tion-not merely appending "s" or "es" to singular word forms-and automatic alternative spellings to assist ESL and NS searchers. Areas of Future Study These conclusions come with the usual cautionary note that the size of the sample group (N = 76) should be consid- ered before suggesting generalizations. Nonetheless, the relative homogeneity of the user level (undergraduate users who were not Information Science stu- dents) reflects typical problems novice undergraduate end-users may experi- ence. The results may have more exter- nal validity when applied to this growing user population than other studies using more diverse levels of us- ers, or those using students in Informa- tion Science courses as subjects. The results warrant future investigation us- ing much larger sample groups. Although the language backgrounds of the ESL students were representative of students at the City University of New York, the samples were too small to ex- amine significant differences within the ESL group. Where the two groups dif- fered significantly may be a factor based on a specific native language. An exami- nation of users grouped by their native language backgrounds may produce findings that identify those ESL students more likely to encounter difficulties with CD-ROM searching. Expansion of our sample size should enable a more thor- ough investigation. The underutilization of some tech- niques by all students suggests a need for more research on which specific tools are actually useful to novice users. It was particularly telling that techniques fea- tured prominently in the Quick Reference CD-ROM Search Techniques 57 Guid~the field search, the thesaurus, and the index-were underutilized by the users in this sample. While these are powerful techniques for the skilled re- searcher who needs to perform searches that are both precise and exhaustive, they may be conceptually too difficult for the autonomous novice user who does not need such high quality searches. For the novice user, bibliographic instruction that focuses on simpler techniques and addresses some of the anomalies pre- sented here might prove more useful. Recommendations CD-ROM databases contain many fea- tures that make them attractive to autonomous end-users. Nonetheless, In- formation Science professionals must become aware of the implications of pro- viding computerized information to an ever-increasing number and variety of users. One of these implications is the need for bibliographic instruction for all users in the problem areas discussed in this paper with specialized, if limited, additional assistance for ESL users. Automatic pluralization, alternative spellings, uniform truncation commands, and synonym searching options will improve the searching capabilities of all users. Additionally, when developing future interface designs and standards, pro- ducers and vendors of "user-friendly" CD-ROM databases should consider some of the problems searchers face. Automatic pluralization, alternative spell- ings, uniform truncation commands, and synonym searching options will im- prove the searching capabilities of all users. Although the adoption of the Z39 .50 protocol or other standards will provide greater standardization across a variety of interfaces, a better goal would be to provide an interface that provides a high level of usability across a variety of users. 58 College & Research Libraries January 1995 REFERENCES AND NOTES 1. U.S. Department of Education, National Center for Education Statistics, The Condition of Education: Post-Secondary Education (Washington, D.C.: Govt. Print. Off., 1991), 2: 21-23. 2. U.S. Department of Commerce, Bureau of the Census, Census of Population and Housing · [CD-ROM], File 3A, Table P-028 (Washington, D.C., 1990). 3. James Murphy, "Baruch Examines International Student Work Rules," interview by Sue- Peng Chus, in The Ticker, (college newspaper) 63, no. 3 (1992): 1. 4. For an analysis of the freshmen classes entering Baruch College in the fall semesters of 1991 and 1992, see the Annual Freshmen Survey(s) from the Office of the Vice President for Student Development, Baruch College, CUNY prepared by Shehbal Teilmann. 5. Ibid. 6. Tim Bucknall and Nikki Mangrum, "U-Search: A User Study of the CD-ROM Service at the University of North Carolina at Chapel Hill," RQ 31 (Summer 1992): 542-53. 7. Ibid. See also Domenica M. Barbuto and Elena E. Cevallos, "End-user Searching: Program Review and Future Prospects," RQ 31 (Winter 1991): 214-27, and Cindy Faries, "User's Reactions to CD-ROM: The Penn State Experience," College & Research Libraries 53 (Mar. 1992): 139-49. 8. Louise T. Su, "Evaluation Measures for Interactive Information Retrieval," Information Processing & Management 28, no. 4 (1992): 503-16. 9. Susan K. Charles and Katherine E. Clark, "Enhancing CD-ROM Searching with Online Updates: An Examination of End-User Needs, Strategies, and Problems," College & Research Libraries 51 {July 1990): 321-28. 10. Carol Hansen Fenichel," Online Searching: Measures That Discriminate among Users with Different Types of Experiences," Journal of the American Society for Information Science 32 {Jan. 1981): 23-32, and Somporn Puttapithakporn, "Interface Design and User Problems and Errors: A Case Study of Novice Searchers," RQ 30 (Winter 1990).: 195-204. See also Mary E. Brown, "Design for a Bibliographic Database for Non-Professional Users" in Proceedings of 54th Annual Meeting, ASIS (Medford, N.J., Learned Information, 1991) 28: 276-82. 11. WilliamS. Cooper, "Getting beyond Boole," Information Processing & Management 24, no. 3 (1988): 243-48. Also Charles and Clark,"Enhancing CD-ROM Searching," 321-28, Gerard Salton, "The State of Retrieval System Evaluation," Information Processing & Management 28, no.4 (1992): 243-48 and Pat Ensor, "Knowledge Level of Users and Non-Users of Keyword Boolean Searching on an Online Public Access Catalog," RQ 32 (Fall 1992): 60-74. 12. Cf. Bucknall and Mangrum, "U-Search," 542-53, Barbuto and Cevallos, "End-user Search- ing," 214-27, and Faries, "Users' Reactions to CD-ROM," 139-49. 13. Cf. Fenichel, "Online Searching," 23-32 and Puttapithakporn, "Interface Design," 195-204. 14. Gillian Allen, "CD-ROM Training: What Do the Patrons Want?" RQ 30 (Fall1991): 88-93. 15. Deanna Nipp, "Back to Basics: Integrating CD-ROM Instruction with Standard User Edu- cation," Research Strategies, 9 (Winter 1991): 41-47. 16. Linda Stewart and Jan Olsen, "Compact Disk Databases: Are They Good for Users?" Online 12 (1988): 48-52, and Gillian Allen, "Patron Response to Bibliographic Databases on CD- ROM," RQ 29 (Fall1989): 103-10. 17. Sally G. Wayman, "The International Student in the Academic Library," Journal of Academic Librarianship, 9 {Jan. 1984): 336-41. See also Tesfai Kflu and Mary A. Loomba, "Academic Libraries and the Culturally Diverse Student Population," College & Research Libraries News, 51, no. 6 (1990): 524-27; Kwasi Sarkodie-Mensah, "Dealing with International Stu- dents in a Multicultural Era," Journal of Academic Librarianship 18 (Sept. 1992): 214-16; and Ziming Liu, "Difficulties and Characteristics of Students from Developing Countries in Using American Libraries," College & Research Libraries, 54 (Jan. 1993): 25-31. 18. Daniel Liestman and Connie Wu," Library Orientation for International Students in Their Native Language," Research Strategies, 8, no. 4 (1990): 191-96, and Liu, "Difficulties and Characteristics," 25-31. 19. Wayman, "The International Student," 336-41, and Mary Alice Ball and Molly Mahony, "Foreign Students, Libraries, and Culture," College & Research Libraries, 48 (Mar. 1987): 160-66. 20. Liu, "Difficulties and Characteristics," 25-31. CD-ROM Search Techniques 59 21. Mary Beth Allen, "International Students in-Academic Libraries," College & Research Librar- ies, 54 (July 1993): 323-33. 22. American Psychological Association, PsycLIT Quick Reference Guide (Washington, D.C.: American Psychological Association, PsyciNFO User Services, 1992) . . 23. Diane DiMartino, William J. Ferns, and Sharon Swacker,"A Study of CD-ROM Search Techniques by English-as-a-Second- Language (ESL) Students" in Proceedings of 14th Na- tional Online Meeting, May 4-6, 1993, ed. Martha E. Williams (Medford, N.J.: Learned Information), 107-17. 24. William J. Ferns, "Boolean Operators in Query Languages: A Study of the Contradictions between Formal Logic and Natural Language" (master's thesis, Bernard M. Baruch College, CUNY, 1985). 25. William J. Ferns and B. Loerinc Helft, "The Use of Query Language Boolean Operators by Professionals," in Proceedings of the Fourth Symposium on Empirical Foundations of Infor- mation and Software Sciences (New York: Plenum, 1987). 26. DiMartino, Ferns and Swacker," A Study of CD-ROM Search Techniques," 107-17. STATEMENT OF OWNERSHIP, MANAGEMENT AND CIRCULATION College & Research Libraries, Publication No. 121-440, is published bimonthly by the Association of College and Research Libraries, American Library Association, 50 E. Huron St., Chicago, IL 60611-2795. The editor is Gloriana St. Clair, E506 Pattee Library, Pennsylvania State University, University Park, PA 16802. Annual subscription price, $25.00. Printed in U.S.A. with second-class postage paid at Chicago, Illinois. As a nonprofit organization authorized to mail at special rates (DMM Section 424.12 only), the purpose, function, and nonprofit status of this organization and the exempt status for federal income tax purposes have not changed during the preceding twelve months. EXTENT AND NATURE OF CIRCULATION (Average ~gures denote the average number of copies printed each issue during the preceding twelve months; actual figures denote actual number of copies of single issue published nearest to filing date: September 1994 issue.) Total number of copies printed: 12,508; actual 12,430. Sales through dealers, carriers, street vendors and counter sales: none. Mail subscription: average 11,701; actual11,871. Free distribution: average 80; actual103. Total distribution: average 11,781; actual11,974. Office use, leftover, unaccounted, spoiled after printing: average 727; actual456. Total: average 12,508; actual12,430. Statement of Ownership, Management and Circulation (PS Form 3526, Jan. 1991) for 1994 filed with the United States Post Office Postmaster in Chicago, October 12, 1994