Perceived Quality of WhatsApp Reference Service: A Quantitative Study from User Perspectives ARTICLE Perceived Quality of WhatsApp Reference Service A Quantitative Study from User Perspectives Yan Guo, Apple Hiu Ching Lam, Dickson K. W. Chiu, and Kevin K. W. Ho INFORMATION TECHNOLOGY AND LIBRARIES | SEPTEMBER 2022 https://doi.org/10.6017/ital.v41i3.14325 Yan Guo (kguo@connect.hku.hk) is MSc(LIM) Graduate, Faculty of Education, The University of Hong Kong. Apple Hiu Ching Lam (applelamwork@gmail.com) is EdD Candidate/MSc(LIM) Graduate, Faculty of Education, The University of Hong Kong. Dickson K. W. Chiu (dicksonchiu@ieee.org) is Lecturer, Faculty of Education, The University of Hong Kong. Kevin K. W. Ho (ho.kevin.ge@u.tsukuba.ac.jp) is Professor of Management Information Systems, Graduate School of Business Sciences, Humanities and Social Sciences, University of Tsukuba. © 2022. ABSTRACT Academic libraries are experiencing significant changes and making efforts to deliver their service in the digital environment. Libraries are transforming from being places for reading to extensions of the classroom and learning spaces. Due to the globalized digital environment and intense competition, libraries are trying to improve their service quality through various evaluations. As reference service is crucial to users, this study explores user satisfaction towards the reference service through WhatsApp, a social media instant messenger, at a major university in Hong Kong and discusses the correlation between the satisfaction rating and three variables. Suggestions and recommendations are raised for future improvements. The study also sheds light on the usage of reference services through instant messaging in other academic libraries. INTRODUCTION Due to the advancement of new technologies and mobile devices, library resources and services are more accessible.1 Apart from independent searching strategies, the interactions between librarians and users have become an effective method to solve user problems, referred to as reference services.2 According to the Reference and User Services Association (RUSA), reference services include creating, managing, and assessing reference transactions and activities. 3 With the increasing user needs, reference services have become an essential part of library services and commonplace in academic libraries.4 Further, technology development requires reference librarians to possess updated skills, willingness, and interest to deal with user inquiries.5 Recently, due to the COVID-19 pandemic, users have increasingly utilized virtual reference services to help them obtain information required for their academic studies instead of face-to-face modes.6 Some libraries have employed different virtual tools, for example, instant messaging services, to provide reference services to their users. One of the most popular global instant messaging services is WhatsApp.7 Referring to the Digital 2022—Hong Kong report, the most-used social media platform among Internet users aged 16 to 64 in Hong Kong was WhatsApp (84.3%), followed by Facebook (83.7%), Instagram (65.6%), WeChat (55.2%), and Facebook Messenger (50.4%).8 The popularity of WhatsApp in Hong Kong accordingly increases WhatsApp reference service usage in academic libraries. The qualitative study by Tsang and Chiu has identified WhatsApp as one of the most commonly-used and relatively preferred reference services of an academic library in Hong Kong.9 mailto:kguo@connect.hku.hk mailto:applelamwork@gmail.com mailto:dicksonchiu@ieee.org mailto:ho.kevin.ge@u.tsukuba.ac.jp INFORMATION TECHNOLOGY AND LIBRARIES SEPTEMBER 2022 PERCEIVED QUALITY OF WHATSAPP REFERENCE SERVICE | GUO, LAM, CHIU, AND HO 2 Many studies have investigated reference service quality with measurements such as satisfaction rating, perceived gaps in reference services ability to meet user expectations, and other information-seeking behaviors. However, few studies focus on instant messaging reference services compared to traditional services, except for a notable recent qualitative study by Tsang and Chiu.10 Therefore, this research aims to quantitatively evaluate user satisfaction with WhatsApp’s application in reference service of a major university’s library in Hong Kong through three dimensions: affect of service (AS), information control (IC), and library as place (LP), which are detailed in the Research Purpose section. The results can help librarians better understand the effectiveness of applying WhatsApp and other instant messaging to improve reference service quality. By expanding technology-based services, libraries can become more competitive in the digital era and provide better user experiences in the future. Thus, this study deals with the following four research questions (RQs): RQ1. What is the users’ awareness level of the library’s reference services? RQ2. How do users evaluate the WhatsApp application in the library? RQ3. What are the relationships between user satisfaction and the three service dimensions AS, IC, and LP? RQ4. How can academic libraries increase user satisfaction with WhatsApp reference services? LITERATURE REVIEW In the late 1800s, library leaders started to pay attention to the importance of reference services. 11 Since then, reference services have also caught the public’s attention and were introduced into public libraries. Reference services can assist readers in solving problems through various interactions between users and staff.12 Currently, the library is not merely a repository of collections, and librarians can provide more help, particularly fulfilling users’ various information needs rather than just offering directions or physical locations of books.13 Nowadays, librarians strive to solve various user problems and inquiries with their professional skills and information literacy.14 At first, in-person and telephone were the most common ways for reference services. However, with the increasing number of remote users and ubiquitous internet connectivity, face-to-face reference and asynchronous emails can no longer satisfy users’ needs.15 Thus, libraries increasingly explore collaborative software and mobile applications such as instant messaging, online chatting, video sessions, and other methods to serve users, referred to collectively as virtual reference.16 Virtual reference occurs electronically in real time, where users may interact with librarians through smartphones, computers, or other devices without physical presence.17 As libraries began to use the internet, several case studies investigated instant messaging reference services in academic libraries.18 At the same time, librarians and researchers began to investigate reference service quality with designated measurements. Various indicators can help measure user satisfaction levels, such as accuracy, communication skills, user satisfaction, instruction, and user’s willingness to return.19 Although these indicators were originally developed for physical reference services, most principles and methods can still be applied to virtual reference services, as instant messaging has become one of the most frequently used INFORMATION TECHNOLOGY AND LIBRARIES SEPTEMBER 2022 PERCEIVED QUALITY OF WHATSAPP REFERENCE SERVICE | GUO, LAM, CHIU, AND HO 3 channels.20 Some studies have confirmed the effectiveness of instant messaging for reference services for more traditional means, such as phone and email.21 As one of the most popular social media chatting software platforms, WhatsApp has become a powerful tool for connecting librarians and users. A primary difference from traditional phone- based reference services is that WhatsApp can share texts, images, documents, and videos (and their links) at a low cost.22 WhatsApp can run as a mobile application on smartphones or as a web page on desktop browsers named WhatsApp Web. WhatsApp Web users are required to use their mobile phone to scan the QR (Quick Response) code on the computer browser (https://web.whatsapp.com/) for authentication before use. As the functionality of WhatsApp Web is similar to WhatsApp, users can easily adapt to WhatsApp Web on desktop computers. As of March 2020, the number of active WhatsApp users has globally increased to approximately 2 billion and is still growing steadily.23 WhatsApp, by April 2021, had become the most popular messaging application based on the number of monthly active users, compared with other popular messaging applications.24 Studies also indicate that students may use WhatsApp for two to three hours daily.25 Although the essential chat functions of WhatsApp are similar to other instant messaging services such as Facebook Messenger and WeChat, WhatsApp and WeChat have been more popular for Hongkongers and Mainland Chinese, respectively.26 Surprisingly, Howard et al. studied students’ habits of using social media platforms at Purdue University in the US and revealed that respondents rarely use WhatsApp in their daily lives, indicating that residents in different regions may have different social media platform preferences.27 Recently, Odu and Omini have demonstrated a significant relationship between using WhatsApp and library service satisfaction from the student’s perspective.28 Some studies also stressed that many students welcome WhatsApp as an effective reference service platform.29 However, Friday et al. pointed out that some librarians might not be trained and equipped with proper and up -to- date skills in using social media tools to provide library services effectively and efficiently.30 Further, Aina, Babalola, and Oduwole argued that hurdles such as instructional policies, lack of time, and heavy workloads might cause difficulties in using these tools to provide library services.31 As for evaluation, Mohd Azmi, Noorhidawati, and Yanti Idaya Aspura applied RUSA’s guidelines for the behavioral performance of reference and information service providers to evaluate the perceived importance versus actual practices of WhatsApp reference service from librarians’ perspectives.32 They suggested that although librarians expressed their awareness of the importance of RUSA guidelines, they would not fully comply with the guidelines because of time and other constraints. Yet, few studies deal with the satisfaction with WhatsApp reference services of academic libraries from user perspectives. Research Purpose Regarding WhatsApp and library services, a few studies focused on finding the relationship between WhatsApp and service usage, user attitudes toward WhatsApp applications, and the difficulties of using WhatsApp, particularly for reference services.33 Though Mohd Azmi, Noorhidawati, and Yanti Idaya Aspura evaluated librarians’ behavioral performance in providing WhatsApp reference service, it was from librarians’ perspectives instead of users.34 https://web.whatsapp.com/ INFORMATION TECHNOLOGY AND LIBRARIES SEPTEMBER 2022 PERCEIVED QUALITY OF WHATSAPP REFERENCE SERVICE | GUO, LAM, CHIU, AND HO 4 Thus, we studied user satisfaction with the WhatsApp reference service offered by academic libraries by adapting Tsui’s instrument to develop the survey framework.35 Tsui employed three key indicators, i.e., affect of service (AS), information control (IC), and library as place (LP), in LibQUAL+, an online library assessment tool developed by the Association of Research Libraries (ARL).36 AS measures “empathy, responsiveness, assurance, and reliability of library employees,”37 like librarian-user interactions concerning the librarians’ knowledge in inquiry responses and the level of reference service provided.38 IC measures “how users want to interact with the modern library and include scope, timeliness and convenience, ease of navigation, modern equipment, and self-reliance,”39 such as library resource availability and accessibility from user perspectives. 40 LP measures “the usefulness of space, the symbolic value of the library, and the library as a refuge for work or study,”41 such as the availability of adequate facilities and an appropriate physical environment from user perspectives.42 The application of these indicators will be further discussed in the Methodology section. METHODOLOGY This study chose a major academic library in Hong Kong with a long track record of technological advancement. A reference desk is situated near the library’s main entrance for traditional services. The library’s web page shows a clear Ask a Librarian column with diversified methods for reference services, including email, telephone, WhatsApp, and other electronic devices to access the reference services. Notably, the WhatsApp reference service is operated the same as other channels, available Monday to Friday from 9 am to 5 pm (except on public holidays). The library promises an inquiry response in no more than four hours. The mission and vision of such reference services are to • help locate information resources; • assist in searching strategies and research; • deal with queries about the use of facilities and services; and • equip users with information literacy. The library has developed a WhatsApp Business account with a mobile phone in the WhatsApp Business application and uses the WhatsApp Web function to handle user inquiries on desktop computers. One to two library assistants support the WhatsApp reference service on shift seamlessly from 9 am to 5 pm, including the lunch break, and a professional librarian reviews WhatsApp inquiry records weekly. This study used a survey administered through Google Form, both online and offline, to collect user perceptions about the library’s WhatsApp reference service. Online methods for collecting survey responses included email, Facebook, WeChat, and WhatsApp, and offline methods included site delivery at the library entrance and sticking the survey QR code on public notice boards. No incentives were provided for the voluntary data collection. The data collected comprised mainly undergraduate and postgraduate students to represent a general user view of the WhatsApp reference service. Microsoft Excel and IBM SPSS Statistics were used to analyze the data, including bivariate correlation for investigating the relationships between WhatsApp satisfaction and the three variables based on Tsui’s study, AS, IC, and LP.43 Among these indicators, AS focuses on whether WhatsApp is easy to use and supportive; IC evaluates the response speed, accuracy, and accessibility of the WhatsApp reference service; and LP measures the staff attitude and whether WhatsApp helps encourage librarian-user INFORMATION TECHNOLOGY AND LIBRARIES SEPTEMBER 2022 PERCEIVED QUALITY OF WHATSAPP REFERENCE SERVICE | GUO, LAM, CHIU, AND HO 5 communication. The survey also includes demographic information, reference services usage, and user satisfaction with the WhatsApp reference service. Participants were asked to evaluate the quality of WhatsApp reference service from these three dimensions through five-point Likert scales in the satisfaction rating part. Finally, the survey asked for the overall satisfaction and other useful comments about the reference service. DATA ANALYSIS Demographic Information As the main analysis of this study is regression analysis, a check on the minimum number of participants needed for analysis was performed. As explained later in this paper, the regression involved six predictors of satisfaction. Using medium effect size and 0.8 as the statistical power, the minimum sample size should be 97 using an online a-priori sample size calculator for multiple regression (https://www.danielsoper.com/statcalc/calculator.aspx?id=1). The data collection yielded 131 completed responses, with 66% of master’s students and the rest undergraduates. Respondents had diversified academic backgrounds, including education (26.0%), science (14.5%), business and economics (13.0%), engineering (12.2%), liberal arts (10.7%), social science (9.9%), architecture (9.2%), and legal studies (4.6%). For the time spent on instant messaging such as WhatsApp and WeChat, 39% spent more than three to five hours every day, while one-fifth of them would spend one to two hours. 22% of respondents spend five hours or above, and only a small portion of them (19%) would spend less than an hour. In summary, most respondents would spend at least one hour on instant messaging daily. Usage of Reference Service Table 1 summarizes respondents’ usage of reference services with a five-point Likert scale (1 = Never; 5 = Always). As shown, walk-in and email are the most common methods to use the reference service, while WhatsApp is the least frequent. When it comes to the purposes of using reference services (see table 2), databases and e-resources and identifying information sources are the two most common purposes for respondents, followed by service and facility and research assistance. Table 1. Usage frequency of reference service through different methods (n = 131) Methods Walk-in Email Phone WhatsApp Mean score 3.23 3.24 2.53 2.32 Note: 1=Never; 5=Always Table 2. Purposes of using reference service (n = 131) Purposes Service and facility Database and e- resources Identify information sources Research assistance (individual/ group) Other Mean score 3.10 3.36 3.28 3.08 2.31 Note: 1=Never; 5=Always https://www.danielsoper.com/statcalc/calculator.aspx?id=1 INFORMATION TECHNOLOGY AND LIBRARIES SEPTEMBER 2022 PERCEIVED QUALITY OF WHATSAPP REFERENCE SERVICE | GUO, LAM, CHIU, AND HO 6 When asked about their preferred way to use reference services, more than half of the respondents said they would use email (67.9%), followed by walk-in (59.5%), WhatsApp (23.7%), and phone (12.2%). As the traditional method, most respondents considered walk-in, in-person reference the most effective reference method because users could receive instant help from librarians, especially for urgent and complex problems. However, results indicated that despite a gap in users seeking reference services help by instant messaging and email, this gap is smaller than that for face-to-face and telephone.44 The user ratings for reference services through different methods were compared using ANOVA. Our result shows a significant result, F(3, 520) = 30.52, p < 0.01. Walk-in (M = 4.18, SD = 0.71) is the most satisfying method. Post-hoc tests showed the rating of email (M = 3.78, SD = 0.60) and phone (M = 3.73, SD = 0.83) statistically indifferent and lower than walk-in, while both were considered better than WhatsApp (M = 3.27, SD = 0.90). Apart from these ratings, respondents were also asked to leave a few comments and suggestions for the reference service. Notably, most respondents showed a positive attitude to the WhatsApp reference service while suggesting some improvements. For example, one respondent requested “longer office hours for WhatsApp.” At the time of this research, the WhatsApp reference service hours were Monday to Friday from 9 am to 5 pm, while in-person reference service reference hours were Monday to Friday, 8:30 am to 7 pm, and Saturdays from 8:30 am to 7 pm. Therefore, the library should extend the WhatsApp service hours to provide more flexible service time, aligning with the findings of Tsang and Chiu.45 Further, a respondent suggested that librarians should “respond to email more efficiently.” For this issue, WhatsApp could serve to expand user access to reference services instead of emails. Users’ Satisfaction with WhatsApp Reference Service Prior research reported that AS, IC, and LP influenced user satisfaction. This study adapted the instrument developed in Tsui’s prior research (see Appendix) to collect data to investigate these relationships.46 As the Cronbach’s Alpha values for all three constructs are higher than 0.7, it is valid to use the average value of these items for our data analysis.47 Table 3 shows the analysis of whether respondents’ academic level would affect AS, IC, LP, and overall user satisfaction with WhatsApp using ANOVA. Results indicated that academic level affected AS but not the other factors and satisfaction. Further, multiple regression results indicated that IC and LP affected WhatsApp satisfaction. Table 4 tabulates our findings. Table 3. ANOVA results Overall Undergraduate (n = 45) Master’s student (n = 86) F-value Affect of service (AS) 3.380 3.200 3.474 5.712 * Information control (IC) 3.202 3.162 3.223 0.286 Library as place (LP) 3.645 3.550 3.695 0.273 WhatsApp satisfaction (SAT) 3.275 3.200 3.314 0.495 Notes: *** p < 0.001; ** p < 0.01; * p < 0.05. INFORMATION TECHNOLOGY AND LIBRARIES SEPTEMBER 2022 PERCEIVED QUALITY OF WHATSAPP REFERENCE SERVICE | GUO, LAM, CHIU, AND HO 7 As shown in table 4, IC and LP have significant positive impacts on users’ satisfaction with using WhatsApp for reference services. However, considering the academic level (undergraduate = 0; master’s student = 1) in our regression model (i.e., interaction effect), the following effects are notable. First, AS does not affect user satisfaction with using WhatsApp for reference services for undergraduates but positively affects master’s respondents. Second, IC positively impacts satisfaction for both undergraduate and master’s respondents, of which the difference between these two respondent groups is statistically insignificant. Lastly, even though LP also positively impacts satisfaction for both groups, the effect is higher for undergraduates than for master ’s respondents. The different learning needs of the groups may explain such differences, as shown in table 5.48 Table 4. Regression analysis Main effect Interaction effect Independent variables Coefficient t-value Coefficient t-value Affect of service (AS) 0.0933 0.7556 −0.3503 −1.744 Information control (IC) 0.6718 6.736 *** 0.7624 4.178 *** Library as place (LP) 0.6092 6.143 *** 0.9366 6.335 *** AS  academic 0.7817 3.076 *** IC  academic −0.1741 −0.8701 LP  academic −0.5721 −0.3178 *** Intercept −1.412 −3.748 *** −1.423 −3.876 *** R2(adj). 0.5444 0.5742 F-value 52.78 *** 30.21 *** Notes: *** p < 0.001; ** p < 0.01; * p < 0.05 Table 5. Impacts of AS, IC, and LP on different student groups Undergraduate Master’s student Affect of service (AS) Not significant 0.7817 Information control (IC) 0.7624 0.7624 Library as place (LP) 0.9366 0.3645 DISCUSSIONS AND RECOMMENDATIONS Subdivision of the WhatsApp Reference Service into Specialist Subjects Our findings indicated that AS had the strongest correlation with WhatsApp satisfaction for master’s students, while the AS part had the lowest satisfaction with undergraduate students. This reflected that respondents who are undergraduates could not receive adequate supportive help from librarians through WhatsApp, aligning with the findings of Tsang and Chiu.49 A possible reason is that the number of WhatsApp reference librarians with specialist subject knowledge was INFORMATION TECHNOLOGY AND LIBRARIES SEPTEMBER 2022 PERCEIVED QUALITY OF WHATSAPP REFERENCE SERVICE | GUO, LAM, CHIU, AND HO 8 small. Yet, one general reference WhatsApp number on the library website is insufficient compared to other methods, as the library website shows seven telephone numbers of branch libraries to serve different patrons. Through different numbers, users could easily find the required experts accordingly. The WhatsApp reference service had only a single number probably because users mostly ask basic and general questions. Such a process would cost professionals too much time and energy to deal with.50 To further enhance the service, it is necessary to reform the operational policies and add a few more WhatsApp accounts, for instance, creating a WhatsApp Business account for each branch library (a total of six branch libraries) or for each school serving users in different disciplines to connect to corresponding subject librarians via specialized WhatsApp accounts.51 This approach can separate users from the general inquiry number dealing with quick and straightforward information inquiries from those requiring specific domain inquiries.52 Further, the general inquiry WhatsApp service should be extended to cater to various students’ needs by possibly improving to provide 24-hour service.53 To remedy human resources requirements, student helpers, interns, and volunteers can serve on shifts on Saturdays, Sundays, and even public holidays.54 More users may seek troubleshooting services during the holidays, especially long holidays, and recently, under the COVID-19 pandemic and its associated isolation requirements.55 More Staff Training Due to the WhatsApp reference service features, the skills required for online and face-to-face conversations are different, e.g., it is difficult to convey emotions like facial expressions and body language online.56 Further, due to the limited interactions between librarians and users and the lack of visual and audio cues through the WhatsApp reference service, librarians can hardly identify user needs in a short time.57 Therefore, librarians may need further professional training for such scenarios, particularly in answering questions quickly and precisely in real-time chat, because users tend to be more impatient during a chat engagement.58 In addition, unlike face-to- face inquiry, some complex issues often cannot be adequately explained through WhatsApp. Therefore, librarians should make appropriate referrals if some problems cannot be solved through WhatsApp. Reference services through video-based platforms such as Zoom can also help.59 Regular training could offer librarians updated information on using the tool and refresh the skills used in responding to the WhatsApp reference service among various staff members, i.e., librarians, library assistants, student helpers, volunteers, and interns. If the library staff does not acquire well-developed skills and competencies in texting, comprehension, and communication specialized in instant messaging services, they cannot efficiently and effectively understand the inquiries and search, locate, explain, and convey the appropriate information resources to users on the asynchronous WhatsApp reference service in a shorter response time.60 Establishment of WhatsApp Reference Service Guidelines Whether the WhatsApp reference service increases the capability to deal with user problems, it still relies on consistently favorable reference behaviors.61 Mohd Azmi, Noorhidawati, and Yanti Idaya Aspura pointed out that users need timely responses and friendly online contacts from librarians, though librarians might not completely follow the RUSA guidelines due to human resources constraints.62 Therefore, libraries should establish easy-to-follow, concise, and INFORMATION TECHNOLOGY AND LIBRARIES SEPTEMBER 2022 PERCEIVED QUALITY OF WHATSAPP REFERENCE SERVICE | GUO, LAM, CHIU, AND HO 9 WhatsApp-tailored guidelines for appropriately conducting WhatsApp reference services, especially because such skills differ from face-to-face services, as discussed above.63 The studied library has developed a simple series of internal work procedures for using WhatsApp Web, including how to open and close WhatsApp Web and how on-duty staff should handle inquiries. To enhance and standardize the WhatsApp reference service, the library should develop guidelines by offering some polite, brief, and interactive text templates for answering inquiries, such as “I am (name) (job title). What may I help you with, dear user?”, as well as answers to frequently asked questions. Progress reporting messages should be sent to users to acknowledge their searching status.64 The relationship between librarians and users can thus be enhanced by creating a consistent, friendly, and warm atmosphere, using informal conversation and emojis, and incorporating WhatsApp’s features to engage users and establish continued service use.65 Further, such guidelines can save time and energy in training new staff and provide the basis for the future development of artificial intelligence aids such as chatbots.66 Promotion for the WhatsApp Service Most respondents conveyed a positive attitude, considering WhatsApp a convenient way to access the reference service, which is in line with the studies by Ansari and Tripathi and Sudharani and Nagaraju.67 However, it is still not the most frequently used and preferred method in the library. One reason is that users still need help with physical materials and ask for the answers face-to- face. 68 However, this is not the only reason, as many studies showed that library promotional efforts are often weak.69 In addition to the traditional promotional materials such as leaflets and contact cards with WhatsApp numbers, the library can also broaden the promotion through massive emails and social media such as Facebook, Instagram, LinkedIn, Twitter, and Signal.70 In the information era, social media is an effective and efficient channel for reaching the target audience and disseminating information in an accessible way. 71 The library should reform the webpage of the WhatsApp reference service to further attract users. For instance, displaying some sample WhatsApp chat screenshots of librarian-user interactions on the library’s website can increase the attractiveness of the service as images can graphically represent the application’s ease of use for library reference help.72 CONCLUSION The study has investigated user satisfaction with the WhatsApp reference service in a major academic library in Hong Kong and explored the correlations between WhatsApp satisfaction with three quality dimensions AS, IC, and LP. The survey revealed various opinions toward using reference services and preference methods, including inconsistencies between users’ frequently used methods and preferred methods. Moreover, by analyzing the correlation between WhatsApp satisfaction and the three variables, results showed that users emphasized the WhatsApp reference service. The results have led to some practical suggestions for improvement: subdividing the WhatsApp reference service with subject specialists, providing more staff training, establishing staff guidelines and policies, and increasing WhatsApp service promotion. LIMITATIONS AND FUTURE RESEARCH There are still some limitations to the study. Firstly, the survey only collected limited complete responses, which may not represent all users’ views. Additionally, the perceptions of both library INFORMATION TECHNOLOGY AND LIBRARIES SEPTEMBER 2022 PERCEIVED QUALITY OF WHATSAPP REFERENCE SERVICE | GUO, LAM, CHIU, AND HO 10 staff and users should be considered. Secondly, the research evaluation design with three dimensions can be extended to measure other quality and effects. Thirdly, as WhatsApp is just one application among various emerging instant-messaging tools, further studies should cover other instant messaging platforms for similar and different purposes. For instance, as the studied university comprises a significant student population from Mainland China, WeChat could be investigated for its possibility and effectiveness as a WhatsApp alternative for providing reference services and promotion to Chinese students.73 INFORMATION TECHNOLOGY AND LIBRARIES SEPTEMBER 2022 PERCEIVED QUALITY OF WHATSAPP REFERENCE SERVICE | GUO, LAM, CHIU, AND HO 11 APPENDIX: KEY SURVEY ITEMS Item Mean SD. Affect of service (ease of use, supportive) (AS) (Cronbach’s Alpha  = 0.799) AS1. There is a clear introduction teaching library users about how to use the WhatsApp function. 3.18 0.87 AS2. The directories are easy to understand. 3.16 0.85 AS3. Reference service through WhatsApp is easy to use. 3.69 0.94 AS4. I can receive instant help from a librarian through WhatsApp. 3.55 0.78 AS5. I can request service anytime, anywhere. 3.31 0.93 Information control (response speed, accuracy, accessible) (IC) (Cronbach’s Alpha  = 0.707) IC1. Response of inquiry is reliable. 3.66 0.74 IC2. WhatsApp application makes reference services easily accessible for users. 3.25 0.95 IC3. Response of inquiry is accurate. 3.87 0.66 IC4. Using WhatsApp to gain access to reference services can meet my needs. 3.73 0.95 IC5. The quality of response obtained through WhatsApp is inferior to walk-in. (R). 2.60 1.26 IC6. The quality of response obtained through WhatsApp is inferior to email (R). 2.66 1.17 IC7. The quality of response obtained through WhatsApp is inferior to phone (R). 2.63 0.98 Library as place (staff attitude, encourage communication) (LP) (Cronbach’s Alpha  = 0.843) LP1. Reference staff is friendly or pleasant 4.08 0.76 LP2. Using WhatsApp to contact a librarian is convenient. 3.90 0.78 LP3. WhatsApp application in reference service increases my productivity in using online library services. 3.53 0.99 LP4. It provides an efficient channel to communicate with librarians. 3.87 0.89 LP5. I request more reference services after I know about the WhatsApp channel. 3.28 0.86 Note: IC5, IC6, and IC7 are reversed codes. INFORMATION TECHNOLOGY AND LIBRARIES SEPTEMBER 2022 PERCEIVED QUALITY OF WHATSAPP REFERENCE SERVICE | GUO, LAM, CHIU, AND HO 12 ENDNOTES 1 Karen Hiu Tung Yip, Patrick Lo, Kevin K. W. Ho, and Dickson K. W. Chiu, “Adoption of Mobile Library Apps as Learning Tools in Higher Education: A Tale between Hong Kong and Japan ,” Online Information Review 45, no. 2 (2020): 389–405, https://doi.org/10.1108/OIR-07-2020- 0287; Ken Yiu Kwan Fan, Patrick Lo, Kevin K. W. Ho, Stuart So, Dickson K. W. Chiu, and Eddie H. T. Ko, “Exploring the Mobile Learning Needs amongst Performing Arts Students,” Information Discovery and Delivery 48, no. 2 (2020), 103–12, https://doi.org/10.1108/IDD-12-2019-0085; Vanessa Hiu Ying Chan, Dickson K. W. Chiu, and Kevin K. W. Ho, “Mediating Effects on the Relationship between Perceived Service Quality and Public Library App Loyalty during the COVID-19 Era,” Journal of Retailing and Consumer Services 67 (2022): 102960, https://doi.org/10.1016/j.jretconser.2022.102960. 2 Samuel S. Green, “Personal Relations between Librarians and Readers,” Library Journal 1, no. 2 (1876): 74–81. 3 “Measuring and Assessing Reference Services and Resources: A Guide,” Reference and User Services Association, accessed July 25, 2021, http://www.ala.org/rusa/sections/rss/rsssection/rsscomm/evaluationofref/measrefguide. 4 Angel Lok Yi Tsang and Dickson K. W. Chiu, “Effectiveness of Virtual Reference Services in Academic Libraries: A Qualitative Study Based on the 5E Learning Model,” The Journal of Academic Librarianship 48, no. 4 (2022): 102533; Kun Zhang and Peixin Lu, “What Are the Key Indicators for Evaluating the Service Satisfaction of WeChat Official Accounts in Chinese Academic Libraries?,” Library Hi Tech, (2022), ahead-of-print, https://doi.org/10.1108/LHT- 07-2021-0218; Yifei Zhang, Patrick Lo, Stuart So, and Dickson K. W. Chiu, “Relating Library User Education to Business Students’ Information Needs and Learning Practices: A Comparative Study,” Reference Services Review 48, no. 4 (2020): 537–58, https://doi.org/10.1108/RSR-12-2019-0084. 5 Andrew Chean Yang Yew, Dickson K. W. Chiu, Yuriko Nakamura, and King Kwan Li, “A Quantitative Review of LIS Programs Accredited by ALA and CILIP under Contemporary Technology Advancement,” Library Hi Tech, (2022), ahead of print, https://doi.org/10.1108/LHT-12-2021-0442; James Friday, Oluchi Chidozie, and Lauretta Ngozi Chukwuma, “Social Media and Library Services: A Case of COVID-19 Pandemic Era,” International Journal of Research and Review 7, no. 10 (2020): 230–37, https://www.ijrrjournal.com/IJRR_Vol.7_Issue.10_Oct2020/Abstract_IJRR0031.html. 6 Ruth Sara Connell, Lisa C. Wallis, and David Comeaux, “The Impact of COVID-19 on the Use of Academic Library Resources,” Information Technology and Libraries 40, no. 2 (2021): 1–20, https://doi.org/10.6017/ital.v40i2.12629. 7 “Digital 2022: Global Overview Report,” We Are Social and Hootsuite, accessed April 30, 2022, https://wearesocial.com/hk/blog/2022/01/digital-2022/; Tsang and Chiu, “Effectiveness of Virtual Reference Services.” 8 “Digital 2022—Hong Kong,” We Are Social and Hootsuite, accessed April 30, 2022, https://wearesocial.com/hk/blog/2022/01/digital-2022/. https://doi.org/10.1108/OIR-07-2020-0287 https://doi.org/10.1108/OIR-07-2020-0287 https://doi.org/10.1108/IDD-12-2019-0085 https://doi.org/10.1016/j.jretconser.2022.102960 http://www.ala.org/rusa/sections/rss/rsssection/rsscomm/evaluationofref/measrefguide https://doi.org/10.1108/LHT-07-2021-0218 https://doi.org/10.1108/LHT-07-2021-0218 https://www.emerald.com/insight/search?q=Yifei%20Zhang https://www.emerald.com/insight/search?q=Patrick%20Lo https://www.emerald.com/insight/search?q=Stuart%20So https://www.emerald.com/insight/search?q=Dickson%20K.W.%20Chiu https://doi.org/10.1108/RSR-12-2019-0084 https://doi.org/10.1108/LHT-12-2021-0442 https://www.ijrrjournal.com/IJRR_Vol.7_Issue.10_Oct2020/Abstract_IJRR0031.html https://doi.org/10.6017/ital.v40i2.12629 https://wearesocial.com/hk/blog/2022/01/digital-2022/ https://wearesocial.com/hk/blog/2022/01/digital-2022/ INFORMATION TECHNOLOGY AND LIBRARIES SEPTEMBER 2022 PERCEIVED QUALITY OF WHATSAPP REFERENCE SERVICE | GUO, LAM, CHIU, AND HO 13 9 Tsang and Chiu, “Effectiveness of Virtual Reference Services.” 10 Tsang and Chiu, “Effectiveness of Virtual Reference Services.” 11 Green, “Personal Relations.” 12 Green, “Personal Relations.” 13 P. Sankar and E. S. Kavitha, “Ask Librarian to Whatsapp Librarian: Reengineering of Traditional Library Services,” International Journal of Information Sources and Services 3, no. 2 (March– April 2016): 35–40, https://www.researchgate.net/profile/Drkavitha- Es/publication/304466788_Ask_Librarian_to_Whatsapp_Librarian_Reengineering_of_Traditio nal_Library_Services/links/5770958c08ae10de639c0ca3/Ask-Librarian-to-Whatsapp- Librarian-Reengineering-of-Traditional-Library-Services.pdf; Spear Wing Sze Wong and Dickson K. W. Chiu, “Re-examining the Value of Remote Academic Library Storage in the Mobile Digital Age: A Comparative Study,” Portal 23, no. 1 (2023), in press; Tin Nok Leung, Dickson K. W. Chiu, Kevin K. W. Ho, and Canon K. L. Luk, “User Perceptions, Academic Library Usage and Social Capital: A Correlation Analysis under COVID-19 After Library Renovation,” Library Hi Tech 40, no. 2 (2021): 304–22, https://doi.org/10.1108/LHT-04-2021-0122. 14 Syeda Hina Batool, Ata ur Rehman, and Imran Sulehri, “The Current Situation of Information Literacy Education and Curriculum Design in Pakistan: A Discovery Using Delphi Method,” Library Hi Tech (2021): ahead of print, https://doi.org/10.1108/LHT-02-2021-0056; Yew et al., “Quantitative Review of LIS Programs.” 15 Tsang and Chiu, “Effectiveness of Virtual Reference Services”; Zhang and Lu, “What Are the Key Indicators.” 16 James Ogom Odu, and Emmanuel Ubi Omini, “Mobile Phone Applications and the Utilization of Library Services in the University of Calabar Library, Calabar, Nigeria,” Global Journal of Educational Research 16, no. 2 (2017): 111–19, https://doi.org/10.4314/gjedr.v16i2.5. 17 “Guidelines for Behavioral Performance of Reference and Information Service Providers,” American Library Association, June 2004, http://www.ala.org/Template.cfm?Section=Home&template=/ContentManagement/ContentD isplay.cfm&ContentID=26937. 18 Marianne Foley, “Instant Messaging Reference in an Academic Library: A Case Study,” College & Research Libraries 63, no. 1 (2002): 36–45, https://doi.org/10.5860/crl.63.1.36; Tsang and Chiu, “Effectiveness of Virtual Reference Services.” 19 Chun-wai Tsui, “A Study on Service Quality Gap in Remote Service Delivery with Mobile Devices among Academic Libraries in Hong Kong,” (master’s thesis, The University of Hong Kong, 2015), https://doi.org/10.5353/th_b5611574; Leung et al., “User Perceptions”; Zhang and Lu, “What Are the Key Indicators.” 20 Tsang and Chiu, “Effectiveness of Virtual Reference Services.” https://www.researchgate.net/profile/Drkavitha-Es/publication/304466788_Ask_Librarian_to_Whatsapp_Librarian_Reengineering_of_Traditional_Library_Services/links/5770958c08ae10de639c0ca3/Ask-Librarian-to-Whatsapp-Librarian-Reengineering-of-Traditional-Library-Services.pdf https://www.researchgate.net/profile/Drkavitha-Es/publication/304466788_Ask_Librarian_to_Whatsapp_Librarian_Reengineering_of_Traditional_Library_Services/links/5770958c08ae10de639c0ca3/Ask-Librarian-to-Whatsapp-Librarian-Reengineering-of-Traditional-Library-Services.pdf https://www.researchgate.net/profile/Drkavitha-Es/publication/304466788_Ask_Librarian_to_Whatsapp_Librarian_Reengineering_of_Traditional_Library_Services/links/5770958c08ae10de639c0ca3/Ask-Librarian-to-Whatsapp-Librarian-Reengineering-of-Traditional-Library-Services.pdf https://www.researchgate.net/profile/Drkavitha-Es/publication/304466788_Ask_Librarian_to_Whatsapp_Librarian_Reengineering_of_Traditional_Library_Services/links/5770958c08ae10de639c0ca3/Ask-Librarian-to-Whatsapp-Librarian-Reengineering-of-Traditional-Library-Services.pdf https://doi.org/10.1108/LHT-04-2021-0122 https://doi.org/10.1108/LHT-02-2021-0056 https://doi.org/10.4314/gjedr.v16i2.5 http://www.ala.org/Template.cfm?Section=Home&template=/ContentManagement/ContentDisplay.cfm&ContentID=26937 http://www.ala.org/Template.cfm?Section=Home&template=/ContentManagement/ContentDisplay.cfm&ContentID=26937 https://doi.org/10.5860/crl.63.1.36 https://doi.org/10.5353/th_b5611574 INFORMATION TECHNOLOGY AND LIBRARIES SEPTEMBER 2022 PERCEIVED QUALITY OF WHATSAPP REFERENCE SERVICE | GUO, LAM, CHIU, AND HO 14 21 Charlotte Clements, “Implementing Instant Messaging in Four University Libraries,” Library Hi Tech 27, no. 3 (2009): 393–402, https://doi.org/10.1108/07378830910988522. 22 Gunnan Dong et al., “Relationships between Research Supervisors and Students from Coursework-Based Master’s Degrees: Information Usage under Social Media,” Information Discovery and Delivery 49, no. 4 (2021): 319–27, https://doi.org/10.1108/IDD-08-2020-0100; Tsang and Chiu, “Effectiveness of Virtual Reference Services.” 23 “Number of Monthly Active WhatsApp Users Worldwide 2013–2020,” Statista Research Department, accessed July 26, 2021, https://www.statista.com/statistics/260819/number-of- monthly-active-whatsapp-users/. 24 “Most Popular Global Mobile Messaging Apps 2021,” Statista Research Department, accessed July 26, 2021, https://www.statista.com/statistics/258749/most-popular-global-mobile- messenger-apps/. 25 Mohd Shoaib Ansari and Aditya Tripathi, “Use of WhatsApp for Effective Delivery of Library and Information Services,” DESIDOC Journal of Library & Information Technology 37, no. 5 (2017): 360–65, https://doi.org/10.14429/djlit.37.5.11090; Y. Sudharani and K. Nagaraju, “WhatsApp Usage among the Students of SVU College of Engineering, Tirupathi,” Journal of Advances in Library and Information Science 5, no. 4 (2016): 325–29, https://jalis.in/pdf/5-4/Nagaraju.pdf. 26 Jianhua Xu, Qi Kang, Zhiqiang Song, and Christopher Peter Clarke, “Applications of Mobile Social Media: WeChat among Academic Libraries in China,” The Journal of Academic Librarianship 41, no. 1 (2015): 21–30, https://doi.org/10.1016/j.acalib.2014.10.012; Tsang and Chiu, “Effectiveness of Virtual Reference”; “Digital 2022—Hong Kong,” 54; Zhang and Lu, “What Are the Key Indicators.” 27 Heather Howard, Sarah Huber, Lisa Carter, and Elizabeth Moore, “Academic Libraries on Social Media: Finding the Students and the Information They Want,” Information Technology and Libraries 37, no. 1 (2018): 8–18, https://doi.org/10.6017/ital.v37i1.10160. 28 Odu and Omini, “Mobile Phone Applications.” 29 Ansari and Tripathi, “Use of WhatsApp”; Sudharani and Nagaraju, “WhatsApp Usage.” 30 Friday, Chidozie, and Chukwuma, “Social Media and Library Services.” 31 Adebowale Japhet Aina, Yemisi Tomilola Babalola, and Adebambo Adewale Oduwole, “Use of Web 2.0 Tools and Services by the Library Professionals in Lagos State Tertiary Institution Libraries: A Study,” World Digital Libraries – An International Journal 12, no.1 (2019): 1–17, https://content.iospress.com/articles/world-digital-libraries-an-international- journal/wdl12101. 32 Nor Azilawati Mohd Azmi, A. Noorhidawati, and M. K. Yanti Idaya Aspura, “Librarians’ Behavioral Performance on Chat Reference Service in Academic Libraries: Perceived Importance vs Actual Practices,” Malaysian Journal of Library & Information Science 22, no. 3 (2017): 19–33, https://doi.org/10.22452/mjlis.vol22no3.2. https://doi.org/10.1108/07378830910988522 https://doi.org/10.1108/IDD-08-2020-0100 https://www.statista.com/statistics/260819/number-of-monthly-active-whatsapp-users/ https://www.statista.com/statistics/260819/number-of-monthly-active-whatsapp-users/ https://www.statista.com/statistics/258749/most-popular-global-mobile-messenger-apps/ https://www.statista.com/statistics/258749/most-popular-global-mobile-messenger-apps/ https://doi.org/10.14429/djlit.37.5.11090 https://jalis.in/pdf/5-4/Nagaraju.pdf https://doi.org/10.1016/j.acalib.2014.10.012 https://doi.org/10.6017/ital.v37i1.10160 https://content.iospress.com/articles/world-digital-libraries-an-international-journal/wdl12101 https://content.iospress.com/articles/world-digital-libraries-an-international-journal/wdl12101 https://doi.org/10.22452/mjlis.vol22no3.2 INFORMATION TECHNOLOGY AND LIBRARIES SEPTEMBER 2022 PERCEIVED QUALITY OF WHATSAPP REFERENCE SERVICE | GUO, LAM, CHIU, AND HO 15 33 Aina, Babalola, and Oduwole, “Use of Web 2.0 Tools and Services”; Ansari and Tripathi, “Use of WhatsApp”; Friday, Chidozie, and Chukwuma, “Social Media and Library Services”; Odu and Omini, “Mobile Phone Applications”; Sudharani and Nagaraju, “WhatsApp Usage.” 34 Mohd Azmi, Noorhidawati, and Yanti Idaya Aspura, “Librarians’ Behavioral Performance.” 35 Tsui, “A Study on Service Quality Gap.” 36 “What is LibQUAL+®?,” LibQUAL+, accessed May 1, 2022, https://www.libqual.org/home; Tsui, “A Study on Service Quality Gap.” 37 Jessica Kayongo, and Sherri Jones, “Faculty Perception of Information Control Using LibQUAL+™ Indicators,” The Journal of Academic Librarianship 34, no. 2 (2008): 131, https://doi.org/10.1016/j.acalib.2007.12.002. 38 Rachael Kwai Fun Ip and Christian Wagner, “LibQual+® as a Predictor of Library Success: Extracting New Meaning through Structured Equation Modeling,” The Journal of Academic Librarianship 46, no. 2 (2020): 102102, https://doi.org/10.1016/j.acalib.2019.102102; Selena Killick, Anne van Weerden, and Fransje van Weerden, “Using LibQUAL+® to Identify Commonalities in Customer Satisfaction: The Secret to Success?.” Performance Measurement and Metrics 15, no. 1/2 (2014), 23–31, https://doi.org/10.1108/PMM-04-2014-0012. 39 Kayongo and Jones, “Faculty Perception of Information Control,” 131. 40 Ip and Wagner, “LibQual® as a Predictor”; Killick, van Weerden, and van Weerden, “Using LibQUAL® to Identify Commonalities.” 41 Kayongo and Jones, “Faculty Perception of Information Control,” 131. 42 Ip and Wagner, “LibQual® as a Predictor”; Killick, van Weerden, and van Weerden, “Using LibQUAL® to Identify Commonalities.” 43 Tsui, “A Study on Service Quality Gap.” 44 Anabel Quan–Haase, “Instant Messaging on Campus: Use and Integration in University Students' Everyday Communication,” The Information Society 24, no. 2 (2008): 105–15, https://doi.org/10.1080/01972240701883955. 45 Tsang and Chiu, “Effectiveness of Virtual Reference Services.” 46 Tsui, “A Study on Service Quality Gap.” 47 Robert A. Peterson, “A Meta-analysis of Cronbach's Coefficient Alpha,” Journal of Consumer Research 21, no. 2 (1994): 381–91, https://doi.org/10.1086/209405. 48 Ka Po Lau, Dickson K. W. Chiu, Kevin K. W. Ho, Patrick Lo, and Eric W. K. See-To, “Educational Usage of Mobile Devices: Differences Between Postgraduate and Undergraduate Students ,” The Journal of Academic Librarianship 43, no. 3 (2017): 201–8, https://doi.org/10.1016/j.acalib.2017.03.004. https://www.libqual.org/home https://doi.org/10.1016/j.acalib.2007.12.002 https://doi.org/10.1016/j.acalib.2019.102102 https://doi.org/10.1108/PMM-04-2014-0012 https://doi.org/10.1080/01972240701883955 https://doi.org/10.1086/209405 https://doi.org/10.1016/j.acalib.2017.03.004 INFORMATION TECHNOLOGY AND LIBRARIES SEPTEMBER 2022 PERCEIVED QUALITY OF WHATSAPP REFERENCE SERVICE | GUO, LAM, CHIU, AND HO 16 49 Tsang and Chiu, “Effectiveness of Virtual Reference Services.” 50 Aina, Babalola, and Oduwole, “Use of Web 2.0 Tools and Services.” 51 Aina, Babalola, and Oduwole, “Use of Web 2.0 Tools and Services.” 52 Leung et al., “User Perceptions”; Zhang et al., “Relating Library User Education.” 53 Maggie Ka Yin Chan, Dickson K. W. Chiu, and Ernest Tak Hei Lam, “Effectiveness of Overnight Learning Commons: A Comparative Study,” The Journal of Academic Librarianship 46, no. 6 (2020): 102253, https://doi.org/10.1016/j.acalib.2020.102253; Tsang and Chiu, “Effectiveness of Virtual Reference Services.” 54 Wesley Wing Hong Cheng, Ernest Tak Hei Lam, and Dickson K. W. Chiu, “Social Media as a Platform in Academic Library Marketing: A Comparative Study,” The Journal of Academic Librarianship 46, no. 5 (2020): 102188, https://doi.org/10.1016/j.acalib.2020.102188. 55 Parker Fruehan and Diana Hellyar, “Expanding and Improving Our Library's Virtual Chat Service: Discovering Best Practices When Demand Increases,” Information Technology and Libraries 40, no. 3 (2021): 1–9, https://doi.org/10.6017/ital.v40i3.13117; Pui Yik Yu, Ernest Tak Hei Lam, and Dickson K. W. Chiu, “Operation Management of Academic Libraries in Hong Kong under COVID-19,” Library Hi Tech, (2022), ahead of print, https://doi.org/10.1108/LHT- 10-2021-0342. 56 Friday, Chidozie, and Chukwuma, “Social Media and Library Services.” 57 Mohd Azmi, Noorhidawati, and Yanti Idaya Aspura, “Librarians’ Behavioral Performance.” 58 Aina, Babalola, and Oduwole, “Use of Web 2.0 Tools and Services”; Friday, Chidozie, and Chukwuma, “Social Media and Library Services.” 59 Yu, Lam, Chiu, “Operation Management” 60 Tsang and Chiu, “Effectiveness of Virtual Reference Services.” 61 Kirsti Nilsen, “The Library Visit Study: User Experiences at the Virtual Reference Desk,” Information Research 9, no. 2 (2004), paper 171, http://informationr.net/ir/9- 2/paper171.html. 62 Mohd Azmi, Noorhidawati, and Yanti Idaya Aspura, “Librarians’ Behavioral Performance.” 63 Tsang and Chiu, “Effectiveness of Virtual Reference Services.” 64 Mohd Azmi, Noorhidawati, and Yanti Idaya Aspura, “Librarians’ Behavioral Performance.” 65 Tsang and Chiu, “Effectiveness of Virtual Reference Services.” 66 Dessy Harisanty et al., “Leaders, Practitioners and Scientists’ Awareness of Artificial Intelligence in Libraries: A Pilot Study,” Library Hi Tech, (2022), ahead of print, https://doi.org/10.1108/LHT- 10-2021-0356. https://doi.org/10.1016/j.acalib.2020.102253 https://doi.org/10.1016/j.acalib.2020.102188 https://doi.org/10.6017/ital.v40i3.13117 https://doi.org/10.1108/LHT-10-2021-0342 https://doi.org/10.1108/LHT-10-2021-0342 http://informationr.net/ir/9-2/paper171.html http://informationr.net/ir/9-2/paper171.html https://doi.org/10.1108/LHT-10-2021-0356 https://doi.org/10.1108/LHT-10-2021-0356 INFORMATION TECHNOLOGY AND LIBRARIES SEPTEMBER 2022 PERCEIVED QUALITY OF WHATSAPP REFERENCE SERVICE | GUO, LAM, CHIU, AND HO 17 67 Ansari and Tripathi, “Use of WhatsApp”; Sudharani and Nagaraju, “WhatsApp Usage.” 68 Leung et al., “User Perceptions”; Tsang and Chiu, “Effectiveness of Virtual Reference Services.” 69 Ernest Tak Hei Lam, Cheuk Hang Au, and Dickson K. W. Chiu, “Analyzing the Use of Facebook among University Libraries in Hong Kong,” The Journal of Academic Librarianship 45, no. 3 (2019): 175–83, https://doi.org/10.1016/j.acalib.2019.02.007; Foley, “Instant Messaging Reference”; Tsang and Chiu, “Effectiveness of Virtual Reference Services.” 70 Lam, Au, and Chiu, “Analyzing the Use of Facebook”; Tsang and Chiu, “Effectiveness of Virtual Reference Services.” 71 Cheng, Lam, and Chiu, “Social Media as a Platform.” 72 Tsang and Chiu, “Effectiveness of Virtual Reference Services.” 73 Apple Hiu Ching Lam, Kevin K. W. Ho, and Dickson K. W. Chiu, “Instagram for Student Learning and Library Promotions? A Quantitative Study using the 5E Instructional Model,” Aslib Journal of Information Management, (2022), in press, https://doi.org/10.1108/AJIM-12-2021-0389. https://doi.org/10.1016/j.acalib.2019.02.007 https://doi.org/10.1108/AJIM-12-2021-0389 Abstract Introduction Literature Review Research Purpose Methodology Data Analysis Demographic Information Usage of Reference Service Users’ Satisfaction with WhatsApp Reference Service Discussions and Recommendations Subdivision of the WhatsApp Reference Service into Specialist Subjects More Staff Training Establishment of WhatsApp Reference Service Guidelines Promotion for the WhatsApp Service Conclusion Limitations and Future Research Appendix: Key Survey Items Endnotes