Gamified Online Quizzes: Pharmacy Student Perceptions of Learning in an Undergraduate Medicinal Chemistry Course

Abstract

Introduction: In the context of pharmacy education worldwide and in Malaysia, the use of digital technologies to promote higher level thinking and discussions is seen as preparing the millennials as pharmacists in the 21^st century. Together with leveraging on millennials’ penchant for mobile technology, gamified online quizzes as an assessment tool that help promote active and collaborative learning in a Medicinal Chemistry course have been used. Objectives: This study investigates students’ perception of the impact of gamified online quizzes on their learning in a Medicinal Chemistry course. Method: This study employs mix method research comprising descriptive analysis, content analysis from informal chats and researchers’ observation to gather the findings for the study. Three gamified online quizzes using Quizizz, were implemented outside classroom time, in place of traditional quizzes. Multiple attempts were allowed within a stipulated time. As interventions, post-quiz discussions were conducted during class time. Students completed an end-of-the-course survey. Results: Out of 63 respondents, more than 96% felt that the gamified online quizzes enhanced their learning as they learned from the instant feedback, their mistakes and post-quiz discussions. Overall student performance based on the percentage and accuracy of answering the quiz improved with time. Student qualitative comments on the survey, the course social media (closed group) and informal chats supported the findings from the descriptive data analysis of the study. Conclusions: From students’ perception, the gamified online quizzes were found to be enjoyable and effective in enhancing active, peer learning in an undergraduate medicinal chemistry course outside class time. For instructors, the online quiz served as an efficient tool for formative assessment in a large classroom setting, and could replace traditional classroom quizzes.

Introduction

The use of online and in-classroom digital games and game-based approaches to promote student engagement via active and collaborative learning has gained prominence [1][2][3]. It is of particular relevance during this challenging Covid-19 times when most face-to-face teaching and learning activities moved to online learning–prompting educators to seek approaches that facilitate and increase students’ online engagement [4][5].

Game-based learning provides learners: (a) the environment to take risks, (b) the chance to make mistakes and learn from these mistakes in a low-stake but competitive environment, (c) the opportunities to keep trying and (d) the avenue to be rewarded for successful attempts. These are similar to how people learn to master certain skills or acquire new knowledge in life.

Besides that, game-based learning employs elements such as point systems, scoreboards, winners and eye-catching avatars coupled with exciting music and colourful, user-friendly interfaces that keep learners motivated and engaged in a non-game context [6][7], for instance in learning abstract concepts or dry subjects [8][9][10][11].

In the context of pharmacy education worldwide and in Malaysia, the use of digital technologies to promote higher level thinking and discussion is seen as preparing the millennials as pharmacists in the 21^st century–where skilful communication, collaboration and critical thinking are essential in various pharmacy practices [8][12][13][14][15][16][17][18][19][20][21].

Gamified web-based quizzes e.g. Kahoot, Quizizz and Socrative, are increasingly being used as a pedagogical strategy to conduct classroom teaching and assessment [7]. Despite doubts of its pedagogical effectiveness [22][23], their appeals to young students lie in engaging and motivating learners in game-based, digital learning experiences [18][24].

In previous years, students taking the Principles of Medicinal Chemistry course at the School of Pharmaceutical Sciences, Universiti Sains Malaysia (USM) were assessed in a number of in-class assignments and pop-quizzes. These formed the formative assessment part of the course. Anecdotally, this subject was found difficult amongst many students. To help their learning on the subject, students requested for regular pop-quizzes. In doing so, however, the instructors found that creating questions, marking such formative assessments and providing constructive feedback were time-consuming, particularly for a large class.

Therefore, this study is aimed to seek for a form of formative assessment that would help instructors to securely conduct the assessment, allows efficient grading, analyse student responses and deliver learning analytics. Additionally, this study aimed to evaluate a digital tool that leverages on digital natives’ penchant for mobile technology.

The popularity of various game-based instructions for active learning in pharmacy classroom settings has been reported previously [12][16][19][21][25][26][27][28]; however, its use outside of class time remains relatively unexplored in traditional higher education setting. The use of gamified online quiz by effectively utilising the notional student learning time (SLT) could serve as a novel way towards effective learning. This preliminary study evaluates pharmacy students’ perceptions of learning using a gamified online quiz approach (ie. using Quizizz) with a view of replacing traditional in-class quizzes.

Method

The Principles of Medicinal Chemistry course is a 2-unit course for second year pharmacy students at Universiti Sains Malaysia. It consists of topics on drug design and development that includes structure-activity relationship (SAR), quantitative SAR (QSAR), drug modelling and pharmacokinetics. The course is taught by two instructors to a large group of 116 students. Three unsupervised online quizzes were offered during the semesters, i.e. students were assessed in Week 7, 11 and 14. Each quiz was conducted about 3 weeks apart. Students completed these quizzes as a part of continuous assessment (10%) in this course. The online quiz consisted of 20-30 multiple-choice, randomised questions and answers with progressive difficulties, which would evaluate students’ six cognitive levels based on the Bloom’s Taxonomy, i.e. knowledge, comprehension, application, analysis, synthesis and evaluation. Using images and diagrams, instructors were able to construct more challenging questions by prompting students to critically examine the visuals (e.g. drug-protein interactions, SAR, molecular modelling in Figure 1), thus testing students’ critical thinking and understanding in medicinal chemistry.

The quizzes were implemented based on students’ preferences that were collected using Google Form. Students were made aware of the quizzes through the Facebook closed group of the medicinal chemistry course. Students were alerted to six rules for online quizzes–one such rule requires students to use assigned name codes to keep their anonymity secured in the cyber world and later, for grading purposes (Table 1).

Simple technical support for students (e.g. game pin, access, name repeat) was provided by the instructors in the evenings of the quiz, between 8-10 pm. Instructors were also present online to monitor the running of the online quiz. Student performance reports were downloaded every 15-20 minutes, and at the conclusion of the online quiz. The reports are colour-coded as shown in Figure 2.

6 Rules of Online Quiz

1. Please make sure you are connected to a stable wi-fi, uni-fi or cable internet. The quiz is web-based. You may use mobile devices or laptops.
2. Please use the assigned code names. ANY names different from the assigned code names will be removed. No marks given.
3. DO NOT SHARE the quiz code or quiz questions with anyone or your juniors/seniors.
4. You may repeat the quiz as multiple times–within the allocated time.
5. Your quiz marks will be based on your best performance.
6. The content of the quiz is copyrighted under the instructor. No screenshot or any audio-visual recording is allowed without prior permission.

* The assigned code name will be crucial during the grading process.

This makes it visually easy for instructors to quickly check and plan for student feedback. Post-quiz discussions based on the Quizizz reports were held during the next available lecture hour. The reports were also used for grading purposes.

Criteria and considerations in designing a gamified online quiz

a) Which gamified online quiz would be suitable for a large class?

As mentioned earlier, there are several free, gamified online student response systems available e.g. Kahoot, Socrative and Quizizz. For this initial study, the criteria used for selecting the online quiz are: 1) it is a free digital tool, and remains free for a large number of students (over 100); 2) student- and instructor-friendly (e.g. low learning curve); 3) has built-in elements of gamification; 4) able to generate analytics; 5) set a deadline for students; 6) stable, responsive and reliable during real-time uses for a large-sized class and 7) able to be used on laptops and mobile devices. Based on these criteria, Quizizz was selected as the gamified online tool as a part of the continuous assessment.

b) How many attempts for an online quiz? How long should it be?

At the outset of this online quiz, the instructors had little idea what constitutes a realistic setting for students, in terms of quiz duration and number of attempts. Unlike the traditional pen-and-paper quiz, the instructors were mindful of the time learners need to familiarise themselves with a new technology. Allowing for extra time reduces anxiety in students, a key factor in facilitating learning [28]. Allowing for extra time also provides students time to rectify technical issues (e.g. internet access, browser) possibly encountered, often at the start, and during the online quiz.

Furthermore, the main purpose of offering the gamified online quiz is to help students to learn from their mistakes and that of their peers. Taking these factors into considerations, the first online quiz was opened for 120 minutes. Students could repeat taking the quiz as many times as they wish–within that period. At the end of the 2-hour period, the online dashboard showed a staggering 834 players with an overall 79% accuracy. Considering that there were 116 students registered for the quiz, each student probably attempted close to 7.2 times. The top 10% players took, on average, 1 minute and 25 seconds to answer each quiz question and rose to the top of the scorecard with 100% accuracy. Due to the huge number of attempts, the first instructor discovered that the overall analytics on student performance could not be processed and downloaded from Quizizz website.

The second quiz was administered about 3 weeks later by the second instructor. Initially, the instructor planned for a traditional in-class quiz; but upon overwhelmingly positive student feedback for the first quiz and requests for a second online quiz, the second quiz was also held using Quizizz. The instructors had a discussion to rectify and improve the first setting. Based on our discussion, students were allowed a maximum of three attempts within a 60-minute period. With the duration and number of attempts capped, the second instructor managed to download the final report.

For the third quiz, students requested to increase the number of attempts. Since the purpose of offering the online quiz is “assessment for learning” rather than “assessment of learning”, the first instructor allowed more repeats than the previous quiz. Assessment for learning refers to formative assessments that are focused on providing feedback for improvements in students’ learning, whereas assessment of learning refers to summative evaluations at the end of a course [29]. Therefore, in the third quiz, the maximum number of attempts were capped at 5 times for a 60-minute quiz.

Data Collection

After the final quiz, the students were asked to complete an online questionnaire on a voluntary basis–no rewards were offered for completing the survey. They were asked to provide only their first name and gender. The purpose of the student survey was to gauge the student perceptions on learning via gamified online quizzes. The questionnaire had a section of 14 questions to find out to what extent gamification features, types of questions, flexibility, post-quiz discussions influenced their learning (see Table 2).

The survey used a five-point Likert scale for each item (1=strongly disagree, 2= disagree, 3=neutral, 4=agree, and 5=strongly agree). At the end of the questionnaire, there were blank spaces for students to write further comments and suggest improvements. Informal chats together with students’ feedback on the course Facebook group were taken into account in this study. The data from students’ comments, suggestions and informal chats were analysed and used to support the descriptive data from the survey.

Results

Out of 116 students, 63 students responded to the survey. Forty-four respondents (69.8%) were female students and 19 (30.2%) were male. The results in Table 2 showed that over 95% thought the online quizzes were more fun, enhanced their learning and were more effective than the traditional in-class quiz. Interestingly, 95.9% of respondents believed that online quizzes in the evenings helped to enhance their learning, effectively uses the notional student learning time (SLT). About 98% indicated that taking the online quizzes made it easy for them to remember concepts and principles in medicinal chemistry. Similarly, 98.3% felt that their learning was further enhanced when post-quiz discussions were held. Because the online quizzes were conducted in the evenings, the limited daytime lecture slots were not compromised; instead they were used for post-quiz discussions. Somewhat 39.6% agreed or strongly agreed that the ‘anytime, anywhere’ flexibility of taking the online quiz did not help their learning.

The web-based quiz used in this course, Quizizz, employs game elements e.g. points, live ranking and scoreboard, memes and funny quotes to inject fun and motivate students by rewarding them based on the speed and accuracy of their answers. Two-thirds of the respondents disagreed that memes and funny quotes cause distractions. When queried about the gamification features, specifically on the duration set per questions, varying between 5-30 seconds, about 56.1% of respondents felt that the fixed duration contributed towards enhancing their learning; whereas 70.7% thought speed and accuracy did help their learning. Additionally, 68.7% of the respondents indicated that having the scoreboard helped their learning.

Studies have shown that game mechanics e.g. scoreboards, rewards and rankings encourage engagement in learners and provide social comparisons, thus may influence students’ motivation and performance [30]. Regarding having the quiz in multiple choice questions (MCQ), 43.6% felt it was easy, but the majority thought that having a variety of questions of different difficulties (easy, medium and hard) helped to enhance learning in the course. Overall, the majority of students prefer gamified online quizzes to traditional in-class quiz.

Discussion

The value of game-based quiz for learning has been well-investigated recently in higher education settings [2][31][32][33]. Traditional in-class quizzes, closed or open-book, have been routinely used as a summative assessment. Instructors may or may not discuss the quiz questions. On the other hand, an open-book quiz seems to reduce anxiety, puts less weight on memorisation, encourages deeper engagement with the course materials and is more realistic as it mimics the real-life working environment [33]. Taking a step further, when learning, formative assessment and game elements are combined, it could potentially enhance student engagement in the course, encouraging learning without threatening esteem [30][34]. To this end, many interactive response systems or similar have been shown to promote active learning and peer instruction in lectures [32][35][36][37][38].

The survey results of student perceptions are further supported by voluntary written student feedback using the same survey and on the closed Facebook group of the course. Table 3 lists the total number of positive comments (10 comments) which outnumbered the negative aspects (1 comment).

Among the positive comments, many mentioned “having fun during gamified online quizzes”; discussing and learning from their friends; felt that they could remember better; get quick feedback and learn from their mistakes. Gamified quiz reframes failure as an essential part of the learning process, thus, promoting resilience in learners [23]. Some students also suggested this online quiz be implemented in other courses, while expressing concerns for not having enough time to learn and improve their scores in a 1-hour quiz.

---

Table 3: Comments from students from the survey and Facebook

Positive comments of the gamified online quizzes were

• New way of learning was exciting!
• I wish every quiz is conducted this way.
• This approach makes me remember better.
• This is my first time in life having fun while answering quizzes.
• Good as in online quiz is done with flexibility and can be discussed with friends.
• We can learn in a fun and relaxing way via online quizzes. Do it for all subjects.
• I really love this online quiz! Especially when we can learn from our mistakes and correct them on the spot. I’m looking forward to the next quiz!
• I really have fun doing this quiz! This helps me remember better–l love the avatars.
• This quiz was very fun doing in a group. Really learnt a lot from our own mistakes. Thanks for this fun and worthy online quiz! Looking forward to the next online quiz!
• Online quiz is much better than traditional quiz. I tend to learn from my mistakes. Unlike traditional ones, we have the chance to attempt more than once and therefore, learning from our mistakes.

Negative comments of the gamified online quizzes were

• Just a suggestion regarding the duration of quiz, I think it should be extended to 1.5-2 hours so that students have more time to think and choose the right answer to each question.

---

Informal interviews with students

To gain an insight into how the students took the gamified online quizzes, informal interviews were conducted with several students. The students revealed that they worked in groups. The group size increased from small (3-4 students) to large (9-10 students) as the quiz progressed. They further revealed that before the start of the quiz, they had all the study materials (books, lecture notes, mobile phones and tablets–at hand) ready. When the quiz began, they would attempt the questions as individuals. If they were unable to answer the questions, they paused and checked with their group mates, read the materials and scanned the internet for answers. Since time is the essence, if they still could not find an answer they tend to continue without submitting an answer. Such response is allowed in Quizizz. These were most apparent in the first half hour of the quiz as reported by the analytics.

As the quiz progressed, the first instructor learned that the instant feedback was helpful to student in checking their conceptions. Once the correct or wrong answers were noted down, discussions ensued with them continuing to learn from their own mistakes and others. The anecdotes support the notion that gamified quiz promotes informal, peer learning outside classroom during the notional SLT.

Post-quiz discussions

Post-quiz discussions are an effective form of intervention; these were made during lecture hours. The discussions were based on the analytics generated by Quizizz, which provides a downloadable colour-coded Excel table displaying answers in green (correct) and red (wrong) as shown in Figure 2.

The report helps instructors to quickly pick the questions that needed further clarification and corrections during the post-discussions. The analytics can also be used as starting points for deeper discussions during the lecture hours. Informal chats with students revealed that this form of intervention is rarely conducted in the traditional quiz and large classroom settings, where formative assessment tend to be overlooked [39][40]. It is, therefore, no surprise that 98% of respondents felt the post-quiz discussions enhanced their learning (survey item 12).

Limitations of the study

The online quiz has its limitations; it invites the possibility of cheating, particularly the identity of the participants. Cheating can be minimised by assigning unique student names. Nonetheless, it could have been answered by the same student using 2-3 assigned names, or even by a random person. Having supervised online quizzes could have prevented cheating. Under another course at the School, a similar online quiz using Elearn@USM, the university’s Moodle-based learning management system, had been administered at a computer lab in the School. Since the capacity of the computer lab is limited to about 50-60 students, the online quiz had to be conducted in two consecutive sessions. When one group of students was taking the quiz, the other group was being quarantined until the first group finished. Even though cheating was prevented in this case, conducting such quiz placed greater burden on staffing, facilities and timetabling.

Another limitation for such online quiz is stable internet connectivity, which is stated as one of the rules in the Table 1. Students are able to access stable Wi-Fi or cable internet provided by the university, on-campus or at student accommodation; though, at times overloaded servers may affect the internet connectivity. In one instance, a student reported that she was unable to log into the online quiz using her laptop browsers. She had to switch to her phone using own data plan to attempt the quiz. She finished her third attempt past the deadline. Allowance for time provides a space for any technical difficulties and helps increase student familiarity with technology. This incident highlights the importance of time, which should be included as a part of the design of an online quiz or any assessment using a digital tool and platform.

Conclusion

The objective of this initial study was to find out the students’ perceptions on the employment of gamified online quizzes in a medicinal chemistry course as an alternative to traditional pen-and-paper quiz. Overall students’ perceptions towards the employment of online quizzes were extremely positive. For instructors, it provides an efficient way to conduct formative assessments throughout the course.

Additionally, the gamified quiz environment promotes informal, active and collaborative learning outside lecture hours. Students strongly indicated that receiving instant feedback, learning from mistakes and post-quiz discussions are three key factors that enhanced their learning. They also recommended the adoption of the gamified online quiz in other courses in the pharmacy curriculum and could serve as an alternative to traditional quiz. A comparison between gamified and non-gamified online quizzes would be explored in the future.

Confronted with multiple lockdowns and remote teaching during the Covid-19 pandemic, adoption of alternative forms of assessment in higher education settings using digital tools and platforms are inevitable and on-going. With careful planning, design and selection of digital tools, gamified online quizzes can promote and sustain active and collaborative learning for deeper engagement and social resilience in 21^st century pharmacy education.

Acknowledgement

The authors wish to thank all students who had participated in the survey and the online quizzes.

Conflict of Interest

The authors declare no conflict of interest. This research did not receive any specific grant from funding agencies in the public, commercial or not-for-profit sectors.

References

1. Koutromanos G, Avraamidou L. The use of mobile games in formal and informal learning environments: A review of the literature. EMI Educ Media Int. 2014;51(1).
2. Salas-Morera L, Arauzo-Azofra A, García-Hernández L. Analysis of online quizzes as a teaching and assessment tool. J Technol Sci Educ. 2012;2(1).
3. Daubenfeld T, Zenker D. A game-based approach to an entire physical chemistry course. J Chem Educ. 2015;92(2).
4. Kandri S-E. How COVID-19 is driving a long-overdue revolution in education. World Econ Forum. 2020;
5. Hill K, Fitzgerald R. Student Perspectives on the Impact of COVID-19 on Learning. Aishe-J. 2020;12(2):1–10.
6. Deterding S, Dixon D, Khaled R, et al. From game design elements to gamefulness: Defining “gamification.” In: Proceedings of the 15th International Academic MindTrek Conference: Envisioning Future Media Environments, MindTrek 2011. 2011.
7. Wang AI, Zhu M, Sætre R. The effect of digitizing and gamifying quizzing in classrooms. In: Proceedings of the European Conference on Games-based Learning. 2016.
8. Aburahma MH, Mohamed HM. Educational games as a teaching tool in pharmacy curriculum. Am J Pharm Educ. 2015;79(4).
9. Akl EA, Pretorius RW, Sackett K, et al. The effect of educational games on medical students’ learning outcomes: A systematic review: BEME Guide No 14. Vol. 32, Medical Teacher. 2010.
10. All A, Nuñez Castellar EP, Van Looy J. Assessing the effectiveness of digital game-based learning: Best practices. Comput Educ. 2016;92–93.
11. Hamari J, Shernoff DJ, Rowe E, et al. Challenging games help students learn: An empirical study on engagement, flow and immersion in game-based learning. Comput Human Behav. 2016;54.
12. Sera L, Wheeler E. Game on: The gamification of the pharmacy classroom. Curr Pharm Teach Learn. 2017;9(1).
13. Stewart DW, Brown SD, Clavier CW, et al. Active-learning processes used in us pharmacy education. Am J Pharm Educ. 2011;75(4).
14. Evans S, Lombardo M, Belgeri M, et al. The geriatric medication game in pharmacy education. Am J Pharm Educ. 2005;69(3).
15. Chavez B, Gilliam EH, Pathak R, et al. Popular game shows as educational tools in the pharmacy classroom. Curr Pharm Teach Learn. 2012;4(2).
16. Barclay SM, Jeffres MN, Bhakta R. Educational card games to teach pharmacotherapeutics in an advanced pharmacy practice experience. Am J Pharm Educ. 2011;75(2).
17. Cain J, Piascik P. Are serious games a good strategy for pharmacy education? Am J Pharm Educ. 2015;79(4).
18. Wang J, Hu X, Hu X, et al. Chinese pharmacy undergraduate attitudes toward game-based learning in pharmaceutical education. Indian J Pharm Educ Res. 2013;47(3).
19. Testman JA. The prescribing cascade game: Applying geriatric pharmacotherapy principles in the classroom. Curr Pharm Teach Learn. 2014;6(5).
20. Shah S, Lynch LMJ, Macias-Moriarity LZ. Crossword puzzles as a tool to enhance learning about anti-ulcer agents. Am J Pharm Educ. 2010;74(7).
21. Grady SE, Vest KM, Todd TJ. Student attitudes toward the use of games to promote learning in the large classroom setting. Curr Pharm Teach Learn. 2013;5(4).
22. Blakely G, Skirton H, Cooper S, et al. Educational gaming in the health sciences: Systematic review. J Adv Nurs. 2009;65(2).
23. Lee JJCU, Hammer JCU. Gamification in Education: What, How, Why Bother? Acad Exch Q. 2011;15(2).
24. Mangold K. Educating a new generation: Teaching baby boomer faculty about millennial students. Nurse Educ. 2007;32(1).
25. Rose TM. A board game to assist pharmacy students in learning metabolic pathways. Am J Pharm Educ. 2011;75(9).
26. Bindoff I, Ling T, Bereznicki L, et al. A Computer Simulation of Community Pharmacy Practice for Educational Use. Am J Pharm Educ. 2014;78(9).
27. Pettit RK, McCoy L, Kinney M, et al. Student perceptions of gamified audience response system interactions in large group lectures and via lecture capture technology Approaches to teaching and learning. BMC Med Educ. 2015;15(1).
28. Persky AM, Stegall-Zanation J, Dupuis RE. Students perceptions of the incorporation of games into classroom instruction for basic and clinical pharmacokinetics. Am J Pharm Educ. 2007;71(2).
29. Peeters MJ. Targeting assessment for learning within pharmacy education. Am J Pharm Educ. 2017;81(8).
30. Hanus MD, Fox J. Assessing the effects of gamification in the classroom: A longitudinal study on intrinsic motivation, social comparison, satisfaction, effort, and academic performance. Comput Educ. 2015;80.
31. Kibble J. Use of unsupervised online quizzes as formative assessment in a medical physiology course: Effects of incentives on student participation and performance. Am J Physiol – Adv Physiol Educ. 2007;31(3).
32. Wijtmans M, Van Rens L, Van Muijlwijk-Koezen JE. Activating students’ interest and participation in lectures and practical courses using their electronic devices. J Chem Educ. 2014;91(11).
33. Afshin Gharib, William Phillips, Noelle Mathew. Cheat Sheet or Open-Book? A Comparison of the Effects of Exam Types on Performance, Retention, and Anxiety. J Psychol Res. 2012;2(8).
34. Howard-Jones P, Holmes W, Demetriou S, et al. Neuroeducational research in the design and use of a learning technology. Learn Media Technol. 2015;40(2).
35. Terrion JL, Aceti V. Perceptions of the effects of clicker technology on student learning and engagement: A study of freshmen Chemistry students. Res Learn Technol. 2012;20(2).
36. Schell J, Lukoff B, Mazur E. Catalyzing learner engagement using cutting-edge classroom response systems in higher education. Cutting-Edge Technol High Educ. 2013;6(PARTE).
37. Jann E. Freed. Teaching with Classroom Response Systems: Creating Active Learning Environments (review). Rev High Educ. 2009;33(2).
38. Schultz M. Use of an on-line student response system: An analysis of adoption and continuance. Int J Innov Sci Math Educ. 2013;21(5).
39. Duschl RA, Gitomer DH. Strategies and challenges to changing the focus of assessment and instruction in science classrooms. Int J Phytoremediation. 1997;21(1).
40. C. Jacoby J, Heugh S, Bax C, et al. Enhancing learning through formative assessment. Innov Educ Teach Int. 2014;51(1).