Online assessments during COVID-19 pandemic – A paradigm shift in educational strategy

Objective: Covid-19 pandemic has instigated changes in the teaching and assessment practices of institutions in many parts of the world, resulting in not only online teaching but also online assessments. The present study analyzes the perceptions of students regarding the first online assessments conducted at a medical college in Pakistan. Materials & Methods: This Mixed method cross-sectional study was conducted at Wah Medical College (WMC) from July to November 2020. The questionnaire was developed after the collaboration of the authors and a review of the pertinent literature on online examinations. Universal sampling was done and all students of WMC were invited to participate in the study. Data collection was done by google forms and analysis was performed on SPSS. Results: Students from all five years of the MBBS program participated almost equally in the study. Students of 1st (43.8 %), 3rd (44.8 %), and 4th year (47.1 %) were not contented with their performance in online exams whereas students of 2nd (49.3 %) and 5th year (61.4 %) were satisfied. Most students considered technological issues as a basis for their poor performance and proposed technical training before online exams. They also provided suggestions related to the improvement of examination formats and tools. Conclusion: The majority of the students found the online assessment more effective and helpful in improving their performance in exams. Student perceptions provided valuable insights for improving the online examination system.


Introduction
Assessment plays an important role in the educational process of a medical student as it not only reflects on the student's progress but also affects the learning strategies adopted by the students. 1 Undergraduate medical education today, utilizes a variety of tools and strategies to ensure that the healthcare graduates of tomorrow are satisfactorily competent in the concerned program outcomes. 2,3 These tools and strategies may be employed both formatively and summatively. 4 Previously, there was a lot of reliance on summative assessment for grades and promotion of students, but now weightage has shifted towards formative assessment as along with improving the learning of the students 4 , it has a positive effect on summative assessment. 5 The Covid-19 Pandemic has created a lasting impact on educational delivery and evaluation. In many parts of the world, Higher Education Institutes (HEI) have shifted their teaching activities to online mode [6]. In Pakistan also, the Higher Education Commission (HEC) has provided guidelines for online teaching and learning to all HEIs. 7 With the increased use of online teaching in medical education, online assessment has emerged as an important form of formative assessment. This results in instant feedback to the students thus helping them to identify the educational gaps. It has been shown that if planned correctly, online assessment can influence the learning strategies of the students and can result in the accomplishment of advanced levels of knowledge and educational goals. 8 Online assessment is also an advantageous option for teachers to help them continuously monitor the progress of the students and to create content for the exam in a short time. 9 However, the students who are the end users of these online assessment formats were not asked about their perceptions and experiences. Understanding the experiences and perceptions of students would help to identify how the process of online assessment could be made more palatable and useful for the students Like various other HEIs, Wah Medical College (WMC), also started online classes for MBBS students in March 2020. Students in different years of the MBBS program were taught in both synchronous and asynchronous formats for almost three months. The teaching strategies were evaluated by conducting online assessments. All students of WMC attempted their first online assessments on cloud-based quiz development software 10 , which was chosen due to its user-friendliness and compatibility. The assessment tool used for these evaluations was based on one-best type multiple-choice questions. 11 This present study aims to analyze student perceptions about the first online assessments conducted at WMC and understand the impact on student learning practices

Materials and Methods
This mixed method, cross-sectional study was conducted in Wah Medical College, NUMS from July to November 2020. Ethical approval for the study was obtained from the Ethical review board of WMC (ERC/IRB/0015). The questionnaire was developed after the collaboration of the authors and a review of the pertinent literature on online examinations. It focused on two main aspects. The first part was related to the administrative aspect and focused on instructions for attempting the examination, examination time, software platform, and internet connectivity. The second part of the questionnaire focused on academic aspects. It required the students to explain their perceptions regarding their performance in examinations, the examination content, and their level of understanding, among others. WMC has a total of 515 students studying across five years of the MBBS program. With the initiation of online teaching, it was made mandatory for all students to appear in online assessments. Students were provided training before the exams to familiarize them with the assessment procedure and software platform. Immediately after exam conduction, students were asked to provide feedback to understand their perceptions and incorporate their suggestions for further assessments. A universal sampling technique was employed and all students who provided their informed consent were included in the study. Those who did not mark the informed consent were excluded. The questionnaire was developed on google forms and shared via email and WhatsApp groups with all students. Data Analysis: Data was analyzed using statistical software SPSS version 20. Students evaluated their examination performance scoring on a scale of 1-3, with 1 indicating that the student performed poorly than he or she normally did in previous examinations, 2 for the response that the student was able to perform the same as he or she normally did in previous examinations, and 3 indicating that the student performed better than he or she normally did in previous examinations. The questionnaire also included 9 more questions devised to address students' views about online assessments on the Likert scale ranging from (1=strongly disagree to 5= strongly agree). 12 The total responses ranging from 9 (indicating the worst) to 45 (indicating the best) for each respondent are then recorded on subscales ranging from 1 to 3. The recorded value is 1 (showing disagreement) when the cumulative response for 9 behavior was less than 20, 2 (Neutral) when the cumulative response for 9 behavior was between 20 and 33, and 3 (showing Agreement) when the cumulative response was above 33 or above. This provided an overall (aggregated) scoring of the online assessments by the students. Further, to capture more in-depth information, students were asked to provide open-ended feedback for improvement of the online assessments. Thematic analysis was conducted to generate potential themes.

Results
Quantitative results: The overall response rate for this study was 80% (n=412) with almost equal representation from students in all five years of the MBBS program. 65.8% of the study respondents were females while the rest (34.2%) were males. 1st question asked from all students was regarding their perceptions about their performance in online exams. (Table 1) Table 1 provides the 3×5 contingency table between the performance rate and year of the students. It delineates that majority of the students in the 1st, 3 rd and 4 th years performed worse than they did in traditional examinations, while the majority of the students from the 2 nd and 5 th years of the students were of the opinion that their performance has improved. The Pearson chi-square test was applied and indicated a significant (p=0.02) association between performance improvement and year of study. The next parts of the questionnaire focused on administrative and academic aspects. Table 2 provides a year-wise breakdown of student responses for each behavior statement.   Table 3 provides the cross-tabulation of students' responses to behavioral statements and their year of study. Although the majority of students were neutral towards the behavioral statements the p-value (p=0.04) for Pearson's Chi-square test revealed a significant association between the two variables. Qualitative results: Thematic analysis was done manually for the last part of our questionnaire which was focused on student suggestions to improve the online assessment process. Initially, open line-by-line coding was done and then themes were developed. The majority of the students provided suggestions for improvement of the online process giving a mixed response. Two potential themes were extracted:  Administrative: Most of the respondents were of the view that the online assessment could be improved by using more user-friendly software. They further commented that while conducting online assessments students' problems such as internet connectivity and availability of electricity and devices should be understood and there should be some flexibility in time.
Time allocation for the question should be according to the length and difficulty level of the questions being asked. Few students also suggested increasing the duration for the preparation of online assessments.  Academic: Students also indicated that for online assessments there should be proper online classes with a separate session for student queries. Online assessment should include a variety of formats focusing mainly on conceptbased questions. A proper proctoring system should be in place to reduce the likelihood of cheating in online exams. Few students argued that the clinical aspects of the topics were not assessed effectively in online assessments.

Discussion
Student perceptions regarding their performance in online assessments at Wah Medical College showed a mixed response (Table 1). This response could be attributed to the system of online assessment being implemented for the first time in WMC. Previous literature on online assessments focuses on blended learning and studies conducted when normal face-toface teaching activities were occurring concurrently. 13 Our study focuses on assessments conducted during the turbulent times of the COVID-19 pandemic for academic content that was taught solely online.
The positive outcomes of this study also help to analyze the efficacy of the online teaching system at WMC, which was based on online sessions of Large Group Interactive Sessions (LGIS), Self-Directed Learning (SDL) sessions, and Small Group Discussions (SGD). SDLs and SGDs have proved to be helpful in improving the learning abilities of medical students during e-learning. 9 The advantage of using advanced technology for learning in medical education has been highlighted in the Teaching and Learning report by The Horizon 2020. 10 A major change in our educational culture of medical education is being portrayed in this study. This provides a new way to establish reliability and validate our assessments for ensuring the achievements of the objectives of learning by the students. 9 Some students also agreed that they were provided adequate time to attempt the exam and that the software platform was easy for them to use. 10 One major issue of concern regarding administrative aspects was the issue of internet connectivity for our medical students. Our results in this regard are similar to another recent study by Farooq et al conducted in Pakistan and reiterate that internet connectivity in remote areas of the country does remain an issue of concern for students. 14 The academic aspect of our questionnaire focused on assessment content and its alignment with the Table of Specifications (TOS) provided to the students. Studies have shown that providing a TOS to students prior to assessments helps them to develop focused learning skills. 15 Faculty were advised to develop questions that would test higher-order thinking skills. 16 Designing such questions would also help to decrease academic dishonesty, as all the questions would be time-bound. Students also perceived that the online assessments did not adequately test their clinical skills. This was understandable because the assessment tool used for these assessments was solely one best type of multiple choice questions (MCQs). MCQs in medical education are used mainly for testing cognition-related skills. 17 We also performed thematic data analysis of openended statements received in response to student suggestions. The themes generated were used to further improve the online assessment process and other assessment formats including OSCEs were also included in subsequent online exams. This study has shown a slightly positive attitude of students towards online exams as compared to the traditional methods. Jawaid et al., and Bandele et al have reported slightly different perceptions. 12,13 Martin et al also showed similar observations for online assessments being the ideal based on optimal evaluation approaches. 14 The outcomes of our study have helped us to improve our online assessment process. These results can also benefit other higher education institutions to develop their online assessment strategies. Currently, Covid-19 has caused a paradigm shift in the teaching and learning of medical undergraduates. This is a trend that will continue into the near future. Researchers have suggested that medical educators should adapt to the educational challenges being faced currently and for this, they have to be involved deeply in shifting the traditional learning and examination policies to e-learning and assessments. 15 Conclusion Assessments and examinations are important tools to test the knowledge and skills which are taught to the students during their course of studies. Covid-19 pandemic has forced the institutions to adopt not only online teaching but online assessments and exams as well. It's a new dimension in a developing country like Pakistan and it will continue at least till the current pandemic comes to an end. Our study has shown that online learning is an effective tool for assessment and medical students are satisfied with their performance in online assessment.