Evaluation of students’ results from Quantitative methods in marketing during COVID-19 pandemic: Case study

The subject Quantitative Methods in Marketing is aimed at providing theoretical understanding and practical abilities in statistical methods that may be used in marketing and market research. The graduate of the course acquires theoretical knowledge and can apply the gained practical skills from the preparation and statistical processing of the questionnaire to the interpretation of the results of analyzes. The aim of the research is to analyze the e-learning method of the subject during the COVID-19 pandemic comparing to traditional teaching before pandemic. The partial aim of the research is a comparison of total points from the course’ tests before and during COVID-19 pandemic for each study programme. The paper’s data base consists of points gained from partial tests of the subject Quantitative Methods in Marketing, the results of which are included in the course’s final evaluation. The analyzes were performed for two periods. The pre-covid period is represented by academic years 2017/18 and 2018/19 and the covid period is represented by academic years 2019/20 and 2020/21. Based on the results, we found that the students’ performance has been improving from year to year. From the point of view of comparing the pre-covid and covid perids, the significant improvement of students‘ results was confirmed in both study programmes during e-learning. The improvement of results during online learning could be influenced by better home preparation and provided study materials and also by practical skills acquired from home assignments.


INTRODUCTION
Nowadays, the world is facing the greatest crisis of modern times, caused by the COVID-19 pandemic, which has influenced not only healthcare, world markets and the economy, but also everyday life as such. Many countries worldwide closed education facilities in a global effort to combat COVID-19 pandemic [3]. Over 100 countries over the world implemented nationwide closures, affecting over half of the world's student population [15]. School closures caused significant interruptions in academic activities as well as in career goals. Students and educators were forced to alter the way of teaching to a distance form. Thankfully, a variety of modern methods are available to provide the distance learning [14].
Due to the exponential rise of the internet and information technology, e-learning has undergone substantial changes [4]. New e-learning platforms are being created for tutors to make assessments easier and for students to take part in lectures [1], [12]. The use of elearning tools in higher education means that a larger amount of data can be examined, which improves teaching quality [2], [11], [13].
However, e-learning is also affected by numerous negative factors, such as consuming of time, self-study, and lack of control during examination. Moreover, according to other studies, if some electronic teaching materials are not involved in education process, it appears to be incomplete [8].
The benefits and problems of large data analysis in higher education have been conducted by many researches in recent years [16]. According to Gasević et al. [5], time management strategies had significant correlations with academic success. Aiding students with the management of their learning resources is according to Jovanović et al. [9] crucial for the regularity of their learning strategies. Private chat messages, team chat messages, twoparticipant calls, and multi-person-meetings are the most popular remote collaboration methods [4].
Learning effectiveness, cost effectiveness, and institutional commitment, as well as access, faculty satisfaction, and student satisfaction, are all aspects that affect the success of elearning. The quality of learning material or online e-learning content becomes more challenging as it is the major criteria in teachers' up-to-date skills and student learning quality among the four components of e-learning, which comprise contents, learning management systems, communication, and evaluation [10]. During COVID-19 pandemic, the remote learning process in Faculty of Economics and Management in Slovak University of Agriculture has been provided by using Microsoft Teams. According to students' responses on remote learning using Microsoft Teams in other education facilities, some researches showed that students were satisfied with the teacher's presentation of the lecture, but the major issue was a poor internet connection [6].

MATERIAL AND METHODS
The paper's data base consists of points gained from partial tests of the subject Quantitative Methods in Marketing, the results of which are included in the course's final evaluation. The analyses were performed in four periods of the following academic years: 2017/18, 2018/19, 2019/20, and 2020/21. Students' results in the study programs Agrarian Trade and Marketing (AOM) and Quantitative Methods in Economics (KME) were also compared. During the traditional teaching periods (the first two analyzed periods), students took two partial tests over the course of the semester, each worth a maximum of 100 points. During the online teaching periods, only one test was written. For this reason, for comparability, the results of the first two periods were recalculated as a percentage on the same basis. The individual analyzes were performed in the statistical software SAS 9.4.
From the methodological point of view, the following statistical methods were used in the paper: • Shapiro-Wilk's test is used to verify the normality of the input data. If the H0 hypothesis is not rejected, it is assumed that the breach of the assumption of normality has not been sufficiently demonstrated. This means that it can be assumed that the analyzed data come from a basic file with a normal distribution.
• The Mann Whitney U test is a nonparametric form of the two-means match test, that is based on the order of values. With this test we verify the hypothesis whether there are statistically significant differences between the results of students before and after the pandemic.

RESULTS AND DISCUSSION
The subject Quantitative Methods in Marketing is specified as compulsory for students of study programme Agrarian Trade and Marketing and as a compulsory elective for students of the study programme Quantitative Methods in Economics. In the researched periods, both full-time and part-time students completed the course. Since teaching in external form does not fully correspond to the full scope of teaching full-time form, the analyzes in the given paper are performed only for full-time students. The requirement for the successful completion of the course is to theoretically and practically understand selected statistical procedures that can be used in evaluation of marketing survey. In addition to the final written test, students' knowledge is verified during the through partial tests.
The following Tab.1 shows the total number of students enrolled in the course during analysed academic years. In addition to the number of students, Table 1 illustrates the course attainment. The overal success is computed as the percentage of students who completed the subject successfully outof the total number of students. The subject attainment for each study programme is also measured. As shown in Table 1, students performed better during the period when teaching due to COVID-19 pandemic was carried out online. Students of study programme KME achieved better attainment than AOM's students. Based on the results, we analyze the evaluation of partial tests as follows: 1. Comparison of test results in pre-covid period and covid period, regardless of the study programme.
2. Comparison of test results for each study programme -Agrarian Trade and Marketing, Quantitative Methods in Economics. Figure 1 shows the information on the achieved scores from partial tests expressed by the arithmetic mean.

Comparative analysis of the student results in pre-covid and covid period
In the first part of study, we analyze study results from the subject Quantitave Methods in Marketing according to the period in which teaching took place. The first period was the period before the covid, within which the teaching took place as standard, so the issues taken over in the lectures were put into practice. In case of problems, the teacher could help the student immediately. The second analysed period is represented by pandemic COVID-19, when learning process took place online. During the academic year 2019/20, students were given study materials from which they had to complete homework assignments. Later, they also had video recordings explaining the issues discussed. MS Teams was used to carry out exercises in the academic year 2020/21, based on an online lecture. Students were given individual calculations to complete. Throughout the class, the teacher was available to them, and he attempted to assist them with any problems they had with the computation via the shared screen. By the end of the lesson, all completed assignments were to be entered into the information system. Descriptive statistics for each analyzed period are shown in Table 2. In the pre-covid period was subject Quantitative Methods in Marketing successfully completed by 307 students. Students in this period received an average of 60.65 points, with 50% of them getting more than 62 points. Because the majority of students received 74 points, the point distribution can be considered as negative asymmetric. The test results varied ± 19.56 points around the average from 3 to 99 points. The variability of the result was at the level of 32%. On the contrary, 255 students successfully completed the subject during the pandemic. The overall mean of 69.07 points was 8.42 points higher than in the pre-covid period. Half of the students received more than 71 points, while the majority of them received 86 points. That means, that also in this period was the modus value higher then median and mean, so it is a negative asymmetry, while the peak of the distribution is shifted more to the right in this case. The results in covid period varied ± 20.03 points around the mean value from 13 to 100 points, in relative terms, the variability was lower (29%) compared to the first period (Tab.2). Distribution of results in both analyzed periods is illustrated in Figure 2. Based on the descriptive characteristics, according to which we can state that students achieved better results online in the covid period, we further verified the statistical significance of these differences. Before the test itself, we first verified the normality of the researched distributions.
According to the Shapiro-Wilk normality test (Figure 2), we can state that the results of students in pre-covid period and covid period do not correspond to the normal distribution. For this reason, we used the nonparametric Mann Whitney U test to verify the hypothesis that students' results in the covid period are better than in the pre-covid period. Based on the given results ( Figure 3) we can conclude that the results in the covid period are significantly better than the results during traditional teaching in the pre-covid period (pvalue < 0.001).

Figure 3
Mann Whitney U test of pre-covid and covid period Source: own processing

Comparative analysis of the student results in pre-covid and covid period based on the study programme
In the previous part of study, we have confirmed that the results achieved during covid period are significantly better, than the results from pre-covid period. In this part of study, we analyze the students' results in pre-covid and covid period according to the study programme. Label AOM represents study programme Agrarian Trade and Marketing and label KME represents study programme Quantitative Methods in Economics. Descriptive statistics for pre-covid and covid period based on the study programme are shown in Table 3. Table 3 Descriptive statistics of students' results for pre-covid and covid period based on the study programme Source: own processing The average results from tests are higher in covid period in both study programmes. The students of Quantitave Methods in Economics (KME) achieved the greatest average results during the covid period (80.26 points) from all analyzed perods. The results varied around average ± 15.49 points. Most of the students of KME during pandemic achieved 87 points and 50% of them got more than 84.50 points. On the contrary, students of study programme Agrarian Trade and Marketing (AOM) achieved the lowest average results (59.32 points) from all analyzed groups in pre-covid period. The results varied around average ± 19.45 points. Most of them got 74 points and 50% of that students achieved more than 62 points from the course ( Table 3). The variability of test results is approximately the same in the studied periods (around 30%), except for KME students in covid period, which had significantly lower variability (19%). Distribution of results for both study programmes in pre-covid and covid period is shown in Figure 4. In this case, too, we subsequently verified the statistical significance of the presumed differences between the covid and nocovid periods. A normality test was performed before the test itself. Based on the p-values of Shapiro-Wilk normality test (Figure 4), we can conclude, that non-rejection of H0 hypothesis can by stated only in pre-covid period for study programme Quantitative Methods in Economics. However, the H0 hypothesis about normal distribution was rejected in other analyzed periods. For that reason we again used the nonparametric Mann Whitney U test to compare the differences between the analyzed periods for both study programmes.

Figure 5
Mann Whitney U test of pre-covid and covid period (programme Agrarian Trade and Marketing) Source: own processing Figure 5 illustrates the output from Mann Whitney U test for the study programme Agrarian Trade and Marketing. Based on the results of the test (p-value=0.0031) and boxplots, we can conclude that in the case of this study programme, the study results during e-learning have significantly improved compared to the study results during standard teaching.

Figure 6
Mann Whitney U test of pre-covid and covid period (study programme Quantitative Methods in Economics) Source: own processing The output from Mann Whitney U test for study programme Quantitative Methods in Economics is shown in Figure 6. As well as for students of Agrarian Trade and Marketing, the results of the analyzed group of students has also improved significantly during the pandemic (p-value < 0.001). To summarize this section, we can therefore confirm the improvement of the results of both study programs during the pandemic compared to the period before the pandemic. Our results are in line with the Gonzales et al. [7] study about influence of COVID-19 confinement on students' performance in higher education. Authors studied the differences in assesments in two groups of students. First group corresponded to academic years 2017/2018 and 2018/2019 and the second group corresponded to academic year 2019/2020. Authors concluded, that COVID-19 pandemic improved the efficiency of learning strategies between students. For that reason, the better scores in students' assessment are expected during the pandemic.

CONCLUSIONS
The aim of the study was to evaluate the students' results of the subject Quantitative Methods in Marketing during the COVID-19 pandemic. We studied the differences in students' results for each study programme during the pandemic compared to pre-pandemic period, when the subject was taught standardly. We found that students scored an average of 69.07 points during the pandemic, with values fluctuating by ± 20.03 points around the average. This result was 8.42 points better then for students in the pre-pandemic period. The hypothesis that results of students from the pandemic period were better than the performance before the pandemic was confirmed through nonparametric Mann Whitney U Test. Subsequently, we verified whether there is a statistically significant difference in the results according to the study programmes during the pandemic period compared to pre-covid period. We found that the best average results (80.26 points) were achieved by students of Quantitative Methods in Economics during the pandemic. The students of Agrarian Trade and Marketing achieved the worst average score (59.32 points) before the pandemic. We can state that the average results of both study programmes improved during the pandemic, which was confirmed by Mann Whitney U test. In conclusion we can state, that while test results have improved, it is impossible to say categorically that students' knowledge levels have improved. We are aware that the results of the tests may also reflect the fact that the tests were written by students at home in the covid period, without direct supervision by a teacher. Despite the above, based on interviews with students, the positives of online teaching were that students had more teaching materials and had the opportunity to consult their issues through MS Teams.