«is (SN Svetlana M. Stanišić Stojić, Gordana Dobrijević, Nemanja Stanišić, and Nenad Stanić Singidunum University, Serbia Abstract This paper ...»
Characteristics and Activities of Teachers on
Distance Learning Programs that Affect Their
Svetlana M. Stanišić Stojić, Gordana Dobrijević, Nemanja Stanišić, and Nenad Stanić
Singidunum University, Serbia
This paper presents an analysis of teachers’ ratings on distance learning undergraduate
study programs: 7,156 students enrolled in traditional and 528 students enrolled in
distance learning studies took part in the evaluation questionnaire, assessing 71 teachers. The data were collected from the Moodle platform and from the Singidunum University information system, and then analysed with SPSS statistical software. The parameters considered as potentially affecting teacher ratings are: number of teachers engaged in a particular course, total number of courses in which the teacher is engaged, teacher’s gender and age, total number of the available resources, and so forth. The results imply that scores assigned to individual teachers are consistent in both traditional and distance learning programs. The average rate was perceived to be lower when there were several teachers in a single course; such an effect was enhanced in cases where there was a significant age discrepancy among them. The other factors considered did not show a significant association with teacher ratings. Students’ main remarks about the work of the teachers have been summarised at the end of this paper.
Possible explanations and implications of the results are discussed and recommendations are given.
Keywords: Distance learning; teacher rating; student satisfaction Characteristics and Activities of Teachers on Distance Learning Programs that Affect their Ratings S. Stojić, Dobrijević, Stanišić, and Stanić Introduction Distance learning is becoming increasingly recognised as a suitable and valuable educational experience (Davies, Howell, & Petrie, 2010). Many universities across the globe offer distance learning courses through the introduction of learning management systems that allow them to have both on- and off-campus students. Since recruitment and retention of students has a significant financial impact on today’s universities, there have been numerous studies on student satisfaction and academic achievements from distance learning programs (e.g., Endres, Chowdhury, Frye, & Hurtubis, 2009; Eom, Wen, & Ashill, 2006).
One of the main factors influencing student satisfaction is the quality of teaching. This is an especially important consideration for university managers and decision makers, who can organise distance learning in a way that allows them to provide ongoing guidance and improvement strategies for teaching staff. On the contrary, some other factors are not easily influenced and managed, such as student self-motivation, student learning style, required/elective courses ratio, and so forth. Rating teachers should be a valuable procedure for students as well, because it can lead to improvement of teaching quality, based on the stated opinions of the students (Marzano, 2012; Nargundkar & Shrikhande, 2012). The study of Taylor and Tyler (2012) strongly confirms the opinion that teachers develop skills and otherwise improve due to student evaluation. They found that teachers were more successful at improving student achievement during the school year when they were evaluated than in the years before the evaluation, and were even more successful in the years after the evaluation. However, some reports (e.g., Toch & Rothman, 2008) show that teacher evaluations have not helped in the formation of highly skilled teachers. We presume that teachers’ ratings could influence their skills development provided that their tenure depends on the rating, the rating is publicly announced (which puts additional pressure on the teacher), and there are some consequences for the teachers with lowest scores. Since there are no widely accepted standards of teacher performance, universities around the world mainly use student evaluation of teachers as the principal source of data on teachers’ performance and quality of teaching (Lalla, Frederic, & Ferrari, 2011; Carr & Hagel, 2008). Some previous studies have dealt with the issue of whether and to what extent the evaluation results truly reflect students’ attitudes. According to most authors, teacher rating proved to be a good indicator of teaching effectiveness (e.g., Beran & Violato, 2005; Nargundkar & Shrikhande, 2012; Wiers-Jenssen, Stensaker, & Grogaard, 2003). Beran and Violato (2005) also found that teacher rating is, to a lesser extent, biased by some factors that are not related to the teachers themselves, such as students’ grade expectations, attendance, and types of courses being evaluated. For example, teachers of elective courses have always been rated higher than teachers of compulsory courses, as shown by all findings to date. In previous research on teacher-related factors that affect ratings, interaction, teacher feedback, and communication appear most often (e.g., Kuo, Walker, Belland, & Schroder, 2013; Loveland, 2007; Rothman, Romeo, Brennan, & Mitchell, 2011). Some other factors having the highest impact on learner satisfaction in online
education are: teacher knowledge and facilitation, course structure (Eom et al., 2006), appropriateness of readings and tasks, technological tools, course organisation, clarity of outcomes and assignments, and content format (Rothman et al., 2011). According to the community of inquiry framework (Garrison, Anderson & Archer, 2000, 2010), a deep and meaningful online learning experience happens through the evolution of three interconnected elements: cognitive, social, and teaching presence. In an educational environment, teaching presence is normally the main responsibility of the teacher. It includes selection, organization, and presentation of the course content, and the design and development of learning activities. Although the teacher’s personality is known to be a crucial factor in establishing a successful online learning atmosphere (Northcote, 2010), the most apparent teachers’ characteristics, such as their qualification experience, do not always appear to be associated with teaching quality (Hanushek & Rivkin, 2010). Still, expressing the personality beyond the provision of mere resources is important because just the interaction is not enough to achieve a sense of teacher presence in online learning contexts (Garrison,Cleveland-Innes, & Fung, 2010).
Our research was aimed at defining the characteristics and activities that will influence the rating of teachers on distance learning study programs. The data from the evaluation questionnaire, conducted regularly for the quality control of teaching and study processes at Singidunum University, have been used for the research. Singidunum University is the largest private university in Serbia, with more than 10,000 students enrolled in degree programs in the fields of finance, banking, accounting, marketing and trade, tourism and hotel management, engineering management, computer science, and electrical and computer engineering. Six percent (about 600 students) of this number attend distance learning programs from all the abovementioned fields except computer science and electrical and computer engineering. The research results are expected to benefit teaching staff as well as the management of higher education institutions, which aims to improve the quality of teaching delivery and increase satisfaction of students on distance learning programs.
Method The research covered all four years of various study programs. The data were collected from both the University information system and the Moodle platform, then crossanalysed by using the SPSS statistical software.
Numerous parameters were analysed as potential correlates of teacher rating, some of them being the number of teachers engaged per course, number of courses assigned to a particular teacher, gender and age of a teacher, total number of the available resources, and students’ activities, and so forth. Available resources and students’ activities were considered collectively as well as individually against the following categories: files, quizzes, forums, lessons, assignments, labels, and dictionaries. The Pearson correlation was used to measure the ratio between the two variables. Linear regression was used to Vol 15 | No 4 Creative Commons Attribution 4.0 International License Sept./14 Characteristics and Activities of Teachers on Distance Learning Programs that Affect their Ratings S. Stojić, Dobrijević, Stanišić, and Stanić define the relationships between the variables, while t test was used to test the differences in the average values. The method of Spearman’s rank-order correlation was used to determine the association between variables that are not normally distributed.
In order to test for association between two non-normally distributed variables while simultaneously controlling for the effect of already known confounders, the method of partial Spearman’s rank correlation was used.
In order to create a comparison with the traditional studies, this research was conducted on a sample of 71 teachers, simultaneously engaged with both traditional classroom programs and distance learning studies. The total number of students that assessed the work of teachers in traditional classroom programs was 7,156, whereas 528 students assessed the work of teachers on distance learning studies. Each student rated the teachers only during the term s/he was currently in attendance, awarding grades from 1 to 5, 1 being the lowest, and 5 the highest. The total number of polls completed was 36,151 on traditional studies and 2,675 on distance learning study programs. The
evaluation questionnaire comprised five questions:
Students were also able to leave a comment at the end of the poll, related to the teacher’s performance. As the mean evaluation score for the teachers involved more than one course, we used the grand mean they received for all the courses they were involved in. At the end of this paper, we also summarised students’ remarks, which were usually given descriptively in the polls.
Results and Discussion Because of the differences between online and traditional courses, some researchers have examined if and to what extent the same student ratings can be used in online courses. For example, students of traditional studies establish more frequent personal contact with their teachers, as opposed to the students of distance learning studies. Due to the lack of personal contact and empathy, the rating of distance learning teachers is often more reflective of the quality of the course content than the quality of the overall work of the teacher. This can provide misleading results, particularly with the courses Vol 15 | No 4 Creative Commons Attribution 4.0 International License Sept./14 Characteristics and Activities of Teachers on Distance Learning Programs that Affect their Ratings S. Stojić, Dobrijević, Stanišić, and Stanić involving more than one teacher. For these reasons, we divided the courses into modules, with the name of the author clearly given for each online resource. A large number of recorded lectures were also uploaded; the communication on each of the courses was personalised so that the students could easily master the materials and have a clear idea of whom they were communicating with.
Figure 1 represents the distribution of scores that the students awarded to the teachers on traditional studies and distance learning studies. The average values, standard deviations and the sample size are shown in this graph. The obtained results show that there is a positive association between the score awarded to the teacher on traditional studies and the score awarded to the teacher on distance learning studies (r(69) = 0.575, p 0.01).
Figure 1. Scatter plot of mean evaluation scores received from students participating in the two programs with “rug” density plots on margins.
The linear model has the following form:
Average evaluation score from students participating in the online distance learning program = 0.880 + 0.756 x average evaluation score from students participating in the traditional classroom program The reliability analysis has also confirmed that the evaluation scale items are reasonably interrelated (5 items; Cronbach's α = 0.82). However, Sijtsma (2009) reminds us that a high value of Cronbach’s alpha does not provide evidence of unidimensionality.
Although distance learning involves reduced learner-tutor interaction due to asynchronous computer-mediated (ACM) conferences, it is obvious that certain teachers are able to meet the requirements, regardless of format or mode of delivery. This points to a very important conclusion: "good teaching" is just "good teaching" regardless of the medium. Comparing traditional and online courses, Beattie, Spooner, Jordan, Algozzine, and Spooner (2002) found similar results across course, teacher, and general ratings, regardless of the mode of delivery. Also, in the IDEA survey 2002-2008 by Benton, Webster, Gross, and Pallett (2010), students’ ratings of teachers, both of online and traditional courses, as well as ratings of the courses themselves, were all very much alike. They found some minor differences (e.g., that teachers of online courses were perceived as using educational technology more effectively), which is expected given the specifics of the mode of educational material delivery.
According to Alonso Díaz and Blázquez Entonado (2009), teachers’ roles in online education are not very different from those of traditional courses. In both teaching modes teachers still have to deal with facilitating the teaching/learning process, combining activities with theoretical content, and encouraging student interaction.
Kelly, Ponton, and Rovai (2007) also compared students’ ratings of overall instructor performance and overall satisfaction of online and traditional courses. They found no significant difference in students’ ratings between online and traditional courses. The authors propose that teachers of online courses should be fair and unbiased, and enthusiastic and helpful, and they should show real interest in student progress and needs.