Portal de Periódicos da CAPES

Use of recorded interactive seminars in orthodontic distance education

2007; Elsevier BV; Volume: 132; Issue: 3 Linguagem: Inglês

10.1016/j.ajodo.2007.03.015

1097-6752

Kenneth T. Miller, Wallace M. Hannum, Tarrl Morley, William R. Proffit,

Problem and Project Based Learning

Introduction: Our objective was evaluate the effectiveness and acceptability of 3 methods of instructor interaction during distance learning with prerecorded seminars in orthodontic residencies and continuing education. Methods: After residents at 3 schools (Sydney, Australia; Winnipeg, Manitoba, Canada; and Manchester, United Kingdom) viewed a recorded interactive seminar, they discussed its content with the seminar leader at a distance via video conferencing, audio-only interaction by telephone, and Internet chat with Net Meeting software (Microsoft, Bellevue, Wash). The residents then completed evaluations containing both closed- and open-ended questions. In addition, attendees at the Iranian Orthodontic Congress also viewed a recorded seminar, had questions answered via an interpreter in a video conference, and completed summary evaluations. Results: Video conferencing received the highest ratings and was never cited as the least favorite method of interaction. Telephone interaction was a close second in mean scores, and Internet chat was a distant third. All residents stated that they would like to be taught through distance education again. However, the Iranian orthodontists were less enthusiastic. Conclusions: Distance learning based on observation of recorded seminars and follow-up interaction is an acceptable method of instruction that can allow residents and practicing orthodontists access to various materials and experts, and perhaps help to ease the strains of current faculty shortages. More data are needed to determine whether video conferencing is worth the additional cost and complexity over audio-only interaction. Introduction: Our objective was evaluate the effectiveness and acceptability of 3 methods of instructor interaction during distance learning with prerecorded seminars in orthodontic residencies and continuing education. Methods: After residents at 3 schools (Sydney, Australia; Winnipeg, Manitoba, Canada; and Manchester, United Kingdom) viewed a recorded interactive seminar, they discussed its content with the seminar leader at a distance via video conferencing, audio-only interaction by telephone, and Internet chat with Net Meeting software (Microsoft, Bellevue, Wash). The residents then completed evaluations containing both closed- and open-ended questions. In addition, attendees at the Iranian Orthodontic Congress also viewed a recorded seminar, had questions answered via an interpreter in a video conference, and completed summary evaluations. Results: Video conferencing received the highest ratings and was never cited as the least favorite method of interaction. Telephone interaction was a close second in mean scores, and Internet chat was a distant third. All residents stated that they would like to be taught through distance education again. However, the Iranian orthodontists were less enthusiastic. Conclusions: Distance learning based on observation of recorded seminars and follow-up interaction is an acceptable method of instruction that can allow residents and practicing orthodontists access to various materials and experts, and perhaps help to ease the strains of current faculty shortages. More data are needed to determine whether video conferencing is worth the additional cost and complexity over audio-only interaction. Small-group seminars and clinical discussions between residents and faculty are important in postprofessional specialty training programs such as orthodontic residencies, as they are for higher-level education generally.1Bender T. Discussion-based online teaching to enhance student learning: theory, practice and assessment. Stylus Publishing, Sterling, Va2005Google Scholar, 2Abraham R.A. Upadhya S. Torke S. Ramnarayan K. Clinically oriented physiology teaching: strategy for developing critical-thinking skills in undergraduate medical students.Adv Physiol Educ. 2004; 28: 102-104Crossref PubMed Scopus (20) Google Scholar, 3Hannum W.H. Research into educational uses of distance learning in orthodontics.Proceedings of the American Dental Education Association. 2006Google Scholar, 4Bernard R.M. Abrami P.C. Lou Y. Borokhovski E. How does distance education compare with classroom instruction? A meta-analysis of the empirical literature.Rev Educational Research. 2004; 3: 379-439Crossref Scopus (808) Google Scholar High-speed Internet connections now make it feasible to involve distant faculty in seminars and clinical conferences. This allows experienced faculty to contribute to residency programs at other schools as well as their own, and perhaps also to provide individualized continuing education. A recent study of orthodontic distance education showed that, although orthodontic residents preferred live interaction to observation of an interactive seminar, they appeared to learn almost as much from observation, especially if they could interact afterward.5Bednar E.D. Hannum W.H. Firestone A. Silveira A.M. Cox T.D. Proffit W.R. Evaluation of distance learning to interactive seminar instruction in orthodontic residency programs.Am J Orthod Dentofacial Orthop. 2007; (in press)Google Scholar The purpose of this project was to follow up and further evaluate the potential for using prerecorded seminars and follow-up discussion in distant learning in both orthodontic training programs and continuing education. First-year residents at 3 schools, the University of Sydney (Australia), the University of Manitoba (Winnipeg, Canada), and Manchester University (United Kingdom), were invited to prepare for and then watch 3 previously recorded seminars on basic topics in orthodontics and then participate in live discussion with the seminar leader immediately afterward. Seminar preparation included readings and 1 assignment to view a special experimental program that included complex graphics on a commercial web site. Three types of communication in the postseminar discussion were evaluated, with a different type of interaction after each seminar: audio interaction via telephone; chat room interaction via Net Meeting software (Microsoft, Bellevue, Wash); and live video interaction with high-speed Internet connections. In addition, a recorded seminar with live video interaction afterward was presented and evaluated at the 2006 Iranian Orthodontic Congress. The seminars had been recorded during distance learning sessions in an earlier experiment, by using dual-streaming video conferencing equipment and Internet2 connections.5Bednar E.D. Hannum W.H. Firestone A. Silveira A.M. Cox T.D. Proffit W.R. Evaluation of distance learning to interactive seminar instruction in orthodontic residency programs.Am J Orthod Dentofacial Orthop. 2007; (in press)Google Scholar, 6Engilman W.D. Cox T.N. Bednar E.D. Proffit W.R. Equipping orthodontic departments for interactive distance learning.Am J Orthod Dentofacial Orthop. 2007; 131: 651-655Abstract Full Text Full Text PDF PubMed Scopus (11) Google Scholar The recordings showed various views of the seminar leaders, their data and slides (Powerpoint, Microsoft, Redmond, Wash), and residents in the 2 groups who were being taught simultaneously. The recordings also captured the interactions among the instructor and the residents at remote locations. To make the recordings available to distant schools, they were digitized with RealVideo (RealNetworks, Seattle, WA) and uploaded to a website at the University of North Carolina (UNC) that was accessible by user name and password. The seminar subjects were eruption problems and their management, the development of modern fixed appliances, and new information about the biology of orthodontic tooth movement. For the Iranian orthodontists, a recorded seminar on changes in treatment of Class III malocclusion was used. After each session, the residents were asked to evaluate the seminar on its perceived effectiveness (how good was the learning experience?) and acceptability (how well did you like it?). A 7-point Likert scale was used to rate specific aspects of each seminar and the interaction that accompanied it. Differences in ratings of the interaction methods were evaluated statistically by using repeated-measures t tests. After the 3 seminars and at the Iranian meeting, participants were asked to evaluate the entire experience using a combination of closed- and open-ended questions, and descriptive statistics were compiled. Although UNC and the distant schools had Internet2 availability, the connection speed for the postseminar video conference was in the range of regular high-speed Internet because of "last mile" limitations. Nevertheless, all 3 schools and the Iranian orthodontists could access the videos on the web site and watch them with few complications. The video from the UNC web site was seen at the distant locations with no delay except for Australia, where a barely perceptible delay (perhaps half a second) was noted. The more complex program from the commercial web site was seen without problems at all 3 participating schools, but there were problems with access to a similar site from Iran, probably because of bandwidth limitations. The residents rated the video quality of the recorded program positively (Fig 1), despite minimal postproduction editing of the original programs. The quality of the recordings was hampered by relatively crude camera selection and a recorder with less than optimal quality. Nevertheless, the residents thought that the videos were adequate for teaching purposes, with the content explained clearly and at a suitable pace. This was consistent with previous studies showing that videos need not be technologically advanced to be successful from an education standpoint.7Marsh C.M. Hannum W.H. Trotman C.A. Proffit W.R. Design and effectiveness of a computer-based continuing education program for orthodontists.Angle Orthod. 2001; 71: 71-75PubMed Google Scholar For the live discussion immediately after each seminar, it proved more difficult to establish acceptable chat-room interaction than it did with audio or video conferencing, because of problems with university wireless networks and firewalls. Despite a successful test, chat-room interaction after the seminar was impossible in Sydney, and video conferencing was substituted. The other interactions occurred as scheduled. In evaluating the overall experience with recorded seminars and follow-up real-time interaction, the residents were positive (Fig 2). They agreed that the overall experience was an effective and efficient way to learn, and better than they expected, and that the technology was not a major distraction. In general, the residents reported themselves more attentive and the experience more enjoyable than a traditional classroom, although they were close to neutral when asked if they could interact better than in a classroom. This indicated that the residents did not perceive less interaction during the distance learning experience when compared with a classroom experience. The Iranian orthodontists, who participated through an interpreter in a follow-up video conference, disagreed that their experience was as good as a traditional classroom and rated their experience below the residents in nearly every category (Fig 2). Several remarked that they could not get all of their questions answered. Because more than 85 orthodontists participated in this session, the 30 minutes allotted for the video conference was inadequate, especially because additional time was needed for questions and responses to be translated. Seminar evaluations by interaction type are shown in Table I. The evaluations were favorable for all types of interaction, with video conferencing rated highest. However, the differences were not statistically significant. Table II, which lists comments that were made more than once, helps to place the ratings in perspective, particularly with regard to the impact of the technology. It might not have been a major distraction, but it was noticed as a disadvantage, especially when problems arose.Table ISeminar evaluations (7-point Likert scale: 7, strongly agree; 1, strongly disagree)Questions: Degree of agreement⁎1, The video presentation was arranged so that it was easy to follow; 2, I found that the teaching methods used in the video presentation were effective in helping me learn; 3, The video was presented at a suitable pace to assist my learning; 4, The content of the video presentation was presented and explained clearly; 5, Watching the video presentation was a productive and effective use of class time; 6, I was attentive and involved during the video presentation; 7, The ability to interact with the instructor after watching the presentation was helpful in increasing my understanding of the content; 8, This form of interacting with the instructor was conducive to learning; 9, It was easy to communicate with the instructor; 10, When compared with a typical classroom, this form of interaction made it more likely of me to ask questions; 11, It was easy for me to get my questions answered by the instructor; 12, My questions were answered as well as if I was in a classroom with the instructor; 13, I was enthusiastic about this seminar; 14, I was actively engaged in learning during this seminar; 15, Overall, I was satisfied with quality of the seminar; 16, This approach was effective as a teaching tool.Postseminar discussion method12345678910111213141516SydneyManitobaManchesterOverallVideo conference Mean6.266.336.416.486.486.566.446.416.333.786.045.936.596.486.416.486.586.615.666.21 SD0.710.620.640.640.700.580.700.690.781.760.901.270.640.750.750.750.930.911.021.06Telephone Mean6.236.316.56.466.466.356.195.885.463.125.625.546.466.236.086.046.085.756.025.93 SD0.510.550.510.510.580.690.631.031.421.371.301.330.580.760.931.180.771.61.021.21Internet chat Mean5.565.725.785.895.895.61.5.675.285.114.445.004.675.835.675.615.566.005.255.45 SD1.291.271.060.960.831.141.031.131.131.501.141.501.041.191.141.041.231.131.21 1, The video presentation was arranged so that it was easy to follow; 2, I found that the teaching methods used in the video presentation were effective in helping me learn; 3, The video was presented at a suitable pace to assist my learning; 4, The content of the video presentation was presented and explained clearly; 5, Watching the video presentation was a productive and effective use of class time; 6, I was attentive and involved during the video presentation; 7, The ability to interact with the instructor after watching the presentation was helpful in increasing my understanding of the content; 8, This form of interacting with the instructor was conducive to learning; 9, It was easy to communicate with the instructor; 10, When compared with a typical classroom, this form of interaction made it more likely of me to ask questions; 11, It was easy for me to get my questions answered by the instructor; 12, My questions were answered as well as if I was in a classroom with the instructor; 13, I was enthusiastic about this seminar; 14, I was actively engaged in learning during this seminar; 15, Overall, I was satisfied with quality of the seminar; 16, This approach was effective as a teaching tool. Open table in a new tab Table IIAdvantages and disadvantages mentioned during seminar evaluationsAdvantagesNumber of times mentionedDisadvantagesNumber of times mentioned1. Learn from prominent leaders in field151. Technological difficulties92. Points of view from all around world82. Unable to ask questions during video63. Enjoyable73. Impersonal at times34. More engaged during video and interaction54. Typing skills needed for chat interaction25. Interactive3 Open table in a new tab Although the seminars were rated highly regardless of the type of interaction (Fig 3), video conferencing received the highest scores, with audio a close second. The average score for audio was actually a little higher in Manchester, and 2 students commented that they thought it was as good as video conferencing. Despite this, all but 1 resident chose video conferencing as their favorite approach, including those in Manchester who had rated audio slightly higher (Fig 4), and 5 voted for audio their least favorite method. Internet chat received the lowest average ratings; 10 voted it as their least favorite.Fig 4Number of residents who ranked each type of interaction their favorite and least favorite. No one ranked video as least favorite or audio as favorite.View Large Image Figure ViewerDownload (PPT) Technically, it is much less demanding to observe a digital recording from a distant web site and then participate in a postseminar discussion than it is to participate in a live distance seminar with dual streaming of data and video. In these experiments, at connection speeds that were well within the capacity of the regular Internet, all 3 schools were able to view the recorded seminars from a distant web site, although Sydney experienced 1 interruption. A postseminar video conference discussion also reduces demand on the Internet connection, because there is no need to stream data and video. It is clear that orthodontic programs without Internet2 access can participate in and benefit from this approach to distance learning. The video conference worked well in Sydney and Winnipeg. It was problematic in Manchester despite a successful previous test, probably because of the amount of Internet traffic at that time of day. This difficulty probably contributed to Manchester's lower evaluation of the video conference experience. Audio quality is critically important when audio-only or video conferencing is the mode of interaction. Good seminar room microphones and speakers become important parts of the necessary equipment. Residents at 1 school complained of difficulty in hearing during their audio discussion; this might have affected the ratings of this method. We did not expect technical problems with chat-room interaction, but they prevented its use with Sydney. The other residents remarked that chat was slow and cumbersome. The residents only "slightly agreed" that it was easy to have their questions answered through chat interaction. Questions and answers are traded only as fast as they can be typed, and it is difficult to keep questions and answers in sequence. Another comment was that the answers received to their questions during the Internet chat were abbreviated compared with those with audio-only or video conference. One resident, however, said he cited Internet chat as his favorite interactive method because it allows those who typically are not outspoken to feel comfortable interjecting questions. For these people, who are reluctant to interact with the instructor for fear of some reaction from others, chat might be a desirable mode of interaction. An important question becomes whether the postdiscussion video conference was worth the extra equipment and cost compared with audio-only or chat-room interaction. In this study, the residents overwhelmingly preferred the video conference but did not rate their seminar with it significantly higher than the ones with audio or chat interaction. Additional data are necessary to determine whether their preference for videoconferencing results in greater learning gains. All residents and the Iranian orthodontists who participated in this study rated the learning experience as effective and enjoyable (Fig 2), and all residents stated that they would like to be taught in this manner again. In the postseminar interaction, residents typically were called on individually if they did not volunteer questions and comments. Many residents said the forced interaction was a plus because it made them pay closer attention to the recorded video and during the follow-up conference. Some residents, particularly those in Manchester, said they felt intimidated when they were questioned, particularly during the video conference. An instructor there commented that his students were more used to group discussions of posed questions, rather than responding individually. Almost everyone was apprehensive the first time they were on camera, and the effect was magnified if there was no previous experience with interactive discussions. Distance education can easily be adapted to overcome problems of this type. As with other forms of instruction, students need to develop skills as learners in courses taught by video conferencing. Most learners need time to acclimate to the cameras and to seeing and hearing themselves on the monitors. Typically, learners without experience of distance learning that uses 2-way video are reluctant to enter into the discussions initially, but, in time, most "warm up" and ignore the cameras and the microphones. Once this happens, their interactions are more frequent and more natural. Distance education with recorded interactive seminars accompanied by real-time postseminar discussion has 2 potentially important advantages: it can be acceptable to orthodontic residents and practitioners, and it does not require bandwidth beyond what is available with the regular Internet. For that reason, it also could be used in continuing education for orthodontic practitioners, perhaps by making recorded seminars available to be viewed in an orthodontist's office at his or her convenience, and following up with an interactive session soon afterward. Chat-room interaction would be particularly well suited to this application, and some of the problems with chat-room access within the university would not exist. These data suggest that further experiments with recorded seminars in both orthodontic residency programs and continuing education are warranted. It appears that adding a real-time interactive discussion after viewing the recorded video can strengthen the overall quality of the learning experience while constraining costs. We thank James Ackerman and David Hall for their participation in some postseminar discussions, Thomas Cox for consultation and help on technical issues, Ceib Phillips for statistical consultation, David Sarver and Lloyd Ketchum of Libertech (Birmingham, Ala) for preparation and hosting of the special web program, and the residents and orthodontists who participated in and evaluated these experiments.