Vol
11:4 pp1-10 |
Learning to care on the Internet:
evaluating an online introductory social work course
|
Characteristics | % |
Aboriginal Ancestry | 8 |
Age 19-25 | 56 |
Age 26-35 | 24 |
Age 36-45 | 8 |
Age 46-55 | 12 |
Women | 80 |
These characteristics would tend to mirror the general characteristics of an introductory social course. As well, only two learners had previously taken an Internet delivered course, while 52% had previously taken an ITV (instructional television) course. Of those who responded, 64% categorized themselves as either having beginner (4%), novice (12%) or average (48%) computer skills. There is a danger in this type of study of only advanced computer users self-selecting to complete a questionnaire on computers and learning. The above would indicate that this is not the case for this study with participants having a variety of computer skill levels.
The open-ended questions asked learners to state in their own words, their perceptions and preferences for the online course (see Appendix A).
Responses to questions 1 and 2 portrayed overwhelmingly positive initial reactions to the course which remained toward the end. Six learners were negative or concerned about starting the course, but in the end turned positive. For example, two learners were concerned that 'it would be complicated and they wouldn’t learn anything', but after experiencing the course found that it 'was very clear and well laid out … making it easier to learn and understand'. Some others were overwhelmed by the amount of information, but once they got into it they found that it was 'not bad at all'. One learner was frustrated by not having the proper programs on their computer for audio and video, but was very pleased once they received help from their spouse. Another learner found that they 'kept getting sidetracked' by the hyperlinks, but was not sure if this was a negative or positive aspect. Only two learners had negative comments: one was 'totally bored' using the online course, frustrated by not having the correct software to run the audio and video, and one found the statistics from Statistics Canada 'just boring'. Another stated that the course 'lacked interactive things online'.
The positive comments included the following:
The questions concerning a particular component of the course that help the learner learn (question 3) revealed several critical components. The three most commonly mentioned components that learners found helped them learn was the glossary, the hyperlinks to additional material, and the varying levels and depth with which the material was covered. Other learners mentioned the benefits of the video clips, or particular exercises that they liked. One learner thought that the lack of a classroom component was a problem, stating that 'I don’t think I`d ever be happy without classroom discussion and stimulation to challenge schemas I’ve developed and to keep studies about people - human'.
The three components that were emphasized in helping learners learn (glossary, links, and variety of levels) illustrate how the capabilities of the Internet can benefit learning social work. Hyperlinks within the course and out to additional material are key features of the Internet itself. The notion of having a variety of levels for the learner to approach the material is a function of the learning model chosen.
The learning models emphasize three different learning modes: didactic learning which mirrors what a text (multimedia in this case) would accomplish, experiential learning which was contained in the exercises such as the poverty game, and reflective learning which allows the learner to extend their knowledge to other cases or apply concepts. Since the consultations with learners found this to be an ideal model for introductory social work, it should be of no surprise that in the evaluations the benefits of this were emphasized.
Learners did not emphasize the importance of the multimedia aspects of the online course. In fact, in answer to the question 'multimedia (audio & video) was essential in the course' 16% disagreed, 28 % strongly agreed and 56% agreed. Given that multimedia7 has been found to increase learning due to dual coding8 or multi-sensory input it may be surprising that this element was not more strongly emphasized. For example, Bagui (1998) found multimedia allows a learner to absorb information using two channels, reducing the cognitive load on a person’s memory, and thereby improving learning. Perhaps students failed to emphasize the importance of the multimedia elements of the course because multimedia was not used effectively. As one student stated, 'the video should be interesting and talk more directly about what is being displayed on the screen'. Additionally, perhaps the quality of the video affected its importance (due to limitations inherent in today’s Internet).
Another surprising finding from this question is that no one mentioned the benefits of the communication and collaboration aspects of the course. This is unexpected since the initial consultations determined that this was an important aspect which should be included in the online course. There could be several reasons for the lack of mention of this element. First, learners did not have to communicate with each other to make use of the online course, or perhaps more importantly to pass the course. Secondly, learners were asked to try communicating with others, but it was not a requirement of the course, nor were any grades given for participation in online discussions. Finally, assignments did not require learners to collaborate in teams. A recent study (Hmelo, Guzdial & Turns, 1998) of computer-support for collaborative learning found that collaboration needs to be anchored. Students need to know what to talk about (online talk), when to talk, and that it is important to talk. They found that learners need to ‘buy-in’ to the need to collaborate, they need to see collaboration as intregral to a project, and see the benefits of being able to communicate online across space and time. Based on the pre-development consultations it would seem that this is indeed an important element of an online social work course, but we can also surmise that without the necessity to communicate and collaborate built into course delivery, learners do not benefit from the feature.
Question 4 asked learners whether or not they feel the computer is appropriate to use in learning about social work and social welfare. Analysis of the responses shows that two thirds of the respondents feel that a course based entirely on Internet-mediated delivery is appropriate for a social work course. They believed that chat rooms, discussion groups, interactive activities, the video and audio, and the self-paced nature of the course were advantages that made the online course appropriate for social work. The other one third feel that personally they would prefer a face-to-face or classroom element, particularly since it is a social work course which is, as a profession, based on interpersonal communication. Even those with a negative response believed that the Internet was 'good as an aid' or 'a useful tool to assist learning as it reinforces the information given in class'. Many of the criticisms stemmed from concerns over getting help from the instructor when they had questions or required explanations. Not surprisingly learners had very similar responses to question 5 which asked them if they thought they could learn an entire course using the Internet tools. Again two thirds said they could learn an entire course using the Internet while one third said no they could not. Of those who could not learn this way, all except one, stated that attending a class is necessary. One other said they would also want a textbook to fall back on.
The answers to the next question contained a variety of suggestions to make the program better. Only 15% of the learners did not have a suggestion to improve the course. There were a range of suggestions, but they can be categorized as follows: 'more exercises and activities', 'option to print the text and audio material', and increased methods for learners to communicate with the instructor using chat rooms, ICQ (an online real-time chat tool), or email. For example, one learner said, 'I myself would incorporate more exercises and activities. I’ve always found a course is more interesting when you can do ‘hands on’ types of things... especially if you are finding the material hard to understand. It also gives you a clearer picture on what you’ve already learned'. Another learner emphasized the importance of instructor feedback saying, ' I guess putting questions to learners through the movie clips. It would feel more alive for me if an oral question was asked, my answers sent off via e-mail, and then responded to. Maybe then I wouldn’t feel so cut off. Instructor feedback is a main reason for my paying for a course'. Finally, one learner had a suggestion illustrating a solution to the problem of learner isolation from the instructor, “In distance education, I find learners tend to feel isolated, and that in class learners have special access to instructors and to other learners. I think that if the instructor of an Internet course continues to be as quick to reply on ICQ, and is as quick with email as he is with ICQ...then that will alleviate this trouble. I would actually make ICQ a recommended addition to the course, as it is a wonderful way to make sure you get answers rather quickly'.
Likert scale questions
Learners were asked to select among 'strongly agree', 'agree', 'disagree'
or 'strongly disagree' according to a number of variables concerning the
features and preferences for online learning (see Appendix A). The overall
response to the online course was very positive with a large majority
responding with strongly agree or agree to all questions. A small percentage
answered disagree to 6 of the 8 questions, and no one answered strongly
disagree to any question.
Two questions, it was easy to move through the course' and 'online self-testing is an important aspect of the course' found 100% of respondents answering with strongly agree or agree. Although still very positive the weakest ratings were awarded to 'multimedia (audio & video) was essential in the course' and 'the activities in this course gave me sufficient practice and feedback'. Follow-up questions via email with several learners revealed that some thought these aspects of the course should be improved, and not that they should be deleted. One learner believed that the video should be more interesting with the instructor asking questions that could be answered via email. Several people thought that online activities are critical for the course, both to make it interesting and to help reinforce their learning. They wanted more interactive learning simulations like the poverty game (a student is given an actual welfare cheque amount and is challenged to survive for a month by selecting among various purchases). The course intended to contain one simulation per module or week of learning. This finding may indicate that additional simulations are required for an online social work course. Such activities are difficult to conceive and time consuming to develop, but the data shows that the additional effort is desirable and necessary. Perhaps because social work learning demands more experiential and reflective types of learning these types of activities are more necessary.
The question 'I would recommend that others take a course that is delivered via the Internet' is another important measure of social work learners' preferences for online learning. This question had 88% agreeing and 12% disagreeing. This disagreement finding is lower than the result for the question concerning the appropriateness of the computer for learning about social work (66% agreed and 33% disagreed). This may be due to those learners (33%) believing that while Internet-mediated learning may not be appropriate for them that they would still recommend it for someone else. In the end, however, 12% just do not like it, and would not recommend it to someone else.
Question
|
Strongly Agree % |
Agree % |
Disagree % |
I would recommend that others take a course that is deliverd via the Internet | 44 | 44 | 12 |
The resources on the Internet enhanced my learning | 40 | 52 | 8 |
It was easy to move through the course | 60 | 40 | 0 |
Multimedia (audio & video) was essential in the course | 28 | 56 | 16 |
Online activities stimulated my learning | 48 | 44 | 8 |
The activities in this course gave me sufficient practice and feedback | 20 | 68 | 12 |
Online self-testing is an important aspect of the course | 64 | 36 | 0 |
The online course was easy to use | 60 | 32 | 4 (4% did not answer) |
Frequency distribution for Likert Scale Questions
Chart 1 (overleaf) indicates an interesting finding for instructors developing courses that are not intended for distance education. Learners were asked what type of course delivery they would prefer for a non-distance education course: ITV, Internet only, classroom only, or classroom with Internet supplements.
Chart 1 clearly shows that learners prefer some kind of Internet component to a course even when it is not a distance education course. Sophisticated computer users all preferred Internet-mediated learning over classroom based learning, and all those with beginning computer skills preferred classroom only. This shows that the level of computer skills of a social work student significantly affects their preference for classroom or Internet based learning10 . The Pearson Chi-square test shows that the relationship between computer skills and delivery preferences for non-distance education course is significant (chi-square = 70.112; CV = 32.9 at .1% level of significance). Overall 15% preferred Internet only delivery, 5% preferred classroom only delivery and a large majority (80%) preferred a mix.
Chart 1 Delivery preference for non-distance social work learning
This has important implications for social work educators whether they teach at a distance or not. The data illustrates that many students clearly want an Internet component for their courses, and that very few prefer to learn by classroom instruction only. In fact, more students would preferto learn only via the Internet than would want to learn only via the classroom, and this is for courses that are local campus delivered courses. As the above chart shows, if your class is composed entirely of beginner computer users then classroom only instruction may be appropriate, but if you have a group composed of novice, average or experienced computer users an Internet componentshould be considered.
Age of learners also has an important bearing on delivery preference for non-distance social work education. All those who preferred classroom only were over age 36. All those who preferred Internet only were age 26-35, and those between 19 and 34 preferred Internet and classroom combined. Several conclusions can be surmised from this data. First, learners who wanted a classroom only option tended to be older with low computer skills. Second, young learners (19-34) did not opt for the Internet only choice - instead it was the next age group (26-35) that chose the Internet only option. This may be due to the desire for starting students to have more direct contact with teachers while those in the age group 26-35 would tend to be more inclined to choose convenience, due to family and other obligations. Further, no Aboriginal participants preferred classroom only for non-distance education (25% preferred Internet only and 75% combination).
Learners were also asked their preference for distance education social work learning. For this question we wanted to determine whether learners who have taken a course via ITV (television) preferred television or the Internet. Of the respondents 28 learners had taken a course via ITV. Of these 6 or 21 % preferred ITV to Internet delivery. Seventy-nine (79%) percent preferred Internet-based delivery over ITV. This is an important finding for educational institutions, such as Carleton University, that deliver numerous courses via ITV. A large majority (79%) of social work learners who have taken a course via ITV would prefer taking their courses using the Internet. As well, all those who preferred ITV were average computer users, while novice and experienced computer users tended to prefer the Internet11 . The Pearson Chi-square test shows that the relationship between computer skills and delivery preferences of ITV learners for distance education courses is significant (chi-square = 32.4 which is more than the CV at .1 % level of significance which =27.877).
Chart 2: Delivery preference of ITV learners for distance course
Chart 2 shows that while there is a direct relationship between computer skills and preferences for Internet delivery over ITV for distance education, the data does not show that it is beginner users that prefer ITV to the Internet. Relating age of the learner with preferences indicates that those who prefer the Internet over ITV are all under age 35. Interestingly all Aboriginal respondents preferred the Internet to ITV.
A series of questions dealt with the features and display of the online course. These questions were intended to assist in improving the appearance and operation of the course and yielded little of research interest. Overall, learners found the features to be easy to use, the appearance pleasing and adequate and the arrangement to be logical. Four percent (4%) found the time it took for material to display to be somewhat frustrating. This was an encouraging finding showing that the intended learners do have sufficient bandwidth. We did, however, develop the video and audio with a 28.8 baud modem in mind. Everyone found the system either very easy (64% answered 9) or somewhat easy to learn and operate.
Overall, the statistical analysis confirmed the findings in the open-ended qualitative data. Social work learners responded overwhelmingly positively to online learning in general, and to the course in particular. The data confirmed what has been found in studies examining Internet-based learning in other fields whereby students appreciated the ability to learn at their own pace12 and found the wealth of links to information on the Internet useful and informative13. All told, a clear majority of the participants (64%) would take a social work course such as this offered entirely over the Internet, and even more would recommend it to others (88%). The quantitative data also corroborates the open-ended responses by learners concerning the centrality of having Internet resources available for all courses.
This paper has, at least, illustrated to those in social work who may resist online educational technologies in social work education that learners tend to want the Internet, and they do want to learn how to work with technology. Friedman, Ward and Biagianti (1998) argue that the practice of social work will be heavily affected by technology, and Gingerich and Green (1996) go as far to say that by the year 2000 those who chose social work because it was not viewed as a technical field of practice will be faced with either learning about technology or choosing another profession. While this may be going too far, the literature does illustrate that using and learning new technology such as the Internet is important for social workers.
This study suggests that some social work students enjoy and want Internet-mediated learning whether for a distance education course or not. They also believe that it improves their learning. The finding that the learners surveyed prefer delivery with an Internet component (80%) or only through the Internet (15%) for a non-distance education course should signal to social work educators the strong aspiration by some students for Internet-mediated education delivery. For distance education courses the finding is even more resolute with 79% preferring Internet only delivery to video or television delivery. These findings were further confirmed by the responses to other questions where 88% stated that they would recommend an Internet delivered course to others or where 2/3 believed that 'a course based entirely on Internet-mediated delivery is appropriate for a social work course'. It is not surprising, however, that learning social work via the Internet is not for everyone. Twelve percent (12%) would not recommend it to someone else, and 8% found that it did not enhance their learning.
The study also confirms the design features of the tested course and may provide a model for others interested in introductory human service education via the Internet. Students found the features and navigation easy to use and appropriate. Learners also correctly emphasized the importance of careful consideration when developing online learning activities, and the significance of thoughtfully integrating online communication and collaboration into the course. In many ways, it may be the capability of the learning model used that impacts student reaction, as much as the delivery mode. In developing online learning, instructors are forced to be systematic in course construction, and this may have benefits that have little to do with the Internet.
The analysis demonstrates the value of Internet technologies in social work education and the desire of social work learners for the integration of the Internet into their learning environment. Opponents to online educational technologies, such as David Noble (1998) argue that it is university administrators and commercial partners who are pushing new technology into education against the wishes of students and faculty. He believes (although no data exists to confirm it) that “the few times students have been given a voice, they have rejected the initiatives (web technology in their courses) hands down”. This study, and others such as Sosabowski (1998) show that students not only support such initiatives for distance education, but also prefer enhanced web usage for campus-based courses. This is not to deny that educators and learners must be watchful of both the commercialization of higher education and the commodification of instruction, as Noble discusses, but it is clear that the challenge is to engage new technology to enhance pedagogical possibilities that have results of human value and quality education, and not simply to reject online educational technology. Internet-mediated social work education is probably not appropriate for all types of content, especially in social work practice where human interaction and discussion is required. It may be suitable for some of the content for these courses, but not all. As well, some communities, such as remote areas, may derive expanded benefits from Internet delivery. To answer these important questions more research is needed on student reactions to other types of online social work courses, and the benefits that could be derived by remote communities.
Bagui, S. (1998)
Reasons for increased learning using multime dia. Journal of Educational
Multimedia and Hypermedia, Vol. 7(1), pp. 3-18.
Friedman, B.D., Ward, D., & Biagianti, A. (1998) Using technology to forge
new allegiances in social work education. New Technology in the Human
Services, Vol. 11(2), pp. 13-18.
Gingerich, W.J. & Green, R.K. (1996) Information Technology: How Social
Work is Going Digital. In Raffoul, P.R. and McNeece, C. A. (Eds.) Future
Issues for Social Work Practice. Boston, MA: Allyn & Bacon.
Hmelo, C.E., Guzdial M. & Turns, J. (1998) Computer-support for collaborative
learning: learning to support student engagement. Journal of Interactive
Learning Research. Vol. 9 (2), pp. 107-129.
Kulik, J.K., Kulik, C.C., & Cohen, P.A. (1980) Effectiveness of computer-based
College teaching: A meta-analysis of findings. Review of Educational Research,
Vol. 50:4, pp.525-544.
Morgan, A. (1996). First year social work students and the impact of information
technology, New Technology in the Human Services, Vol. 9(4), pp. 2-11
Noble, D.F. (1998) Digital diploma mills: the automation of higher education.
First Monday. Vol. 3(1).
Rafferty, J. (1998) Social work and information communication technologies:
the tortoise and the hare? New Technology and the Human Services, Vol.
11 (2), pp. 10-12
Russell, T. (1998) The “No Significant Difference” Phenomenon: A comparative
Annotated Bibliography on Technology for Distance Education. North Carolina:
North Carolina State University, Office of Instructional Telecommunications.
Sciven, M. (1967) The methodology of evaluation, pp. 39-83. In Tyler,
R.W., Gagne, R.M. & Sciven, M. (Eds) Perspective in curriculum Evaluation.
Chicago, Rand McNally.
Smith, B L & MacGregor, J T (1992) What is Collaborative Learning? In
Collaborative Learning: A sourcebook for Higher Education, p10-29, A.S.
Goodsell, M, Maher R & Tinto ed., University Park, PA, National Centre
on Postsecondary Teaching
Sosabowski, M.H., Herson, K., & Lloyd, A.W. (1998) Enhancing learning
and teaching quality: integration of networked learning technologies into
undergraduate modules. Active Learning, Vol. 8.
Footnotes
1 No
books have been written and the social work or human service journals
covering technology related issues have only a few articles on education,
most of which discuss general issues or are think pieces.
2 The study was made possible with funding support from
the Office of Learning Technologies (OLT), Human Resources Development
Canada.
3
Information concerning instructional design and learning models can be
found at http://www.cudenver.edu/~mryder/itc_data/idmodels.html
4 Kulik, J.K., Kulik, C.C., & Cohen, P.A. (1980) meta-analysis
study of 59 evaluations of computer-based college teaching found that
computer-based instruction made small but significant contribution to
course achievement.
5 Russell, T. (1998) have used meta-analysis in an attempt
to bring together findings describing the effects of computers on learning
from previously published research in hundreds of journal articles. This
technique enables a statistical summary of quantitative data from research
reports; the most useful of which is said to be the effect size or measure
of impact of a particular treatment expressed as a percentage of a standard
deviation. Thomas Russell reports on 355 comparative evaluation research
reports and papers that find that there is no significant difference in
test scores or learning outcomes between classroom learners and the technology
mediated learning group. Available online at http://www2.ncsu.edu/oit/nsdsplit.htm
6 Steve Hick’s informal discussion with Aboriginal social
work educators indicates that they would prefer Internet delivery combined
with classroom discussion where necessary over student travel to campus
courses or teacher travel to communities for intense blocks of classroom
work. They generally prefer to have learners stay and study in the community,
and find bringing educators into the community for condensed segments
of instruction to be a problem.
7 Multimedia is the use of text, graphics, animation,
pictures, video and sound to present information.
8
Dual coding refers to the notion that people take in information through
all the different senses and by using voice and visuals a person is able
to more easily process the information for long term memory.
10 The Pearson Chi-square test shows that the relationship
between computer skills and delivery preferences for non-distance education
course is significant. The chi-square value was 70.112 which is more than
the CV at .1 % level of significance which =32.909.
11 The Pearson Chi-square test shows that the relationship
between computer skills and delivery preferences of ITV learners for distance
education courses is significant. The chi-square value was 32.359 which
is more than the CV at .1 % level of significance which =27.877.
12 For example, a study by the University of Saskatchewan
funded by the Office of Learning Technology found that self-pacing was
an important feature of their Internet-based math readiness course. See
University of Saskatchewan, Extension Division, Evaluation of the Internet-based
Math Readiness Course (MRC) Trail, Office of Learning Technologies, Human
Resources Development Canada. Available online at http://olt-bta.hrdc-drhc.gc.ca/publicat/MRCexece.html
13 A recent evaluation of online learning for undergraduate
pharmacy-related teaching found that the resources on the Internet provided
added value and empowered learners to take greater responsibility for
their own learning. See Sosabowski, M.H., Herson, K., & Lloyd, A.W. (1998)
Enhancing learning and teaching quality: integration of networked learning
technologies into undergraduate modules. Available online at http://www.cti.ac.uk/publ/actlea/al8.html#/contents
This tool is for a formative evaluation of an Internet-mediated course developed by Steven Hick for an introductory social work course. The following questionnaire will be completed online by volunteer participants after completing selected portions of the online course.
Questionnaire
Now that you have experienced the Introduction to Social Work and Social Welfare online course, please candidly complete this survey about the program. We are looking for both positive and negative comments about your experiences with, and opinions on, the function and usefulness of, the course. Remember, your comments are provided anonymously to ensure that you may write your honest opinions. What you say will influence future versions of this course.
Demographic Information
Age: 18 19-25 26-35
36-45 46-55 56-65 65
Sex: Male / Female
Aboriginal Ancestry YES / NO
How would you rank your computer skills? Beginner | Novice | Average |
Experienced | Sophisticated
Have you ever taken an ITV course? YES / NO
Have you ever taken an online course before? YES / NO
Email Address __________________
Part 1: Learner Satisfaction and Perception of Online Course
Please check the boxes below that most appropriately reflect your feelings about using the online course.
Question
|
Strongly Agree |
Agree |
Disagree |
Strongly disagree |
It was easy to move through the course | ||||
Online activities stimulated my learning | ||||
Multimedia (audio & video) was essential in the course | ||||
The activities in this course gave me sufficient practice and feedback | ||||
Email was an important part of the course communication? | ||||
The online course was easy to use | ||||
Online self-testing is an important aspect of the course | ||||
I would recommend that others take a course that is deliverd via the Internet | ||||
The resources on the Internet enhanced my learning |
Please check the boxes that most appropriately reflect your impressions about using the online course. If you feel an item is Not Applicable, check NA.
Overall Reactions | 1 | 2 | 3 | 4 | 5 | 6 | 7 | 8 | 9 | NA | ||
Please describe your overall reaction to the online course | Terrible | Wonderful | ||||||||||
The time it took for course material to display | Frustrating | Satisfying | ||||||||||
The mix of text, images, audio and video | Dull | Stimulating | ||||||||||
The way information was presented | Rigid | Flexible | ||||||||||
Screen | 1 | 2 | 3 | 4 | 5 | 6 | 7 | 8 | 9 | NA | ||
Characters on the computer screen | Hard to Read | Easy to Read | ||||||||||
Images and photographs | Fuzzy | Sharp | ||||||||||
Character of shapes (fonts) | Not Legible | Very Legible | ||||||||||
Screen layouts make tasks easier | Never | Always | ||||||||||
Amount of information that can be displayed on screen | Inadequate | Adequate | ||||||||||
Arrangement of information on screen | Illogical | Logical | ||||||||||
Colors used in screen design | Displeasing | Pleasing | ||||||||||
Performing an operation leads to a predictable result | Never | Always | ||||||||||
Computer responds fast enough | Never | Always | ||||||||||
Learning | 1 | 2 | 3 | 4 | 5 | 6 | 7 | 8 | 9 | NA | ||
Learning to operate the system | Difficult | Easy | ||||||||||
Getting started | Difficult | Easy | ||||||||||
Time to learn to use the system | Slow | Fast | ||||||||||
Exploration of features by trial and error | Discouraged | Encouraged | ||||||||||
Exploration of Features | Risky | Safe | ||||||||||
Discovering new features | Difficult | Easy | ||||||||||
Remembering names and use of commands | Difficult | Easy | ||||||||||
Remembering specific rules about entering commands | Difficult | Easy |