Wednesday, March 26, 2008

Week 4 - Evaluation Paradigms and Models

Quite a bit to take in this week but I found Gordon's table to be a good aid. When comparing two paradigms that I would associate with my area, I feel the following theories could apply:

Constructivist Hermeneutic Interpretivist Qualitative

This paradigm appears to be parallel with the constructivist theory that learning is constructed from previous knowledge or experience and that it is through the research of ideas that knowledge is obtained. This paradigm would lend itself I feel to the research and investigation elements required for many of the educational areas in which we operate.

Eclectic-Mixed Methods-Pragmatic Paradigm

This paradigm seems to have the advantage over some of the others in that it mixes some techniques associated with other paradigms and appears to encompass most methods to achieve an outcome. I can particularly relate to the ‘pragmatic’ aspect which reflects the fact that nothing is ever perfect in education but can be improved through design and recognition of possible flaws. Eclectic-Mixed Methods-Pragmatic Paradigm almost appears to be a parody of the other paradigms in that it does not rely on one particular belief.

Saturday, March 15, 2008

Week 3 - eLearning guidelines for Quality

Below are three of the guidelines that would be appropriate to my working area. Many eLearning issues could possibly fit into more than one guideline or even overlap depending on the category of guideline.

TD4 What makes for an effective online discussion

Despite efforts to introduce the discussion board to students at level 2, they are really only forced to use it at level 3. It has been found that when tasks or dialogue on the discussion board are optional, very little activity is evidenced. This is primarily due to the fact that no-one actually facilitates the discussion board at level 2 and as such, students are left to organise themselves. The fact that no-one appears to be monitoring the discussion board generates low motivation in this area and few students actually participate. Those that do use the discussion forum just publish meaningless comments or inappropriate material. If a discussion board facility is to be utilised in a meaningful way, then some feedback needs to be provided together with appropriate activities to keep students motivated and on track. The discussion board is a tool that enables distance learning students to make contact with and share thoughts using an asynchronous medium. However, without guidance and supervision it becomes sadly lacking.


TD10 Should students present work using online discussion tools?


Following on from above (TD4), Level 3 students are expected to submit work for summative assessment using the discussion board. This applies to one module only and is facilitated by the assessor. Two or three activities are provided with the coursework which involves a certain amount of research and prepares the student for their final submission. Students are encouraged to post their findings on the discussion forum and share ideas and findings with their group. Each intake is spaced one month apart and students for each intake are grouped accordingly. However, there is some resistance to submitting using this means. Students feel that due to the high numbers passing through this course, plagiarism between student postings will occur. Many students have already been picked up for ‘copying and pasting’ direct from the net without reference or little acknowledgement to “own work”. Comments from the assessor frequently appear on the discussion board to this end. The answer here maybe is to set specific tasks for discussion and formative assessment with the final assignment to be submitted using a more secure method thus promoting sharing without the inherent plagiarism where it counts.

TT3 Is there evidence of timely, accurate and well targeted feedback to students?

Feedback is adequate considering the volume of students. Some online assessment material is in the form of multiple choice questions which provides automatic feedback and enables the student to see immediately how many questions they have got right or wrong. Students are encouraged to attempt the quiz as many times as they wish until they achieve 100%. The only problem here sometimes is that some questions can be quite ambiguous (often for a purpose) resulting in students seeking help through other channels to achieve the answer thus taking away the independent nature of on-line learning. The turnaround of feedback used to be quite poor due to the fact that this was carried out by only one or two people and the burden was quite onerous. As such, positive feedback was not always plausible. However, this has now been addressed by the hiring of more assessors who are now able to take a more proactive approach to monitoring and feedback. The result of this is that students are now able to progress through the modules more quickly and most students are happy with the turnaround of work submitted.

TO4 Are retention rates reviewed and evaluation done on why students did not complete the course?

It is recognised that any structured course that contains several modules and/or is self paced, the highest drop-out will occur within the first few weeks. The drop-out on our level 2 courses used to be huge and this has been reduced greatly by the introduction of monitoring systems. The majority of the students can be classed as ‘mature’ and as such work and family commitments impinge on their ability to commit to the number of hours required. Students that appear to be inactive are telephoned periodically to ensure that they are still on target and intend to continue with their studies. This has been welcomed by many of the students as it demonstrates our desire for them to succeed and continue their studies whilst at the same time, giving them the opportunity to air any concerns on a personal level.

Sunday, March 9, 2008

Week 2 - The Importance of Evaluation

Sorry to be so late with this post....... but briefly:

Why is Evaluation Important to me and how do I define it?

I would define evaluation is being a means to measure effectiveness of learning. Also to check that goals are being achieved and if not, why?

In the past when I have designed and delivered courses, most of the evaluation I have carried out has been to merely measure the effectiveness of the material and how it was received by the students. The feedback obtained has enabled me to go back and tweak various aspects, thus improving on previous ideas and adapting accordingly which has proved to be invaluable. A boring but necessary topic for example, can be improved on and measured at the formative stage before it falls at the last hurdle. But that is easy when ownership of a course sits in the lap of one person, which of course is quite often not the case. Institutions are primarily seeking results and achievements and I believe evaluation is an important part of ensuring success, especially within elearning environments where important feedback to tutors is delayed, the results of which unfortunately are quite often evidenced by drop-out or non-achievement.


What Sort of Evaluations Mentioned on the Presentation Are Familiar to You Already and Why?

My experience within various organisations is that evaluation is more than often done at the end - when frustration and disappointment with a course had already set in. The key here it would seem would to have not only learned from the feedback for the future, but to put into practice some ongoing evaluation process that addresses this problem.

The model demonstrated in the presentation would be a good guideline I think. Quite often though analysing feedback only leads to published statistics which become both meaningless and historic.

Why is Quality Important in eLearning?

I have learned from previous papers on this course that transferring a face to face course to an online learning environment takes a lot of time and effort. Attempting to 'sell' the benefits of elearning can be difficult and often met with resistance from many. Ensuring quality within elearning would overcome some of the difficulties and promote this type of learning for the future. Development time put aside for this type of learning should I feel, include evaluation as well as testing of resources.