Artifact
Formative and Summative Evaluation Final Project
Description of Artifact
The final project was completed by a group of four students including my self. Our self-selected task was to evaluate Moodle as a learning tool. The final project reflects both formative and summative evaluation components. The formative evaluation plan uses alpha/beta testing, questionnaires, usage tracking, and focus groups. The summative evaluation plan is similar with one exception, the plan uses pre and post tests instead of alpha/beta testing.
Explanation
This artifact demonstrates my competency with conducting needs assessment and evaluating learning systems.
The results of an evaluation phase of the ADDIE model are used to determine the effectiveness of the intervention. The evaluator determines the effectiveness by determining if performance improved to close the gap between expected performance and actual performance. Even though the evaluation phase comes at the end of the ADDIE model, true evaluation should include both formative and summative forms. A brief explanation of the difference is that formative evaluation is like a cook tasting the soup while it is being prepared. Summative evaluation happens when the prepared soup is set on the table to be tasted by the guests. Information from a formative evaluation is used to provide feedback to an instructional designer as the designer works through all phases of the ADDIE model. Summative evaluation provides feedback to the instructional designer if the final product met expectations in terms of performance.
The artifact supports this competency even though it is just a plan for an evaluation. Our group did not carry an actual evaluation out. The details of the evaluation demonstrate my ability to think through an evaluation and prepare a plan that would provide detailed data to support or refute claims of the effectiveness of Moodle as an instructional tool.
Process
We completed parts of the evaluation throughout the semester. Each part had a separate deadline. Dr. Caplow provided feedback for each assignment to be included in the final product. At the end of the semester, we combined the separate reports into one seamless document. The six assignments are listed below:
- Background
- Purposes of both formative and summative evaluation
- Audiences and decisions
- Questions
- Methods
- Introduction, sample, logistics, budget, timeline, and instrumentation
We completed the assignments by taking turns being the leader per assignment. The leader was charged with completing final edits and turning the assignment end. Often, as a team, we would each contribute parts of the assignment to the group and the leader would combine the parts into a coherent document. We used Microsoft Word to complete each written assignment. There was an attempt to use the chat function on Blackboard, but that failed because many of us would be shut out at various times. Our main form of communication was the discussion board on Blackboard.
Framework
The framework we used to construct our evaluation plan can be found in Interactive Learning Systems (Reeves and Hedburg, 2003). The template shows all thirteen steps to the evaluation plan.
Roles and Responsibilities
Each person in the class took turns being the lead of the group on different assignments. We did not have the best coordination, but somehow we got it to work, mainly because each of us kept looking for holes in our work and volunteered to fill them.
Applications and Programs
- Blackboard discussion board and chat
- Microsoft Word
Formative and Summative Evaluation
We were given full credit for each smaller report that we turned in and received near full credit on our final project.
Reflection
Formative and Summative Evaluation was taken my first semester as a Learning Systems Design and Development student. I had some experience with online learning prior to starting this degree program, but little to no experience working in a group in an online class. The discussion board module of Blackboard is not very conducive to getting to know classmates. In addition, one of our group members was halfway around the world in Korea. This plus other group members on dial up connections, made the possibility of a synchronous chat to be close to nothing. We attempted this anyway with little success.
Our group managed to complete all assignments on time. While our coordination of tasks was not very efficient or organized, our work was judged to be of superior quality. I believe that was due to each one of us trying to look for gaps and filling in as needed. If I had to do it over again, I would have used Skype and Google docs. Skype in order to have a better chat function and maybe even a conference call among group members. Google docs because as each of us edited the document, we would post copies to the drop box in Blackboard. That caused naming issues and one had to be careful which version of a document one chose to download. I think improved coordination of tasks and better collaborative word processing tools would have improved our work and lessened our stress in completing the project.