Week 3: Evaluating Programs & Human Performance Technology

• Chapters in Section III discuss evaluation in instructional design and provide you with three evaluation models: CIPP, Five-Domain, and Kirkpatrick’s Training Evaluation model. Search for at least two other models used for evaluation and summarize these models. Describe how you would use them to evaluate your instruction.

Brinkerhoff’s Success Case Method deals with having to determine what works and what doesn’t work or what needs to be fixed in instruction. According to its founder, Robert Brinkerhoff, the success case method (SCM) “is a quick and simple process that combines analysis of extreme groups with case study and storytelling…to find out how well some organizational initiative is working (Reiser & Dempsey, 2012). In order to determine what works, there would have to be a comparison between case studies that are similar that were successful and the ones that were not successful. There is a five step process in Brinkerhoff’s Success Case Method. In step 1, you must work with stakeholders in order to determine what program is going to be observed and evaluated and what approach is going to be taken in order to make a plan a successful case study. An important thing in this step is coming up with a budget and time frame for the study. Everyone has to come to an agreement about everything in step 1 in order to be successful with the rest of the steps. In step 2, a visual impact model will be made that points out the goals, objectives, impacts, and outcomes that are expected along the way. In the 3^rd step, a research survey will be conducted. The survey will consist of the best and worst cases. Data from the surveys will be collected and analyzed to see what the surveyors had to say about some of the different cases that they found to be successful and unsuccessful for them. The 4^th step is to set up phone interviews with multiple success cases and a few unsuccessful cases. The purpose of the 4^th step is to gain detailed information necessary for documenting, with empirical evidence, the success case stories (Reiser & Dempsey, 2012). During the interviews, it is best to make note of what was used that worked, what outcomes happened, what was the value, what other factors helped, and any suggestions. The last step is to write out what all information was found from the evaluations, data, and surveys.

Patton’s Utilization-Focused Evaluation (U-FE) was discovered by Michael Patton in the 1970s. The purpose of U-FE is to ensure that evaluation processes and findings get used. U-FE is “evaluation done for and with specific intended users for specific, intended uses (Reiser & Dempsey, 2012). There are nine steps that should be used in the process and if the evaluation is not used with all the steps, then it is useless. The first step is to conduct a readiness assessment. Second, identify the primary intended users and develop a working relationship with them. Third, conduct a situational analysis. Fourth, identify the primary intended uses. Fifth, focus on the evaluation. Sixth, design the evaluation. Seventh, collect, analyze, and interpret the evaluation data. Eighth, continually facilitate evaluation use. Lastly, the ninth step is to conduct a metaevaluation - an evaluation of the evaluation. An evaluation that is not used in an organization serves no purpose.

Both of the methods, in my opinion, are beneficial to the classroom because it provides everyone to be a part of the process and the implementing of the evaluation models that will be used in their environment.

• Reflect on what other questions that instructional design evaluation should address besides whether the instructional design leads to comparable amounts of learning and learner satisfaction as traditional methods. Should return on investment and management of resources also be considered in evaluations of instructional programs? What other measures should/could be considered?

If I understood this question correctly, investments and managing resources plays a big part of the evaluations of instructional programs. I think that most of the time, districts buy things that they think the schools need and don’t really give it as much thought as they probably should or they don’t look at all of the fine print before agreeing to buy something. If a program proves to be highly effective, then the district should present it to the teachers and if there is a majority of the teachers who want to implement the evaluation programs, then the district should then consider it and do the research on it. Once the district buys new programs, there should definitely be more trainings available or more training manuals. I’m speaking off what I hear teachers around me complain about. If the teachers get new programs, and don’t know how to use them, then they won’t, and then that is just wasted money. I also think that if the schools adopt new programs, they should somehow, somewhere be implemented in the curriculum to see how they are being used.

• Section IV focuses on human performance, performance support systems, knowledge management systems, and the concept of informal learning. Not all problems in learning and/or performance require an instructional one. Many times a non-instructional approach is a more appropriate solution. Identify a performance problem in your area of work and identify non-instructional solutions that may help solve the problem. Would better performance support systems, knowledge management systems, or opportunities for informal learning solve the problem?

I work as a classroom assistant in the special education department (not really the teacher) in a high school setting. I have been working on average with class sizes of no more than 12 in a class. All of the students in the classes are low. There may be a couple that are high level for a class or some that are high level but in the low level category, if that makes sense. A lot of the times, when you have a group of low students, they tend to have bad behavior. That was the case in these classes. In these classes, the teachers gave them work assignments that were easy enough for them to complete for an easy grade. Whenever there was a behavior issue, the student got a threat to go to their Assistant Principal or get a referral. In the department I work in, they didn’t really focus on things like solving the problem because of the type of students and backgrounds they had, unfortunately.

References

Reiser, R. A., & Dempsey, J. V. (2012). Defining the Field. In Trends and Issues in Instructional Design and Technology (pp. 1-34). Boston: Pearson.