University of Massachusetts at Boston
Graduate College of Education
Critical & Creative Thinking Program
Seminar on

Evaluation of Educational Change

CrCrTh693
Spring 2006

Syllabus

Instructor: Peter Taylor, Critical & Creative Thinking Program
Email: peter.taylor@umb.edu
Phone: 617-287-7636
Office: Wheatley 2nd flr 143.09 (near Counseling & School Psychology)
Class: M 4-6.30, in McC 2-628C
Office/phone call hours: M 1.40-3.40 by sign up or by arrangement.
Course Website: http://www.faculty.umb.edu/pjt/693-06.html
General email: Emails sent to cct693@yahoogroups.com go to everyone in the course.
E-clippings: Clippings from the internet sent to cct693clips@yahoogroups.com will be archived for all to read at http://www.yahoogroups.com/group/cct693clips

CATALOG DESCRIPTION

This course covers techniques for and critical thinking about the evaluation of changes in educational practices and policies in schools, organizations, and informal contexts. Topics include quantitative and qualitative methods for design and analysis, participatory design of practices and policies, institutional learning, the wider reception or discounting of evaluations, and selected case studies, including those arising from semester-long student projects.

COURSE DESCRIPTION for Spring 2006

Theme: Cycles and epicycles of Action Research

First note that in this course "educational change" is construed broadly to include organizational change, training, and personal development, as well as curricular and school change.
This course builds on the question: "If you have good ideas how do you get others to adopt and/or adapt them?" (in other words, how do you build a "constituency" around your idea). This concern can lead you into evaluating how good the ideas actually are (with respect to objectives you formulate) so you can demonstrate this to others. It can also lead you to work with others to develop the idea so it becomes theirs as well and thus something they're invested in. In any case, the first person in your constituency is yourself!
To prepare you for such work, class activities introduce tools for group facilitation, participatory planning, reflective practice, and systematic evaluation. We do all this within a framework of "Cycles and epicycles of Action Research."

PREREQUISITES: CrCrTh601 and 602, or permission of instructor.
For CCT students, this course is best taken in your third last semester (before the Practicum and Synthesis).

ACCOMMODATIONS: Sections 504 and the Americans with Disabilities Act of 1990 offer guidelines for curriculum modifications and adaptations for students with documented disabilities. If applicable, students may obtain adaptation recommendations from the Ross Center (287-7430). The student must present these recommendations to each professor within a reasonable period, preferably by the end of the Drop/Add period.

Students are advised to retain a copy of this syllabus in personal files for use when applying for certification, licensure, or transfer credit.
This syllabus is subject to change, but workload expectations will not be increased after the semester starts. (Version 3 May. '06)

SECTIONS TO FOLLOW IN SYLLABUS

TEXTS and MATERIALS

Calhoun, E. F. (1994). How to Use Action Research in the Self-Renewing School. Alexandria, VA: ASCD.
Schmuck, R. (1997). Practical Action Research for Change. Arlington Heights, IL: Skylight. (used copies available via amazon.com)
Readings for the course consist primarily of individual articles and book chapters. Most of these can be downloaded from the Healey Library's Electronic Reserves (marked ERes on the syllabus, docutek.lib.umb.edu/ (path: Electronic reserves and Course Materials | select cct693, enter password provided by instructor) or e-journals).

Recommended: portable storage (e.g., zip disk or flash drives), synchronization & bibliographic software. (For more info see http://www.cct.umb.edu/competencies.html)

Additional materials linked to the course website include:

REQUIREMENTS

More detail about the assignments, expectations, and rationale is provided in the Notes on Teaching/Learning Interactions and Rubrics (see links above) and in handouts that will be linked to this website. (Alert the instructor if a link does not work to a handout you need.)

Written assignments and presentations (2/3 of grade)

A. Action Research assignments (four) and Evaluation Clock assignment (one).
B. Design Project: Design an Action Research Process related to a change or intervention in a specific classroom, workplace or personal teaching/learning practice, an educational policy, an educational institution, or a social policy. Your design should include how you will evaluate the existing situation, how you would facilitate the reflective and/or collaborative process in which a constituency shapes a change or intervention, and how you would evaluate the outcome with a view to expanding further the constituency for adopting/adapting the change or intervention. (If you actually carry out some of the design, that will deepen the project, but it is not required.)
A sequence of 5 assignments is required--initial description, notes on research and planning, work-in-progress presentation, complete draft report, and final (1500-2500 words) report.

Participation and contribution to the class process (1/3 of grade)

C. Building learning community through prepared participation and attendance at class meetings (=13 items)
D. Personal/Professional Development (PD) Workbook submitted for perusal before week 7 (with worksheet in week 7) & at the end of the semester (=2 items)
E. Minimum of two in-office or phone conferences on your assignments, PD workbook, and project -- one before mid-semester break; the other before week 10 (=2 items)
F. Peer commentary on another student's draft report (with copy submitted to PT or included in PD workbook)
G. Assignment Check-list maintained by student and submitted week 12
H. Process Review on the development of your work, included with your PD Workbook at end-of-semester perusal.

SCHEDULE OF CLASSES

Class 1 (1/23) Introductions
The framework of Action Research Cycles and Epicycles is introduced through a compressed example.
Critical Incident Questionnaire

Homework tasks include:
Class 2 (1/30) Action Research Session 1
Reading: Schmuck, p. vii-29
Feedback on Critical Incident Questionnaire
Questions on Syllabus and course mechanics

From ill-defined case through structured brainstorming to defining problems, tasks, and lines of cross-communication for the week ahead (handout). Continue Action Research work if time permits.
Additional reading: Greenwald, "Learning from Problems."
*A* Asmt. 1: Contribution and response to AR wiki by 2/4

Class 3 (2/6) Action Research Session 2
Pre- or post-class reading on Focused Conversations: Stanfield, 6-29.
Focused Conversation on Action Research experience to date (handout)
Work in small groups or individually, with coaching by instructor: review tasks undertaken; plan presentations (including coordinating with others); define tasks remaining, and lines of cross-communication for the week ahead; generate invitation list.

Class 4 (2/13) Action Research Session 3
*A* Asmts. 2 & 3: Oral presentation and Draft written report from each student.
Presentation to peers and guest panel (invited by Action Researchers)
Post-class reading: To reflect on your experience, start early on reading for classes 5 and 6.

2/20 No class (Presidents' Day)
Narrative: You can now examine what others have written in light of your own Action Research experience.

Class 5 (2/27) Comparing your Experience as Novice Action Researchers with the Considered Formulations from Other Sources, I
Reading: Schmuck, pages 29-146
Dialogue Process session

Class 6 (**Thursday 3/9**, place McC 2-628C) Comparing your Experience as Novice Action Researchers with the Considered Formulations from Other Sources, II
Reading: Calhoun, especially chapters 1-3
[cancelled: Guest Panel of school change action researchers]
Small group work on guidelines for small group work and on compare-contrast

*A* First conference must be completed by 3/10 to discuss Action Research experience, the course thus far, and your PD workbook (bring to conference)
*A* Submit worksheet on PD workbook and research organization (as part of participation item on PD workbooks)
*A* Schedule second conference before 4/21 to discuss your projects and use of evaluation clock

3/13 No class (Spring break)

Class 7 (3/20) Formulating informative comparisons as a basis for evaluations, I
Reading: Weiss, chapter 1.
Comparison steps (2-4) in the evaluation clock, used to analyze a clipping on the effects of a smoking ban
Critical Incident Questionnaire on course to date
Post-class reading: PT's precis of Pietro, Evaluation Sourcebook, p. 22-23 (on evaluation clock) and p. 12-17, & 21 (to provide context) (handout); Guide to the Evaluation clock (handout)

*A* Amst. 4 due: Reflection paper (500-1000 words) relating your Action Research experience to points made by at least one of the following readings
Class 8 (3/27) Strategic Participatory Planning, applied to personal course and life projects
Reading: Weissglass, "Constructivist Listening," Spencer, chaps. 5 & 7; also Review Project reports from previous semesters (online using password protected site or borrowable from PT.)
Feedback on Critical Incident Questionnaire
Supportive Listening (a variant of constructivist listening) on one's hopes/fears re: educational change
Strategic personal planning workshop (about the educational/organizational change you want to facilitate/promote) (handout)
In-class drafting of initial description of AR design project
Post-class reading: Materials on Strategic Participatory Planning from ICA Facilitators Manual

*A* Asmt. 5a. Use the comparison steps (2-4) in the evaluation clock to analyze a clipping on an evaluation or related study (chosen during class 7, handout)
Reading: Hitchcock & Hughes, Chapter 5, Designing, planning and evaluating research.

Class 9 (4/3) Formulating informative comparisons as a basis for evaluations, II
Introduction to statistical formulations of comparisons and background assumptions
Peer coaching on Evaluation clock assignment and its extension to students' projects
Additional readings: More from Patton, Weiss, Stark, or precis of Patton or Weiss
*A* Asmt due by email by 4/8: Initial Project Description (revised in response to PT's comments)

Class 10 (4/10) Participatory Action Research
Reading: CEDAC, Our Economy, Taylor, "Epilogue," 204-210, others TBA
Video segment on Myles Horton and the Highlander Center, a longterm source of Participatory Action Research

*A* Asmt. 5b due: Full evaluation clock used to analyze the chosen clipping and plan the missing pieces of the study.

4/17 No class (Patriots' Day)
*A* Asmt due by email by 4/17: Notes on Research and Planning for Individual Student Projects

Class 11 (4/24) Work-in-progress Presentations on Student Projects
Work-in-progress Presentations and peer/instructor evaluations
*A* Asmt due: Work-in-progress Presentation on Project
Titles of Projects

Class 12 (5/1) Politics and Theories of Evaluation and Educational Research
(handout)
Readings--at least one from each category:
a. Case studies of Woodhead, "When psychology," Hunt, "The dilemma," Metcalf, "Reading between the lines."
b. Action Research as an alternative to Positivist and Interpretivist approaches: Carr & Kemmis, Becoming Critical, CEDAC, Our Economy, Greenwood, "Action science and organizational learning," Taylor, "Epilogue"
"Jig-saw" discussion of readings
Additional reading: McLeod, et al., "Changing how we work," Senge et al., "Fostering communities"

*A* Assignment Check-list maintained by student, with incomplete contract if needed
*A* Asmt due: Complete Draft of Design Project (2 copies)

*A* Make comments on another student's draft, and email them to the person by 5/7.

Class 13 (5/8) Taking stock of course & of change: Where have we come & where do we go from here?
Reading: Tuecke, "Creating a wall of wonder," Cashin, "Student ratings of teaching"
[Historical/Future Scan of Educational change in its wider world context]
Visioning further personal and professional development as agents of educational change
GCOE & CCT course evaluations (see previous semesters' evaluations)
Additional reading: Stanfield, Courage to Learn, Stanfield, The Workshop Book (selections TBA)

5/15
Due before 5pm 5pm, Initial meeting for fall synthesis students (tentative)

BIBLIOGRAPHY

ERes indicates reading on electronic reserve.
# indicates additional texts on evaluation, action research, or facilitating group process (to be borrowed from the library, interlibrary loan, or instructor.
## indicates usefull readings to help in writing and revising.

Backer, T., J. Chang, A. Crawford, T. Ferraguto, D. Tioseco and N. Woodson (2002). "Case study and analysis: The Center for the Improvement of Teaching, University of Massachusetts, Boston."(PPR)
Brookfield, S. D. (1995). Becoming a Critically Reflective Teacher. San Francisco: Jossey-Bass Publishers (eRes)
Calhoun, E. F. (1994). How to Use Action Research in the Self-Renewing School. Alexandria, VA: ASCD.
Carr, W. and S. Kemmis (1986). Becoming Critical: Education, Knowledge and Action Research. Geelong: Deakin University Press., chapters 6 & 7 (up to p. 200 on eRes)
Cashin, W. E. (1990). "Student ratings of teaching: A summary of the research." Management Newsletter 4(1): 2-7. (eRes)<
CEDAC (Community Economic Development Advisory Committee) (1995). Our Economy: Our Future, Final Report. York, Ontario: City of York. (eRes)
Conlin, M. L. (2002). "The basics of writing: Process and strategies," in Patterns Plus: A Short Prose Reader with Argumentation. Boston: Houghton Mifflin, 1-11. (eRes)##
Couto, R. A. (2001). "The promise of a scholarship of engagement." The Academic Workplace 12(2): 4, 6.
Daniel, D., C. Fauske, P. Galeno and D. Mael (2001). Take Charge of Your Writing: Dicovering Writing Through Self-Assessment. Boston: Houghton Mifflin.##
Elbow, P. (1981). Writing with Power. New York: Oxford Univ. Press##
Entin, D. (2001). "Review of The Reflective Practitioner: How Professionals Think in Action." The Academic Workplace 12(2): 13, 18.
Greenwald, N. (2000). "Learning from Problems." The Science Teacher 67(April): 28-32. (eRes)
Greenwald, N. (2000). Science in Progress: Challenges in Problem-based Learning for Secondary Schools
Greenwood, D. J. and M. Levin (1998). Introduction To Action Research: Social Research For Social Change. Thousand Oaks, CA: Sage. (pp. 187-202 on eRes)
Hitchcock, G. and D. Hughes (1995). Research and the Teacher: A Qualitative Introduction to School-based Research. New York: Routledge.(pp. 77-112 on eRes)
Hunt, M. (1985). "The dilemma in the classroom: A cross-sectional survey measures the effects of segregated schooling," in Profiles of Social Research: The Scientific Study of Human Interactions. New York: Russell Sage, 51-97. (eRes)
Institute of Cultural Affairs, n.d., Facilitators Manual (excerpts on Strategic Participatory Planning). Toronto: Canadian Institute of Cultural Affairs.
Isaacs W. (1999) Dialogue and the Art of Thinking Together. New York: Currency.#
Jenkins, M. (2000). "Action learning: Taking the time it takes." Paper presented to the International Association of Facilitators, Toronto, April 27 2000.
Kanar, C. (2002). "Improving your paragraph skills," in The Confident Writer. Boston: Houghton Mifflin, 60-88.##
Madison Metropolitan School District (2001). "Classroom action research." http://www.madison.k12.wi.us/sod/car/carhomepage.html viewed 25 Jan. 03.
Madison Metropolitan School District (2001). "Classroom action research starting points." http://www.madison.k12.wi.us/sod/car/carstartingpoints.html viewed 25 Jan. 03.
McLeod, M., P. Senge and M. Wheatley (2001). "Changing how we work." Shambhala Sun(January): 29-33. (eRes)
Metcalf, S. (2002). "Reading between the lines." The Nation(Jan. 28): 18-22. (eRes)
Nelson, J. (2001). The Art of Focused Conversation for Schools. Toronto: Canadian Institute of Cultural Affairs.
Patton, M. Q. (1982). Practical Evaluation. Newbury Park, CA: Sage.#
Perelman, L., J. Paradis, E. Barrett (n.d.) The Mayfield Handbook of Technical and Scientific Writing. https://mit.imoat.net/handbook/home.htm##
Pietro, D. S. (Ed.) (1983). Evaluation Sourcebook. New York: American Council of Voluntary Agencies for Foreign Service.#
Rokovich, M. A., M. Stevens and J. Stallman (2000). "Implementing change at SJUSD: An unfinished case study." Presented to the International Association of Facilitators, Toronto, April 27 2000. (PPR)
Schmuck, R. (1997). Practical Action Research for Change. Arlington Heights, IL: Skylight.
Senge, P., N. Cambron-McCabe, T. Lucas, B. Smith, J. Dutton and A. Kleiner (2000). "Fostering communities that learn," in Schools That Learn. New York: Currency,459-465. (eRes)
Spencer, L. J. (1989). Winning Through Participation. Dubuque, Iowa: Kendall/Hunt (eRes)
Spina, S. U. (2002). "Six key principles of action research." (handout)
Stanfield, B. (Ed.) (1997). The Art of Focused Conversation. Toronto: Canadian Institute of Cultural Affairs. (pp.30-37 on eRes)
Stanfield, B. (2000). The Courage To Lead: Transform Self, Transform Society. Gabriola Island BC: New Society Publishers.#
Stanfield, R. B. (2002). The Workshop Book: From Individual Creativity to Group Action. Toronto: Canadian Institute of Cultural Affairs.#
Stark, J. S. and A. Thomas (Eds.) (1994). Assessment and Program Evaluation. Needham Heights, MA: Simon & Schuster. (#, in Healey stacks)
Taylor, P. J. (2005). "Epilogue," in Unruly Complexity: Ecology, Interpretation, Engagement. Chicago, University of Chicago Press: 203-213. (PPR)
Tuecke, P. (2000). "Creating a wall of wonder with the TOP environmental scan." International Association of Facilitators, Toronto, Canada, April 27 - 30 (iaf-world.org/iaf2000/Tuecke.PDF).
Turabian, K. L. (1996). A Manual For Writers of Term papers, Theses, and Disertations. Chicago: Univ. of Chicago Press (in Healey reference section)##
Weiss, C. H. (1998). Evaluation. Upper Saddle River, NJ: Prentice Hall.#
Weissglass, J. (1990). "Constructivist listening for empowerment and change." The Educational Forum 54(4): 351-370. (eRes)
Winter, R. (1989). Learning from Experience: Principles and Practice in Action Research London: Falmer.#
Woodhead, M. (1988). "When psychology informs public policy." American Psychologist 43(6): 443-454. (eRes)