University of Massachusetts at Boston
Graduate College of Education
Critical & Creative Thinking Program
Seminar on

Evaluation of Educational Change

CrCrTh693
Spring 2007

Syllabus

Instructor: Peter Taylor, Critical & Creative Thinking Program
Email: peter.taylor@umb.edu
Phone: 617-287-7636
Office: Wheatley 2nd flr 143.09 (near Counseling & School Psychology)
Class: M 4-6.30, in McC 2-628C (facing out into the main corridor)
Office/phone call hours: M 2-3.40 & 6.40-7 by sign up, or by arrangement.
Course Website: http://www.faculty.umb.edu/pjt/693-07.html
Course wiki: http://cct.wikispaces.com/693
General email: Emails sent to cct693@lists.umb.edu go to everyone in the course.
E-clippings: Clippings from the internet sent to cct693clips@yahoogroups.com will be archived for all to read at http://www.yahoogroups.com/group/cct693clips

CATALOG DESCRIPTION

This course covers techniques for and critical thinking about the evaluation of changes in educational practices and policies in schools, organizations, and informal contexts. Topics include quantitative and qualitative methods for design and analysis, participatory design of practices and policies, institutional learning, the wider reception or discounting of evaluations, and selected case studies, including those arising from semester-long student projects.

COURSE DESCRIPTION for Spring 2007

Theme: Cycles and epicycles of Action Research

First note that in this course "educational change" is construed broadly to include organizational change, training, and personal development, as well as curricular and school change.
This course builds on the question: "If you have good ideas how do you get others to adopt and/or adapt them?" (in other words, how do you build a "constituency" around your idea). This concern can lead you into evaluating how good the ideas actually are (with respect to objectives you formulate) so you can demonstrate this to others. It can also lead you to work with others to develop the idea so it becomes theirs as well and thus something they're invested in. In any case, the first person in your constituency is yourself!
To prepare you for such work, class activities introduce tools for group facilitation, participatory planning, reflective practice, and systematic evaluation. We do all this within a framework of "Cycles and epicycles of Action Research."

PREREQUISITES: For CCT students, this course is best taken after the Practicum: Processes of Research and Engagement.

ACCOMMODATIONS: Sections 504 and the Americans with Disabilities Act of 1990 offer guidelines for curriculum modifications and adaptations for students with documented disabilities. If applicable, students may obtain adaptation recommendations from the Ross Center (287-7430). The student must present these recommendations to each professor within a reasonable period, preferably by the end of the Drop/Add period.

Students are advised to retain a copy of this syllabus in personal files for use when applying for certification, licensure, or transfer credit.
This syllabus is subject to change, but workload expectations will not be increased after the semester starts. (Version 5 April '07, with incomplete links)

SECTIONS TO FOLLOW IN SYLLABUS

Texts and Materials
Requirements
Schedule of Classes and Preparation
Reports from Students' Projects
Bibliography

TEXTS and MATERIALS

Calhoun, E. F. (1994). How to Use Action Research in the Self-Renewing School. Alexandria, VA: ASCD.
Schmuck, R. (either 1997 or 2006). Practical Action Research for Change. Arlington Heights, IL: Skylight. (Used copies of old editions available via amazon.com)
Readings for the course consist primarily of individual articles and book chapters. Most of these can be downloaded from the Healey Library's Electronic Reserves (marked ERes on the syllabus, docutek.lib.umb.edu/ (path: Electronic reserves and Course Materials | select cct693, enter password provided by instructor) or e-journals).

Recommended: portable storage (e.g., flash drives), synchronization & bibliographic software. (For more info see http://www.cct.umb.edu/competencies.html)

Additional materials linked to the course website include:

Notes on Teaching/Learning Interactions (including guidelines for assignments and links to previous students' work)
Rubrics (for course as a whole & for individual assignments), including Objectives for Thoughtful and Responsive Educators
Handouts, some non-copyrighted Readings and other Resources, and examples of student reports from previous classes. (If link doesn't work for a handout that you need, notify the instructor.)

REQUIREMENTS

More detail about the assignments, expectations, and rationale is provided in the Notes on Teaching/Learning Interactions and Rubrics (see links above) and in handouts that will be linked to this website. (Alert the instructor if a link does not work to a handout you need.)

Written assignments and presentations (2/3 of grade)

A. Action Research written assignments (eight)
B. Action Research work-in-progress presentations (two)
Design an Action Research Process related to a change or intervention in a specific classroom, workplace or personal teaching/learning practice, an educational policy, an educational institution, or a social policy. Your design should include how you will evaluate the existing situation, how you would facilitate the reflective and/or collaborative process in which a constituency shapes a change or intervention, and how you would evaluate the outcome with a view to expanding further the constituency for adopting/adapting the change or intervention. (If you actually carry out some of the design, that will deepen the project, but it is not required.)
A sequence of 10 assignments is required-reflection on class demonstration action research, initial description (based on strategic personal planning), KAQ, evaluation clock, work-in-progress presentation, notes on research and planning, narrative outline, second work-in-progress presentation, complete draft report, and final (1500-2500 words) report.

Participation and contribution to the class process (1/3 of grade)

C. Building learning community through prepared participation and attendance at class meetings (=13 items)
D. Personal/Professional Development (PD) Workbook submitted for perusal before end of week 6 (with worksheet in week 6) & at the end of the semester (=2 items)
E. Minimum of two in-office or phone conferences on your assignments, PD workbook, and project -- one before mid-semester break; the other before week 10 (=2 items)
F. Peer commentary on another student's draft report (with copy submitted to PT or included in PD workbook)
G. Assignment Check-list maintained by student and submitted week 12
H. Process Review on the development of your work, included with your PD Workbook at end-of-semester perusal.

SCHEDULE OF CLASSES

Class 1 (1/29) Introduction to Action Research Cycles and Epicycles, I
The framework of Action Research Cycles and Epicycles is introduced through a compressed example during this class and the next.
Critical Incident Questionnaire

Homework tasks include: review the syllabus and overview, get set-up to use the internet and computers, download Notes on Teaching/Learning Interactions (including guidelines for assignments and links to previous students' work) and Rubrics (for course as a whole & for individual assignments), peruse vision charts and evaluations from previous semesters, begin your PD workbook, and sign up for first conference (to which you should bring your PD workbook).

Class 2 (2/5) Introduction to Action Research Cycles and Epicycles, II
Reading: Schmuck, 1997, p. vii-29; 2006, p. ix-29
Pre- or post-class reading on Focused Conversations: Stanfield, 6-29.

Feedback on Critical Incident Questionnaire
Questions on Syllabus and course mechanics
Continuation of activity from class 1
Focused Conversation on compressed Action Research experience to date (handout)

Class 3 (2/12) Strategic Participatory Planning, applied to personal course and life projects
Reading: Final project by alum of the course (Jan Coe; from password protected site - see below), CEDAC, Our Economy, Taylor, "Epilogue," 204-210, Schmuck on "cooperative" action research, Spencer, chaps. 5 & 7, Weissglass, "Constructivist Listening," also Review Project reports from previous semesters (online using password protected site.)

Interview an alum of the course about their experience developing an Action Research process.
Supportive Listening (a variant of constructivist listening) on one's hopes/fears re: educational change
Strategic personal planning workshop (about the educational/organizational change you want to facilitate/promote) (handout,pdf)
In-class drafting of initial description of AR design project

Post-class reading: (for those interested in Strategic Participatory Planning) Materials from ICA Facilitators Manual (PPR)

*A* Asmt. 1: 1st Reflection on your Experience as Novice Action Researchers in relation to the Considered Formulations from Other Sources (Schmuck)

2/19 No class (Presidents' Day)

Class 4 (2/26) Opening up problems and focusing in on inquiry
Reading: Greenwald, "Learning from Problems."
Use of KAQ framework.

*A* Asmt. 2: Initial Project Description (revised in response to PT's comments by email on in-class draft)

Class 5 (3/5) Formulating informative comparisons as a basis for evaluations, I
Comparison steps (2-4) in the evaluation clock, used to analyze a clipping on the effects of a smoking ban, then to design evaluation as part of students' projects
Reading: Clipping (PPR) Guide to the Evaluation clock (handout (before and/or after class)

*A* Asmt. 3: KAQ assignment

Class 6 (3/12) Formulating informative comparisons as a basis for evaluations, II
Introduction to statistical formulations of comparisons and background assumptions
Peer coaching on Evaluation clock assignment and its extension to students' projects

*A* Asmt. 4a. Use the comparison steps (2-4) in the evaluation clock to design evaluation as part of your project.
*A* First conference must be completed by 3/15 to discuss your Action Research ideas, the course thus far, and your PD workbook (bring to conference)
*A* Submit worksheet on PD workbook and research organization (as part of participation item on PD workbooks)
*A* Schedule second conference before 4/25 to discuss your projects and use of evaluation clock

3/19 No class (Spring break)

Class 7 (3/26) Work-in-progress presentations, I
*A* Asmt 5: Work-in-progress Presentation I on Project
*A* Asmt 6: Notes on Research and Planning for Student Projects (to be pasted onto the wiki by 3/25 at latest so everyone can see how other students are approaching their projects)

Class 8 (4/2) Reflection on your Experience as Novice Action Researchers with the Considered Formulations from Other Sources
Readings: Schmuck, pages 29-146, Calhoun, How to Use Action Research (especially chapters 1-3), Weiss, chapter 1, and (optional) Weiss, chapters 2 &4.

Dialogue Process session
Critical Incident Questionnaire II on course to date

*A* Asmt. 4b due: Use the full evaluation clock to design the evaluation part of your project.

Class 9 (4/9) Reflection on your Experience as Novice Action Researchers with the Considered Formulations from Other Sources, II
Readings: At least three from Hitchcock & Hughes, Chap. 3, "Access, ethics, and objectivity," Chapter 5, "Designing, planning and evaluating Research"; Greenwood & Levin, Chaps. 8 & 11, "Action research cases," & "Action science and organizational learning"; Rokovich, et al., "Implementing change"; Jenkins, "Action learning"; CEDAC, Our Economy; Greenwald, Science in Progress; Madison Metropolitan School District, "Classroom action research" (and linked pages), study of CIT
Feedback on Critical Incident Questionnaire II
Small group work on guidelines for a) small group work and b) Action Researchers
Video segment on Myles Horton and the Highlander Center, a longterm source of educational and social change

4/16 No class (Patriots' Day)

Class 10 (4/23) Politics and Theories of Evaluation and Educational Research, I: Case studies
(handout)
Readings--at least one of:
Woodhead, "When psychology," Hunt, "The dilemma," Metcalf, "Reading between the lines."

"Jig-saw" digestion and discussion of readings

*A* Asmt 7: Narrative Outline for Project Report

Class 11 (4/30) Work-in-progress presentations, II
*A* Asmt 8: Work-in-progress Presentation II on Project

Titles of Projects
Julie Barrett, "Schedule Migration and Problem Based Learning at Massachusetts College of Art"
Elizabeth Burch & Renee Plouffe, "To foster a more inclusive school environment for children with special needs"
Virginia De La Garza, "Improving The Effectiveness of a Faculty Training Program at a University in Mexico"
Elliot Frank, "Evaluation of Metacognition in Biology"
David Frias, "How Can Lower Level Employees Affect Upper Level Management Decision Making?"
Scott Goddess, "An inquiry into establishing productive, sustainable, coaching partnerships through making connections "
Marnie Jain, "Separating Myself from My Art and Becoming the Artist I Want to Be"
Jane Lachance, "My Use of Story as a Facilitating Tool in a Small Heatlh Care Workers Group "
Kathleen Leavitt, "Supporting Staff who are without a Nurse in Charge"
Michael Lihon, "Action research to continuously motivate participation, of production workers, on changes to support process improvement "
Elizabeth Naylor, "Action Research to develop curriculum for teachers to become Ordinary Inspirers and encourage their students to be Moral Learners"
John Quirk, "Teachable Leadership and the Junior Class: A Plan to Develop Effective Student Leadership through Regenerative, Reflective Action Research Practices"
Joshua Reyes, " Electricians Online Review: A Community of Learners"

Class 12 (5/7) Politics and Theories of Evaluation and Educational Research, II: Action Research as an alternative to Positivist and Interpretivist approaches
Readings--at least one of:
Carr & Kemmis, Becoming Critical, CEDAC, Our Economy, Greenwood, "Action science and organizational learning," Taylor, "Epilogue"
Additional reading: McLeod, et al., "Changing how we work," Senge et al., "Fostering communities"

Dialogue Process session on theory and practice

*A* Assignment Check-list maintained by student, with incomplete contract if needed
*A* Asmt due: Complete Draft of Design Project (2 copies)

*A* Make comments on another student's draft, and email them to the person by 5/12.

Class 13 (5/14) Taking stock of course & of change: Where have we come & where do we go from here?
(handout)
Reading: Tuecke, "Creating a wall of wonder," Cashin, "Student ratings of teaching"
Historical/Future Scan of Educational change in its wider world context
GCOE & CCT course evaluations (see previous semesters' evaluations)
Additional reading: Stanfield, Courage to Learn, Stanfield, The Workshop Book (selections TBA)

5/21
Due before 5pm *A* PD workbook submitted for perusal, to be picked up after 5/28 from Department of Curriculum & Instruction office, W-2-93
*A* Final Project report (printed copy as well as by email attachment or on disk) and Process Review 5pm, Initial meeting for fall synthesis students (tentative)

BIBLIOGRAPHY

ERes indicates reading on electronic reserve.
# indicates additional texts on evaluation, action research, or facilitating group process (to be borrowed from the library, interlibrary loan, or instructor.
## indicates usefull readings to help in writing and revising.

Backer, T., J. Chang, A. Crawford, T. Ferraguto, D. Tioseco and N. Woodson (2002). "Case study and analysis: The Center for the Improvement of Teaching, University of Massachusetts, Boston."(PPR)
Brookfield, S. D. (1995). Becoming a Critically Reflective Teacher. San Francisco: Jossey-Bass Publishers (eRes)
Calhoun, E. F. (1994). How to Use Action Research in the Self-Renewing School. Alexandria, VA: ASCD.
Carr, W. and S. Kemmis (1986). Becoming Critical: Education, Knowledge and Action Research. Geelong: Deakin University Press., chapters 6 & 7 (up to p. 200 on eRes)
Cashin, W. E. (1990). "Student ratings of teaching: A summary of the research." Management Newsletter 4(1): 2-7. (eRes)
CEDAC (Community Economic Development Advisory Committee) (1995). Our Economy: Our Future, Final Report. York, Ontario: City of York. (eRes)
Conlin, M. L. (2002). "The basics of writing: Process and strategies," in Patterns Plus: A Short Prose Reader with Argumentation. Boston: Houghton Mifflin, 1-11. (eRes)##
Couto, R. A. (2001). "The promise of a scholarship of engagement." The Academic Workplace 12(2): 4, 6.
Daniel, D., C. Fauske, P. Galeno and D. Mael (2001). Take Charge of Your Writing: Dicovering Writing Through Self-Assessment. Boston: Houghton Mifflin.##
Elbow, P. (1981). Writing with Power. New York: Oxford Univ. Press##
Entin, D. (2001). "Review of The Reflective Practitioner: How Professionals Think in Action." The Academic Workplace 12(2): 13, 18.
Greenwald, N. (2000). "Learning from Problems." The Science Teacher 67(April): 28-32. (eRes)
Greenwald, N. (2000). Science in Progress: Challenges in Problem-based Learning for Secondary Schools
Greenwood, D. J. and M. Levin (1998). Introduction To Action Research: Social Research For Social Change. Thousand Oaks, CA: Sage. (pp. 187-202 on eRes)
Hitchcock, G. and D. Hughes (1995). Research and the Teacher: A Qualitative Introduction to School-based Research. New York: Routledge.(pp. 77-112 on eRes)
Hunt, M. (1985). "The dilemma in the classroom: A cross-sectional survey measures the effects of segregated schooling," in Profiles of Social Research: The Scientific Study of Human Interactions. New York: Russell Sage,51-97. (eRes)
Institute of Cultural Affairs, n.d., Facilitators Manual (excerpts on Strategic Participatory Planning). Toronto: Canadian Institute of Cultural Affairs.(PPR)
Isaacs W. (1999) Dialogue and the Art of Thinking Together. New York: Currency.#
Jenkins, M. (2000). "Action learning: Taking the time it takes." Paper presented to the International Association of Facilitators, Toronto, April 27 2000.
Kanar, C. (2002). "Improving your paragraph skills," in The Confident Writer. Boston: Houghton Mifflin, 60-88.##
Madison Metropolitan School District (2001). "Classroom action research." http://www.madison.k12.wi.us/sod/car/carhomepage.html viewed 25 Jan. 03.
Madison Metropolitan School District (2001). "Classroom action research starting points." http://www.madison.k12.wi.us/sod/car/carstartingpoints.html viewed 25 Jan. 03.
McLeod, M., P. Senge and M. Wheatley (2001). "Changing how we work." Shambhala Sun(January): 29-33. (eRes)
Metcalf, S. (2002). "Reading between the lines." The Nation(Jan. 28): 18-22. (eRes)
Nelson, J. (2001). The Art of Focused Conversation for Schools. Toronto: Canadian Institute of Cultural Affairs.
Patton, M. Q. (1982). Practical Evaluation. Newbury Park, CA: Sage.#
Perelman, L., J. Paradis, E. Barrett (n.d.) The Mayfield Handbook of Technical and Scientific Writing. https://mit.imoat.net/handbook/home.htm##
Pietro, D. S. (Ed.) (1983). Evaluation Sourcebook. New York: American Council of Voluntary Agencies for Foreign Service.#
Rokovich, M. A., M. Stevens and J. Stallman (2000). "Implementing change at SJUSD: An unfinished case study." Presented to the International Association of Facilitators, Toronto, April 27 2000. (PPR)
Schmuck, R. (1997). Practical Action Research for Change. Arlington Heights, IL: Skylight.
Senge, P., N. Cambron-McCabe, T. Lucas, B. Smith, J. Dutton and A. Kleiner (2000). "Fostering communities that learn," in Schools That Learn. New York: Currency,459-465. (eRes)
Spencer, L. J. (1989). Winning Through Participation. Dubuque, Iowa: Kendall/Hunt (eRes)
Spina, S. U. (2002). "Six key principles of action research." (handout)
Stanfield, B. (Ed.) (1997). The Art of Focused Conversation. Toronto: Canadian Institute of Cultural Affairs. (pp.30-37 on eRes)
Stanfield, B. (2000). The Courage To Lead: Transform Self, Transform Society. Gabriola Island BC: New Society Publishers.#
Stanfield, R. B. (2002). The Workshop Book: From Individual Creativity to Group Action. Toronto: Canadian Institute of Cultural Affairs.#
Stark, J. S. and A. Thomas (Eds.) (1994). Assessment and Program Evaluation. Needham Heights, MA: Simon & Schuster. (#, in Healey stacks)
Taylor, P. J. (2005). "Epilogue," in Unruly Complexity: Ecology, Interpretation, Engagement. Chicago, University of Chicago Press: 203-213. (PPR)
Tuecke, P. (2000). "Creating a wall of wonder with the TOP environmental scan." International Association of Facilitators, Toronto, Canada, April 27 - 30 (iaf-world.org/iaf2000/Tuecke.PDF).
Turabian, K. L. (1996). A Manual For Writers of Term papers, Theses, and Disertations. Chicago: Univ. of Chicago Press (in Healey reference section)##
Weiss, C. H. (1998). Evaluation. Upper Saddle River, NJ: Prentice Hall.#
Weissglass, J. (1990). "Constructivist listening for empowerment and change." The Educational Forum 54(4): 351-370. (eRes)
Winter, R. (1989). Learning from Experience: Principles and Practice in Action Research London: Falmer.#
Woodhead, M. (1988). "When psychology informs public policy." American Psychologist 43(6): 443-454. (eRes)