University of Massachusetts at Boston
Graduate College of Education
Critical & Creative Thinking Program
Seminar on

Evaluation of Educational Change

CrCrTh693
Spring 2003

Syllabus

Instructor: Peter Taylor, Critical & Creative Thinking Program
Email: peter.taylor@umb.edu
Phone: 617-287-7636
Office: Wheatley 2nd flr 143.09 (near Counseling & School Psychology)
Class: M 4-6.30, McC 4-272
Office/phone call hours: M 1.30-3.30, or by arrangement
Email office hours: M & Th 7.30-9am
Course Website: http://www.faculty.umb.edu/peter_taylor/693-03.html
General email: Emails sent to cct693@yahoogroups.com go to everyone in the course.
E-clippings: Clippings from the internet sent to cct693clips@yahoogroups.com will be archived for all to read at http://www.yahoogroups.com/group/cct693clips

CATALOG DESCRIPTION

This course covers techniques for and critical thinking about the evaluation of changes in educational practices and policies in schools, organizations, and informal contexts. Topics include quantitative and qualitative methods for design and analysis, participatory design of practices and policies, institutional learning, the wider reception or discounting of evaluations, and selected case studies, including those arising from semester-long student projects.

COURSE DESCRIPTION for Spring 2003

Theme: Facilitation and Evaluation towards Educational Change*

*In this course educational change is construed broadly to include organizational change, training, and personal development, as well as curricular and school change.
One side of the course explores an Action Research approach to formulating possible educational innovations and building a constituency for them. Activities introduce tools for group facilitation, participatory planning, and reflective practice. Whereas this exploratory side of the course "opens up and out," the other side of the course "focuses in" on tight evaluations, that is, on a) demonstrating the effectiveness of specific changes in educational practices and policies in schools, organizations, and informal contexts (so as to help promote changes); and b) identifying needs in such settings (so as to design the changes proposed).
These two sides of course and the tension between the them are further explored through student projects and case studies, which also highlight the wider political reception or discounting of evaluations and educational change initiatives.

PREREQUISITES: CrCrTh601 and 602, or permission of instructor.
For CCT students, this course is best taken in your third last semester (before the Practicum and Synthesis).

ACCOMMODATIONS: Sections 504 and the Americans with Disabilities Act of 1990 offer guidelines for curriculum modifications and adaptations for students with documented disabilities. If applicable, students may obtain adaptation recommendations from the Ross Center (287-7430). The student must present these recommendations to each professor within a reasonable period, preferably by the end of the Drop/Add period.

Students are advised to retain a copy of this syllabus in personal files for use when applying for certification, licensure, or transfer credit.
This syllabus is subject to change, but workload expectations will not be increased after the semester starts. (Version 26 Jan '03)

SECTIONS TO FOLLOW IN SYLLABUS

TEXTS and MATERIALS

Calhoun, E. F. (1994). How to Use Action Research in the Self-Renewing School. Alexandria, VA: ASCD.
Schmuck, R. (1997). Practical Action Research for Change. Arlington Heights, IL: Skylight.
A set of readings are available on reserve for personal photocopying.
Books and additional readings in the bibliography are available on reserve. (Arrange time in your schedule to read or photocopy relevant selections in the Healey Library.)
Recommended: ZIP disk & drive and synchronization & bibliographic software. (For more info see http://www.cct.umb.edu/competencies.html)

Additional materials linked to the course website include:

REQUIREMENTS

More detail about the assignments, expectations, and rationale is provided in the Notes on Teaching/Learning Interactions and Rubrics handouts.

Written assignments and presentations (2/3 of grade)

A. Action Research assignments (four) and Evaluation Clock assignment (one).
B. Design Project: Design EITHER an evaluation of a change or intervention in a specific classroom, workplace or personal teaching/learning practice, an educational policy, an educational institution, or a social policy OR your facilitation of a reflective and/or collaborative process to shape such a change or intervention. A sequence of 5 assignments is required--initial description, notes on research and planning, work-in-progress presentation, complete draft report, and final (1500-2500 words) report.

Participation and contribution to the class process (1/3 of grade)

C. Building learning community through prepared participation and attendance at class meetings (=13 items)
D. Personal/Professional Development (PD) Workbook submitted for perusal before week 7 (with worksheet in week 7) & at the end of the semester (=2 items)
E. Minimum of two in-office or phone conferences on your assignments, PD workbook, and project -- one before mid-semester break; the other before week 10 (=2 items)
F. Peer commentary on another student's draft report (with copy submitted to PT or included in PD workbook)
G. Assignment Check-list maintained by student and submitted week 12
H. Process Review on the development of your work, included with your PD Workbook at end-of-semester perusal.

SCHEDULE OF CLASSES

Narrative: A quick overview of the course is provided in week 1, but it is assumed that this will make most sense if you directly experience the issues raised. So from week 2 you are thrown right into a month-long unit in which you explore what Action Research entails by working on student-defined problems arising from a issue in organizational/educational change. Each week also introduces an other approach to group facilitation.

Class 1 (1/27) Introductions
--to the need for evaluation (incl. activity digesting a clipping)
--to the idea of Action Research, incl. AR spiral and the ideal of Participatory Action Research
--to the general experience of the course, through interviewing an alum (tentative)
--to Action Research unit on "Enhancing diversity in the CCT Program"
--to formative evaluation, through an end-of-class Critical Incident Questionnaire

Homework tasks include: review the syllabus and overview, get set-up to use the internet and computers, download Notes on Teaching/Learning Interactions, peruse vision charts and evaluations from previous semesters, begin your PD workbook, sign up for first conference, warm-up for Action Research sessions (see handout).

Class 2 (2/3) Action Research Session 1
Reading: Schmuck, p. vii-29
Feedback on Critical Incident Questionnaire
Questions on Syllabus and course mechanics
From ill-defined case through structured brainstorming to defining problems, tasks, and lines of cross-communication for the week ahead (handout)
Additional reading: Greenwald, "Learning from Problems."

Class 3 (2/10) Action Research Session 2
Reading: Taylor, Overall lessons learned from Action Research unit in 2002 (handout)
Introduction to Co-operative Group work, including discussion of video and modeling of small group activity (procedures)
Small group work: review tasks undertaken; reformulate direction and tasks; adjust the people to communicate with and check-in for the two weeks ahead; verbal preliminary reports to the whole class.
Post-class readings: Notes on facilitating collaboration (handout)

2/17 No class (Presidents' Day)
*A* Asmt. 1: Email PT by 2/18 a 250-500 word report on task(s) you have taken on and how they relate to the specific tasks others who you are communicating with have taken on.

Class 4 (2/24) Action Research Session 3
Pre- or post-class reading on Focused Conversations: Stanfield, 6-29.
Focused Conversation on Action Research experience to date (handout)
Small group work: review tasks undertaken; plan presentations (including co-ordinating with others); define tasks remaining

Class 5 (3/3) Presentations by Action Researchers
Presentation to peers and guest panel (invited by Action Researchers)
*A* Asmts. 2 & 3: Oral presentation and Draft written report from each student.
Post-class reading: To reflect on your experience, start early on reading for class 7. (See PT's lessons learned, Action Research Cycles)

Narrative: You can now examine what others have written in light of your own Action Research experience. While you are doing this (in preparation for class 7 & asmt. 5), we shift to the focusing in side of the course through the tool/discipline of the evaluation clock.

Class 6 (3/10) Formulating informative comparisons as a basis for evaluations
Reading: Introductory/overview sections from either Patton, Weiss, Stark, or precis of Patton or Weiss until you get a feel for the contrast between Action Research and evaluation.
Introduction to statistical formulations of comparisons and background assumptions
Comparison steps (2-4) in the evaluation clock, used to analyze a clipping on the effects of a smoking ban
Post-class reading: PT's precis of Pietro, Evaluation Sourcebook, p. 22-23 (on evaluation clock) and p. 12-17, & 21 (to provide context) (handout); Guide to the Evaluation clock (handout)

*A* First conference must be completed by 3/14 to discuss Action Research experience, the course thus far, and your PD workbook (bring to conference)
*A* Schedule second conference before 4/21 to discuss your projects and use of evaluation clock

3/17 No class (Spring break)

*A* Asmt. 4a due 3/18 by email (or by mail to 41 Cornell St., Arlington, MA 02474). Use the comparison steps (2-4) in the evaluation clock to analyze a clipping on an evaluation or related study (chosen during class, handout)

Class 7 (3/24) Comparing your Experience as Novice Action Researchers with the Considered Formulations from Other Sources
Reading: Selection from Calhoun, How to Use Action Research; Hitchcock & Hughes, Chap. 3, "Access, ethics, and objectivity"; Greenwood & Levin, Chaps. 8 & 11, "Action research cases," & "Action science and organizational learning"; Rokovich, et al., "Implementing change"; Jenkins, "Action learning"; CEDAC, Our Economy; Greenwald, Science in Progress; Madison Metropolitan School District, "Classroom action research" (and linked pages), study of CIT
Small group discussions and reports back to the class
Guest Panel of school change action researchers
Critical Incident Questionnaire on course to date
*A* Submit worksheet on PD workbook and research organization (as part of participation item on PD workbooks)
*A* Amst. 5 due: Reflection paper (500-1000 words) relating your Action Research experience to points made by at least one of the readings for class 7.

Narrative: In order to complete a satisfying course project you need to focus on something tight and do-able, but Strategic Personal Planning allows you to find this focus by first opening out and acknowledging a wide range of factors and wishes that your work could (should?) take into account. Strategic Personal Planning provides a glimpse of Strategic Participatory Planning, which can be used to facilitate group endeavors.

Class 8 (3/31) Strategic Participatory Planning, applied to personal course and life projects
Reading: Weissglass, "Constructivist Listening," Spencer, chaps. 5 & 7; also Review Project reports from previous semesters (on reserve and online).
Feedback on Critical Incident Questionnaire
Supportive Listening (a variant of constructivist listening) on one's hopes/fears re: educational change
Strategic personal planning workshop (about the educational/organizational change you want to facilitate/promote) (handout)
In-class drafting of initial description of design project
Post-class reading: Materials on Strategic Participatory Planning from ICA Facilitators Manual, CEDAC, Our Economy

*A* Asmt. 4b due: Full evaluation clock used to analyze the chosen clipping and plan the missing pieces of the study.

*A* Asmt due by email by 4/4: Initial Project Description
Reading: Hitchcock & Hughes, Chapter 5, Designing, planning and evaluating research.

Narrative: The combination of Opening out and Focusing in always seems hard for most CCT students and others who take CCT courses, so we take another look at use of the Evaluation Clock and Strategic Personal Planning. Meanwhile and for the weeks ahead you concentrate on development of your own projects and supporting each other to complete them.

Class 9 (4/7) More on Evaluation Clock and Strategic Personal Planning
Peer coaching on Evaluation clock assignment and its extension to students' projects
Additional readings: More from Patton, Weiss, Stark, or precis of Patton or Weiss

Class 10 (4/14) Work-in-progress Presentations on Student Projects I
Work-in-progress Presentations and peer/instructor evaluations
*A* Asmt due: Work-in-progress Presentation on Project (or class 11)
Titles of Projects
Senait Fesseha, Developing and Maintaining Creative Writing
Mary Frangie, Consistency of Care Among Massage Therapists
Kris Hanks, Digging In The Dirt: Connecting With a New Community Through Gardening
Gloria Hicks, Creating a culture of empowerment in nursing
Barbara Huscher, A future action research plan for one's self through depression
Robert Lingley, Evaluating the Narrative of a Screenplay
Eden Medaglia, TBA
Melissa Moynihan, Facilitating group process in support of family-friendly work places
Robert Norris, A Facilitation of Reflective Creative Writing
Benjamin Okafor, Action Planning And Evaluation On The Methods To Enhance The Participation Of Nigerians In Boston Towards The Eradication Of Corruption In Nigeria
Matthew Puma, Outlines of an Evaluation System for the Adult Education Programs of CEI
April Rucker, The Use of Effective Assessment and Mobilizing a Group Process to Develop a New Portrayal of Women in Advertising
David Rudewick, The Enemy Within (Action research to change communities with persisting prejudice)
Maryann Scheufele, Better Breakfast Builders: dedicated to developing better breakfast habits
Terry Smith, Creating a change in my environment and/ or my approach to goal achievement
Heidi Straghan, Facilitation of a Group Process Resulting in a House Rehabilitation Team
Jenne Todd, Evolution Exhibit Evaluated
Kathleen Walsh, Checking in: Creating a listening process for kindergarten music classrooms using elements of the dialogue process
Nicole Weber, Environmental Education Program in 2 Protected Areas of Honduras
Anna Yangco, Transforming into reflective teachers dedicated to facilitating more independent learners

4/21 No class (Patriots' Day)
*A* Asmt due by email by 4/22: Notes on Research and Planning for Individual Student Projects

Class 11 (4/28) Work-in-progress Presentations on Student Projects II
Work-in-progress Presentations and peer/instructor evaluations
*A* Asmt due: Work-in-progress Presentation on Project (or class 10)

Narrative: Although it is not expected that your projects tackle the larger political context of making changes in education or draw on sophisticated theories about evaluation and educational change, the next class serves to put these areas on your maps. The final class takes stock of the knowledge and experiences you have gained from the course, and looks towards your future research and engagement around educational change.

Class 12 (5/5) Politics and Theories of Evaluation and Educational Research
(handout)
Readings--at least one from each category:
a. Case studies of Woodhead, "When psychology," Hunt, "The dilemma," Metcalf, "Reading between the lines."
b. Action Research as an alternative to Positivist and Interpretivist approaches: Carr & Kemmis, Becoming Critical, CEDAC, Our Economy, Greenwood, "Action science and organizational learning," Taylor, "Whose trees," & "Constructing heterogeneous webs"
"Jig-saw" discussion of readings
Video segment on Myles Horton and the Highlander Center, a longterm source of Participatory Action Research
Additional reading: McLeod, et al., "Changing how we work," Senge et al., "Fostering communities"

*A* Assignment Check-list maintained by student, with incomplete contract if needed
*A* Asmt due: Complete Draft of Design Project (2 copies)

*A* Make comments on another student's draft, and email them to the person by 5/11.

Class 13 (5/12) Taking stock of course & of change: Where have we come & where do we go from here?
Reading: Tuecke, "Creating a wall of wonder," Cashin, "Student ratings of teaching"
Historical/Future Scan of Educational change in its wider world context
GCOE & CCT course evaluations (see previous semesters' evaluations)
Additional reading: Stanfield, Courage to Learn, Stanfield, The Workshop Book (selections TBA)

Due 5/19 before 5pm
*A* PD workbook submitted for perusal, to be picked up after 5/26 from Department of Curriculum & Instruction office, W-2-157
*A* Final Project report (printed copy as well as by email attachment or on disk) and Process Review

BIBLIOGRAPHY

Backer, T., J. Chang, A. Crawford, T. Ferraguto, D. Tioseco and N. Woodson (2002). "Case study and analysis: The Center for the Improvement of Teaching, University of Massachusetts, Boston." .
Brookfield, S. D. (1995). Becoming a Critically Reflective Teacher. San Francisco: Jossey-Bass Publishers
Calhoun, E. F. (1994). How to Use Action Research in the Self-Renewing School. Alexandria, VA: ASCD.
Carr, W. and S. Kemmis (1986). Becoming Critical: Education, Knowledge and Action Research. Geelong: Deakin University Press., chapters 6 & 7 (up to p. 200)
Cashin, W. E. (1990). "Student ratings of teaching: A summary of the research." Management Newsletter 4(1): 2-7.
CEDAC (Community Economic Development Advisory Committee) (1995). Our Economy: Our Future, Final Report. York, Ontario: City of York.
Conlin, M. L. (2002). "The basics of writing: Process and strategies," in Patterns Plus: A Short Prose Reader with Argumentation. Boston: Houghton Mifflin, 1-11.
Couto, R. A. (2001). "The promise of a scholarship of engagement." The Academic Workplace 12(2): 4, 6.
Daniel, D., C. Fauske, P. Galeno and D. Mael (2001). Take Charge of Your Writing: Dicovering Writing Through Self-Assessment. Boston: Houghton Mifflin.
Elbow, P. (1981). Writing with Power. New York: Oxford Univ. Press
Entin, D. (2001). "Review of The Reflective Practitioner: How Professionals Think in Action." The Academic Workplace 12(2): 13, 18.
Greenwald, N. (2000). "Learning from Problems." The Science Teacher 67(April): 28-32.
Greenwald, N. (2000). Science in Progress: Challenges in Problem-based Learning for Secondary Schools
Greenwood, D. J. and M. Levin (1998). Introduction To Action Research: Social Research For Social Change. Thousand Oaks, CA: Sage.
Hitchcock, G. and D. Hughes (1995). Research and the Teacher: A Qualitative Introduction to School-based Research. New York: Routledge.
Hunt, M. (1985). "The dilemma in the classroom: A cross-sectional survey measures the effects of segregated schooling," in Profiles of Social Research: The Scientific Study of Human Interactions. New York: Russell Sage, 51-97.
Institute of Cultural Affairs, n.d., Facilitators Manual (excerpts on Strategic Participatory Planning). Toronto: Canadian Institute of Cultural Affairs.
Jenkins, M. (2000). "Action learning: Taking the time it takes." Paper presented to the International Association of Facilitators, Toronto, April 27 2000.
Kanar, C. (2002). "Improving your paragraph skills," in The Confident Writer. Boston: Houghton Mifflin, 60-88.
Madison Metropolitan School District (2001). "Classroom action research." http://www.madison.k12.wi.us/sod/car/carhomepage.html viewed 25 Jan. 03.
Madison Metropolitan School District (2001). "Classroom action research starting points." http://www.madison.k12.wi.us/sod/car/carstartingpoints.html viewed 25 Jan. 03.
McLeod, M., P. Senge and M. Wheatley (2001). "Changing how we work." Shambhala Sun(January): 29-33.
Metcalf, S. (2002). "Reading between the lines." The Nation(Jan. 28): 18-22.
Nelson, J. (2001). The Art of Focused Conversation for Schools. Toronto: Canadian Institute of Cultural Affairs.
Patton, M. Q. (1982). Practical Evaluation. Newbury Park, CA: Sage.#
Pietro, D. S. (Ed.) (1983). Evaluation Sourcebook. New York: American Council of Voluntary Agencies for Foreign Service.
Rokovich, M. A., M. Stevens and J. Stallman (2000). "Implementing change at SJUSD: An unfinished case study." Presented to the International Association of Facilitators, Toronto, April 27 2000.
Schmuck, R. (1997). Practical Action Research for Change. Arlington Heights, IL: Skylight.
Senge, P., N. Cambron-McCabe, T. Lucas, B. Smith, J. Dutton and A. Kleiner (2000). "Fostering communities that learn," in Schools That Learn. New York: Currency, 459-465.
Spencer, L. J. (1989). Winning Through Participation. Dubuque, Iowa: Kendall/Hunt
Spina, S. U. (2002). "Six key principles of action research." (handout)
Stanfield, B. (Ed.) (1997). The Art of Focused Conversation. Toronto: Canadian Institute of Cultural Affairs.
Stanfield, B. (2000). The Courage To Lead: Transform Self, Transform Society. Gabriola Island BC: New Society Publishers.
Stanfield, R. B. (2002). The Workshop Book: From Individual Creativity to Group Action. Toronto: Canadian Institute of Cultural Affairs.
Stark, J. S. and A. Thomas (Eds.) (1994). Assessment and Program Evaluation. Needham Heights, MA: Simon & Schuster. (in Healey stacks)
Taylor, P. J. (1999) "Constructing Heterogeneous Webs in Socio-Environmental Research." ms.
Taylor, P. J. (2001). Excerpt from "'Whose trees are these?' Bridging the divide between subjects and outsider-researchers," in R. Eglash and G. DiChiro (Eds.), Appropriating Technology: Vernacular Science and Social Power. Minneapolis: University of Minnesota Press.
Tuecke, P. (2000). "Creating a wall of wonder with the TOP environmental scan." International Association of Facilitators, Toronto, Canada, April 27 - 30 (iaf-world.org/iaf2000/Tuecke.PDF).
Turabian, K. L. (1996). A Manual For Writers of Term papers, Theses, and Disertations. Chicago: Univ. of Chicago Press (in Healey reference section)
Weiss, C. H. (1998). Evaluation. Upper Saddle River, NJ: Prentice Hall.#
Weissglass, J. (1990). "Constructivist listening for empowerment and change." The Educational Forum 54(4): 351-370.
Winter, R. (1989). Learning from Experience: Principles and Practice in Action Research London: Falmer.
Woodhead, M. (1988). "When psychology informs public policy." American Psychologist 43(6): 443-454.
# See also the compilation of chapter precis of these books on reserve.