University of Massachusetts at Boston

Graduate College of Education

Critical & Creative Thinking Program

Seminar on


Evaluation of Educational Change


CrCrTh693

Spring 2003



NOTES ON TEACHING/LEARNING INTERACTIONS

including guidelines for assignments and examples of previous students' work



Refer to the Rubrics handout for a check-list of expectations for the assignments and other requirements.
Homework and other preparatory tasks are not part of the required assignments, but are recommended to make classes more valuable to you.
Examples of previous students' work are linked to this document. They can be consulted to indicate the range of ways students tackle an assignment; they are not models to be copied. If the link does not work, the example has not yet been uploaded.

Written assignments and presentations

Note: If you get behind, ask for an extension or skip the assignment/item-the intended learning rarely happens if you submit a stack of late work.
A.
Action Research assignments
The expectation for these assignments is that you will pull together the work you have done for your Action Research tasks and reflect on the experience in ways that fit the tasks you take on and your own style. Moreover, provided you submit the assignment on time, my responses will be designed to help you develop your contribution to the AR still remaining.
The interim report is a work-in-progress report. As such, it's OK to write about the process but make sure you give detail on your research re: what you have learned (and the KNFY that made you investigate that), what new Qs have emerged, and what your specific plans are for investigating those new Qs. If you don't have such Qs, do KNFY to generate some.
If you compose the interim report as something that someone outside the course could understand -- not as a report to me -- this will help you clarify your own thinking. Ditto for the request that you relate your tasks "to the specific tasks others who you are communicating with have taken on." To address this it's important simply to be clear how what they are finding out affects what you are asking (and vice versa)? If someone else has done some K, what NFY follows for you (and vice versa)?
[examples of previous students' work, 1, 2, 3, 4]
The final report should touch the following bases: Research-what did you find out that you didn't know before, which includes what would convince others of something you already know.
Action proposals-what actions follow from/ are supported by your research.
Constituency-Action Research is research that helps you take action, which includes convincing or enlisting others to take action that respects the resources, possible limited, that they have. Who are the people who you want to convince or enlist in carrying out the action proposals.
Open questions-what more do you need to find out and how would you propose to do that research. (This can include research on proposals you have thought of, but did not do the research to support.) All these items should be framed in relation to the case, "Enhancing diversity in the CCT Program," with which the Action Research unit began.
In order to make such a report convince or enlist your constituency it is generally best to be very succint and make the connections among the four bases above easily grasped. Most readers do not make time to digest long narratives about what you did, nor to fill in the thinking that is omitted (e.g., about how a proposal follows from your findings). So try to make sure what you select to include counts in your efforts to influence someone to do something differently. (Also do not begin with "My task/job/role was to...," which tends to convey the impression that someone other than you was dictating what you did.) Use an appendix for details that are not essential to convincing or enlisting your constituency to take up your proposals.
Finally, the assignment due with the presentation is a draft final report, on the non-judgemental assumption that there'll be room for you to strengthen your report after having it read by someone with more experience in KNFYing and other aspects of AR. Suggestions I make should be seen as part of "Dialogue around written work" (see NOTES ON TEACHING/LEARNING INTERACTIONS).
[examples of previous students' work, 1, 2, 3, 4]
Reflection paper (500-1000 words) relating your Action Research experience to points made by at least one of the readings for class 7.
[examples of previous students' work, 1, 2, 3, 4]

Evaluation clock
The expectation for this assignment is that you will not get it right the first time, but will need coaching to produce the focused comparson steps and the recursive full clock.
[examples of previous students' work, 1, 2, 3, 4]

B. Stages of development for design project
The design project should not be seen as a "term paper," but as a process of development that involves i) dialogue with the instructor (see below) and other students and ii) revision (re-seeing) in light of that dialogue. To facilitate that process, a sequence of five assignments and peer commentary is required. The goals of each stage are described below. The design you propose must relate to evaluation and/or facilitation of educational change (broadly construed) that you are doing or interested in undertaking. Previous semesters' projects are available for viewing on reserve and/or through link to the course website [2002].
6. Initial description
Building on your in-class strategic personal planning, compose an initial overview of your design project. This overview may, several revisions later, end up setting the scene in the introduction of your project. In one-two prose paragraphs (not disconnected points a.k.a. "bullets"), an overview should convey subject, audience, and your reason for working on this project.
[examples of previous students' work, 1, 2, 3, 4]

7. Notes on research and planning
Pull together notes on your reading and your thinking and present it in a form organized so it can elicit useful comments from a reader (in this case, me). To show your planning, you should submit an updated overview and an outline and/or evaluation clock. To show that you are finding out what others have been doing in your area of interest, you should include annotated bibliography of readings done or planned. Record the full citations for your sources, including those from the WWW. I recommend using a bibliographic database-Endnote can be downloaded for a 30 day trial from http://www.endnote.com
[examples of previous students' work, 1, 2, 3, 4]

8. Work-in-progress presentation
Preparing presentations, hearing yourself deliver them, and getting feedback usually leads to self-clarification of the overall direction of your project and of your priorities for further work. In this spirit, 15 minute presentations of your work-in-progress are scheduled early in your projects and are necessarily on work-in-progress. Convey the important features of work you have already done and, to elicit useful feedback during 3-5 minutes of Q&A, indicate also where additional investigation or advice are needed and where you think that might lead you.
9. Complete draft report
See guidelines for final report. The draft must get to the end to count for this assignment, even if some sections along the way are only sketches.
[examples of previous students' work, 1, 2, 3, 4]

10. Final report (1500-2500 words, plus bibliography of references cited)
Whatever form your report takes, make sure you explain why you have designed this evaluation or facilitation. You should also include material that conveys your process of development during the project and in the future. The report should not be directed to the instructor, but conceived as something helpful to readers like your CCT student colleagues.
For the report to be counted as final, you must have revised in response to comments from instructor and peers on complete draft. Allow time for the additional investigation and thinking that may be entailed.

A & B, Dialogue around written work
I try to create a dialogue with each student around written work, that is, around your writing, my responses, and your responses in turn. Central to this teaching/learning interaction are requests to "Revise and Resubmit." The idea is not that you make changes to please me the teacher or to meet some unstated standard, but that as a writer you use the eye of others to develop your own thinking and make it work better on readers. I may continue to request revision when I judge that the interaction can still yield significant learning. Such a request does not mean your (re)submission was "bad"-even when the first submissions of written assignments are excellent, angles for learning through dialogue are always opened up.

In my comments I try to capture where the writer was taking me and make suggestions for how to clarify and extend the impact on readers of what was written. After letting my comments sink in, you may conclude that I have missed the point. In this case, my misreading should stimulate you to revise so as to help readers avoid mistaking the intended point. If you do not understand the directions I saw in your work or those I suggest for the revision, a face-to-face or phone conversation is the obvious next step-written comments have definite limitations when writers and readers want to appreciate and learn from what each other is saying and thinking. Please talk to me immediately if you do not see how you are benefitting from the "Revise and resubmit" process. I am still learning how to engage students in this in ways that take into account your various backgrounds and dispositions and my own.

Students should submit two copies of all typed assignments because I want to give you one back with my comments so you have it when you read them. (I keep the other plus a carbon copy of my comments to refer back to.)

C. Building learning community -- Prepared participation and punctual attendance at class meetings are expected, but allowance is made for other priorities in your life. I do not require you to give excuses for absence, lateness, or lack of preparation. Simply make up the 80% of participation items in other ways (D-H).

D. Personal/Professional Development Workbook
In your workbook keep records or products of homework and Action Research tasks, preparation for assignments, weekly journal-type reflections on the course and classes, notes on readings, clippings, e-clippings. Explore, when appropriate, the relationship between, on one hand, your interests and possible projects and, on the other hand, the readings and activities. I encourage those of you who find it hard to make space for journalling/reflection to stay 10 minutes after class and write while your thoughts are fresh. If you are using the workbook effectively, it should convey your developing process of preparing to practice the tools and of critical thinking about course readings, activities, and discussions. When you first show me the PD workbook for perusal, I will let you know if you need to show more processing and organize it better. The worksheet submitted in week 7 will allow me to make further suggestions about your PD workbook and research organization. When I peruse the PD workbooks, bind together pages with post-its or otherwise indicate which bits you do not want me to look at.

Clippings and E-clippings
To keep up with current developments-and get you into the habit of this for your lifelong learning- look for articles related to evaluation and facilitation of educational change in newspapers, magazines, journals, and websites. The education column in NY Times on Wednesdays is often good. Write the full citation on each article, unless it is already included. Use large post-its to add your own reflections on specific points in the articles you choose. Aim for one/week. Include these in your PD workbook, including copies of items from the WWW posted to cct693Clips. For clippings you find on the web submit the URL and brief annotation to cct693clips@yahoogroups.com. These can be viewed at http://www.yahoogroups.com/group/cct693Clips. Use the search box to find clippings on specific topics.

E. Conferences
for discussion of comments on assignments (see Dialogue around written work, above), ideas for course projects, your PD workbook, and the course as a whole. They are important to ensure timely resolution of misunderstandings and to get a recharge if you get behind.

F. Peer commentary
After the draft report is completed, you should comment on another student's draft. Send me a copy by email and/or include in PD workbook. Keep Elbow, Writing with Power, chapters 3 & 13 in mind when you decide what approaches to commenting you ask for as a writer and use as a commentator. In the past I made lots of specific suggestions for clarification and change in the margins, but in my experience, such suggestions led only a minority of students beyond touching up into re-thinking and revising their ideas and writing. On the other hand, I believe that all writers value comments that reassure them that they have been listened to and their voice, however uncertain, has been heard.
[examples of previous students' work, 1, 2, 3, 4]

G. Assignment check-list
Please keep track of your assignments and revisions submitted and when they are returned marked OK/RNR. To gauge whether you are on track for at least a B+, simply note whether you have submitted 80% of the assignments by the dates marked and attended 80% of the classes.

H. Process review
Identify 4-6 examples that capture the process of development of your work and thinking about facilitation and evaluation of educational change. Journaling, freewriting, drafts, etc. may be included, that is, not simply your best products. Explain your choices in a 250-500 word cover note and through annotations (large post-its are a good way to do this). Submit with your PD workbook, or extract into a portfolio.
[examples of previous students' work, 1, 2, 3, 4]

Other Teaching/Learning Tools

Rationale for the Assessment system
The rationale for grading the different assignments simply OK or R&R (revise & resubmit) and granting an automatic B+ for 80% satisfactory completion is to keep the focus of our teaching/learning interactions on your developing through the semester. It allows more space for students and instructor to appreciate and learn from what each other is saying and thinking. My goal is to work with everyone to achieve the 80% satisfactory completion level. Students who progress steadily towards that goal during the semester usually end up producing work that meets the criteria for a higher grade than a B+ (see Rubrics). Use the Assignment Check-list to keep track of your own progress. You are free to do more than 80% of the assignments and fulfill more than 80% of the participation items, but it does not hurt your grade to choose strategically to miss some in light of your other work and life happenings. Ask for clarification if needed to get clear and comfortable with this system.

Learning Community and email group/list
Individually and as a group, you already know a lot about educational change. You can learn a lot from each other and from teaching others what you know. The email group or list (emails sent to cct693@umbmap.cc.umb.edu) can be used to help the community develop.

Activities for "self-affirming" learning
Students already know a lot. If this knowledge is elicited and affirmed, they are more able to learn from others. Activities such as freewriting bring to the surface students' insight that they were not able, at first, to acknowledge. Over the course of the semester, students are encouraged to recognize that there is insight in every response and share their not-yet-stable aspects. The trust required takes time to establish.

Tools for Group Process
Each week introduces a different tool or practice for facilitating group process. Handouts on the tools are linked to the course website when they are ready.

Guided freewriting
In a freewriting exercise, you should not take your pen off the paper. Keep writing even if you find yourself stating over and over again, "I don't know what to say." What you write won't be seen by anyone else, so don't go back to tidy up sentences, grammar, spelling. You will probably diverge from the topic, at least for a time while you acknowledge other preoccupations. That's OK-it's one of the purposes of the exercise to get things off one's chest. However, if you keep writing for ten minutes, you should expose some thoughts about the topic that had been below the surface of your attention-that's another of the aims of the exercise. Reference: Elbow, chapter 2.

Think-pair-share
After preparing your thoughts, you pair up with another person, and, through sharing ideas, you refine your own and prepare to share a key part of them with the whole class (time permitting).

Taking stock during semester ("formative evaluation")
Through activities, such as the Critical Incident Questionnaire, I encourage students to approach this course as a work-in-progress. Instead of harboring criticisms to submit after the fact, we can find opportunities to affirm what is working well and suggest directions for further development.

Taking stock at end of semester involves multiple angles on course evaluation (including written evaluations during class, Process reviews and planning for your ongoing PD):
a) to feed into your future learning (and other work), you take stock of your process(es) over the semester;
b) to feed into my future teaching (and future learning about how students learn), I take stock of how you, the students, have learned.