Bureau of Educational and Cultural Affairs

OFFICE OF ENGLISH LANGUAGE PROGRAMS
Home > English Language Programs > English Teaching Forum > Volume 42 > Number 3

Building Consensus on Teacher Assessment

Paul Michael Chandler (United States) and Stael Ruffinelli de Ortiz (Paraguay)

Teacher Assessment (TA) is a broad term for various procedures used to study teachers’ classroom performances. Other terms sometimes used include evaluation and appraisal. Besides providing the occasional snapshot of instructional practice, TA gives teachers and educational institutions means for improving instruction in an ongoing fashion. But, as Penny Ur points out: “Few institutions have systematic teacher-appraisal systems, and where these do exist, they are very often for hiring and firing purposes rather than to assist professional improvement and learning. The effect may therefore be stressful and demoralizing rather than helpful” (Ur 1996, 322).

When schools do assess teachers, observations tend to be completed by an administrator, such as a director or headmaster. In some instances, a more experienced colleague or supervisor assesses the performance of newer teachers. Observation reports should be unemotional, factual accounts of what happened throughout the observed class (e.g., Class began promptly; Ten students were present; The warm-up activity lasted ten minutes.). Positively worded critiques are the mainstay of sound observation practices. Even problems can be worded positively to focus on improvement (e.g., Your pronunciation of final consonant clusters has improved, but keep working on the troublesome diphthong ou, as in focus).

Many EFL programs offer a variety of levels of instruction (from novice to advanced) for learners of all ages (from toddlers to adults), creating a need for different types of teacher assessment. In other words, the assessment formats and instruments used should be varied according to the circumstances of the educational setting.

To facilitate an open, collaborative process of TA, we developed an easily implemented workshop that can be used at a teacher-training session or in a faculty meeting. We conducted the workshop at a private English language institute that teaches English to all levels and ages. The workshop required about two hours (see approximate timetable in Appendix 1). In this article we describe the TA procedures considered, illustrating the process with feedback from administrators, teachers, and students. As teacher trainers and supervisors of language programs, we are stakeholders in the assessment process.

Teacher assessment formats and instruments

Formats (e.g., observations, class summaries) and instruments (e.g., evaluation forms, journals) of TA vary widely. Some institutions use a fixed, standardized form for each class observation report. Others prefer more open-ended reports, such as a summary of the class observed along with pedagogical suggestions and additional comments. With teacher trainees, discussion of the class and the lesson plan both before and after the observation is strongly recommended. We believe that trainees benefit from discussions with the observer before the class is observed, while experienced teachers need only provide an overview of the upcoming class to the observer. In some cases, the second or third classroom observation is unannounced so that the teacher can be observed in a “normal” or typical performance. Since our trainees work in teams, they become accustomed to the presence of other teachers in the classroom. Because of the positive way we provide feedback and share teaching ideas, the trainees are not unduly nervous about classroom visits in general, although there is usually some level of anxiety.

In between the fixed and open-ended TA instruments, there is the practical checklist, which includes a final section of open-ended and specific comments from the observer, or an attached letter from the observer to the teacher being observed. The checklist is meant to orient the observer—who is not always a language teaching specialist—to the goals of the program by highlighting specific features of teacher and student behaviors. The openended items, or attached letter, allow for qualitative feedback from the observer.

Learners are the other important source of TA data. Feedback from students is crucial, and procedures for student evaluation of faculty vary depending on the level and age of the students. The focus of the assessment also varies from the individual task or activity to an entire course or program (see Figure 1). For instance, to evaluate a specific task, a teacher might obtain student feedback with a simple request: “Please write your personal reaction to the listening activity we just completed.” To assess teaching performance over time, surveys, journals, or open-ended essays result in more detailed responses from students. Such responses can be elicited by making the following request of students: “Please assess my teaching of this course during this academic period, stressing areas where I have excelled and offering constructive criticism of any ways in which I may improve.”

Student course evaluations often take the form of a five-item lickert scale ranging from “strongly agree” to “strongly disagree,” or the evaluation may include a list of desired, observable behaviors (e.g., “The teacher gives clear instructions” or “The reading materials are interesting.”). Specific items usually address behaviors, goals, and skills, such as student behaviors, teacher behaviors, the skills practiced, language use, and error correction. Teacher-student journals may be used to build students’ sense of ownership in the learning process. The teacher may sometimes provide a couple of guiding questions, while at other times leaving the topic open.

Videotaping classes for assessment purposes can be a less stressful alternative for teachers, especially if they are insecure or new to the profession. When videotaping, the camera should be placed in the back of the classroom, focused primarily on the teacher, but set at a wide enough angle to record the entire class. (To enhance the visual aspects of the recording, the camera should not face bright windows.) Students will soon ignore the camera. How the video is used for assessment purposes can vary. Viewing of the video may be restricted to the videotaped teacher, who then writes a personal report assessing his or her own performance. Some videotaped teachers may prefer to watch and discuss the videotape with a trusted colleague whose role is to offer constructive criticism. A more threatening scenario is to view the video with a superior. In all cases, the adoption of specific procedures, such as a summary with comments, a checklist, or a list of new goals, ensures teachers’ familiarity with and acceptance of the procedures.

For trainees, peer teaching and peer lesson planning as professional development activities can enhance working relationships and decrease nervousness about collegial observations. Teachers get accustomed to collaborative, shared efforts. In addition, Harmer recommends the formation of small teacher groups to discuss issues that surface during instruction (2001, 349). These groups can be used to reinforce means of expressing positive criticism.

Participants

We conducted two separate workshops. In the first workshop, we collected data from twelve participants who had spent six months in the CertTESOL initial training course. All but three were under age twenty. This training program emphasizes awareness of the teacher’s role. Because the teacher sets the climate, tone, and environment of the classroom, the workshop sensitizes trainees to their responsibilities. For example, they must understand the magnitude of the teacher/student power differential, be aware of the dynamic nature of the pedagogical process, and develop open and honest relationships with their students.

The second workshop consisted of eighteen certified, qualified teachers whose classroom experience ranged from one to fifteen years. Most work full time, either at a private institute or at K-12 schools under contract with the institute. Five are tutors with the Trinity CertTESOL program and have given professional workshops.

Phase 1: How threatening is teacher assessment?

We generated a list of forms of TA to discuss with workshop participants, providing a wide range of possibilities that included observations by administrators, colleagues, and student teachers; informal classroom visits by parents; student evaluations; reports from collaborative or action-research projects; student-teacher journals; and videotaped classes. (The entire list of formats and instruments discussed with the teachers is shown in Figure 2.) In addition to the twelve options we generated, the teachers-in-training added three more: open class discussions, role plays in which the students imitate the roles of teacher and students, and an interview of the students by a colleague of the teacher who later provides a typed summary of the students’ comments. After discussing the list, we had pairs of teachers further discuss (see Appendix 2) and rank the items from least to most threatening (see Figure 2).

These teachers chose as less threatening procedures self-assessed videotapes and written reflective journals. There was a general tendency to rank supervisor and director visits as the most threatening option. In their preference for role-play, we believe the trainees were reflecting what we call the “imposter” syndrome. When new teachers first enter the classroom, they often report that for some time they feel as though they are students who are just “acting” like teachers. These feelings should be discussed during the training (or individually) to help them more comfortably assume new roles in the classroom.

Although most teachers listed Supervisor visit or School director visit as the most threatening forms of TA, many still wrote in their reflections and/or requested orally that they be observed by a supervisor because they value greatly the professional feedback they receive. We believe that this is due to the open and supportive nature that the institute attempts to maintain between the administration and the teaching staff. In other words, the teachers consciously requested the most threatening form of TA in spite of the fact that it makes them nervous. Both groups also considered parent observations and discussions rather threatening.

Phase 2: The benefits of teacher assessment

In the second phase of the workshop, we wanted to move beyond the risk-taking nature of TA to discuss its benefits or consequences. We listed eight possible benefits and had each teacher rank them from 1, most important, to 8, least important (see Appendix 3).

For some of the twelve teachers-in-training, there were differences of opinion (see Figure 3). Nevertheless, some clear tendencies are apparent at the high and low ends of the spectrum. Note the obvious cluster of three items at the top of the chart. Half of the group listed Improvement of students’ learning as the most important benefit of teacher assessment. Besides a strong concern for maximizing learning, most of the trainees clearly exhibit altruistic tendencies along with their understanding of how important professional teaching development is to them. At the lower end of the chart, the remaining items clustered near each other. The cluster reflects less concern in general with monetary rewards, professional recognition, and financial security, though one person gave a fairly high ranking of 3 to Promotion, and three teachers gave a ranking of 4 to Recognition. Thus, some of the respondents appreciate knowing when people believe they are doing a good job in the classroom. Four teachers gave a ranking of 8 to Salary improvement. This may be due in part to the poor salaries in most Paraguayan schools (i.e., it is unlikely that teacher assessment will affect a teacher’s salary).

The more experienced teaching staff expressed very strong agreement that Improvement of students’ learning was the most important benefit of teacher assessment (see Figure 4). At the same time, they consistently deemphasized the importance of Recognition, and many gave the lowest ranking to Salary improvement and Job retention.

Phase 3: Assessment and the professional teaching portfolio

In the third part of the workshop, we brainstormed with the teachers about what, besides some form of TA, should be included in a professional teaching portfolio (see Figure 5). We divided the chalkboard in half and had them generate a list of items, discussing whether they should be considered obligatory or optional. We facilitated the discussion, sometimes playing devil’s advocate to get teachers to consider the importance of the items. By the end of this phase of the workshop, the teachers had come to a consensus regarding the general makeup of the professional teaching portfolio. A summary of that consensus was provided to teachers to ensure uniformity of portfolios.

Phase 4: Follow-up reflection

Once we had completed the above phases of the workshop, we asked the participants to revisit the forms of TA and select their top three preferences (see Figure 6). As can be seen, the two groups differ slightly. While the more experienced teachers preferred to use written student evaluation forms (probably the norm at most institutions), the trainees preferred self-assessed videotapes. Both groups selected collegial observation as their second choice. And, as mentioned earlier, in spite of the threatening nature of supervisor visits, both groups strongly value feedback from their superiors.

Phase 5: Student preferences

While some educational institutions gather data on learning and instruction via end-of-term student evaluations, learners usually are not directly involved in the establishment of the procedures. We believe that all institutions, public and private, should develop students’ sense of voice and ownership in the educational process, based on the ages and abilities of the students. In line with this belief, we took the six forms of assessment most preferred by the teachers to eight different groups of students at the language institute to ask for their feedback (see Appendix 4). Depending on their schedules, either the school director or a supervisor visited students at different levels (high elementary through superior) and of different ages (nine-year-olds to adults). First the types of TA were discussed; then students were asked to discuss and write down their thoughts about each option. Finally, at the bottom of the page, they indicated their top preference.

Wide variation in opinion was apparent. The most frequent preference was for teachers to talk openly with the students, while the second preference was for students to write evaluations (30 percent made Talking with students their top choice; nearly 25 percent chose Students write evaluations). The third choice was also for a written instrument, the use of student- teacher journals (20 percent). Thus, 44 percent were in favor some type of written assessment. Obviously, we have different learning styles in our student population. Teachers always need to keep this in mind when selecting TA procedures and carrying out assessment.

Students’ oral and written comments were informative. In one class, a few students commented that they might not be truthful in an open class discussion. But several students in the same group begged us to avoid using journals for TA because they do not enjoy writing. Although they wanted their teacher to be observed, some students expressed fear that the class would not be normal (typical). Other concerns expressed by the students, either orally or in writing, included questions regarding anonymity, nervousness, and truthfulness.

Conclusion

At the beginning of the project we brainstormed, reviewed the professional literature on teacher assessment, and discussed options. We then continued the discussion with other teachers to consider different perspectives, ultimately deriving a list of options from which the teaching staff could choose. Finally, we talked with our students to get their perspective on the various types of TA we were considering. Reactions to this process have been quite positive. The teachers, trainees, and students provided important reactions to the range of assessments discussed. These were instrumental in formulating our decisions regarding assessment.

Based on these workshops, the institute is implementing the forms of TA described below.

• Among other things, teachers will include in their portfolios: (1) final student evaluations (A set of statements were developed for children five to nine years of age in which they indicate their opinions by marking a smiling, neutral, or frowning face [based on Lipton 1992]), (2) a collegial observation report, (3) a self-assessment of a videotaped class, and (4) at least one supervisor observation.
• At the end of the year, teachers will be invited to make an appointment for an office visit with the supervisor or to include in their portfolio a written reflection/ summary on the academic year.
• Student evaluations may take one of three forms: (1) open discussions, (2) opentopic student-teacher journals, or (3) open-ended essays.
• Teachers must produce evidence of their participation in faculty development workshops and presentations and document in their portfolios student performance, cooperative and collaborative work with their colleagues, and any innovative materials they created for their classes, such as a reading unit or strategy instruction.

Informed and shared decision making are key to successful TA. Once all parties understand the procedures, frequency, benefits, and goals of the assessment, implementation will be easier.

The final stakeholders in the TA process are the parents. When teachers prepare professional teaching portfolios that include several forms of ongoing assessment, both they and administrators have important documentation at hand to demonstrate to parents how quality of instruction is addressed and maintained. In this way, our efforts in these workshops help everyone. Developing an ongoing and flexible set of TA procedures has helped all of us grow as administrators, teachers, and students.

Because we valued teacher and student feedback, included all stakeholders in the process, and shared data freely, we found this experience very rewarding. The collaborative nature of the project gave us new insights and enriched our own professional development as researchers and supervisors.

We wish to make one final comment about having two separate workshops, one for trainees and another for more experienced teachers. We believe this was a wise choice because trainees might feel intimidated in the same workshop with their superiors and, in most cases, with current or former teachers. We recommend that, whenever possible, the separate workshop format be used in similar situations.

Our hope is that this multi-phase workshop can begin to fill the need for “systematic teacher-appraisal systems,” called for by Ur (1996). The institute will revisit assessment procedures annually to consider whether revisions are necessary. This will help maintain the flexible and ongoing nature of teacher assessment that has been established with the collaboration of our faculty and students.

References

Harmer, J. 2001. English language teaching. 3rd ed. Essex: Longman.

Lipton, G. 1992. Practical handbook to elementary foreign language programs. 2nd ed. Lincolnwood, Illinois: NTC Publishing.

Ur, P. 1996. A course in language teaching. Cambridge: Cambridge University Press.

Figure 1: Instructional Unit Being Assessed

Figure 2: Teacher Rankings of the Least to Most Threatening Forms of Teacher Assessment

* The experienced teachers did not mention three items suggested by the teachers in training.

Figure 3: Teachers-in-Training Ranking of the Benefits of TA

Figure 4: Experienced Teaching Staff Ranking of the Benefits of TA

Figure 5: Teacher Preferences for Portfolio Contents

Figure 6: Top Assessment Preferences by Group

Appendix 1: Approximate Timetable: Two-Hour Workshop

• 30 minutes to discuss teacher assessment formats and instruments and to designate how threatening they are (completed in pairs)
• 30 minutes to discuss the benefits and consequences of TA and to rank them in order of importance (completed individually)
• 30 minutes to discuss the contents of the professional teaching portfolio (brainstorming items as a group, organized on chalkboard as obligatory vs. optional elements)
• 0 to 15 minute break if desired (our teachers declined to break, preferring to finish)
• 10 to 15 minutes to choose top choices of TA (the final written activity)

Appendix 2: Judging the Nature of Teacher Assessment

Names: _______________ and _________________

Work in pairs. After discussing the teacher assessment instruments listed below, please rank them on a continuum from least to most threatening.

____ Student evaluations (fixed questionnaire of specific items)
____ Student evaluations (open-ended essays assessing the class and your teaching)
____ Supervisor visit
____ School director visit
____ Self-assessment of videotaped class (only you view it)
____ Written reflective journal on specific class(es) or unit of instruction
____ Videotaped class assessed with a supervisor
____ Collegial observation (friendly teacher that you invite)
____ Fixed-item student-teacher journal
____ Open topic student-teacher journal
____ Parent class observation and discussion
____ Student teacher observation and discussion

Appendix 3: Ranking the Benefits of Teacher Assessment

Names: _______________

Read the following alphabetical list of possible “benefits” or “consequences” of teacher assessment. Please be as honest as possible as you rank them from 1 to 8 in their order of importance to you, with 1 being the most important and 8 being the least important.

____ Expansion of duties (teaching new classes or levels, new responsibilities)
____ Improvement of your students’ learning
____ Intrinsic rewards (personal satisfaction, happiness, feeling of accomplishment)
____ Job retention (maintaining or increasing your employment)
____ Professional development (improving your knowledge and skills)
____ Promotion (improving your position at the school)
____ Recognition through awards or certificates
____ Salary improvement (earning a bonus or a salary increase)

Appendix 4: English Student Survey

Teacher’s name: _______________
Level: ____
Your age: ____

The Institute is considering which forms of teacher assessment (TA) to use. The teachers already chose their preferred TA procedures. Now we want you, the students, to consider these options. Which ones do you think will help our teachers improve? Why do you prefer them?

1. The teacher videotapes the class and writes an evaluation. (Only the teacher sees the video.)
2. Supervisor visit. (The supervisor visits the class, discusses it with the teacher, and writes a report.)
3. Students write evaluations of their teachers. (Example: Describe how you feel about your English class.)
   Do you agree or disagree with each sentence below? (Circle one.)
   agree    disagree    I like the way we learn to read.
   agree    disagree    My teacher gives clear instructions.
   agree    disagree    The teacher helps me improve my pronunciation.
4. Collegial observation. (The teacher being observed invites another teacher to visit the class, take notes, discuss the class, and write a report.)
5. Talk with students. (The teacher has an open discussion with the students about his or her teaching.)
6. Student-teacher journals. (Students are free to write their opinions about anything related to the teaching of the class.)

Of these six options, which do you most prefer? Write its number: ___

Thanks for your cooperation!

Paul Michael Chandler supervises teacher development and beginning Spanish language instruction at the University of Hawaii at Manoa. In 2003 he was a Fulbright Scholar at the Universidad del Norte in Paraguay.

Stael Ruffinelli de Ortiz is owner and director of a language institute in Asunción, Paraguay, where she has provided Trinity CertTesol Training since 1993. She also provides teacher development assistance to the Paraguayan Ministry of Education.

Back to the top