| . |
. |
. |
|
Current
Proficiency Testing
|
|
|
|
EFL proficiency tests did not
use to be based on any particular syllabus but were designed to measure different levels
of language ability in general. This made it difficult for teachers to prepare students
for a proficiency test since there were no identified learning objectives that could serve
as guidelines for selecting appropriate teaching materials and techniques. In recent
years, however, research in the field of testing has focused more on the relationship
between testing and the teaching which precedes the test—referred to in the
literature as the “washback” effect. As a result, some proficiency test
specifications—what language is tested and how it is tested—now more closely
resemble syllabus specifications of ELT coursebooks that reflect current methodological
principles. This development brings proficiency tests more in line with what we expect of
achievement tests, thus, reflecting teaching.
However, in order for teachers to exploit the positive link between teaching and testing
effectively, they need to analyze the objectives of the test and then be able to select
appropriate materials and procedures that will serve these objectives. This is not an easy
task for teachers who have not had recent training in the principles of testing. It may be
helpful, then, to see in practice how test objectives can be inferred from a close
analysis of an actual language examination, and, from there, to see how these objectives
relate to teaching.
|
|
|
|
This article will look at the washback effect, using as an example the new University of
Michigan Examination for the Certificate of Competency (ECCE), designed for intermediate-
to upper-intermediate-level learners. The ECCE can be considered an example of the
movement towards achievement-oriented proficiency tests.
|
|
|
ECCE: Teaching and testing
relationship
|
|
|
Current language teaching methodology views
language use as a communicative, interactive process taking as its starting point the
notion of communicative competence. Since the development of communicative competence is
the main objective for teaching, determining what we teach and how we teach, it would
appear that a model of communicative competence would help us identify testing objectives
and serve as an appropriate framework for evaluating the washback effect, provided, of
course, that the test under review has been designed on the basis of the principles of
communicative competence.
What is tested in the ECCE: Components of communicative competence. The term
“communicative competence” originated in sociolinguistics with Dell Hymes
(1972). Communicative teaching is guided by the model put forward first by Canale and
Swain in 1980 and revised by Canale in 1983. According to this model, communicative
competence de-scribes the learner’s ability to use the language in listening,
speaking, reading and writing in terms of the following competencies:
|
|
|
- linguistic competence (knowledge of form)
- sociolinguistic competence (ability to use language appropriately in different contexts)
- discourse competence (cohesion and coherence)
- strategic competence (knowledge of verbal and non-verbal communication strategies).
|
|
|
Each of these components is tested in the ECCE
in a variety of ways, both separately and in relation to one another.
Linguistic competence is tested with a multiple-choice format in the grammar section. This
section tests the recognition of a wide range of basic grammatical structures including
tenses, word form, prepositions, etc. Linguistic competence is also tested indirectly in
the listening and reading sections through comprehension of gist and specific information
which may require the understanding of a specific grammatical item. It is tested as well
in the productive skills sections of the test (speaking and writing ) where candidates are
required to produce grammatically correct basic sentences in their compositions and oral
interviews. Most integrated skills course books aim to develop linguistic competence
through skills development as well as controlled grammatical exercises which focus on
recognition of form. However, teachers tend to over-emphasize the latter in their teaching
mainly because it is a more concrete way of teaching grammar and because it also easily
lends itself to self study.
Sociolinguistic competence is tested to some extent in the vocabulary section, where
candidates are required to choose the word that best completes the sentence. Very often
collocations are tested, i.e., words which are firmly associated with each other; e.g.,
make a mistake, apply for a job. Words are tested in a sentence context, which means that
candidates can apply the strategies they have developed from classroom instruction for
guessing words in context using clues such as synonyms, antonyms, punctuation, and
discourse markers.
Discourse competence and linguistic competence are tested in the reading and listening
sections. Comprehending gist involves making connections between different parts of the
text (discourse competence) while comprehending specific detail may involve linguistic
contributory skills referred to earlier. Although an integration of these comprehension
sub-skills is essential to the learner’s ability to function in real communication,
it is an accepted pedagogical practice to break down the comprehension process into
separate skills for both teaching and testing.
All four competencies are tested integratively in the writing and speaking sections, where
the candidates need to produce language typically used in everyday contexts. The language
must not only be accurate (evidence of linguistic competence) but also appropriate
(evidence of sociolinguistic competence) and coherent (evidence of discourse competence).
A candidate also has the opportunity to demonstrate his or her strategic competence in the
oral interview; here strategic competence is essential for maintaining the flow of
communication when the candidate lacks a particular word or phrase, or needs to ask for
clarification.
How is communicative competence tested in the ECCE? Certain sections of the
test make use of real-life tasks as an appropriate vehicle for assessing communicative
language ability. Most of these tasks are representative of those included in EFL
coursebooks which aim at developing communicative competence.
In the first task of the listening section, the candidates listen to short dialogues or
announcements related to everyday situations. Candidates need to listen for gist or
specific information and respond to a set of pictures. There is no reading involved in the
task, and the skills tested are appropriate for this type of text. This design is intended
to enhance its authenticity as a communicative activity. In the second section candidates
listen to a ten-minute interview which is presented in short segments followed by two to
four questions after each section. Candidates are allowed to take notes, which is an
appropriate task for this type of text. Note-taking is also an authentic sub-skill which
most candidates need to develop.
The reading tasks assimilate real tasks in order to test different approaches to reading.
As can be seen below, the different tasks are matched with appropriate text-types (See Figure 1 ).
The speaking section of the test is an oral interview in which candidates talk about
personal background information and ask questions about a set of two or three photographs.
The questions have a purpose: to help candidates come to a decision, which they must then
explain. This role-play facilitates the use of certain features of spoken discourse that
are prevalent in real communication such as asking for clarification and forming
questions. Moreover, the fact that there is an outcome in such an activity increases its
communicative potential.
The writing task represents a real-life task, in that a purpose is created for writing:
Candidates write a composition or a letter in response to a reading prompt. The topics
relate to the life experience of young adults so that they are not stuck for ideas and can
concentrate on expressing themselves well.
Many of the features that contribute to the creation of real-life tasks have the following
characteristics:
|
|
|
- They are contextualised and relate to every day situations.
- They test authentic skills.
- They provide a purpose for communication.
- They are appropriately matched to text type.
- They have an outcome
|
|
|
On the whole, ECCE reflects current methodology
in teaching by focusing on testing the components of communicative testing discretely and
integratively using all four skills. It also makes an attempt to test students’
ability to use language through real-life tasks. There is a plethora of coursebooks to
choose from which aim to develop communicative competence in all four skills. The
teacher’s manual of such books usually includes a description of its objectives,
which should make explicit reference to the teaching of communicative transactions in all
four skills. However, as we have seen from the analysis of the ECCE, many of the test
activities test real world abilities and therefore could be used as teaching materials not
only to help the students with strategies for the actual test but also to help students
develop genuinely useful language skills.
|
|
|
Many teachers who are under pressure to prepare
students effectively for a particular test expose students to a number of practice tests
in the hope that these will increase competency, which in turn will enable the students to
pass the test. This is a misconception, particularly if the students receive limited
feedback from the teacher, such as simply supplying the correct answer. The following sug-
gestions for using the ECCE practice tests as a teaching tool, however, could lead to
effective language skill building and development of specific test-taking strategies
through active involvement of the learner.
ECCE listening section. In the following example the students listen to a
short dialogue or announcement, which is then followed by a comprehension question. The
students respond to the question by selecting the appropriate picture from three choices.
Students will hear:
|
|
|
Man: Here’s the delivery. You owe me
$20. I’d prefer you pay by credit card.
Woman: But I already paid. Just a minute. I’ll show you.
Woman: What does she need to get for the man? (comprehension question)
|
|
|
Students will see pictures (See Figure 2 ).
The above item requires the student to listen for specific information which requires
understanding of a particular structure (al-ready paid). Teachers can help students
develop this skill by introducing the following procedures:
|
|
|
- Make students aware of the need to predict the content which will help them to process
the information more easily. This can be done by discussing the pictures before they
listen.
- Provide a pre-listening question which will help students focus on the structure. For
the above item the question “Will the woman pay now?” would serve this purpose.
|
|
|
The following item from the same section
requires the student to understand the gist of the conversations:
Students will hear:
|
|
|
Man: This is terrible. I had no idea the
road would be so crowded.
Woman: I know. Let’s get off as soon as we can. Turn off at the next exit.
Woman: Where are they now? (test question).
|
|
|
Students will see pictures (See Figure 3 ).
Listening for gist can be practiced with the test materials in the following way:
|
|
|
- Before listening, the teacher can provide examples of contexts on the b/w board which
the students match with the pictures, e.g., someone giving instructions, a weather report,
description of person or place and sequence of events. Alternatively, students can suggest
contexts themselves.
- Students then listen to the dialogues and confirm their predictions.
- Students discuss specific language which helped them understand the gist of the
conversation.
|
|
|
In the second part of the listening students
have to understand the main points of a longer text usually in the form of an interview in
which someone is talking about a place of interest. The students hear the interview in
several parts, and after each part they are asked to respond to two to five questions by
indicating, “Yes, that’s true,” or, “No, that’s not true.”
They are allowed to take notes while they are listening. Therefore, activities which focus
on note-taking and listening for specific information would help the students
considerably.
Introduce the topic and elicit from the students key words related to the topic. This
develops predicting skills.
Provide written statements related to the text, some of which are the main points of the
text and others which are irrelevant. Students check the main points. Alternatively,
provide a while-listening activity which will help students focus on the main points. An
example would be a chart which indicates important dates about the place of interest. This
would help students follow the sequence of events and help them to organize their notes.
Grammar. Grammar is tested in a multiple-choice format and is contextualised
in short conversations or in sentences that reflect spoken discourse. Students need to
understand the meaning of the item and be able to recognize the correct form.
A: “Did you go to class yesterday?” B: “Yes, and you _______ gone,
too!”
|
|
|
- shouldn’t have
- would have
- wouldn’t have
- should have
|
|
|
The above item requires the student to
understand the language expressing the function of criticism. Teachers can make students
aware of the relationship between form and function with procedures such as the following:
Before the students look at the options, the teacher asks the following questions about
the meaning of the exchange.
|
|
|
Did A go to class? Did B go to class?
|
|
|
Do you think B is agreeing with A, advising A,
or criticizing A?
Then the students can check the options and match the function with the correct form.
|
|
|
“John went to the game -______ the bad
weather.”
|
|
|
- despite
- in spite
- despite that
- in spite of that
|
|
|
The above item requires the student to recognize
the correct form since all the options have the same meaning. Students can be helped with
development of accuracy by working with this item in the following way:
|
|
|
- Students write a sentence with each of the options before looking at the stem.
- Students discuss in groups the word order of their sentences.
- Students then choose the correct option for the given sentence.
|
|
|
Vocabulary. Vocabulary is tested
at the sentence level with the focus on recognition of correct use. This test activity can
be used to build up specific word-attack skills which will also be useful for reading
comprehension. For example, students can be guided to guess words from context by focusing
their attention on clues such as synonyms, antonyms, and reference words.
Example 1:
|
|
|
This is a real diamond, not _________.
|
|
|
- a compromise
- a conversion
- an alienation
- an imitation
|
|
|
Do you like my coat? I got it at _____ price.
|
|
|
- a bargain
- a bonus
- an easy
- a simple
|
|
|
As can be seen in example 2, high frequency
collocations in everyday contexts are tested. The best way to help students with
collocations in exam preparation material is to do the following:
|
|
|
- Students try to complete the sentence before looking at the options.
- Students then look at the options and make comparisons with their own choices.
- Students discuss the distractors with the teacher and come up with appropriate contexts
for them.
|
|
|
Reading. The first reading task is
expository prose between 220 and 250 words in length. The purpose of the text is to inform
the reader about a topic of interest such as environmental issues or human behavior of the
type you would find in a magazine or a leaflet. The multiple-choice questions check the
students’ ability to understand the main idea, specific details, and to make
inferences. Multiple-choice questions can be very effective for training interpretative
skills if the teacher exploits them in the following way:
|
|
|
- Make students aware that distractors for reading items are written with students’
likely mistakes in mind.
|
|
|
- They contain words from the text but the overall meaning is different.
- They often exaggerate the meaning by using all, never, or always.
- They may say something that is possibly true but not stated in the text.
|
|
|
- By understanding the nature of the distractors, the students can then discuss the
justification for their choices in groups and then present their arguments to the whole
class.
- For further practice in interpretative skills, the teacher can produce a number of
statements that can be inferred from the text and a number of statements that could be
true but are not stated in the text. This activity could also generate useful group
discussions.
|
|
|
The second and third reading tasks require the
student to skim and scan the text quickly. The following guidelines could be useful for
assisting students to retrieve information quickly and accurately:
|
|
|
- Direct students to read the questions first so that they read with a purpose.
- Make them aware of the importance of rubrics to locate information quickly.
- Time the task so that students are forced to skim and scan and not waste time reading
every single word.
|
|
|
Writing. In the writing section students need to
write a letter or composition in response to a prompt (letter or short article). They have
only thirty minutes to write a coherent text so there is obviously no time for the
students to write more than one draft. However, students can be trained to apply a process
approach to writing in which they manage their time effectively. For example, they should
be encouraged to work out how much time they need for each stage of their writing in the
following examination conditions:
|
|
|
- reading of prompt and underlining key words (2 minutes)
- noting down ideas and useful language (4 minutes)
- organizing ideas into paragraphs (2 minutes)
- writing of composition (20 minutes)
- editing (2 minutes)
|
|
|
Speaking. The students’
speaking ability is assessed through a role-play performed with the examiner. Frequent
practice with roleplays in the classroom helps the student to develop fluency and
accuracy. Controlled practice with typical question forms preceding the activity may help
to develop the students’ accuracy, but students also need to develop strategic
competence for managing conversations more effectively and giving an impression of
fluency. Practice with the following categories of words and expressions may help to
achieve this end:
|
|
|
- asking for clarification’ What do you mean exactly?
- hesitating’ Let me see.
- fillers’ O.K., fine, right
|
|
|
The above suggestions are by no means
exhaustive. The important thing to remember, however, is that this approach for teaching
exam practice test material is much more instructional, and teachers should spend just as
much time preparing these lessons as regular lessons.
|
|
|
In this article I have analyzed the ECCE and
illustrated how it could be considered an example of the movement towards
achievement-oriented proficiency testing. The analysis shows that objectives of the test
resemble syllabus specifications of current ELT coursebooks which aim to develop
communicative competence. Furthermore, the test activities can be exploited effectively as
classroom activities which not only help the students to become familiar with the format
of the examination but also help them to develop useful language skills. In order to
maximize the washback effect, however, teachers need to be empowered with a greater
understanding of the principles involved in communicative testing. It is therefore
essential that all teacher education programs contain a testing component which includes
an analysis of various proficiency tests and practical ideas to illustrate the positive
effects of the relationship between testing and teaching.
|
|
|
CHRISTINE IRVINE-NIAKARIS is Director of Education at the Hellenic American Union, Athens,
Greece. |
|
|
Return
|
|
|
- Canale, M. and M. Swain, 1980. Theoretical basis of communicative approaches to second
language teaching and testing, Applied Linguistics, 1. pp. 1–47.
- Canale, M., 1983. On some dimensions of language proficiency. In Issues in Language
Testing Research, ed. J. W Oller. Rowley, MA: Newbury House.
- Hymes, D. H. 1972. On communicative competence. In Sociolinguistics, eds. J. B. J. Pride
and Holmes. Harmondsworth: Penguin Books.
- Weir, C. J. 1990. Communicative Language Testing. London: Prentice Hall International
(UK) Ltd.
- ———. 1995. Understanding and Developing Language Tests, Phoenix ELT.
|
|
Figure 1
| Text task and purpose |
Text type |
|
|
| Intensive reading skills to
obtain through comprehension |
Expository prose |
| Skimming and scanning to make a
decision, e.g., selecting a school, holiday |
4 or 5 short texts which can be
compared and contrasted to find the information desired |
| Skimming and scanning to find
relevant information |
A longer text in the form of a
brochure or manual |
|
|
Figure 2
| PAID INVOICE |
| CASH |
| CREDIT CARD |
|
|
Figure 3
| CROWDED BUS |
| CROWDED THEATER |
| CONGESTED HIGHWAY |
|
|
Back to Article
|
. |
. |