A critical analysis of selected IELTS preparation materials

Abstract

Commercially produced IELTS preparation materials are becoming increasingly available for use by students in self-study programs and by teachers as classroom resources in IELTS preparation courses. This study investigated the appropriateness of the content, organisation, learning approaches and presentation of the listening and reading components of six of the most widely used of these publications.

A numerical analysis was conducted of item-types and a criterion-based evaluation of passages/texts and items. The latter used a purpose-designed set of criteria to facilitate comparisons across texts and tests. Results of analyses were tabulated, and these tables form the basis of discussion in the body of the report. Further analysis considered more subjective perspectives gained from users through student surveys and structured teacher interviews in three Australian language centres. The publications were found to provide a wide range of resources suitable both for self-study and in the classroom. A number of recommendations referring particularly to the reading texts are made, but there are also some observations made concerning the practice tests in general and the need to eliminate elements which may lead to confusion and frustration on the part of students.

The study also considered the broader issue of the role of the publications in preparing students for the IELTS and in the development of language skills in general, with particular reference to preparation for further study. The study finds that IELTS preparation materials should include more texts and tasks that would contribute to the social and academic acculturation of students. As the IELTS was found to have increasingly significant impact on teaching programs and student self-study programs, then publications purporting to prepare students for the IELTS have a responsibility to not only simulate practice tests but to also contribute positively to the development of overall language, and learning skills.

2.1.2 Framework for comparison

The report used the following criteria to compare features of the listening and reading practice tests and preparation materials.

Figure 1a: Criteria used in discussing and comparing features of the listening practice tests and preparation materials

Text characteristics:

• Topics: range and relevance
• Range of text-types and linguistic features
• Level of difficulty relating to text delivery, pace of delivery, clarity of enunciation, and other features of oral language, which affect the students’ ability to identify essential information
• Level of linguistic complexity and organisational difficulty of the text: text structure including repetitions, re-wording, summary sentences, potentially confusing contradictions, difficult or unfamiliar vocabulary and other features which assist or hinder the learner in fulfilling the task

Test features:

• Test layout: clarity of layout and consistency with IELTS Specimen Materials (1997)
• Accessibility of instructions to students
• Reading and vocabulary demands of tasks
• Range of question-types
• Variety of task-types: the range of skills learners are required to display
• Complexity of tasks: the relationship between the text and the question and the level of skills required to complete the task

Test management strategies:

• Test-taking skills: hints and teaching points on managing the test as a whole
• Strategies for managing individual tasks: explanation of, practice exercises and teaching points focused on particular questions and task-types
• Other: other features of the book, including layout and accessibility of scripts, extra exercises, answer keys

Figure 1b: Criteria used in analysing features of the reading practice tests and preparation materials

Text characteristics:

• Topics: range and relevance
• Range of text-types and linguistic features
• Level of linguistic complexity of sentence and text structure, vocabulary contextualisation, in-text definitions, concrete or abstract concepts

Test features:

• Test layout: clarity of layout and consistency with IELTS Specimen Materials (1997)
• Accessibility of instructions to students
• Range of question-types
• Variety of task-types: the range of skills learners are required to display
• Complexity of tasks: the relationship between the text, the question and the level of skills required to complete the task

Test management features:

• Test-taking skills: hints and teaching points on managing the test as a whole
• Strategies for managing individual tasks: explanation of, practice exercises and teaching points focused on particular question and task-type
• Other additional features of the material, including layout and accessibility of scripts, extra exercises, answer keys

5.0 Test management features

Each of the publications under review contains additional information and IELTS preparation material to accompany the practice tests. The following section contains a short overview of the additional preparation material contained in the publications.

5.1 Test-taking skills and strategies for managing individual tasks

Each of the publications contains general information about the contents, procedure and format of the IELTS test. In each, the authors give instructions concerning the use of the books and advice on obtaining the most benefit from the book. All the publications give advice on how best to prepare for the IELTS test.

Deakin (1996, p. 4) includes a section on general test-taking strategies. The paragraph on listening strategies explains the way the listening test is conducted with advice on how to follow the listening texts. The paragraph on reading (p. 4) emphasises the importance of keeping to the recommended time limits and general advice about checking answers and other details. Also included in Deakin (p. 6) is a section summarising the key listening strategies and the key skills being tested. The reading strategies outlined are those central to the test-taking process, with emphasis on skills relating to time-management. The advice and information is clear and concise. No examples or practice exercises are included.

Gibson, Rusek and Swan (1996) include a user-friendly page of practical hints and instructions for the actual test day. Also included is a set of questions and answers regarding the listening and reading tests. There are no examples or practice activities before the practice tests and no guidance in managing the readings, apart from the general advice already mentioned.

Jakeman and McDowell (1996), provide the general information on test-taking skills and being prepared on the day of the test, with the requirements of each module of the test clearly laid out and explained. Also included in the answer key is an outline of the tasks and the skills being tested. For each group of questions, a suggested approach is given. These additions are both helpful for individual learners but also useful for generating discussion in class, and enabling learners and teachers to focus on appropriate reading strategies. Sahanaya and Lindeck (1998), and Sahanaya, Lindeck and Stewart (1998), include general information about the IELTS test with particular emphasis on the instructions and how to follow them correctly. Also included are a number of practice exercises and examples with explanations of both correct and incorrect answers in the key.

Todd and Cameron (1996) contains general information about the IELTS test, advice for students on exam preparation and various practice strategies. There are a number of helpful hints regarding the day of the test and advice on using the various units of the book. There are no explanations of answers nor is there guidance in managing particular tasks.

van Bemmel and Tucker (1994) contains a general description of the test with a detailed description of each module, including the kinds of tasks students may expect to encounter. Also included are a number of practice activities for the listening and reading modules with a comprehensive outline of the skills needed to approach the various tasks. The answers to the practice activities are accompanied by explanations. There are some general hints for the reading modules, as well as some advice on dealing with unfamiliar vocabulary. The additional material is useful both for individual learners and as a basis for teaching sessions in an IELTS preparation course.

5.1.1 Other

All the publications are well laid out and include answer keys, although some of these contain occasional errors. All the publications, except for Deakin (1996,) contain tape scripts for the listening practice tests. One observation that could be made is that most of the publications would be more accessible with a more detailed contents page. If the materials are to be used as classroom resources the various items could be more quickly located if the contents page contained more details, such as reading passage titles and page numbers for modules and sections of the tests. This is especially so in the texts where the practice tests are laid out as complete tests, as in Jakeman and McDowell (1996). A useful addition in Gibson, Rusek and Swan (1996) is the page (p. 2) which outlines the reading test passages by number, title, theme and page number. This same publication also has a large number of reading passages providing additional practice for students.

Some difficulties were encountered in understanding the format of the first section of Sahanaya, Lindeck and Stewart (1998). The explanation of question-types is useful and the deconstruction of the reading passages with an attempt to enable the learner to understand why the answers are correct is particularly helpful. However, the lack of a separate answer key and the repetition of some readings, without a clear indication in the first reading that help is available for that text, may mean that some of the valuable information is overlooked.

Apart from the above observations, there were few comments to be made on the publications as a whole. Teachers obviously had their preferred publications and were familiar with those they had chosen to work with.

6.0 Teacher perspectives

As previously mentioned, 15 teachers from three different centres were interviewed (see Section 2.1.4). The interviews were semi-structured, using a set of prompts previously faxed to the participating students. The prompts for the teacher interviews can be found in Appendix 4. While the discussions mainly focused on the IELTS preparation materials, the issues raised were those central to teaching English as a second language. The teachers’ most important concern was to help students prepare for the IELTS test without jeopardising the coherency and pedagogic rationale of the English language teaching program.

6.1 Teacher comments on particular materials

Teachers were asked to comment on the publications included in the study (see Section 2.1.1). As few teachers were familiar with all the publications on the list, few comparisons between the books were made. However, there were many relevant comments made on the various publications. In general, teachers tended to rely on those books they were familiar with, while at the same time, looking for new materials to add to their resources.

Teachers made particularly favourable comments about the Cambridge Practice Tests for IELTS (Jakeman and McDowell 1996), although some teachers expressed the opinion that the listening tests were a little too easy. Teachers also commented favourably on the useful strategies included in IELTS to Success: Preparation Tips and Practice Tests (van Bemmel and Tucker 1997) and IELTS Preparation and Practice: Academic Reading and Writing Module (Sahanaya and Lindeck 1998) IELTS Preparation and Practice: Listening and Speaking Module (Sahanaya and Lindeck and Stewart 1998).

Prepare for IELTS: Academic Modules (Todd and Cameron 1996) was also a text that teachers were familiar with and regarded as a useful and interesting resource. However, once again it was felt that the listening texts and some of the readings were not as challenging as those in the IELTS test or IELTS Specimen Materials (1997). Not all teachers regarded the fact that the listening texts were not at a level of difficulty consistent with the IELTS or the IELTS Specimen Materials (1997) as constituting a problem. This issue is more fully discussed in the section relating to learner proficiency levels (see Section 6.5). IELTS Practice Now (Gibson, Rusek and Swan 1996) received mixed comments. Some teachers found the readings both useful and interesting, especially as there is such a large number of readings ranging over a variety of topics. However, some teachers had found some of the tasks quite difficult and confusing. The listening tasks in this publication were also found to be quite difficult and one teacher claimed to have discontinued use of the book because of the difficulty of the listening passages.

Practice Tests for IELTS (Deakin 1996) was felt to be a good basic text with consistent levels of difficulty throughout the tests but obviously not as useful as a teaching resource as the texts with more explanations and strategy training activities.

A number of teachers used the IELTS Specimen Materials in the versions available in their own centres, as they felt this gave the students a realistic idea of the requirements of the IELTS.

The two publications not included in this study which were mentioned most positively by teachers were IELTS: Strategies for Study (Garbutt and O’Sullivan 1996) and Prepare for IELTS: The IELTS Preparation Course (Cameron 1999). Teachers expressed the opinion that the approach to teaching strategies for language learning and for managing the IELTS in both these publications was pedagogically sound and helpful to both teachers and learners.

6.2 Skills developed by IELTS preparation materials

All the teachers interviewed acknowledged the benefits of using the practice tests to provide practice for students in following instructions, developing time-management skills, and becoming less reliant on the dictionary. Clearly, the practice tests familiarise students with the test format and the general expectations of the test and help students improve their overall test-taking skills. One teacher expressed the view that the practice tests enabled the students to become ‘psychologically ready’ for IELTS. In developing concentration and an awareness of the demands of the test, students are less likely to become flustered on the day of the test.

However, teachers expressed concern about the futility of excessive practice without the teaching of the strategies or the language skills needed for operating in academic English. The opinion was expressed that some students “become so obsessed that they want to only focus on IELTS rather than on the language skills they need to develop and in the absence of knowing how else to prepare for the IELTS, they cling to the practice tests”. Views expressed by all teachers reflected to varying degrees the concern expressed by Hayes and Watt in their investigation entitled ‘An IELTS preparation course for Asian students: When practice doesn’t make perfect’ (1998 p. 15), ie. “simply providing students with material from IELTS practice books contributed little to their overall language development and consequent IELTS performance”. This realisation by the authors led to the “development of a 350 hour course of topic-based units, consisting of activities designed to raise language awareness, develop language learning strategies, and familiarise students with the exam”. Consequently, teachers tended not to rely on the materials to teach the skills they felt were needed, but chose the materials to compliment their own teaching programs. Those books with strategies for managing the tasks were regarded as useful in developing such programs.

Another concern expressed by teachers regarding the IELTS preparation materials in general was that there were too many comprehension-check activities and too few tasks aimed at helping students gain a broader understanding of the texts. This issue relates to the overall concept of test practice and is discussed in Section 8 Summary of key issues and recommendations.

6.3 Factors affecting teachers’ choice of tasks and practice materials

Teachers were asked what factors influenced their choice of tasks and practice materials for individual teaching sessions. Choice of texts and tasks was most influenced by the range of questions and task-types included in the materials, with consideration given to the level and interests of the class. Those teachers who had developed a range of strategies they intended to teach, chose texts through which those strategies could be most successfully practised. Teachers also endeavoured to choose tasks they felt were consistent with the level of difficulty in the IELTS.

6.4 Difficult question and task types

Teachers were also asked about the question and task-types which students appeared to have the most difficulty with. No distinction was made between the practice test materials, and the specimen materials, and it appeared, from student feedback, that students also found the same question-types difficult in the IELTS. The following remarks apply to those practice test items teachers had used in class. Most teachers commented on multiple-choice questions, the choices for which are often ambiguous and in some cases have such a fine distinction that it is difficult to justify the correct answers. The choice of one answer or another often does not appear to reflect the learner’s real language ability. Tasks involving the answers ‘yes’, ‘no’ or ‘not given’ were also regarded as difficult and often confusing. Tasks requiring the matching of headings and paragraphs were also often described as difficult and confusing.

Close and summary-type tasks were also mentioned by teachers as causing difficulty for students, but it was recognised that these kinds of tasks, when well constructed, provide valuable practice in managing the language. However, teachers felt that the tasks that were too difficult were not helpful at all and caused frustration for both students and teachers. These kinds of questions may also promote incorrect study techniques, in which students read to answer the question rather than to gain an understanding of the text. Also, teachers gave the opinion that, if the questions appear unfair or the answers unjustifiable, the students tend to lose confidence in the whole process, perhaps even

including the IELTS scores. For these reasons then, the practice tests have a responsibility to be set at the appropriate level and to be as clear and manageable as possible.

6.5 The impact of the proficiency level of the learner on IELTS preparation courses

Teachers were asked about the impact the initial proficiency level of the learner had on the learner’s rate of progress and on the IELTS preparation course. Predictably, those teachers teaching classes that imposed a minimum entry level found the preparation for IELTS both more satisfactory and more successful. Most teachers were comfortable with the idea of IELTS preparation sessions consisting mainly of training in the strategies needed to manage IELTS, with few language skills included. This was particularly so in cases where the IELTS sessions were part of an Academic English Program.

Where students were not required to achieve a minimum level of English before entering the class, teachers expressed more concerns about the students with lower levels of proficiency being unable to grasp the strategies being taught. Some teachers felt that the development of some strategies at least enabled students to demonstrate the language skills they did have, but in most cases it was felt that the difficult texts confused them and eroded their confidence.

In teaching students with lower levels of proficiency, teachers felt that the easier practice tests were more useful in helping the students prepare for IELTS and in developing language skills than the more difficult tests. While there needs to be some recognition of the fact that the practice tests may not be of the same standard as the IELTS, teachers felt that such practice was more beneficial than practice on tests which may be largely incomprehensible.

6.6 Measurement of student progress

Teachers were asked how student progress was measured in IELTS preparation courses and whether the results achieved in the practice tests were consistent with the IELTS scores received by the students. Most teachers responded that in the limited time available the students were not assessed but the practice tests were often used to provide informal feedback to students. The differences in the levels of the various tests did not allow for patterns to emerge over the short time of most IELTS preparation courses but teachers indicated that those students who were comfortable with the tests and could manage most of the question-types usually perform satisfactorily in the IELTS. Students struggling with the question types and having difficulty understanding the complexities of the test were much less likely to achieve satisfactory results in the IELTS.

In general however, teachers were of the opinion that students achieved higher scores in the practice tests than in the IELTS. While it was felt by some that the practice listening tests were easier than the IELTS, in fact the higher scores achieved in practice tests may be attributable to other factors, including a more relaxed testing situation, with more explanations provided by the teacher and more time given for completion of the practice test.

Teachers expressed the opinion that IELTS preparation courses should also undertake the role of helping students to assess their own level of proficiency and to gauge their own readiness to manage the IELTS. Certainly if the materials are called ‘practice tests’ the students tend to put their faith in them, so the tests need to be of a sufficiently high standard.

6.7 The place of IELTS preparation materials in the broader dimension of preparing learners for further study

Teachers were asked whether they felt that IELTS preparation courses should have the broader focus of helping students to prepare for further study. The responses to this question were mixed. Most teachers felt that students did not make connections between the IELTS and further study and the study plans of the students were so varied that it was not realistic to expect IELTS texts to be relevant to the specific subjects of further study. However, some teachers did feel that the materials should include more academic texts with appropriate tasks, while others felt that more reading and listening tasks dealing with current issues should be included. Teachers felt that, considering the amount of time and energy expended in preparing for IELTS, more could be achieved in promoting the social and the academic acculturation of the students.

6.8 Other comments and suggestions

Teachers were asked to comment further on any of the issues raised previously or suggest changes for future IELTS publications. Most teachers indicated their opinion that future materials should include the strategies needed to manage individual item-types, as already mentioned in connection with the publications by Cameron (1999) and Garbutt and O’Sullivan (1996), as well as the test-taking skills already included in a number of publications. Publications could also provide more assistance in developing the language skills necessary to manage the reading texts by including more deconstruction of texts and explanation of the language features involved.

The opinion was also expressed by several teachers that the reading texts in the practice tests should be more closely related to the IELTS writing and speaking tasks. As students usually have some difficulty in generating ideas, it was felt that it would be of assistance to students if the reading texts were concerned with the same kinds of issues as the writing and speaking modules.

There was some difference in the way the practice materials were viewed by the teachers involved in language support programs at university. These teachers tended to be more critical of the reading texts and felt that more discrimination could be employed in choosing texts which could provide models for academic writing and which could be generally more relevant.

7.0 Student perspectives

7.1 Numerical information

Two student surveys were administered. The first, completed by 52 students, comprised a series of questions directed at students preparing for IELTS. The second, completed by 26 students, was a shorter set of questions, addressed to students who had already taken the IELTS (see Appendix 5). The purpose of both surveys was to elicit students’ opinions of and experiences with the IELTS preparation materials. Points of interest in the first survey included: the features students looked for when choosing study materials; the way in which the materials are used by the students; students’ perceptions regarding the difficulty of practice tests; students’ opinions on the extent to which IELTS training should relate to further study; and students’ suggestions on changes or additions which could be implemented in future publications.

The second survey was aimed at obtaining feedback on the IELTS materials from a post-IELTS point of view. Students who had already taken the IELTS were asked to: compare the results of the practice test to their IELTS scores; explain how the IELTS scores differed from the practice test scores; and comment on the practice listening and reading materials.

Of the students surveyed the main first language groups were: Chinese (30%); Indonesian (21%); Korean (17%), Japanese (9%) and Thai (9%). Of the remaining 10% of students less than 4%, ie. two students, were from non-Asian speaking countries. Almost 60% of the students were between the ages of 23 and 30, with 25% of students over 30 years of age and only 15% 22 years of age or younger.

There was a range in the length of time students had been studying in Australia and in the time they had been enrolled in their current IELTS preparation course. The length of time students reported studying in Australia varied from one month to more than eight months, with a fairly even distribution over the time periods outlined in the survey. Over 60% of students had been enrolled in their current IELTS preparation course for less than two weeks; almost 39% for five to eight weeks and only 7% for more than eight weeks.

The main reasons given for taking IELTS were for the purposes of further study. Over 90% of students were taking IELTS as a pre-requisite for entry to university, two students were planning to continue their study at TAFE, one person was taking IELTS for immigration purposes and one person to check their English level.

7.2 Students’ choice of publications and practice tasks

Students were asked what they looked for when they chose IELTS practice tasks. The most important requirement for students was clear instructions. Familiar and/or interesting topics were also considered very important to students. New-looking materials and recently published materials were also rated highly. A number of other criteria were mentioned, including the organisation of the book, the level of difficulty, tasks containing exercises and books outlining strategies to manage the tasks. By far the largest number of responses, however, indicated the necessity for clear instructions and familiar or interesting topics.

7.3 Practice test conditions

Students were asked whether they did practice tests in the time-limit given, without rewinding the tape during the practice test and without the use of a dictionary. Over 75% of students indicated that they did take the practice tests under these real-test conditions. However, a number of students indicated that they extended the time for reading tests and listened to the tapes for listening tests two or three times before being able to answer the questions. Some students felt that the practice tests were a good opportunity to learn new words and language structures, which could not be done within the time limits set for the test.

7.3.1 Consistency of practice test scores

Students were asked whether their practice test scores were consistent, ie. were scores obtained in practice tests always similar? There was almost an equal number of yes/no responses to this question. Students accounted for the differences in practice scores in a number of ways. A small number of students attributed lower scores to loss of concentration and their feelings on the day. Quite a number of students indicated that familiar topics and vocabulary made some tests easier, while “others are more difficult because of lack of background knowledge or unfamiliarity about them” and also, “some tests are easier to guess”.

Other comments referred to the fact that some tests were difficult to understand and “those with guide answers are clearer and easier”. A small number of students indicated that they had found the speaking “too fast”. It is obvious from the students’ comments that they do need support and guidance, both in selecting materials and in using those materials to develop the skills they need, to perform to the best of their ability in the IELTS.

7.4 IELTS training and further study

Students were asked whether they thought that IELTS training should help them prepare for further study. Over eighty percent of students felt that the training should help them prepare for further study. None of the comments indicated that students felt the training to be useful for broader study purposes, except to the extent that it assisted their reading and writing skills. One student commented “IELTS training can only help me gain an ideal score”, while a number of comments indicated that the training should be more practical. One person commented that “materials are very useful but there are sometimes materials with very strange topics”. Another student wrote, “I think EAP class which I took was more useful, such as lecture class, notetaking and academic reading and writing”.

Obviously, there is scope for a more academic focus to both the IELTS training courses and publications, particularly in those centres that have a large percentage of students taking IELTS for the purpose of entering university.

7.5 Students suggestions for future publications

Students were asked to suggest any additions that would make future IELTS listening and reading materials more useful in preparing for the test. Student comments were quite varied in this section, but common concerns included: the need for more listening materials, with some students feeling a need for more availability of listening materials within their centre; more guidance in managing reading tasks; and more similarity between practice tests and the IELTS.

Other issues raised were more to do with the content of the texts. Comments indicated that more “interesting and useful topics” were wanted; and “both in listening and reading materials, there should be more materials that are concerned with the latest information, such as the latest news and magazine articles”, and there should be more variety to choose from. These comments appear to be more related to the materials available within the centre in question than to the selection of publications listed in this study.

Comments specifically relating to listening materials suggested that listening materials should “include some more usual talk” and “listening materials should have more varied questions”.

As we can see from the comments, concerns expressed by students are reflected in the issues discussed in the study and are similar to concerns raised by teachers.

7.6 Phase 2 Survey: Post-IELTS Viewpoint

Student comments in the second survey tended to be more focused and the materials were rated differently. In both surveys students were asked to indicate which materials from the list provided they had found most useful. In the first survey this was done by completing a table with space against each publication for them to mark materials according to the availability of the publication, their use of it and its and usefulness. In the first survey, students noted the two publications - Todd and Cameron (1998) and Jakeman and McDowell (1996) - most frequently and most positively. These two publications were also identified as being the most readily available for those students. In the second survey, in which the respondents were identified only as having already taken the IELTS, the two publications rated most highly as being useful in preparing for the IELTS were Gibson, Rusek and Swan (1996) and Jakeman and McDowell (1996). In this survey, students also volunteered the titles of four additional test preparation publications they had found useful.

7.6.1 Comparing practice test scores with IELTS scores

Students were asked whether the scores they had received in their IELTS test were similar to those they obtained in practice tests. Only 50% of students reported similar results in the reading module and slightly more in the listening module.

Students were asked to explain what the differences were in their scores and make comments on the practice materials, comparing the practice materials to the IELTS. Comments made by the students indicated that most students had found the listening practice tests easier than the IELTS. Other students remarked on the effect of exam conditions on the listening test. One student commented “the listening test is not clear in exam conditions”, and another stated, “It is better if we can use the language lab in the listening test. It is very important!”.

The reading in the practice tests was also rated as easier than the IELTS, but not to the same extent as the listening test. The publication IELTS Practice Now (Gibson, Rusek and Swan 1996) was singled out by one student as being “good for reading. The other books are easier than the IELTS test”.

While most students had found at least some of the practice tests easier than the actual test, a number of students acknowledged the benefits of using the practice tests to prepare for IELTS. Comments included the following: “They’re all good for practice”; “The materials are easier than the IELTS test but they give a very good strategy for preparing yourself”.

In brief, the comments made by the students, rather than providing new or contradictory information tended to highlight such issues as: the need for guidance in managing the tests and understanding the demands of the test; the need for more focus on strategy development; the importance of an academic focus; and comparative difficulty of practice tests and the IELTS.

8.0 Summary of key issues and recommendations

A number of issues has been raised and discussed in this evaluation of the practice tests. Paramount is the issue of the place of IELTS practice materials in developing English language skills and the relation of the materials to the IELTS itself.

From one point of view, there should be no need for practice materials other than the official IELTS Specimen Test. However, unlike practice materials, the main function of the Specimen Materials is to model the items, texts and format of IELTS. They do not set out to be a pedagogical tool, as such. Practice materials reside more clearly in a pedagogical context and are seen as learning tools by students and teachers. Indeed the very existence of these materials is evidence of a positive educational trend, in that, used effectively, they mostly reflect educationally desirable models of reading and listening practice. However, the tendency for students to use practice tests ineffectively, in a repetitive, rote-learning process, suggests that inclusion of additional pedagogical guidance and explicit learning strategy guidance is desirable. Materials conceived as ‘test preparation’ resources are more likely to do this than materials conceived as ‘practice’ materials.

There was a satisfying process of validating the project team’s insights through the teacher and learner input to the project. Issues that became most apparent in the analysis of the texts and tasks were in most cases also commented on by the teachers. However, most teachers tended to comment in a general way on the practice tests, and introduced the features of the texts in a random order throughout the interviews. By establishing a set of criteria and constructing a framework for the evaluation of the practice tests, it has been possible to establish a context for, and substantiate observations made about, the practice tests.

It is clear from the evaluation tables that the layout of the publications is generally user-friendly. The information for teachers and students about the test and on maximising the use of the practice test material is also generally sufficient and accessible. However, it is clear that some practice tests are substantially more difficult than others. The evaluation tables can be used to pinpoint the features of the texts and tasks contributing to their level of difficulty.

There is also clearly an issue of the relation between EAP programs and IELTS preparation programs, and the extent to which these are complementary. In their investigation into the methods of preparing students for IELTS, Hayes and Watt (1998) come to the conclusion that using the IELTS material in a topic-based syllabus, with additional vocabulary and skills practice in the same context, had a significantly more positive effect on the learners’ progress than did using a selection of unrelated texts. Any texts which introduce students to new topics and new vocabulary may be considered useful for developing skills in coping with unfamiliar vocabulary. However, the vocabulary dimension is only one aspect of language learning. Texts which can be linked to a broader educational program and focus must be considered more appropriate in an EAP classroom, as these support more comprehensive language learning and language skills development by students. From this point of view, preparation programs organised solely on the basis of practice tests may be restricted to developing test-taking skills, at the expense of academic English skills. Teachers need to make a considered judgement as to how and how often to use practice tests in their test preparation programs.

8.1 Listening texts and tasks

In general, the topics covered by the listening texts were found to be both relevant and appropriate. A variety of text-types was included, displaying a wide-range of linguistic features. In regard to the features of individual texts, there were few criticisms. The informal structure of some texts may cause difficulties for students and some texts appeared to have quite long, unbroken sections compared to the IELTS Specimen Materials (1997). However, despite these difficult texts, from both the analysis of the texts and tasks, and feedback from teachers, it appeared that many of the listening practice tests were easier than in the IELTS listening sub-tests. The easier texts appeared to have more repetitions of essential information and the pace of delivery was often slower than in the IELTS Specimen Materials (1997).

Also, there was found to be an overall lack of authentic lecture-type tasks. There were very few expository texts in which students were required to understand more abstract ideas. In the more difficult texts some abstract ideas exist, but the difficult vocabulary tends to obscure the meanings for the learners. There is definitely a need for more of the kind of lecture topics, including abstract ideas and discussion, which would be encountered on entering university study. Such texts would need to be accompanied by appropriate notetaking tasks and should have the potential to generate class discussion and interaction.

Recommendation 1

Listening practice tests can be improved by including scripts of a more normal pace of delivery with appropriate strategies to help students manage those texts.

Listening practice tests should contain more authentic lecture-type tasks with appropriate academic note-taking tasks.

8.2 Reading texts and tasks

In considering aspects of field, mode and tenor in the texts within the various practice materials evaluated in this study, it is clear that many reading texts conform to broadly accepted characteristics of academic writing and are thus providing models consistent both with texts in the IELTS Specimen Materials (1997) and the IELTS Specifications for Item Writers (1998). These texts are also broadly consistent with texts generally encountered in reading related to university study, though not necessarily with specialist registers within specific disciplines. Further, they can form a base for effective teaching and learning activities in the IELTS preparation classroom. However, as discussed in Section 4.0, not all reading tests were found to meet the criteria.

Quite a number of texts reflected a journalistic style, the features of which were discussed earlier, rather than discussion at an academic level. While there are sufficient texts of an explanatory nature, there need to be more texts of a discussion or argumentative type, particularly as this is the kind of text students of non-English speaking background have the most difficulty in managing. The inclusion of such texts would also tend to exclude trivial topics and should generate more of the tasks appropriate to academic reading.

Recommendation 2

Reading texts for IELTS practice should be academic in both content and structure. Rather than focusing on providing a wide range of question-types, tests should focus on tasks that both extend and test the language skills students will need in their programs of further study. The topics of the texts should be suitable for EAP classes and should provide material that promotes discussion and interaction within the classroom. Students preparing to take the academic version of the IELTS usually have a serious purpose for undertaking current studies, and so should be given as much assistance as possible in their social and academic acculturation. Texts that open avenues for discussion on recent issues and topics, related to tertiary studies, constitute useful teaching resources.

8.2.1 Practice materials for training in or modelling IELTS tasks

Most recent publications contain teaching materials on listening and reading strategies. However, there is definitely a need for more guidance for both students and teachers in coming to terms with reading texts of this nature. As mentioned above, the use of the term ‘practice test’ serves to lead the learner away from the notion of learning new strategies and towards a less meaningful process of repetitive trialing. Strategies to manage large sections of text and an awareness of text structure are important to an understanding of the whole text and its purpose. A deconstruction of the reading texts and an explanation of the features of the texts would enable students to identify the patterns of text structure. Accompanied by an explanation of the particular reading strategies which would be appropriate for that particular text, this would be very useful in developing both strategies and language skills. From each set of readings studied, students would gain knowledge of the texts and an understanding of how best to approach them. From the discussion above, it is apparent that repetitive practice on random texts does not markedly increase learner proficiency. A coherent teaching or self-directed learning program is needed, if learners are to improve their language skills through IELTS preparation activities.

Recommendation 3

Reading practice materials should incorporate a coherent program of skills development as well as model-test passages and questions.

8.3 Assumptions made about teacher familiarity with the IELTS

All publications were found to contain a range of information about the IELTS. In general, the more recent publications contain more information about the test and the test procedure than do earlier materials. However, even for teachers and students completely unfamiliar with the test, each of the publications contains sufficient information to prepare students to manage the format and procedure of the test.

Practice tests which differ in level of difficulty from the IELTS may cause some problems for those unfamiliar with the IELTS or the IELTS Specimen Materials (1997). It may be that some acknowledgment needs to be made in the publications that the materials are intended to provide practice in various skills at various levels but may not fully replicate the level of difficulty of the IELTS.

8.4 Development process for practice materials

While the overall feedback on the practice materials and accompanying tasks was positive, there were comments on the number of tasks, particularly those accompanying the reading texts, which were found to be confusing. Both teachers and students expressed frustration over ambiguous answer choices, and answers which were not clear. Had the practice tests been thoroughly trialed in a classroom situation some of these difficulties could have been eliminated. To reduce confusion further, for tasks requiring an in-depth understanding of the text, explanations of the correct answers would be helpful to users of the materials.

Recommendation 4

All practice tests be thoroughly trialed to reduce the number of ambiguous and unclear answers.

Difficult texts, with tasks requiring a finer discrimination of meaning, should be accompanied by explanations.

9.0 Conclusion

This study has divided its attention between exploring the relation of practice materials to the IELTS test and exploring the uses of practice test materials by teachers and learners preparing for the IELTS. The former concept led to the generation and discussion of the numerical analysis of test-items (detailed in the Appendices), while the latter concept has informed much of the discussion of features of the test materials in the body of the report and the Evaluation Tables (also contained in the Appendices).

The analyses were carried out in part with a view to assisting teachers to understand the potential and shortcomings of these commercially-produced materials as test preparation vehicles. Ultimately, one does not expect the practice materials to impact on the design of the IELTS. Indeed, the IELTS Specimen Materials (1997) and the IELTS Specifications for Item Writers (1998) were used as benchmarks for the criteria discussed in this study. Nevertheless, the study has served to tease out how teachers and learners perceive the test and test preparation contribute to achieving longer term student goals of entering tertiary study and/or becoming permanent residents in Australia. The study has thus tried to speak to a diverse audience, including IELTS developers, language teachers, language materials writers, IELTS administrators and examiners, and, indirectly to a small extent, to English language learners. We hope the study has also contributed useful insights to the ongoing discussion of the relationship between language testing and language learning.

Acknowledgements

We wish to acknowledge the support and advice of the members of the Project Steering Committee (Robyn Muldoon, Ingrid Wijeyewardene and Chandini Tilakaratna), in the implementation of the project, the design of the evaluation tables and the preparation of the report. Their insights and ideas have enriched our thinking throughout the project and resolved many difficult questions along the way. We would also like to thank Sharon Durrant for her patience, persistence and eye for detail in the word-processing and preparation of the final report. The remaining errors and omissions are ours alone.

Bibliography

• Brown, J. D. J. (1998). ‘An Investigation into Approaches to IELTS Preparation, with particular focus on the Academic Writing Component of the Test’, IELTS Research Reports 1998, ELICOS Association, Vol. 1, p. 20 - 37.
• Cameron, P. (1999). Prepare for IELTS: The IELTS Preparation Course, University of Technology: Sydney.
• Coleman, G. and Heap, S. (1998). ‘The Misinterpretation of Directions for the Questions in the Academic Reading and Listening sub-tests of the IELTS Test’, IELTS Research Reports 1998, ELICOS Association, Vol. 1, p. 38 - 71.
• Deakin, G. (1996). Practice Tests for IELTS. Indonesia Australia Language Foundation and Hawthorn Institute of Education: Melbourne.
• Flowerdew, J. and Miller, L. (1997). ‘The Teaching of Academic Listening Comprehension and the Question of Authenticity’, English for Specific Purposes, Vol. 16, p. 27 - 42.
• Freez, S. (1998) Text-based Syllabus Design, NCELTR: Sydney.
• Garbutt, M. and O’Sullivan, K. (1991). IELTS Strategies for Study, NCELTR: Sydney.
• Gerot, L. (1995). Making Sense of Text, Antipodean Educational Enterprises: Sydney.
• Gibson, C., Rusek, W. and Swan, A. (1996). IELTS Practice Now. CALUSA: Adelaide.
• Hamp-Lyons, L. (1998). ‘Ethical Test Preparation Practice: The Case of the TOEFL’, TESOL Quarterly, Vol. 32, p. 329 - 337.
• Hamp-Lyons, L. (1999). ‘The Author Responds …’ TESOL Quarterly, Vol. 33, p. 270 - 274.
• Hayes, B. and Watt, L. (1998). ‘An IELTS preparation course for Asian students: When practice doesn’t make perfect’, EA Journal, Vol 16, p. 15 - 21.
• Hughes, A. (1989). ‘Testing Reading’, Testing for Language Teachers, Cambridge University Press: Cambridge.
• IELTS Specimen Materials April 1997. University of Cambridge, The British Council and IDP Education Australia: Cambridge.
• Jakeman, V. and McDowell, C. (1996). Cambridge Practice Tests for IELTS. Cambridge University Press: Cambridge.
• McNamara, T. and Hamilton, J. (1991). ‘EAP Tests as Performance Tests’, Proceedings of the 4th Annual Education Conference, ELICOS Association: Sydney.
• Mikulecky, B. S. (1990) A Short Course in Teaching Reading Skills, Addison-Wesley: New York.
• Nunan, D. (1995). ‘Reading: A Discourse Perspective’, Language Teaching Methodology, Prentice Hall In: Englewood Cliffs, N. J.
• Sahanaya, W. and Lindeck, J. (1998). IELTS Preparation and Practice Listening and Speaking: Oxford University Press: Oxford.
• Sahanaya, W. Lindeck, J. and Stewart, R. (1998). IELTS Preparation and Practice: Reading and Writing Academic Module. Oxford University Press: Oxford.
• Todd, V. and Cameron, P. (1996). Prepare for IELTS Academic Modules.: Insearch Language Centre, UTS: Sydney.
• Tomlinson, B. (1999). ‘Developing Criteria for Evaluating L2 Materials, IATEFL Issues, Issue 147, p. 10 - 13.
• van Bemmel, E. and Tucker, J. (1997). IELTS to Success: Preparation Tips and Practice Tests: Hawthorn Institute of Education Jacaranda Wiley Ltd: Melbourne.
• Wadden, P. and Hilke, R. (1999). ‘Polemic Gone Astray: A Corrective to Recent Criticism of TOEFL Preparation’ TESOL Quarterly, Vol. 33, p. 263 - 269.

Appendix 1.0

Note: Appendix 1.0, the project application, has not been reproduced in this publication, as it duplicates the outline of purpose and methodology in the body of the report. Please contact the authors for a copy of the original report and Appendices.

Appendix 2.1: Numerical analysis tables for listening item-types

Table 1a Numerical analysis of listening item-types Deakin (1994). Practice Test for IELTS.

Table 1b Numerical analysis of listening item-types Gibson, Rusek and Swan (1996). IELTS Practice Now.

Table 1c Numerical analysis of listening item-types

Jakeman and McDowell (1996). Cambridge Practice Tests for IELTS.

Table 1d Numerical analysis of listening item-types Sahanaya, Lindbeck and Stewart (1998). IELTS Preparation and Practice.

Table 1e Numerical analysis of listening item-types

Todd and Cameron (1996). Prepare For IELTS: Academic Modules.

Table 1f Numerical analysis of listening item-types van Bemmel and Tucker (1997). IELTS to Success: Preparation Tips an Practice Tests.

Table 1g Numerical analysis of listening item-types IELTS Specimen Materials (1997).

Appendix 2.2: Evaluation tables for listening tests

Table 2a Evaluation table for listening tests Deakin (1996). Practice Tests for IELTS

Table 2b Evaluation table for listening tests Gibson, Rusek and Swan (1996). IELTS Practice Now.

Table 2c Evaluation table for listening tests Jakeman and McDowell (1996). Cambridge Practice Tests for IELTS.

Table 2d Evaluation table for listening tests Sahanaya and Lindeck (1998). IELTS Preparation and Practice: Listening and Speaking.

Table 2e Evaluation table for listening tests Todd and Cameron (1996) Prepare For IELTS: Academic Modules

Table 2f Evaluation table for listening tests van Bemmel and Tucker (1997). IELTS to Success: Preparation Tips and Practice Tests.

Table 2g Evaluation table for listening tests IELTS Specimen Materials 1997

Appendix 3.1: Numerical analysis tables for reading item-types

Table 3a Numerical analysis of reading item-types Deakin (1994). Practice Tests For IELTS.

Table 3b Analysis of reading item-types Gibson, Rusek and Swan (1996). IELTS Practice Now.

Table 3c Numerical analysis of reading item-types Jakeman and McDowell (1996). Cambridge Practice Tests For IELTS.

Table 3d Numerical analysis of reading item-types Sahanaya and Lindeck (1998). Preparation and Practice.

Table 3e Numerical analysis of reading item-types Todd and Cameron (1996). Prepare For IELTS: Academic Modules.

Table 3f Numerical analysis of reading item-types van Bemmel and Tucker (1997). IELTS To Success.

Table 3g Numerical analysis of reading item-types IELTS Specimen Materials (1997).

Appendix 3.2: Evaluation tables for reading tests

Table 4a Evaluation table for reading tests Deakin (1994). Practice Tests for IELTS

Table 4b Evaluation table for reading tests Gibson, Rusek and Swan (1996). IELTS Practice Now

Table 4c Evaluation table for reading tests Jakeman and McDowell (1996). Cambridge Practice Tests for IELTS.

Table 4d Evaluation table for reading tests Sahanaya, Lindeck and Stewart (1998). Preparation and Practice.

Table 4e Evaluation table for reading tests Todd and Cameron (1996). Prepare For IELTS: Academic Modules.

Table 4f Evaluation table for reading tests van Bemmel and Tucker (1997). IELTS to Success: Preparation Tips and Practice Tests

Table 4g Evaluation table for reading tests IELTS Specimen Materials (1997)

Appendix 4.0: Prompts for teacher interviews

Issues for discussion were listed in the letter forwarded to participating teachers before the teacher interviews.

Areas in which we are interested in obtaining feedback include the following.

• Comments on the particular materials you have used.
• The skills the IELTS preparation materials do/or do not help to develop, eg. time management, following instructions, managing unfamiliar vocabulary.
• The impact of the proficiency level of the learner on the training program.
• Measurement of student progress.
• Factors you keep in mind when choosing which particular tasks and practice materials are used.
• Your views regarding the place of IELTS preparation materials in the broader dimension of preparing learners for further study.
• Any comments of your own about the materials or suggestions for additions to future IELTS preparation materials.

Appendix 5.0: IELTS Preparation, Reading and Listening Materials - Student Survey 1

Student Survey 1

1. Please tick the appropriate boxes.

   Age:

   • < 18 yrs
   • 18 - 22 yrs
   • 23 - 25 yrs
   • 26 - 30 yrs
   • > 30 yrs

   Nationality: __________

   First Language: __________

2. What is your reason for taking the IELTS?

   • For entry to university
   • TAFE
   • Other institution: __________
   • Other reasons: __________

3. How long have you been studying English in Australia? __________

4. When did your current IELTS preparation course begin? __________

   day / month / year

5. What do you look for when you choose an IELTS practise task? (eg. clear instructions, colourful, new looking materials, familiar or interesting topics etc)

   ---

6. Do you usually do the practice tests in the time limit given, without rewinding the tape (for listening tests) and without using a dictionary?

   Response: Yes / No

   If not please explain how you usually use the practice reading and listening tests.

   ---

7. Are the scores you obtain in your practice tests always similar.

   Response: Yes / No

   If not can you give reasons why some practice tests are easier / more difficult than others?

   ---

8. As well as helping you prepare for your IELTS test do you think IELTS training materials should help you prepare for further study?

   ---

9. Can you suggest anything that could be added to IELTS Reading or Listening materials to make them more useful in preparing for the test?

   ---

Appendix 6.0: IELTS Preparation, Listening and Reading Materials - Student Survey 2

Student Survey 2

(Survey questions for students who have already taken the IELTS test)

1. Please give the place and date of your last IELTS test.

   Place: __________

   Date: __________

   IELTS Project Book List

   1. Prepare for IELTS Academic Modules by Vanessa Todd and Penny Cameron. Insearch Language Centre, UTS
   2. IELTS Practice Now by Carol Gibson, Wanda Rusek and Success Preparation Tips and Practice Tests by Eric van Bemmel and Janina Tucker. Hawthorn Institute of Education Jacaranda Wiley LTD
   3. IELTS Preparation and Practice by Wendy Sahanaya and Jeremy Lindeck. Oxford
   4. Practice Tests for IELTS by Greg Deakin. Indonesia Australia Language Foundation and Hawthorn Institute of Education
   5. Cambridge Practice Tests for IELTS by Vanessa Jakeman and Clare McDowell. Cambridge University Press
   6. IELTS Specimen Materials April 1995 University of Cambridge The British Council IDP Education Australia

2. From the book list can you write down the number of any materials you found particularly helpful in preparing for IELTS.

   ---

3. Can you name any other books or materials which were particularly useful to you in preparing for IELTS?

   ---

4. Approximately how many practice tests did you do before you took the IELTS test?

   ---

5. Were the scores you received in your IELTS test similar to those you received in your practice tests?

   ---

6. If there were any differences in either the listening or reading scores of your IELTS test to those in your practice tests can you explain what those differences were?

   ---

7. Can you make any comment on the practice listening and reading materials and how well you think they compare to the IELTS test?

   ---