Attitudes of tertiary key decision-makers towards English language tests in Aotearoa, New Zealand: Report on the results of a national provider survey

Abstract

IELTS research grant awarded Round 9, 2003

This study surveys all tertiary institutions in Aotearoa New Zealand to understand the attitudes of decision-makers towards the English language tests used as entry to their programs.

A survey of all public and private tertiary institutions in Aotearoa New Zealand was conducted to investigate the attitudes of tertiary decision-makers towards English language tests being used as entry to their programs. Results showed that the extensive and rapid changes in the tertiary environment in Aotearoa New Zealand means that it is now characterised by diversity, resulting in a wide variety of types of English language requirements for different courses. Many tertiary institutions are actively recruiting international students, and are moving away from rigid gate-keeping English language policies to more flexible ‘pathways’ to proficiency. There is an increased interest in the use of a range of English language assessment methods to give a fuller picture of students’ abilities. IELTS is the most frequently used test, and the IELTS ‘brand’ is well-known, but its use in high-stakes situations means that it has taken on a symbolic value beyond its function as an indicator of language proficiency.

Decision-making on English language thresholds is made in a wide variety of ways and at a wide variety of institutional levels. A number of decision-makers at tertiary institutions have said they would appreciate more information about test results from test providers. There is potential for greater liaison on language proficiency issues between course providers and external industry standard-setting bodies.

Note: Aotearoa New Zealand is the bilingual name for New Zealand. It is becoming more frequently used, particularly in contexts which emphasise the inclusiveness of all New Zealanders. The Māori name ‘Aotearoa’ is translated as ‘Land of the long white cloud’.

1 INTRODUCTION

Language testing currently has a high profile in Aotearoa New Zealand, for a number of interrelated reasons linked to changes in the structure of the tertiary education sector, changes in the secondary schools qualifications system, and changes in immigration. Each of these changes will be explained in turn, to show that this high profile has raised the awareness, but not necessarily the understanding, of the position of IELTS as the most frequently used language proficiency test.

1.1 Changes in the structure of the tertiary education sector

The 1989 Education Act was the start of significant changes in the education sector in Aotearoa New Zealand, setting up a competitive model of tertiary education provision. Since then, tertiary education providers have been in a state of flux, with campus mergers among Public Tertiary Institutions (PTIs); the expansion of Private Training Establishments (PTEs), at which a wide range of academic and applied subjects are taught; and Wānanga, tertiary institutions built on Māori philosophies. These institutions are all funded by the government at different rates, with the greater amounts going to PTIs and Wānanga. The competitive nature of the tertiary environment has also been in a context of decline in university funding in comparison to the other public tertiary providers and the private providers (Kinnear 2005). The distinction between PTIs and PTEs is therefore of intense interest to the sector, and has been included as a main variable in this research.

In 1990 restrictions were lifted on students from mainland China which had been imposed after the 1989 Tiananmen Square massacre (Gamble and Reid 2002), and New Zealand was endorsed by China as an acceptable education destination (Smith and Rae 2004, pp 6). This resulted in high numbers of Chinese students, particularly in English language schools which went through rapid, unsustainable growth (Quigley 2004). A number of the schools failed, and combined with the rapid rise in the New Zealand dollar in 2004, there was a fall in student numbers. In 2005 this affected tertiary institutions, which had been reliant on recruiting from the local language schools (Quigley 2004).

The numbers of foreign fee-paying students in 2003 and 2004 can be seen in Table 1. It shows the greatest fall in schools, which dropped 12.5% from 20,938 in 2003 to 18,322 in 2004. Private tertiary providers dropped 4.1% from 52,465 in 2003 to 50,334 in 2004. Public tertiary providers increased 11% from 37,407 in 2003 to 41,528 in 2004, before the downturn trend moved through to degree level.

Table 1: Number of foreign-fee paying students 2003, 2004 (Data from Education New Zealand 2005)

The numbers of students from different countries of origin for 2005 can be seen in Figure 1.1, which lists countries with over 1,000 student visas. This shows that the highest numbers come from non-English speaking Asian countries of China (39,260), followed by South Korea (13,229) and Japan (4,868).

Figure 1.1: Student visas at 1 January 2005 by country (over 1,000) (Data from Education New Zealand 2005)

To maintain quality in the sector, a Code of Practice for the Pastoral Care of International Students was introduced in March 2002 (Ministry of Education 2003). This makes it mandatory for any education provider wishing to enrol international students to be a signatory to the Code, which includes regulations about the provision of information. For example, clause 4.2.5 of the Code requires that English language proficiency requirements (if applicable) must be provided to prospective international students.

The increased numbers of international students (and pressure to maintain these numbers), together with the increased clarity required by the Code, have consequently resulted in an increased importance for providers in determining their English language requirements. Read and Hirsh (2005, pp 4) noted that IELTS has been the preferred test in Aotearoa New Zealand since it was first administered in 1991, although tertiary institutions will generally accept a range of other tests. At the same time, the diversity in the education sector has also resulted in a number of different pathways into tertiary studies opening up for international students. These include enrolment in secondary school; intensive English courses in private language schools or language centres of tertiary institutions; foundation studies at tertiary institutions; and support during their academic studies (Read and Hirsh 2005, pp 2).

In addition, there has been a growing feeling that relying on a proficiency test score alone to assess the English language competence of international students is limiting (Read and Hirsh 2005, pp 2). This has resulted in what has been described as a ‘prevailing negative perception’ of IELTS (Barkhuizen and Cooper 2004, pp 104). Concerns have also been expressed about the levels of literacy in the student population, highlighted by the new proportions of students from non-English speaking backgrounds (Kirkness 2003, pp 47). The view seems to be held by students and staff that because IELTS is the main gate-keeping mechanism for international students it is - or should be - an indication of readiness for academic study as well as a test of English language proficiency (Skyrme 2004, pp 100; Barkhuizen and Cooper 2004, pp 105). It can be said that the same ‘moral and financial need to ensure that fee-paying international students who undertake university education are capable of succeeding’ outlined by Feast (2002, pp 70) in Australia in relation to IELTS scores, also applies in Aotearoa New Zealand. Research has shown the complexity of the relationship between English proficiency and indications of future academic success; this literature has been well-canvassed elsewhere (for example Dooey 1999; Kerstjens and Nery 2000; Coleman, Starfield and Hagan 2003; Read and Hirsh 2005).

1.2 Changes in the secondary education qualifications system

Another set of changes since 2000 was the phased introduction of the National Certificate of Educational Achievement (NCEA) - a controversial standards-based assessment framework which replaced the former qualifications from 2002 and reached the final year of secondary study in 2004. Students coming from New Zealand secondary schools into tertiary study now have to conform to the literacy standards included in this framework, and can not ‘opt out’ by choosing a proficiency test such as IELTS instead. There are no literacy standards set in the NCEA for second language learners, and many language teachers regard the standards as inappropriate and impossible to achieve (Parker 2004, pp 8). There is anecdotal evidence that some secondary students are choosing to sit IELTS in Australia for re-entry as a new international student (Parker 2004, pp 8). However, another view is that the ‘extra dimensions of academic culture’ provided through the NCEA may go some way towards mitigating the effect of the one-off high stakes test (Skyrme 2004, pp 101).

1.3 Changes in immigration

Changes to immigration to increase the number of skilled migrants to Aotearoa New Zealand have resulted in a high profile and controversial focus on IELTS - which is the test used as part of the points system to determine immigration eligibility by the New Zealand Immigration Service. For example, a Sunday newspaper article on a mock IELTS examination in 2002 implied that the standards were unrealistically high (Catherall and Laugesen 2002, pp A4). (The mock test was carried out on a group of people including a New Zealand-born housewife and student, two multi-millionaire businessmen who had migrated to New Zealand 10 years before, and a receptionist originally from India. The two businessmen walked out without completing the test, and although the others completed the test they ‘appeared quite drained at the end of the ordeal’.)

The immigration issue has also impacted on international students because many students arrive with the possibility in mind of applying for immigration after completing their studies (Richardson 2005, pp 23).

2 AIMS OF THE STUDY

There were two main aims of this study.

1. To investigate the English language assessment methods and threshold levels used in the different types of tertiary institutions in Aotearoa New Zealand.

The importance of the consequences of uses and interpretations of tests, or ‘consequential validity’, is an important part of construct validity (Davies 2003, pp 362). It is therefore of interest to explore the setting of English language thresholds with those who make the decisions at tertiary institutions in Aotearoa New Zealand. Research carried out by McDowell and Merrylees (1998) investigated the range of tests used in Australia, and attitudes towards IELTS as a test. They found a wide range of tests were used, although IELTS was the preferred test in Australia, and the most commonly used. More recently, Coleman, Starfield and Hagan (2003) surveyed a cross-section of staff in an Australian and UK university on their perceptions and knowledge of IELTS. They found that one-third or fewer of staff believed that the threshold was sufficiently high for success in university study, and that language support staff believed that staff may attribute student difficulties to the test rather than the entry standards set by the institutions (Coleman, Starfield and Hagan 2003, pp 190).

2. To explore the criteria used in determining the English language threshold levels.

It was also of interest to investigate the criteria for decision-making, and the methods involved. McDowell and Merrylees (1998, pp 133-134) investigated the policy-makers of English language thresholds, and the consultation involved in the process. They found that a combination of bodies was most frequently involved, and this was most often carried out through consulting a combination of sources (ESL professionals, other institutions and the literature). Coleman, Starfield and Hagan (2003, pp 193) asked staff about the means through which they believed entry scores were established, and found that only two-fifths of staff believed that ‘a careful consideration of the appropriate English language ability necessary of university study explained the setting of IELTS entry scores’. The level was attributed to inter-university competition by 22% of staff. The current research project extended these previous studies into the Aotearoa New Zealand context, with a focus on the decision-making process for the thresholds used for IELTS or other tests.

A central issue is the degree to which tertiary institutions understand the nature of the IELTS test, and the implications of the test results for student performance. Connected to this is the need for IELTS to provide information to institutions about using the IELTS test scores most appropriately. A strong finding in Coleman, Starfield and Hagan’s study (2003, pp 202) was the lack of awareness among staff of the meanings of IELTS scores. They suggested regular communication with staff about IELTS. Our study provides information about how this might be achieved most effectively.

3 METHODOLOGY

The methodology for this research was a census of decision-makers in all tertiary providers in Aotearoa New Zealand, carried out by means of a postal questionnaire supplemented by telephone enquiries. Supplementary interviews with authorities mentioned by respondents were also held for clarification.

3.1 Survey frame

The frame of target respondents for the questionnaire was based on the lists of signatories to the Code of Pastoral Care for International Students (see 1.1 above), available on the Ministry of Education website (Ministry of Education 2004a). All public and private institutions except private language schools were included. This gave lists of 33 public tertiary institutions and 171 private tertiary institutions. See Appendix 1A and 1B.

To ensure a good response rate, the questionnaire was sent to named individuals. This required extensive website searches followed by telephone calls to every institution to try and locate the decision-makers on language levels. After detailed enumeration, telephone calls were made to 51 identified decision-makers in public institutions, and 68 decision-makers in private institutions. In addition, 52 calls were made to administration or other staff who identified another staff member as the decision-maker.

3.2 Questionnaire development and administration

As well as identifying the appropriate person to be the target respondent for the questionnaire, and explaining the project, discussions during these telephone calls identified the following issues relating to English language levels:

• Threshold levels are sometimes decided at institutional level, and sometimes by each program or course leader. This is not related to the numbers of students involved; eg both systems are operating at different universities and polytechnics with large numbers of international students.

• It is often not clear who decides or who has decided on the threshold levels within the institution; the levels have been decided by previous administrations and inherited by current staff who may or may not have any understanding of the processes involved.

• Some decision-makers identified bodies such as the Immigration Service, Vice-Chancellors Committee or Nursing Council as setting the levels. This was followed up in interviews to supplement the questionnaire data (see 3.4 below).

• Many decision-makers would like more information from the test producers about what the levels mean. They felt that, as non-language experts, they were making decisions without adequate understanding of the levels.

• Some decision-makers who are also involved in preparing students for language tests are concerned about the stress for students of high stakes one-off language tests.

• Some decision-makers have been ‘caught out’ by students who had achieved the level set but whose English proficiency levels were not adequate for the courses.

Where appropriate, these issues were incorporated into the questionnaire, a preliminary draft of which was developed from the researchers’ background knowledge of the tertiary environment in Aotearoa New Zealand, with reference to McDowell and Merrylees’ survey (1998) of Australian universities and TAFE (Technical and Further Education) Colleges, and Coleman, Starfield and Hagan’s survey (2003) of tertiary staff in Australian, UK and Chinese tertiary institutions.

To maximise the response rate, it was necessary that the questionnaire be short (Converse and Presser 1986), and it was decided to keep it to two sides of a page. This required careful consideration of wording and layout. There was a mixture of closed questions, questions with Likert-type five-point rating scales, and open questions for comments. The benefit of a restricted space for comments meant that all comments made could be thoroughly analysed. The questionnaire was printed on pale green paper (Kaskad ‘leafbird green’) for ease of recognition amongst other papers.

In-depth, iterative trials of the preliminary draft of the questionnaire were conducted with two non-target but parallel informants, and then in turn with three target respondents at public institutions and three target respondents at private institutions (including one by post). Talk-aloud protocols were used where appropriate, and the feedback was incorporated into the following drafts of the questionnaire. After each trial, the questionnaire was adjusted in response to observations made on the basis of answers to the questionnaire itself. The final questionnaire, developed after these eight phases, is Appendix 2.

Questionnaires were printed with identification codes using the computer’s mail merge function, and sent out with reply-paid envelopes and letters of introduction between October and December 2004 (see Appendix 3). Follow-up telephone calls and emails were made, and/or a second copy of the questionnaire was sent out in February 2005.

Responses were entered into a spreadsheet and questions were analysed individually; where appropriate each one was also analysed according to provider type (public or private). A four-digit identifying code was allocated to each respondent as a means of maintaining confidentiality, and these have been used where comments are included with the results below.

3.3 Survey response rate

The respondent frame was divided into public and private providers, as developed from the original Ministry of Education lists in Appendices 1A and 1B. Each had different response rates.

There were 33 institutions on the original list of public providers; two of these institutions had merged by the time the final list was compiled. Informants from two other providers reported that local language schools took care of the language testing of students. Responses were obtained from 27 of the remaining 30 institutions, to make an institutional response rate of 90.0%. Nine providers had more than one decision-maker, which resulted in 81 questionnaires being sent out, and 57 responses received back, of which six were not eligible. The individual response rate adjusted for estimated eligibility was therefore calculated as follows:

Estimated number eligible responses (public institutions)

Estimated individual response rate (public institutions)

The institutional response rate of 90.0% and individual response rate of 79.8% therefore showed very good coverage of the public institutions.

Only one questionnaire was sent to each of the private institutions, so the difference calculated between individual and institutional response rates for public providers did not apply. From the 171 on the original list of private providers there were 115 replies. Of these, 31 respondents replied that their institutions were ineligible, 17 said that they did not recommend or set the levels, four refused to take part in the survey, and 63 responded with eligible responses. This left 56 non-responses. The response rate to allow for estimated ineligibility was therefore calculated as follows.

Estimated number ineligible responses (private institutions)

Estimated response rate (private institutions)

This response rate of around 50% for private institutions was lower than for public institutions, but the private institutions had comparatively fewer numbers of international students (and there is some uncertainty as to whether additional non-response should have been included with the ‘ineligible’ responses). Therefore, there was an overall good coverage of institutions.

3.4 Supplementary interviews

In response to comments made in the trials and on the questionnaires, two interviews were held with representatives of authorities mentioned: the Nursing Council of New Zealand; and the Vice-Chancellors’ Committee. In both cases, the interviewees were adamant that these bodies did not set English language levels for programs of study. The Nursing Council English language requirements only applied to nurses on application for registration; and the Vice-Chancellors’ Committee role was to observe the levels set by the universities (although this was not entirely clear on their website).

4 RESULTS AND DISCUSSION

The survey results are generally presented in order of the topics asked in the questionnaire, except that the background information which was asked at the end of the questionnaire (in line with accepted questionnaire design practice) is presented in the early sections of the report. Where comments are reported, they include the confidential code number allocated to respondents for data analysis purposes.

4.1 Key decision-makers of language levels

The questionnaire started with a ‘filter’ question to identify the correct respondents in the institution (as a further check to the advance telephone calls which had been made for this purpose).

Are you responsible for recommending or setting the English language proficiency levels for entry by ESL (English as a Second Language) students in your institution, or for a program - other than language courses - in the institution?

Yes, I set the levels. Please answer the questions below.

Yes, I recommend the levels, which are set by: ____.

Please answer the questions below.

No. Please return this questionnaire in the envelope provided.

The results from this question are presented in Table 2, which shows that 33 of the 114 respondents were the key decision-makers. A further 70 recommended the levels, the majority of which were then set by academic boards or committees (28), or by the faculties or programs (13). Some were set by

more senior individuals in the institutions such as the Chief Executive Office or Dean (7), and others by owners or those in a governance role such as trustees or a council (6). These results indicate that a similar combination approach to that found in Australian institutions by McDowell and Merrylees (1998, pp 133) occurs in the majority of cases.

It was interesting to see that four respondents identified regulatory authorities such as the Nursing Council or Teachers’ Council as setting the levels, although those bodies themselves saw their role as setting levels for entry to the profession rather than to programs of study (see 3.4 above). Three other respondents identified the language tests themselves as setting the standards, rather than as indicators of proficiency at particular levels.

Six respondents reported that they were ineligible, even though some of them had identified themselves as decision-makers in the initial telephone calls. This confirmed the lack of clarity in many institutions about processes for decision-making of English language requirements.

Table 2: Responsibility for setting English language proficiency levels

To find out the roles of people who made decisions about language levels in the institutions, respondents were asked:

Q 17 What is your title/role in your institution?

The responses have been grouped into 12 categories (plus ‘other’ and ‘no response’) in Table 3. These categories were decided according to a mixture of hierarchy (for example, Director and Assistant Director) and area of expertise (for example, Director and International Director); although the groupings were necessarily somewhat arbitrary as the titles and roles themselves mean different things in different organisations.

The results show that the majority of respondents were at the senior level of organisations, such as Director (21), Principal (19), Academic Director (16), Program Manager (10) and International Director (10). It was also interesting to see the number of decision-makers who were in administrative roles, summarised under Administration Manager (8). The same number of respondents identified their roles as primarily concerned with English language issues: English Program Manager (5) and English Lecturer (3).

Table 3: Roles of respondents in their institutions

4.2 Levels taught

The levels taught in each program were divided into three categories in the questionnaire: pre-degree or practical; undergraduate; and postgraduate. The numbers of each in the respondents’ programs were elicited in the first question:

Q 1 Which of the following levels are taught on your program?

Pre-degree/practical

Undergraduate

Postgraduate

The numbers at each level are listed in Table 4. It shows that the largest number of respondents were in programs at pre-degree/practical level (52), followed by undergraduate level (17), all three levels (16), pre-degree/practical and undergraduate (11) and undergraduate and postgraduate (10). Only one program was reported as being pre-degree/practical and postgraduate. These results are expected, given that the total group of tertiary institutions includes a large number of private providers focusing on the teaching of practical courses such as aviation, horse-riding or acupuncture.

Table 4: Program levels

4.3 ESL students enrolled in tertiary programs

Information about the numbers of ESL (English as a Second Language) international students studying in each program was elicited in the following questions:

Q 13 Approximately how many ESL international students are studying in your program this year? Q 14 What percentage of the current total student body in your program are ESL international students? Q 15 Approximately how many other (permanent resident) ESL students are studying in your program this year? Q 16 Do you predict the number of ESL international students to change significantly in future years?

The numbers of ESL international students in the respondents’ programs can be seen in Figure 4.1, as percentages of types of provider. The highest percentages of public providers had 21-100 ESL international students (28.0%), 1-10 students (20.0%), and 1001-5000 students (16.0%).

The pattern for private providers was similar but more pronounced, with the highest percentage having 1-10 ESL international students (49.2%), followed by 21-100 students (23.8%), and 11-20 students (9.5%). A fifth of public providers and half of private providers have very few ESL international students, and around a quarter of both types have between 20 and 100.

(Percentages by provider type, N=114)

Figure 4.1: ESL international students in programs, by provider type

The percentage of ESL international students in the total student body can be seen in Figure 4.2. This shows that 54.9% of public providers and 49.2% of private providers, around half of both types, have ESL international students as 1-10% of their total student body. A further 15.7% of public providers and 17.5% of private providers have ESL international students as 11-20% of their student body. Nearly 8% of private providers have ESL international students as 91-100% of their student body, even though English language schools were not included in this survey.

(Percentages by provider type, N=114)

Figure 4.2: ESL international students as percentage of total student body, by provider type

The number of other ESL (permanent resident) students studying in tertiary programs can be seen in Figure 4.3. This shows that respondents from 42.0% of public providers and 55.6% of private providers reported having 1-100 permanent resident ESL students in their programs, whereas 8.0% of public providers and 20.6% of private providers reported having none. A further 26.0% of respondents from public providers and 15.9% from private providers were unsure about the numbers. This is probably because the data is often not formally collected, although it might be assumed in some cases from data on ethnicity.

(Percentages by provider type, N=114)

Figure 4.3: Permanent resident ESL students in programs, by provider type

Figure 4.4, the predictions of changes in numbers of ESL students, shows that a slightly smaller proportion of public providers thought that the number will change significantly (14.0%) than think it will not (16.7%), whereas a slightly higher proportion of private providers thought it will change significantly (20.2%) than thought it will not (18.4%). The same proportion of respondents from public and private providers was unsure (12.8%).

Figure 4.4: Predictions of changes in ESL international students, by provider type

Comments for this question also showed a mixed view of the future. Of the 27 comments, 13 indicated that they expected an increase:

Likely to increase. Many new immigrant international students are English speakers. (2011)

Developing new programs specifically for international students. (1019)

Modest increase at postgrad level. (1036)

However, another 10 indicated that they expected a decrease:

The number is dropping nation-wide. (1024)

Less as NZ becomes less popular because of cost. (2038)

Not unless Govt agencies - NZIS (New Zealand Immigration Service), MOE (Ministry of Education), NZQA (New Zealand Qualifications Authority) etc - get their act together. (2026)

A further three were unsure which way it would go:

As we deliver. (2060)

Could they be any lower? (1028)

Let’s see where the exchange rate goes. (2040)

These comments reflect the uncertainties in the export education market. Where they were negative they did not reflect problems with the reputation of Aotearoa New Zealand as an education destination, but rather issues of cost caused by the exchange rate for the New Zealand dollar.

4.4 English language assessment methods accepted

The type of English language assessment method accepted for each level was elicited in the second question, which asked about the language tests most commonly included on website information by providers.

Q 2 Please tick the English language assessment methods you accept for entry of ESL international students:

Pre-degree / practical Undergraduate Postgraduate

A NZ school results

B English teachers’ assessment

C Interview with staff

D Your institution’s test

E Prior tertiary studies through English

F IELTS - General

G IELTS - Academic

H TOEFL - Paper

I TOEFL - Computer

J Other Please specify

The results have been presented separately for public and private providers in Figure 4.5 and Figure 4.6, divided into practical/pre-degree, undergraduate and postgraduate. These were scaled by the numbers of ESL international students in the programs for each respondent, taking the mid-point of each category (see 4.2 above).

These figures show that for both public and private providers, IELTS Academic was accepted by the highest proportion of providers at all levels: for public providers 90.5% of those teaching at pre-degree/practical level; 96.1% at undergraduate level; and 100.0% at postgraduate level. For private providers: 98.7%; 97.8%; and 100.0% respectively. The second most accepted assessment method for public providers was prior studies through English, but higher proportions of private providers accepted TOEFL Paper or Computer at all levels. The institution’s own test was accepted at undergraduate level by 53% of public providers and 82.2% of private providers.

These results confirm the importance of IELTS and may reflect the increasing emphasis on Foundation studies by a number of institutions (for a list of university and polytechnic Foundation Studies programs, see Read and Hirsh 1995, pp 57-58). For both types of providers, proportions accepting school results at undergraduate levels are low, but this may change as the impact of the new secondary school qualifications NCEA (see 1.2 above) begins to be felt.

(Percentages by qualification type, scaled by numbers of students)

Figure 4.5: Proportion of public providers accepting English language assessment methods, by tertiary levels

(Percentages by qualification type, scaled by numbers of students)

Figure 4.6: Proportion of private providers accepting English language assessment methods, by tertiary level

The six other assessment methods identified by respondents for this question are presented in Table 5 with their levels. These included general assessments such as a statement of ability from the student, school results from other countries, and ‘IELTS equivalent’, as well as mention of specific tests such as Cambridge (Cambridge ESOL Examinations 2006), Oxford Placement Tests (Oxford University Press 2004), and TOEIC (Test of English for International Communication) (TOEIC 2006). This shows that while some other assessment methods were being used, those identified in Figure 4.6 above were the most frequently accepted (however, for frequency of use, see 4.7).

(Frequencies)

Table 5: Other assessment methods used

4.5 English language entry levels accepted (IELTS and TOEFL)

After initial website searches and telephone calls with potential respondents, it had been expected that IELTS and TOEFL (Test of English as a Foreign Language) (TOEFL 2006) were the most commonly used English proficiency tests, so the following question was included in the questionnaire to elicit the threshold levels used by providers accepting the tests: Q 3 If you accept these tests, what entry levels are required (please indicate the range as appropriate)?

The results are presented in Table 6 on the next page. This shows some variation in entry levels accepted at different institutions at the same level. These IELTS levels correspond to those reported in Read and Hirsh (2005, pp 55-56) obtained from the websites of public providers.

TWES Test of Written English Score ERS Essay Rating Score

Table 6: Entry levels required by institutions which accept IELTS or TOEFL

It was interesting to note that some respondents used IELTS as the benchmark, and indicated that, although they accept TOEFL, they were unfamiliar with its scoring system. However, the variation indicated by those respondents who did specify a TOEFL score indicates that a number of decision-makers are familiar with the results of the tests. Publicity around the imminent launch of the ‘next generation’ TOEFL may increase awareness further.

4.6 Reasons for different levels

Some programs had a range of acceptable entry levels according to the course, and the next question aimed to elicit the reasons for the differences:

Q 4 If you have different language levels for the same level of courses, what are the reasons?

There were 32 comments in response, of which a number reiterated the differences. Some noted that the differences reflected the different decision-makers within the institutions, in particular that some had higher levels:

Faculty’s decision based on necessary English language competency required. (1050)

Often disciplines demand/request higher entry requirements which must be justified. (1002)

Some postgraduate programs have set higher levels and others are considering this. Arts faculty is considering raising levels. Reason: to guarantee higher level of language fluency on entry to demanding programs. (1047)

Others had program level support in some courses for students with lower proficiency levels:

We do allow entry to ‘supported stream’ courses for the Bachelors Degree for students with 5.5 (no band <5.0). (2026)

That students of lower level will concurrently study further English while they study their chosen subject discipline. (1028)

The 500 assessment for pre-degree is for Foundation Studies only because this program of study is ‘English rich’. (1048)

We have a small number of Japanese, who don’t require IELTS as we have two Japanese instructors. (2034)

Our focus is on art making - a language in its own right. (2025)

Some had particular reasons for requesting higher levels:

We require 6.0 for some 2-week intensive programs as ability to read and write fairly quickly is necessary. (2016)

Diploma in culinary arts requires a 6.0 because of English and French requirement. (1006)

High language levels/content in some programs. Industry requirements for professional qualifications. (1041)

These comments reflect the range of types of programs which are now being offered in the diversified tertiary environment, with very different language requirements.

4.7 Most-used English language assessment method

As the previous questions had focused on the range of assessment methods accepted, the following question was included to identify the most frequently used method:

Q 5 Which English language assessment method is the one used most often for entry to your program?

The results are shown in Table 7. It can be seen that IELTS - including Academic and General and in conjunction with other assessment methods - was reported by over half of the respondents (73) as being the assessment method most often used in their program. Similar numbers of institutions were reported as using interviews and their own institution’s test as the most frequent assessment method. No-one identified using TOEFL in this question, in spite of it being identified as an acceptable entry method by a large number of respondents in Question 2 (see 4.4 above).

Table 7: Most-used English language assessment method

4.8 Attitudes towards assessment method accuracy

In the telephone discussions that formed the first part of this survey, a number of potential respondents indicated a lack of confidence in the results of the tests. The next question therefore aimed to identify the respondents’ attitudes towards each of the tests they used:

Accuracy scale: Very accurate to not at all accurate; undecided; N/A.

The results can be seen in Figure 4.7, which shows that the respondents gave the highest ratings of accuracy to IELTS Academic and interviews with staff. However, it should be noted that the students for which each assessment applies may be different in level, even within institutions.

(Percentages, N=114)

Figure 4.7: Attitudes towards assessment method accuracy The ‘other’ assessment methods which had been identified in Question 2 (see 4.4) are shown with their ratings in Table 8. An additional method, ‘referee’s assessment’, was also reported for this question. A range of ratings for the accuracy of these methods was given, particularly for the Cambridge tests, and the middle rating was given for TOEIC.

Table 8: Accuracy of other assessment methods used

4.9 Criteria in choosing assessment methods

A number of criteria used in deciding on the choice of English language assessment method were identified in initial telephone calls and questionnaire trials:

Figure 4.8 shows the results, which identify the criterion rated as ‘very important’ by the highest percentage of respondents as ‘reliability of results’ (51.8%), followed by ‘clarity of results’ (33.3%). These results again highlight the importance for test producers of ensuring that the results are clear for the end-users of the test in institutions.

The least important criterion was ‘cost to students’, which was rated as ‘very important’ by 6.1% of respondents, and ‘not at all important’ by 20.2%.

Three other criteria were noted for this question: ‘Student’s fear of failing’ was rated as ‘very important’, and both ‘Recency of results’ and ‘Security’ were rated as the next category of importance.

(Percentages, N=114)

Figure 4.8: Attitudes towards criteria

4.10 Attitudes towards English language levels

A further question investigated the attitudes of a range of stakeholders towards the language tests. This included staff in the institutions, who anecdotally have felt under pressure to cope with large numbers of ESL students at administration and academic levels. Students themselves were included to see whether staff had had feedback about whether the students felt the entry scores were sufficient for them to cope with their studies. Attitudes of non-ESL students were queried in response to comments made that some resent changes in lecturing style necessitated by high numbers of ESL students. (No other stakeholders were identified for this question.)

The results can be seen in Figure 4.9, which shows that most respondents rated the majority of stakeholders as somewhere in between ‘very satisfied’ and ‘not at all satisfied’. The highest amount of satisfaction was perceived in the international ESL students themselves, although only 4.4% of respondents thought that they were ‘very satisfied’ with their level of English. This supports Skyrme’s (2004, pp 99) case study findings with Chinese international students who had entered university with the minimal entry levels, and who found in the first semester of their studies that they were having to implement new strategies to cope with the language demands of their courses. However, Ward and Masgoret’s (2004, pp 37-38) survey of 2,736 international students in Aotearoa New Zealand found that the majority described their academic progress as satisfactory, without a significant amount of difficulty.

Next highest was the satisfaction of the respondents themselves, which is understandable for those who were responsible for setting the levels. However, these scores still had the majority in the middle ranking (37.7%), indicating that there may be pressures on the decision-makers to keep the levels lower than they would otherwise choose. Coleman, Starfield and Hagan (2003, pp 190) found that Australian and UK staff believed the English proficiency levels of students were too low; only one third believed the English levels were high enough to assure success.

(Percentages, N=114)

Figure 4.9: Satisfaction with levels of English of ESL students

4.11 Likelihood of change in entry levels

A question was included to examine whether the institutions were feeling a need to respond to the numbers of international students; either by lowering their requirements in response to falling numbers of international students, or by raising their levels in response to levels of satisfaction with the levels (as investigated in the previous question):

Q 9 Is it likely that your English language entry levels requirements will change in the near future?

Yes

No

Unsure

Comments:

Respondents’ opinions about whether English language entry levels requirements were likely to change in the near future can be seen in Figure 4.10. This shows that while $17.7\%$ of respondents in public providers thought they would change, another $51.0\%$ thought they would not, and $29.5\%$ were unsure. There seemed to be less likelihood for change among respondents from private providers, of whom $9.5\%$ thought they would change, $61.9\%$ thought would not change, and $25.4\%$ were unsure.

These results do not indicate the strong dissatisfaction with entry levels which might have been predicted from the anecdotal evidence and literature reported above.

(Percentages, N=114)

Figure 4.10: Likelihood of change in English language entry levels

Comments were provided by 17 respondents for this question. Two were related to the changes in the secondary curriculum (see 1.2 above):

International students studying at schools here may get ‘appropriate’ L2 NCEA. (1043)

NCEA supersede IELTS. (1039)

Two other respondents were critical of the tests themselves:

IELTS is out of touch with NZ standards. (2036)

Problem is that IELTS/TOEFL do not necessarily test the things we need to know - too general, too easy to prepare for test once system is known. We are increasingly (through our own ESL school) developing internal test pathways. (1047)

Others indicated that review was part of the system:

Many review levels. (2033)

It’s a dynamic environment! (1029)

A range of external and internal factors were also identified:

Hairdressing ITO now recommend IELTS Level 6. (2057)

It may be application of entry criteria rather than the levels that is at issue. (1001)

But an English competency requirement is being built into degree for all students. (1038)

4.12 Factors in choice of entry levels

The factors underlying the choice of entry levels were examined more closely in the following question:

Q10. How important is it to ensure the following when you make decisions on the English language entry levels?

The results can be seen in Figure 4.11, which shows that the strongest concern of respondents was the ability of lecturers and teachers to teach and assess all students, with 50.0% of ratings at ‘very important’. This was followed by the importance of successful study by ESL students through English, which 49.6% of respondents also rated as ‘very important’.

External pressures such as competitiveness with other institutions was rated as less important, although it was rated by 14.0% of respondents as ‘very important’, and 18.4% with the next rating. This is higher than Coleman, Starfield and Hagan’s (2003, pp 193) finding of 0.6% of staff who strongly agreed - and 22.0% who agreed - that inter-university competition was a prime motivation in setting IELTS levels. However, the numbers of international students were rated by 12.3% of respondents as ‘very important’.

One respondent felt strongly enough to add a comment to their response:

Quality and success is more important than $$ and numbers. (1014)

Two ‘other’ factors were identified for this question: ‘National benchmark’ was rated as ‘very important’, and ‘wide consultation’ was noted but not rated.

(Percentages, N=114)

Figure 4.11: Factors in choice of entry levels

4.13 Attitudes towards English language ability at the start of tertiary studies

A question was included to investigate the respondents’ attitudes towards students’ English ability at the start of subject area tertiary studies. This was based on a similar question in Coleman, Starfield and Hagan’s (2003, pp 221) study. (Their question was: ‘I think a student’s language ability should be fully formed before they begin a university degree’ - strongly agree, agree, not sure, disagree, strongly disagree. However, they do not appear to have included analysis of the question in their discussion.)

Q 11 Do you believe a student’s English ability should be fully formed before they begin subject area studies at tertiary level?

Yes

No

Unsure

Comments:

The responses can be seen in Figure 4.12, which shows that slightly under half of respondents from public providers (47.1%) thought that their ability should be fully formed before beginning subject studies, whereas a third of respondents (33.3%) thought it should not. The reverse pattern held for respondents from private providers, of whom just over a third (34.9%) thought their ability should be fully formed before beginning subject studies, whereas slightly over half (52.4%) thought it should not.

(Percentages, N=114) Figure 4.12: Attitudes towards English language ability at the start of tertiary studies

Comments from 41 respondents gave further information about their attitudes. Some thought it should be fully formed because of the particular needs of the subject area:

Especially important for the study of law. To accept students who are less than competent in English is to do them a disservice. (1036)

Especially in our case, teaching aviation. (2035)

In teacher education programs, students start working very early into the start of their program. (1035)

Others saw the need to keep the levels higher for all students:

This avoids any attempt to talk down or over-simplify matters to tedium level for others. (2063)

We run a highly specialised course with a lot of technical language and all bar one of our international students are from English-speaking countries. All are adult learners. (2006)

Some who thought student ability did not need to be fully formed gave developmental reasons:

There is an ideal level at which focus should shift from language as object to language as medium - around intermediate level. Prior to this, distinctions between Academic English and General English are fairly meaningless. (See Krashen on sheltered subject courses). (2026)

In technical subjects impossible to achieve this (eg chemistry etc). (1043)

Might have good everyday English but need to pick up technical jargon as course progresses. (2011)

The subject study can motivate students to learn English very quickly. (2054)

Others had course-specific reasons why the students did not need high levels of English language proficiency:

First language material available. (2053)

Some of our short courses (group booking) bring interpreters to get information across. (2060)

In our case, we don’t need more than a basic competency. (2029)

Some respondents felt that there was an institutional responsibility to help students continue to gain English language ability:

We give ongoing compulsory support throughout undergraduate studies. (2027)

A set of courses can be structured to handle improved English. (2008)

Some disciplines and programs need to take some responsibility for continuing to develop students’ capabilities, especially writing to the discipline and academic literacy. (1034)

Some comments across the rankings noted that this issue depended on a number of factors:

It depends on the level of support available at the institution. (1042)

Depends on levels and courses chosen. (2019)

Depends on definition of ‘fully’ - Don’t think many international, or Kiwi students have achieved that. (2030)

These comments show a range of opinions about the levels of English necessary before students start studies in different practical and academic subject areas. There is an implication that a wide range of English proficiency entry levels are required, even within program levels, necessitating a good understanding of English test results.

4.14 Inputs to decisions on entry levels

The last exploratory question in the questionnaire was designed to find out what inputs were used by decision-makers when they decided on the threshold levels for their institution:

Q 12 How important are each of these inputs in helping you make decisions on the English language entry levels?

The results, in Figure 4.13, show that the most important input was the feedback from course lecturers or teachers; 40.4% of respondents rated this as ‘very important’. In contrast, only 14.2% rated the information from test producers as ‘very important’. This shows that decisions were being made from practical experience from the students’ proficiency levels. The high levels of consultation with ESL professionals reported in McDowell and Merrylees (1998, pp 134) did not appear to occur here.

Figure 4.13: Inputs into decisions on entry levels

Other inputs noted as ‘very important’ by respondents were: ‘IELTS “brand” quality’, ‘Organisational policies’, ‘Own experience’, ‘Prior experience with previous requirements’, and ‘Student success rates’. One respondent also noted (but did not rate) ‘Health and safety issues for nurses, students on construction sites, etc’.

4.15 Other feedback to IELTS

At the end of the questionnaire, respondents were invited to note any issues they would like to raise with IELTS, which had been identified clearly as the source of the survey:

Q 18 Is there any other feedback you would like to give to IELTS?

There were 28 comments in response to this question. In addition to requests for further detailed information, some respondents were critical of the Test, mentioning issues of reliability and content. Other comments related to the use of the test:

Many international students (ESL) have an unhealthy focus on passing the test rather than achieving language proficiency. The high demand for IELTS prep courses is testimony to this. If we trained our teachers better in assessment methods their recommendations after a 12-week course should be more informative than a one-off test. (2026)

IELTS are clear on what the test results can tell providers, but results often misused and too much relied on them eg predictive ability. We continually repeat this message here! (1002)

IELTS 7 could not be achieved by many current NZ students. Some professions set too much value on high IELTS. (2036)

International students should be placed initially according to pre-entry test but re-assessed once they have settled (I have strong views on the stress factor in language testing in general which may also affect IELTS/TOEFL). (1002)

Two comments related to the disparity between permanent resident students and international students:

Permanent residents by-pass the IELTS requirement and create problems for faculties - who label them as internationals. (1027)

Currently ESL resident students not required to reach same IELTS level as international students. This needs to change. (1015)

Further comments related to the general context of IELTS testing:

Have never assessed it/used it. We have an IELTS requirement only to make sure that intending international students can cope with the course. We do not actively seek international students. (2006)

Our institution runs courses in outdoor recreation training (eg instructor/leadership). A minority of our students are from overseas and usually English speaking. However we are moving towards providing short experience for young overseas students in outdoor pursuits from Singapore. English is OK here from the information to be put across and we make allowances in other ways eg ask for interpreters for larger group bookings. (2060)

These comments reflect the points made in recent academic literature which emphasise the need for IELTS to link to the academic environment in Aotearoa New Zealand (Barkhuizen and Cooper 2004, pp 105; Rogers 2004, pp 59); it has taken on a symbolic value greater than its role as an indicator of a language proficiency.

5 CONCLUSIONS

The extensive and rapid changes in the tertiary environment in Aotearoa New Zealand have had a number of consequences for language tests such as IELTS:

• The tertiary environment is characterised by diversity, resulting in a wide range of English language requirements for different courses. English language assessment methods therefore need to be able to provide institutions with the particular knowledge they require about students who will study on their programs.

• Many tertiary institutions are actively recruiting international students, and are moving away from rigid gate-keeping English language policies to more flexible ‘pathways’ to proficiency. This increasingly includes foundation programs, and may include academic programs with language support.

• There is an increased interest in the use of a range of English language assessment methods, to give a fuller picture of the student’s abilities. Currently IELTS and TOEFL are the most widely accepted assessment methods, with IELTS the most often used. Interviews also used widely.

• Decision-making on English language thresholds is made in a wide variety of ways and at a wide variety of institutional levels. In some cases decisions are made at institutional level, and in others it is made at program level. In some cases decisions are made by groups of people, who may not have expertise in language testing.

• There is a potential for greater liaison on language proficiency issues between course providers and external standards-setting bodies such as the New Zealand Vice Chancellors’ Committee, the New Zealand Nursing Council, or the Hairdressing Industry Training Organisation.

• The IELTS ‘brand’ is well-known, but as the most frequent English language test used in high-stakes situations it has taken on a symbolic value which is sometimes negative. This double-edged sword means that IELTS needs to maintain dialogue with end-users so that they are able to apply the test most appropriately and effectively to their situation. Decision-makers and other staff are also interested in understanding more about language tests such as IELTS.

ACKNOWLEDGEMENTS

We would like to thank the respondents and interviewees for giving their time to participate in this research.

References

• Barkhuizen, G and Cooper, R, 2004, ‘Students’ perceptions of IELTS preparation: How helpful is it for tertiary study in English?’, New Zealand Studies in Applied Linguistics, vol 10, no 1, pp 97-106
• Bellingham, L, 1995, ‘Navigating choppy seas: IELTS as a support for success in higher education’, TESOLANZ Journal, vol 3, pp 21-28
• Cambridge ESOL Examinations website, 2006, retrieved 4 December 2006 from <http://www.cambridgeesol.org/exams>
• Catherall, S and Laugesen, R, 2002, ‘Testing time for would-be Kiwis’, Sunday Star-Times, 24 November, A4
• Coleman, D, Starfield, S and Hagan, A, 2003, ‘The attitudes of IELTS stakeholders: Student and staff perceptions of IELTS in Australian, UK and Chinese tertiary institutions’ in IELTS International English Language Testing System Research Reports, Vol 5, IDP/IELTS, Canberra, Australia
• Converse, JM and Presser, S, 1986, Survey questions: Handcrafting the standardized questionnaire, Sage, Thousand Oaks, California
• Davies, A, 2003, ‘Three heresies of language testing research’, Language Testing, vol 20, no 4, pp 355-368
• Dooey, P, 1999, ‘An investigation into the predictive validity of the IELTS test as an indicator of future academic success’ in Teaching in the disciplines/ Learning in context, eds K Martin, N Stanley and N Davison, Proceedings of the 8th Annual Teaching Learning Forum, The University of Western Australia, February 1999, retrieved 21 June 2004 from <http://lsn.curtin.edu.au/tlf/tlf1999/dooey.html>, Perth, pp 114-118
• Education New Zealand, 2005, Facts and Statistics, retrieved 2 June 2005 from <http://www.educationnz.org.nz>
• Feast, V, 2004, ‘The impact of IELTS scores on performance at university’, International Education Journal, vol 3, no 4, retrieved 21 June 2004 from <http://www.flinders.edu.au/education/iej>, pp 70-85
• Gamble, W and Reid, G, 2002, ‘Overseas students keeping state schools afloat’, New Zealand Herald, 17 March, retrieved 3 June 2005 from <http://www.nzherald.co.nz>
• IELTS International English Language Testing System website, 2005, retrieved 2 June 2005 from <http://www.ielts.org>
• Kerstjens, M and Nery, C, 2000, ‘Predictive validity in the IELTS test: A study of the relationships between IELTS scores and students’ subsequent academic performance’ in IELTS International English Language Testing System Reports, vol 3, ed R Tulloh, IELTS Australia, Canberra, pp 85-108
• Kinnear, J., 2005, Massey University: A year of transformation, Vice-Chancellor’s presentation to Massey University staff, May, NZ
• Kirkness, 2003, ‘Developing a policy to promote academic literacies in English at tertiary level: A case study’, TESOLANZ Journal, vol 11, pp 45-56
• McDowell, C and Merrylees, B, 1998, ‘Survey of receiving institutions’ use and attitude to IELTS’, IELTS International English Language Testing System Reports, vol 1, ed S Wood, IELTS Australia, Canberra
• Ministry of Education, 2003, Code of Practice for the Pastoral Care of International Students - revised December 2003, retrieved 8 March 2004 from <http://www.minedu.govt.nz>
• Ministry of Education, 2004a, Signatories to the Code of Practice, retrieved 8 March 2004 from <http://www.minedu.govt.nz>
• Ministry of Education, 2004b, Export Education Update, No 5, May, retrieved 2 June 2005 from <http://www.minedu.govt.nz>
• New Zealand Herald, 2004, ‘Students from Europe targeted as China market collapses’, December 29, retrieved 2 June 2005 from <http://www.nzherald.co.nz>
• Oxford University Press, 2004, Oxford Placement Test 1 (New Edition Pack), Oxford
• Parker, K, 2004, ‘ESOL students excluded from tertiary participation’, TESOLANZ Newsletter, vol 13, no 2, pp 8-9
• Quigley, N, 2004, ‘Export Education’, Sunday Supplement, September 23, National Radio
• Read, J and Hirsh, D, 2005, English language levels in tertiary institutions: A report for the Export Levy Program of Education New Zealand, retrieved 25 February 2005 from <http://www.educationnz.org.nz>
• Richardson, A, 2005, ‘Falling by degrees’, Listener, March 26, pp 22-23
• Rogers, S, 2004, ‘Sound preparation: Accents in IELTS listening materials’, TESOLANZ Journal, vol 12, pp 59-79
• Skyrme, G, 2004, ‘Stepping over the threshold: Chinese international students in transition from language centre to university’, New Zealand Studies in Applied Linguistics, vol 10, no 2, pp 93-102
• Smith, L and Rae, A, 2004, Managing international student numbers at New Zealand public universities and polytechnics: A report to the Ministry of Education, Massey University, retrieved 4 April 2005 from <http://www.educationnz.org.nz>
• TOEFL Test of English as a Foreign Language website, 2006, retrieved 4 December 2006 from <http://www.ets.org/toefl>
• TOEIC Test of English for International Communication website, 2006, retrieved 4 December 2006 from <http://www.ets.org/toeic>
• Victoria University of Wellington, 2004, Victoria International, retrieved 10 March 2004 from <http://www.vuw.ac.nz>
• Ward, C and Masgoret, A-M, 2004, The experiences of international students in New Zealand: Report on the results of the national survey, International Policy and Development Unit, Ministry of Education, retrieved 2 June 2005 from <http://www.minedu.govt.nz>

APPENDIX 1A: PUBLIC TERTIARY INSTITUTIONS

Signatories to the Code of Practice for the Pastoral Care of International Students at 17 August 2004 (Ministry of Education 2004a) (N = 33)

APPENDIX 1B: PRIVATE TERTIARY INSTITUTIONS (NON-LANGUAGE SCHOOL)

Signatories to the Code of Practice for the Pastoral Care of International Students at 17 August 2004 (Ministry of Education 2004a) (N = 171)

APPENDIX 2: QUESTIONNAIRE

IELTS Survey

Are you responsible for recommending or setting the English language proficiency levels for entry by ESL (English as a Second Language) students to your institution, or for a programme - other than language courses - in the institution?

1 Which of the following levels are taught on your programme?

Pre-degree / practical: ____ Undergraduate: ____ Postgraduate: ____

2 Please tick the English language assessment methods you accept for entry of ESL international students:

3 If you accept these tests, what entry levels are required (please indicate the range as appropriate)?

4 If you have different language levels for the same level of courses, what are the reasons?

____________________

____________________

5 Which English language assessment method is the one used most often for entry to your programme?

____________________

6 How accurate do you think the following assessment methods are as measures of the English language ability required for your programme?

Accuracy scale: Very accurate to not at all accurate; undecided; N/A.

7 How important are the following criteria when you choose assessment methods for entry requirements?

Please turn over …

8 How satisfied do you think the following people are with the current level of English of the ESL (English as a Second Language) students in your programme?

9 Is it likely that your English language entry levels requirements will change in the near future?

Yes: ____ No: ____ Unsure: ____

Comments: ____________________

10 How important is it to ensure the following when you make decisions on the English language entry levels?

11 Do you believe a student’s English ability should be fully formed before they begin subject area studies at tertiary level?

Yes: ____ No: ____ Unsure: ____

Comments: ____________________

12 How important are each of these inputs in helping you make decisions on the English language entry levels?

13 Approximately how many ESL international students are studying in your programme this year?

None: ____ 1-10: ____ 11-20: ____

21-100: ____ 101-200: ____ 201-500: ____

501-1,000: ____ 1,001-5,000: ____ 5,001 or more: ____

14 What percentage of the current total student body in your programme are ESL international students?

None: ____ 1-10%: ____ 11-20%: ____

21-30%: ____ 31-40%: ____ 41-50%: ____

51-60%: ____ 61-70%: ____ 71-80%: ____

81-90%: ____ 91-100%: ____

15 Approximately how many other (permanent resident) ESL students are studying in your programme this year?

Unsure: ____ None: ____

1-100: ____ 101-200: ____ 201-500: ____

501-1,000: ____ 1001-5,000: ____ 5,001 or more: ____

16 Do you predict the number of ESL international students to change significantly in future years?

Yes: ____ No: ____ Unsure: ____

Comments: ____________________

17 What is your title/role in your institution? ____________________

18 Is there any other feedback you would like to give to IELTS?

____________________

Thank you for participating in this research. Please return the completed survey in the envelope provided.

APPENDIX 3: LETTER OF INTRODUCTION

Systemetrics Research Ltd

P O Box 1507

Palmerston North

Aotearoa New Zealand

T +64 6 353-6357 F +64 6 353-8357 M +64 6 353-607

Dear

Re: IELTS Survey

I am writing to you to seek your help in responding to my survey.

I am carrying out research for the International English Language Testing System (IELTS) as an independent researcher. This is a joint-funded project for Cambridge ESOL, the British Council, and IELTS Australia (Round 9), as shown on the website for IELTS Australia at www.idp.com/globalexaminationservices/article857.asp.

The project is an investigation of the attitudes of tertiary key decision-makers towards English language tests in Aotearoa New Zealand. I am particularly interested in finding out about the English language tests used by tertiary institutions in Aotearoa New Zealand, and how the entry threshold levels are established for international ESL (English as a Second Language) students - and other ESL students - studying at tertiary level.

The survey will be sent to all tertiary institutions (non-language school) which are signed up to the Code of Practice for the Pastoral Care of International Students. Much of the information requested is an extension of information already publicly available under the Code, with additional exploratory questions.

Your participation is of course voluntary, and any information other than that which is publicly available will be confidential to me and the project’s statistical advisor, Professor Stephen Haslett at Massey University. All information which is not publicly available will be presented anonymously in the research report, and no individual person or institution will be able to be identified. In this project I am bound by the ethical protocols for applied linguists as outlined on www.aaal.org/pages/AILApractices.htm. I will send a copy of the results to you when the project is completed.

Please do not hesitate to contact me if you have any queries.

With thanks

Dr Hilary Smith

hilary_smith@xtra.co.nz

Author biodata

Hilary Smith is a language and linguistics teacher and researcher based in Palmerston North, Aotearoa New Zealand. She has lived and worked in Tonga, Papua New Guinea and the Lao PDR. Her major areas of interest include English for speakers of other languages (ESOL), language attitudes, language and education, language and settlement, and education and international development.

Stephen Haslett

Stephen Haslett is Professor of Statistics in the Institute of Information Sciences and Technology and Director of the Statistics Research and Consultancy Centre at Massey University, Palmerston North, New Zealand. He is a Chartered Statistician (UK) and a Fellow of the Royal Statistical Society. He has been involved extensively in collaborative research and statistical consulting projects for the last 25 years, including a number of projects in linguistics.