Investigating stakeholders' perceptions of IELTS as an entry requirement for higher education in the UK

Abstract

This project explores stakeholders’ perceptions of the role of the International English Language Testing System (IELTS) in the admissions processes of UK higher education (HE) institutions. It draws on two pieces of empirical study: a large-scale questionnaire survey of those responsible for admissions decisions in a range of HE institutions in the pre- and post-1992 sectors; and a smaller-scale interview-based qualitative study of a subset of these participants.

The empirical data gathered offered insights into the processes of standards-setting in various contexts, highlighted tensions between standards-setting and a growing economic imperative to recruit, and identified a niche for development opportunities in raising stakeholders’ awareness of the content and process of IELTS to enhance the quality of decision-making in this area. The study offered a number of recommendations for the designers/producers of IELTS, and for HE institutions. It also highlighted a number of directions for further complementary research.

1. INTRODUCTION

Higher education in the UK has seen significant growth over the last 10 years in its international student population (largely comprising students whose first language is not English) and applications to courses from undergraduate to postgraduate and research degree level. In the context of this increasing internationalisation of UK higher education provision, the role and importance of English language qualifications, upon which institutions determine whether or not students have the appropriate level of English language proficiency to enter and to be successful on their programmes, has become increasingly significant. While there is an important and growing literature in the area of assessment in ELT generally, and in the context of assessment designed for higher education entry evaluation purposes, an under-researched area is of how stakeholders in the UK perceive the role and value of such examinations and qualifications for their own entry evaluation purposes. Arguably, the most significant of such assessments and qualifications lie in the IELTS Test of four macro skills, and it is in the specific context of the perception of this Test as a factor in decisions around entry to courses in UK higher education institutions that this research project was located. The project was commissioned by the British Council, IDP: IELTS Australia and the University of Cambridge ESOL Examinations and carried out between March 2007 and March 2008.

To provide more contemporary insights into this new internationalised higher education (HE) context, this research project includes a brief review of key aspects of published research relating to the impact of the IELTS Test on the decision-making process of those academic/administrative staff responsible for application acceptance and rejection. This review includes funded research studies published between 1995 and 2001 (Rounds 1-7) listed in Cambridge ESOL’s Research Notes 8 (May 2002) and later rounds (Rounds 8-10) listed in Research Notes 20 (May 2005). It is worth noting that all these studies have been published in the volumes of Research Reports produced over the years by IDP: IELTS Australia (more recently in collaboration with BC). The review is supplemented by a review of relevant research appearing in key ELT/ESOL-related international refereed journals in the period 2000-2007. More specifically, it provides a critical review of contemporary relevant research into stakeholders’ perceptions of the use and usefulness of the IELTS Test for the HE sector, including key recent work such as Cizek (2001a, 2001b), Rea-Dickins et al (2007), Smith and Haslett (2007), Coleman, Starfield and Hagan (2003), Read and Hayes (2003), and Kerstjens and Nery (2000).

The project then considers by survey the perceived use and usefulness of IELTS among key stakeholders responsible for the acceptance of students whose first language is not English onto academic programmes in UK HE institutions. The research also seeks to identify whether additional EAP (English for Academic Purposes) support is needed for students to successfully complete their programmes of study and, if present, how this support is provided. It further seeks to report and disseminate the findings of this desk-based and survey research in a form useful to both the research-funding providers, and to a wider constituency of stakeholders and EAP practitioners.

The research project also provided an opportunity to raise awareness among stakeholders of the IELTS Scores Explained standards-setting DVD. Initial perceptions of participants regarding the value of this resource were elicited, though a full evaluation of participants’ assessments of the DVD was beyond the scope of this research.

2. INSIGHTS FROM THE LITERATURE

The impact of high-stakes testing has been widely acknowledged in the literature (Cizek 2001a, 2001b, Mehrens and Cizek 2001, Burger and Krueger 2003, Train 2002) though it remains a contested area (Camilli 2003). One example of such high-stakes testing comes with the impact of IELTS (International English Language Testing System), a key English language exam used to assess the capability of candidates wishing to enter programmes in institutions of higher education and for immigration or professional purposes in English-speaking countries. Such testing systems can have a massive impact on the lives and futures of many of those who are users of this system.

The IELTS testing system has a history of ongoing funding of research into all aspects of the system. The test, originally known as English Language Testing Service (ELTS), replaced the English Proficiency Test Battery (EPTB), which had been used since the mid 1960s in gauging potential HE students’ language proficiency. This system continued until the late 1980s when it became clear that some practical administrative issues, largely around the scope of the test, needed addressing. A validation study was commissioned (Criper and Davies 1988, Hughes, Porter and Weir 1988) and this led to the setting up of the ELTS Revision project to design and construct a new test. To enhance the international nature of the test, the International Development Programme of Australian Universities and Colleges (IDP), now known as IELTS Australia, joined British Council and UCLES to form an international partnership. The new test was simplified and shortened and also changed its name to reflect the new internationalisation, becoming known as the International English Language Testing System (IELTS) and went into operation in 1989. During the period 1989-1994, the system was monitored through a raft of research evaluations, and further modifications were introduced in 1995, including the replacement of three subject-specific subtests with one Academic Reading and one Academic Writing module, the removal of the thematic link between the Reading and Writing modules, the convergence of scoring on all modules to nine bands, the introduction of checks on marking consistency, an appeal procedure, new validation procedures, security procedures and computerised administration procedures.

The change from three subject-specific subtests was based on feedback from IELTS administrators and examiners (Charge and Taylor 1997) and from a significant body of research into ESP and second language reading from Caroline Clapham (Clapham 1993, 1995, 1996). Clapham concluded that a single test did not discriminate for or against candidates regardless of their disciplinary areas and that a single test would not hinder accessibility. More specific details of these innovations and the rationale behind them can be found in Charge and Taylor (1997). More recently, continued evaluation of the system led to the introduction in 2001 of a new Speaking test, and in 2005 the introduction of new assessment criteria for the Writing test and the introduction of computer-based testing. A recent and comprehensive overview of the history of the assessment of academic English comes in Davies (2008). Interestingly, Davies notes that calculations of predictive validity in each of the stages of academic language assessment considered (grammar, ‘real-life’ contexts and features of language usage) vary only slightly and so he suggests that the choice of proficiency test needs to be guided not only by predictive validity but also by other factors, one of which is impact on stakeholders, again emphasising the importance of this aspect of language testing research, as realised in our research project. The history of IELTS is therefore one of continual monitoring and enhancement through research and evaluation, and the project reported here was intended to contribute to this consistent chain of development of the testing system.

A number of studies have investigated relationships and correlations between IELTS scores and subsequent academic performance, as reported by Feast (2002) and Davies (2008). The outcomes of these projects generated variable conclusions. A range of studies concluded that there was a weak positive association between academic performance and IELTS scores (Criper and Davies, 1988;

Elder, 1993; Ferguson and White, 1993; Cotton and Conrow, 1998; Hill et al, 2000; Kerstjens and Nery, 2000). Some studies found no statistically significant relationship between IELTS and academic performance (Fiocco, 1987; Graham, 1987; Light, Xu and Mossop, 1987) while others found their results inconclusive (Dooey, 1999). The exception came with a study conducted by Bellingham (1993) which suggested a moderate association between the two variables though this study was unusual in that it included students with a wide range of IELTS scores including some below 5.0.

While there is a significant and growing literature on English language testing (Cheng et al 2004) and on the credibility, reliability and validity of IELTS in particular (Green 2007), other more social and qualitative impacts also deserve consideration (Brown and Taylor 2006; Barkhuizen and Cooper 2004; Read and Hayes 2003; Coleman, Starfield and Hagan 2003). These include the ways in which individual students perceive the value of such suites of exams and, more significantly for this project, the processes through which individuals in institutions make decisions as to the appropriacy of certain scores as indicators of a student’s capability to succeed on a course or their acceptability to participate in such a course. The current context is one of increasing interest in ‘consequential validity’, a concern with the social consequences of testing, and so an increasing emphasis on the ways in which assessments affect learning and teaching practices. In light of this, a body of recent research has focused on impact studies on IELTS, including the consideration of stakeholder attitudes. A key overview of methodological and theoretical issues of such research is presented in Hawkey (2006), which focuses as one of its two case studies on IELTS impact testing. The stakeholders considered in this research include test-takers, teachers, textbook writers, testers and institutions. However, unlike the present study, there was no specific emphasis on admissions gatekeepers, a niche our research aims to fill, while acknowledging that Hawkey (2006) provides an invaluable guide at both a theoretical and practical level to those engaging in impact studies.

Rea-Dickins et al (2007) looked at the affective and academic impacts of the IELTS performance of a group of postgraduate students, and argued there had been little focus in IELTS impact studies on the different IELTS profiles of ‘successful IELTS students’. In relation to this argument, the research project reported here sought to uncover the ways in which stakeholders in admissions roles equate such profiles with IELTS scores, and to further elucidate Rea-Dickins et al’s claim that there is an overwhelming lack of awareness by admissions staff about IELTS.

Smith and Haslett (2007) investigated the attitudes of HE decision-makers in Aotearoa New Zealand towards the English language tests used for admission purposes. They argued that the changing context and growing diversity were leading to consideration of more flexible pathways to entry. IELTS still held a symbolic value beyond its purpose as an indicator of language proficiency, due to its high-stakes function as the best known ‘brand’ of English language testing systems. They reported that a number of decision-makers said they would appreciate more information about test results from test providers and that there was potential for greater liaison on language proficiency issues between course providers and external industry standards-setting bodies. In relation to these assertions, the current project sought to investigate if such perceptions are mirrored in the UK context and to investigate any emerging divergence from Smith and Haslett’s findings.

Coleman, Starfield and Hagan (2003) contrasted stakeholder attitudes to IELTS in Australia, the People’s Republic of China and the United Kingdom. As with the current project, the perceptions and perspectives of university staff and students were measured via quantitative and qualitative methodologies. The researchers argued that students were, on the whole, more knowledgeable than staff on a wide range of themes related to the IELTS Test. Both staff and students indicated that the purpose of the IELTS Test was primarily a functional one in terms of acceptability for entry to a particular course or programme, and that the educational role of language proficiency improvement was a secondary consideration. Participants perceived the IELTS Test to have high validity but staff and student respondents differed over the predictive nature of the IELTS test score in relation to university study. Students tended to have a positive view of IELTS as a predictive indicator of future success whereas staff were less satisfied with the predictive value of the Test and wished to see minimum standards for entry set at a higher level. The current project therefore sought to investigate if such perspectives were still reflected by institutional gatekeepers some four years after the publication of this key piece of research, though the nature of student perceptions was beyond the remit of this study.

Read and Hayes (2003) investigated the impact of IELTS on the preparation of international students for tertiary study in New Zealand. They found that even students who gained the minimum band score for tertiary admission were likely to struggle to meet the demands of English-medium study in a New Zealand university or polytechnic, though teachers generally recognised that IELTS was the most suitable test available for the purpose of admission to HE programmes. The current study sought to ascertain whether the views of gatekeepers at HE institutions in the UK converged or diverged from those positions.

Kerstjens and Nery’s (2000) research sought to determine the relationship between the IELTS Test and students’ subsequent academic performance. They reported that for students at the vocational level, IELTS was not found to be a significant predictor of academic performance, although staff and students were generally positive about students’ capability to cope with the language demands of their first semester of study. The correlation between English language proficiency and academic performance is an issue that has been researched frequently and an overview of this research theme can be found in Davies (2008). The present study therefore examined this relationship and sought the perspectives of HE respondents as to the difficulties students encounter and whether or not IELTS fully meets their needs in terms of addressing language difficulties.

Mok, Parr, Lee and Wylie (1998) compared IELTS with another examination used for purposes similar to the general IELTS paper and McDowell and Merrylees (1998) investigated the range of tests available in Australian tertiary education to establish to what extent IELTS was serving the needs of the receiving institutions. Similarly, Hill, Storch and Lynch (2000) sought to explore the usefulness of IELTS and TOEFL (the two main measures of English language proficiency used for selection to universities in Australia) respectively as predictors of readiness for the Australian academic context. The current research project hoped to uncover whether IELTS was the dominant language testing system in UK HE and if stakeholders view it as meeting their needs, as well as those of their students.

Feast (2002) sought to investigate the relationship between IELTS scores as a measure of language proficiency and performance at university, as measured by grade point average (GPA). Her research revealed a significant and positive, but weak, relationship between English language proficiency and academic performance. On the basis of this research, she recommended raising the IELTS scores required for admission, either globally or on individual papers, while recognising this might result in financial losses in terms of student numbers recruited, and that her recommendations would raise political and financial considerations for university management.

Edwards et al (2007) also highlighted the concerns of university teachers and administrators around the limitations of tests of English used in relation to university admissions, and expressed concerns around the degree to which acceptance of students with levels well below native-speaker competence represented a lowering of academic standards, or a pragmatic response to an increasingly globalised HE market. In the light of this changing economic context, this research project sought to elicit participants’ perceptions regarding any tension between setting language standards and recruitment, and how any such tensions might be resolved.

A key concern of this research was also the relationship and communication between EAP specialists and those responsible for admissions to UK higher education. In acknowledgement of the need for improved communication and to enhance the shared understanding of issues around admissions criteria, BALEAP (British Association of Lecturers in English for Academic Purposes) has produced updated Guidelines on English Language Proficiency Levels for International Applicants to UK Universities (Bool et al 2003). This document suggests that two months of intensive EAP study is the equivalent of one band on the IELTS scale. However, more recent changes in the composition of the international student population have seen research-based challenges to this position (Green 2005, Read and Hayes 2003).

Our project therefore drew on a range of contemporary literature, at both the research design stage and at the analysis and interpretation stage, to consider the degree of convergence and divergence of our findings with those of other projects undertaken in related but distinct contexts.

3. METHOD

As noted earlier, the empirical phase of the research project sought to engage stakeholders and probe their perceptions of the use and value of the IELTS Test as a factor in decision-making processes regarding entry into UK HE institutions. In doing so, the project sought to address a number of specific research questions:

• What IELTS level is the required minimum for student acceptance onto a range of programmes in various UK institutions? How consistent are these requirements in differing sectors of HE provision?
• To what degree do stakeholders consider the IELTS Test a useful indicator of academic English proficiency appropriate for higher education study in the UK?
• What is the process for standards-setting in various HE institutions?
• To what degree is there a tension between setting standards and the need to recruit?
• What degree of additional post-entry EAP support do stakeholders find is necessary that is not indicated by IELTS levels? How do stakeholders respond to any additional identified EAP needs?
• What other English language qualifications do institutions accept as equivalent to IELTS?
• How aware are stakeholders of the process and content of the IELTS examinations and what development needs does this reveal?
• What are the implications of these understandings for IDP: IELTS Australia, the British Council and the University of Cambridge ESOL Examinations in terms of adjustments to, provision of, guidance for, and the marketing of the IELTS Test?

In addition to these core research questions, the project also sought to investigate to what degree the IELTS Scores Explained standards-setting DVD was viewed to be a helpful resource.

3.1 Sample

The research project targeted a sample of academics and administrators in two UK HE sectors: Russell Group Universities and the post-1992 ‘New’ Universities. Through a systematic review of the websites of these institutions, we identified individuals operating in either academic or administrative roles who were responsible for the standards-setting of IELTS scores which were deemed acceptable by the department or the institution for entry onto particular programmes, at both undergraduate and postgraduate levels. Interested individuals from a range of institutions were approached via email. The sample of universities focused on those with high usage of IELTS.

We identified 15 HE institutions from two distinct groups. Seven were from the Russell Group (a collaboration of 20 UK universities that receive two-thirds of universities’ research grant and contract funding in the UK, sometimes referred to as the British equivalent of the Ivy League of the United States, and containing many of the UK’s leading universities with 18 of its 20 members in the top 20 in terms of research funding) and from the 1994 group of ‘smaller research-intensive universities’. Another seven were from the new universities created in 1992 largely from former polytechnics, central institutions, or colleges of higher education that were given the status of universities by the Conservative government in 1992, or institutions that have been granted university status since then.

To further broaden the sample, we also solicited a response from one private university. Within each institution, we identified 15 departments to enable us to investigate some of the complexities that exist within institutions and differing intra-institutional variations in standards setting. The departments were selected in order to achieve a degree of comparability across the institutions. They were also selected to offer a range of subject areas including science, social science, humanities and more professionally and vocationally-focused departments. The intention here was simply to achieve a broad sample rather than to claim ontologically objective and epistemologically positivistic bases for the findings.

We also identified a further group of institutions and departments in both sectors to ensure that we met our target sample, should the initial sampling procedures prove insufficient, but the response rate, after follow-up contact, proved sufficient. The procedure involved approaching participants at the start of the project, distributing questionnaires to the identified recipients and offering a copy of the DVD to those agreeing to participate, along with an invitation to a telephone interview.

We received responses from seven old and seven new universities and the private university. Within these institutions, we received responses representing:

• 14 departments within the old university sector
• 12 departments within the new university sector
• 1 department within the private university.

The findings are based on a sample of 100 questionnaire responses, complemented by 12 follow-up telephone interviews. The questionnaire elicited 104 responses but four responses had to be discarded as those respondents had misunderstood either the purpose of the research or the roles of the participants targeted, and had no experience or awareness of the IELTS testing system. This situation came about where the questionnaire had been forwarded to these respondents by the targeted respondents, under the mistaken impression that the forwarded respondents might be able to contribute meaningfully to the research. The four excluded respondents were thanked for their contribution but the data were excluded as irrelevant to the aims of the project. Coincidentally, this entailed that the achieved sample was exactly 100 and in the analysis section below, quantitative data will be reported in terms of percentages, but with the understanding that, where they relate to the full sample, these percentages also equate to the number of respondents. Both the questionnaire and the telephone

interview schedule covered the research questions stated above which represented the initial research questions, plus others that emerged from the initial phases of data collection and analysis.

3.2 Approach to data analysis

We used a basic statistical analysis of the quantitative data that emerged from the questionnaire and, for the more qualitative data from both questionnaire and interview, we utilised a category analysis by pulling out key insights from the data and collating, analysing and interpreting these under various pre-identified themes (deductive coding) and emergent themes (inductive coding), as outlined in Miles and Huberman (1994) and Glense and Peshkin (1992). These findings were then linked to previous/other findings and the research literature to establish the relevance of the research.

The main qualitative approach taken in the report is to build a narrative around the voices of the respondents. In the results section, in the elements relating to both the questionnaire and interview data, various interpretations are offered and then at least one respondent is quoted to illustrate the point. However, to contextualise these responses, and to link the voices to the participants more coherently, it is necessary to identify and contextualise the background of each respondent. This approach serves to offer differentiation of one voice from another and to ensure that the ideas being reported are not simply idiosyncratic. It also demonstrates the degree to which opinions and positions are shared by other respondents.

In order to achieve this, four levels of differentiation are identified alongside the numerical indicator of each respondent. The first level of differentiation concerns the role of the respondent, coded as either academic tutors (‘ac’) or administrators (‘admin’). The second level codes the institutional sector: ‘old’, ‘new’ or ‘priv’ for private. The third level differentiates the students as either undergraduate (‘ug’) or postgraduate (‘pg’). Where respondents deal with students at both levels, the coding ‘ug/pg’ is used. The final level of coding is for the subject/disciplinary area within which the respondents are located. This level offers the widest degree of potential differentiation and so, in the interests of analytical clarity, is divided into the following four sections:

1. ‘sci’ - pure/applied sciences, eg chemical engineering, mechanical engineering, materials physics, metallurgy, design engineering and computing, speech sciences, informatics.
2. ‘a&h’ - arts and humanities eg English, modern foreign languages, history, applied linguistic studies, languages and European studies.
3. ‘soc sci’ - social sciences eg education, politics, economics.
4. ‘prof/voc’ - professional/vocational studies eg law, medicine, business studies, architecture, health studies.

Occasionally, respondents, particularly administration voices, are not linked solely to one subject/disciplinary area, and in such cases one of the four codes above will be replaced by the code ‘gen’ for general.

So, for example, a quotation from respondent 1, who is an academic tutor, working in an old university with postgraduate students in the area of materials physics, would be identified as follows:

Experience shows that students, even at 6.5, struggle with the course in terms of English proficiency. Q1, (ac, old, pg, sci)

It is, however, important to note that, while an attempt is being made to locate the voices of respondents in their institutional and professional contexts, it would be misleading to claim that these voices are representative of all members of their role-group, institutional, sector, level and subject

contexts, and so a balance is being sought here - to minimise idiosyncratic perspectives while not seeking to misrepresent voices, even shared voices, as generalisations.

3.3. Timetable

The project was divided into a number of phases, as indicated below.

Phase 1

An analytic review of relevant research was carried out and the major aspects which this covered are outlined in the section above on insights from the literature. The samples of the institutions and the individuals for the empirical phase were identified. The questionnaire was piloted with four members of staff from the researchers’ department and from the researchers’ institution’s English language support unit. This process elicited two new questions, and the rewording of three items, to enhance comprehensibility and clarity.

Phase 2

New data to be captured and analysed were both quantitative and qualitative in nature. The quantitative element consisted of an analysis of the responses to the email questionnaire (see Appendix 1). The questionnaire was in part based on the IELTS survey for college/university staff contained within the IELTS Scores Explained standards-setting DVD. The largely quantitative enquiry here was supplemented with additional qualitative questions to probe the reasons behind certain decisions in standards-setting and institutional pressures/requirements, in line with the aims and the specific research questions enumerated above.

Phase 3

The data elicited in both quantitative and qualitative form from the questionnaire were further supplemented by a more qualitative analysis of responses to a range of open questions in a series of telephone interviews. The interview was piloted with three members of staff from the researchers’ department. This process helped in the formulation of some of the follow-up questions and examples indicated in the interview schedule (see Appendix 2). The exact form and scope of these open questions was dependent partly on the initial research questions and partly on the responses to the questionnaire in Phase 2, as well as on the piloting process. The aim of the supplementary questions, in both the questionnaire and interview, was to drill down beyond the existing factual/descriptive data to understand more fully the reasons for individual HE institutions’ and departments’ setting of specific English language requirements and any improvements HE institutions / departments might find useful.

Phase 4

The intention here was to elicit data regarding the degree to which the IELTS Scores Explained standards-setting DVD was viewed to be a helpful resource, and what suggestions participants had as to how the DVD could be revised or improved. This element of the research was limited by the fact that only 17 questionnaire respondents requested a copy of the DVD, and of these respondents, only six accepted the request to be interviewed. Their insights are included in the data analysis and suggestions for enhancing this aspect of the project in future research are included in the conclusions section.

3.4 Ethical considerations

The ethical standards of the British Educational Research Association (2004 version) were followed. The project was approved through the University of Sheffield Ethical Review Procedure.

4. EMPIRICAL FINDINGS

4.1 Insights from questionnaire data

4.1.1 Overview of participants

Of those who responded to the questionnaire, 56% came from old universities, 43% from new universities and 1% from the private university.

Figure 1: Comparison of participating institutions by sector

Within this breakdown, the responses came from 14 departments within the old university sector, 12 departments within the new university sector, and one department within the private university. Seventy nine per cent of responses came from those who identified themselves as academic staff and 21% from those who identified themselves as administrative staff.

Figure 2: Comparison of participants by role

In terms of seniority, within the academic staff sector, 29.1% (23 respondents) identified themselves as junior staff, 48.1% (38 respondents) as mid-level staff and 22.8% (18 respondents) as senior-level staff. Within the administrative staff sector, 47.6% (10 respondents) identified themselves as junior staff, 52.4% (11 respondents) as mid-level staff and none as senior-level staff.

4.1.2 Use of IELTS as an entry requirement

IELTS was used in 96% of the sampled institutions, with the other 4% reporting that they used their own institutional test but that these tests were largely based on the IELTS model. One institution, while accepting IELTS, reported using its own test based on the IELTS Academic module as an alternative for students who could not access an IELTS test centre.

Of the total sample, 12% were working with foundation or pre-sessional students, 24% with undergraduates and 64% with postgraduate students.

Figure 3: Percentage of participants by student level

Within the new university sector, including the one respondent from the private sector, 9% were working with foundation or pre-sessional students, 18% with undergraduates and 73% with postgraduate students. In the old university sector, 11% were working with foundation or pre-sessional students, 23% with undergraduates and 66% with postgraduate students.

Overall, 61% of respondents reported that their institution used the Academic module of the IELTS Test, while 31% responded that their institution accepted either the Academic module or the General Training module. No institution reported using the General Training module alone. Eight per cent of respondents could not respond to this question as they reported they were unaware that there were two types of test eg:

I was not aware that there were two types of test. I expect that most candidates choose to take the IELTS Academic. Q14 (ac, old, pg, prof/voc)

Figure 4: Versions of IELTS test used

Of the 61 respondents who reported that they used the Academic module, the majority (65.5%) reported that the reason for using this test was that the academic nature of the test best fitted with their requirements.

The language for the course is academic and not conversational in nature, therefore more appropriate. Q8 (ac, old, pg, sci)

Need to assess suitability for academic study in English. Q19 (ac, new, pg, sci)

Most appropriate test for assessing English Language for academic purposes.

Q41 (admin, new, pg, sci)

Test for admission onto a postgraduate course, so Academic IELTS is the appropriate one.

Q75 (ac, old, pg, sci)

Other respondents (26.2%) noted it was an institutional requirement.

University/ International Office guidelines favour this test. Q3 (admin, old, pg, soc sci)

University decision. Q23 (ac, new, pg, a&h)

Others (6.5%) noted this decision was one arrived at through experience.

The IELTS is a suitable instrument for us as the decisions based on it have proved viable over the years. I like the different component results, which allow for a particularly useful interpretation of language skills competence. Q2 (ac, old, pg, a&h)

Of those who used either the Academic module or the General Training module, a range of response was given in justification.

To determine the level of English of applicants. Q10 (admin, old, pg, soc sci)

University admissions policy. Q13 (ac, old, ug, a&h)

To gauge students’ general level of ability. We prefer IELTS (Academic) but not all students have the chance to study for it. Q18 (admin, new, ug/pg, gen)

Don’t know which one - the International Office makes this decision, administers the tests and provides me with the results. Q21 (ac, new, pg, sci)

I use whatever is likely to be there or useful. Q25 (ac, old, pg, soc sci)

As noted earlier, we received responses representing 14 departments within the old university sector, 12 departments within the new university sector, and 1 department (English) from the private university. These were as follows:

• Old universities - Economics, Speech Sciences, Politics, Materials Physics, Applied Linguistics, Mechanical Engineering, Metallurgy, Medicine, French, Institute for Lifelong Learning, Business School, Chemical Engineering, Health Economics and Management, Education
• New universities - History, Languages and European Studies, Health Studies, Design Engineering and Computing, Informatics, Business Studies, English, Languages and European Studies, Applied Linguistics, Art and Design, Law, TESOL.

4.1.3 Minimum entry requirements

Respondents were questioned as to their required minimum for student acceptance onto their course or programme. The following table represents their responses. The level of study is indicated as undergraduate (U) or postgraduate (P). The sector is indicated for comparative purposes: Old (O), New (N) and Private (Pri).

Key: Undergraduate (U), Postgraduate (P) University - Old (O), New (N) and Private (Pri)

Table 1: Departmental minimum IELTS grade requirements

The table indicates that there is no clear pattern as to particular faculties or departments systematically choosing higher or lower entry requirements, and that these decisions are taken on a case-by-case basis. There is, however, a tendency for the higher entry requirements of band 7 and above to be related to old universities, unless there is a language element to the course, eg Applied Linguistics or TESOL.

It is perhaps telling of the relative paucity of effective communication between ESP specialists and academic/administrative admissions officers that there was no mention of the BALEAP Guidelines on English Language Proficiency Levels for International Applicants to UK Universities (Bool et al 2003). This is despite over 70 British universities being BALEAP members and 23 of these having courses accredited by the BALEAP Accreditation Scheme. Therefore, it might be the case that, despite recent efforts (eg Green 2005), the dissemination of this and other codes of advice and practice could be more widely publicised by stakeholding organisations such as the sponsors of this research, while acknowledging that the dynamic nature of the composition of the international student population requires such guidelines to be read critically and themselves to be constantly under review. As Green (2005, p 59) notes: …institutions responsible for policy relating to test scores should take the imprecision of test scores, and their derivatives, such as score gains, into account. They should exercise caution in interpreting scores, and should seek multiple sources of evidence of learners’ abilities wherever possible.

It would be interesting in future research to compare the entry requirements for home students to see if there is a correlation between the academic requirements and the language requirements for international students, but such a comparison was beyond the scope of this project. The table indicates some interesting issues, such as the low levels of 5.5 for undergraduate English and postgraduate Art and Design - both programmes with a substantial communicative requirement. Both these courses were from new universities and the rationale for such low entry requirements was one of recruitment and marketing, with respondents indicating that higher entry levels might mean that candidates selected other universities with higher research/teaching ratings. Both here, and for many other aspects of the questionnaire data, insights from the interview data shed further light on the reasoning behind these standards-setting processes.

In response to a question regarding the participants’ perceptions as to whether they felt that entry grades should be higher, lower or remain unchanged, $57\%$ felt they should remain unchanged while $43\%$ felt they should be higher, as argued by Coleman, Starfield and Hagan (2003) and Feast (2002).

Figure 5: Perceptions of entry grade changes desired

This perhaps mirrors the insights offered by Edwards et al (2007) whereby there is evidence of an uncertainty about whether the current lower levels indicate a lowering of standards or a pragmatic response to concerns about recruitment in an increasing marketised higher education context.

No respondents wished to see entry levels lowered. When asked under what circumstances they would accept a student who had not reached their stated requirements, 32% said they would not do so under any circumstances, often ascribing this to notions of the ethics of accepting students who do not have the language capabilities to be successful in the specific course or programme, as illustrated by the following response:

I wouldn’t ever go below the threshold because accepting them onto the programme would be unethical because they would not be able to pass. Q19 (ac, new, pg, sci)

The largest grouping of responses to this question (44%) noted they might accept a student who had not reached their stated requirements if they were to take a pre-sessional programme before the commencement of their studies while 12% would accept the students if they had indicated an acceptable level of language proficiency at interview:

If the IELTS was a few years back and the person had been working in English and could demonstrate a higher level in the interview. Q26 (ac, old, pg, a&h)

Twelve per cent of respondents said the IELTS requirement would be waived if the student had studied to degree level previously in English, either in the UK or overseas. Other respondents offered a range of individual circumstances in which they would allow such students entry:

If they had spent considerable time in the UK since taking their IELTS and there was not time to take another test. Q4 (admin, old, ug, soc sci)

Sometimes, we know that achieving an IELTS of 8.0 depends on a student’s ability to perform well in that test. It is not necessarily indicative of a candidate’s true ability. As a result, we interview candidates for whom a level of, say, 7.5 has been achieved.

Q14 (ac, old, pg, prof/voc)

If convincing reasons for a lower score were given (eg severe illness during test) or if student agreed to undertake additional pre-sessional English tuition. Q35 (ac, old, pg, sci)

Where qualifications in previous exams suggest sufficient competence, motivation and academic ability to close the language gap whilst on the Foundation Programme.

Q72 (admin, old, ug/pg, gen)

If they showed an improved performance on our internal placement test.

Q90 (ac, new, ug, a&h)

Occasionally there was a reflexive acknowledgement that such decision-making processes had been flawed:

We did last year on the recommendation of the English language teaching unit and we regret this as the student will fail due largely to having poor English skills. Q58 (ac, old, pg, sci)

4.1.4 IELTS as an indicator of academic English proficiency

Respondents were asked to indicate whether they felt the IELTS Test to be a useful indicator of academic English proficiency appropriate for the particular course or programme and the vast majority (88%) felt that it was, echoing the findings of Coleman, Starfield and Hagan (2003).

Figure 6: IELTS a useful indicator of academic English proficiency?

Of the 12% who did not, there were indications that these respondents felt that the Test did not accurately predict competence in academic activities that students would be required to participate in at university.

I don’t think the score students come in with reflects their ability to write academic essays or to cope with UK academic culture so in that respect not a very useful indicator of success anyway - might as well stay as it is. Q13 (ac, old, ug, a&h)

Even with achieving overall score of 6, many if not most of these students still struggle with their English language communications, especially with their written assignments, projects, etc, if they don’t fail or leave the course early out of frustration. Q17 (admin, new, pg, sci)

Think IELTS is more applicable to everyday skills.

Q84 (ac, new, pg, sci)

This issue was followed up in more qualitative detail in the interviews and more fine-grained responses and analysis were indicated. These will be discussed in the next section.

4.1.5 Tension between setting standards and the need to recruit

A key issue in this research was a consideration of participants’ perceptions as to whether there was a tension between the setting of standards for entry and the need to recruit, as indicated by Edwards et al (2007). Of the participants, $59\%$ felt there was such a tension while $41\%$ did not.

Figure 7: Tension between setting of standards for entry and need to recruit?

Those responding that they felt there was such a tension were asked how they sought to resolve this. Responses to this question were varied. An example of particular categories of reaction included the provision of additional support:

By promoting the University pre-sessional course, which allows the diagnostic testing of students on arrival and tailor-made language tuition. Q9 (admin, old, pg, a&h)

By running a student success project with tutor who is able to support individuals who are seen to be ‘at risk’ of failure. Also EFL support classes. Q13 (ac, old, ug, a&h)

By supporting international students in English language before and during their courses. Q20 (ac, new, ug, a&h)

A number of respondents pointed to the need to maintain the current levels either for economic reasons.

We keep the 6.5 requirement. An increase would certainly lower our application numbers.

Q3 (admin, old, pg, soc sci)

There is a tendency to recruit.

Q19 (ac, new, pg, sci)

We do not move.

Q26 (ac, old, pg, a&h)

Or by strict and uncritical adherence to institutional policies on admission:

Comply with University/Administrative directives and policies. Q17 (admin, new, pg, sci)

Some respondents felt there was a value, in terms of enhancing the diversity among the student population, that justified a push to recruitment:

By looking at recruitment to increase student numbers but also to internationalise the university and there is much to be gained from having exchange students on short programmes. However, their general level of English tends to be lower than that of direct entry u/gs and we provide them with additional language support. We are constantly assessing the situation and are in the middle of an overhaul of the system we use.

Q18 (admin, new, ug/pg, gen)

Others felt there was a moral dimension to the issue and that the concerns should be less about the issue of recruitment and more around issues of retention and successful outcomes for students:

Despite the pressure to recruit, I am tending this year to be more, not less stringent with language requirements for second language speakers as the University’s support systems are currently not well suited to the needs of many of the second language students I work with at level 0 and we have a duty not to set people up to fail. Q12 (ac, old, ug, soc sci)

Some respondents felt that it was resolved by leaving the decision in the hands of the member of staff responsible for admissions:

Tutors (of which I am one!) would be happy to limit intake to IELTS 7.0 or higher, but the economics of the course wouldn’t work that way. The decision is left to me and IELTS 6.5 seems to be a reasonable compromise. Many 6.5-ers are extremely good students who do very well here. Q15 (ac, old, pg, sci)

Left to discretion of particular Admissions Tutor.

Q27 (ac, new, pg, prof/voc)

One respondent pointed out the essence of the dilemma and the fact that it had not really been identified as an issue to be explicitly addressed within her institution:

It’s not really resolved - it’s part of the rich texture of the daily environment that we work within. Q23 (ac, new, pg, a&h)

The respondents who felt that there was no tension for them between standards-setting and recruitment tended to be either those who, due to the popularity and marketability of their programmes, did not face pressure to increase recruitment:

No - although this may not be true for all taught PG programmes across the University. We are fortunate to have received 668 applications for 75 places (2007 entry).

Q14 (ac, old, pg, prof/voc)

Not in our Department as we can afford to be choosy.

Q25 (ac, old, pg, soc sci)

Or those who indicated that they felt that the maintenance of high academic standards was paramount and had been successful in advocating this position within their institution:

There is no tension now, because we have persuaded the administrators that we have much more to lose than to gain by admitting inappropriately. Q2 (ac, old, pg, a&h)

4.1.6 Additional post-entry English support

Participants in the questionnaire element of the project were asked if they felt that students required additional post-entry English language support, and whether or not the need for any such support was indicated by the IELTS Test.

Of the respondents, 74% felt that students did require additional post-entry support, as argued by Read and Hayes (2003), while 26% did not.

Figure 8: Post-entry English language support required?

Of those who responded that additional support was needed, 64% indicated that they did not feel that the need for such support was indicated by the IELTS Test while 36% felt that the Test played a diagnostic role in indicating post-entry support needs.

Figure 9: Additional English language support needs indicated by IELTS?

Participants were asked how their department/institution responded to additional identified English language needs. These needs were met either by referral to the institutional English language support service:

Students who are in need of English language support are directed to the Tesol Centre and the University English Scheme, where their needs are assessed; they are then placed onto an appropriate course. Q20 (ac, new, ug, a&h)

We would refer them to our Centre for English language education if they needed help. This can be done pre-sessionally or concurrent with our course. Q44 (admin, old, ug, soc sci)

Make them do courses offered internally by the university. Q66 (ac, old, pg, sci)

Or provide departmentally specific support:

We do provide support - academic writing by regular tutors, then language support by a language tutor. This involves about 10% of students, and seems to be unrelated to IELTS. So tutors identify students when they have seen their written work.

Q2 (ac, old, pg, a&h)

The department offers study skills sessions and the student can enrol themselves on the in-sessional English courses offered by our language centre. Q73 (admin, old, pg, soc sci)

Some offered a combination of both internal and external support:

In-sessional English is provided in addition to an academic writing tutorial service, subject-specific English classes and PG writing classes. Q51 (admin, old, ug/pg, gen)

4.1.7 Other language tests accepted for admissions

The questionnaire sought to investigate the status of IELTS as an entry criterion as part of the admissions process, and its relationship with other tests of English proficiency. The pre-eminent position of IELTS as the language proficiency test of choice in UK HE was confirmed by the fact that 100% of respondents indicated that their institution accepted IELTS for admission purposes.

Twelve per cent of respondents commented that IELTS was the only English language testing system they employed, while $88\%$ reported their institutions used other systems as well.

Of the 88% of respondents accepting other tests:

• 80% accepted TOEFL (Test of English as a Foreign Language)
• 29% accepted Cambridge Proficiency or CAE (Certificate of Advanced English)
• 18% employed their own internal test or interview
• 17% accepted GCSE (General Certificate of Secondary Education) English
• 8% accepted the NCUK (Northern Consortium United Kingdom) EAP diplomas
• 7% accepted Trinity Examinations
• 6% accepted the TOEIC (Test of English for International Communication) examination.

Figure 10: Percentages of all institutions accepting various entry qualifications

This pre-eminence for IELTS confirms the value of an increasing research agenda concerning the IELTS Test, the relevance of investigating the perceptions of stakeholders, and the value for the research sponsors (British Council, IDP: IELTS Australia and the University of Cambridge ESOL Examinations) of continuing efforts to raise awareness of their testing system and clarifying misunderstanding around the system through its research outputs.

4.1.8 Additional comments from respondents

Finally, respondents were given the opportunity to add any additional comments they felt might be of use to the project; these comments elicited a range of responses. Some responses indicated a high level of satisfaction with IELTS:

We find it generally useful. After I went on the training provided by the English language unit to help admission tutors make sense of the figures I was then far more aware of what potential students would be able to cope with. Q8 (ac, old, pg, sci)

Of the testing methods I use, I regard IELTS as probably the most reliable, in particular the ease with which we can check the validity of the certificate. Q55 (ac, old, pg, sci)

Some respondents used the opportunity to point out that English language proficiency in general, and IELTS in particular, were not the only considerations when admitting students, and that any indicator had its limitations.

It is a ‘quick and easy’ indicator but I would guess from experience that:

• a low score does not mean they CANNOT write a thesis
• a higher score does not mean that they CAN write a thesis

Oral skills/listening and speaking are also vitally important for Doctoral students as they must undergo a VivaVoce. Q25 (ac, old, pg, soc sci)

… the level of English is only one of the factors involved in ensuring success for students coming to study in the UK from other cultures. A high level of English does not necessarily guarantee success and more preparation needs to be with students to prepare them for UK study. Q63 (ac, old, ug, a&h)

These issues were considered in more critical and qualitative depth in the analysis of the interview phase of the project.

4.2 Insights from the interview data

Twelve questionnaire respondents who had volunteered were interviewed by telephone. Three respondents came from the new university sector, eight from the old university sector and one from the private university. Three of the respondents (Ints 3, 4 and 10) were administrators; the remainder were academic staff. The interview schedule (see Appendix 2) was drawn up from a list of concerns arising from the original research questions and from issues raised through the questionnaire phase that suggested further investigation would be illuminating.

4.2.1 IELTS as an indicator of student’s language capability in subsequent academic performance.

Interviewees were questioned as to whether they felt that IELTS gave a fair impression of a student’s language capability in an academic context and whether it was a useful indicator of subsequent academic performance. One administrator interviewee was unequivocal that the test achieved these purposes:

Yes it is, yeah.

Int 4 (admin, old, pg, soc sci)

Others were generally positive but with some reservations:

It can be a little bit hit and miss but I guess it’s not a bad indicator of what their general language capabilities are likely to be. Int 12 (ac, old, pg, sci)

I believe it offers a ‘ball park’ indicator, especially the academic test. It is useful from this perspective but does not give a precise indication of a student’s likely facility with academic English. In my experience, capability particularly within academic writing can be very varied among students with the same score. Similarly it is not necessarily a reliable indicator of subsequent academic performance. Int 9 (ac, old, pg, soc sci)

Interviewees here seem to be arguing that IELTS has a role to play at the admissions stage, resonating with Bayliss and Ingram’s (2006, p11) conclusion that: IELTS scores can quite accurately predict students’ language behaviour in the first six months of their study programme…which is encouraging not only for IELTS but for stakeholders who rely on the test scores for placement. However, they are also noting their perceptions that its use as an indicator of subsequent academic performance is limited. Indeed this was expressed concisely by one interviewee: It is insufficient as a single indicator (Int 10 admin, old, pg, soc sci) confirming perceptions noted in Coleman, Starfield and Hagan (2003) and in Kerstjens and Nery’s (2000) research.

Another respondent seemed to be more negative, equating the IELTS Test more with the psychometric style of testing used for entry onto particular programmes, such as the Graduate Management Admissions Test (GMAT) and TOEIC (Test of English for International Communication):

It’s not just a language test it’s more akin, it seems to me, to a kind of aptitude test…almost an IQ test…and overseas there’s a whole industry in training people to pass the test.

Int 1 (ac, old, ug, soc sci)

This might indicate a need for IELTS to raise awareness of the nature and rationale of its Test to differentiate itself from other tests with which it is being unfairly conflated, echoing Rea-Dickins et al’s (2007) claim that there has been too little attention paid to the relationship between IELTS profiles and successful students. This issue will also be considered when looking at interview questions around understandings of the nature of the Test itself. Similarly another respondent raised issues about the dangers of conflating a broad language profile into a single band score, and seemed to be arguing for a more qualitative approach to assessment:

I would have said yes until I learned a little bit more about it and then I find that this sort of generalising everything into one number it’s difficult to know what students do or don’t know, where their weaknesses are, and where their strengths are as well… Int 2 (ac, old, pg, sci)

There is an indication of a desire for more in-depth evaluation but, as will be seen later in the section on development opportunities and awareness-raising, this raises a tension in terms of the pressure of time.

In terms of a correlation between IELTS and subsequent academic performance, one interviewee admitted this was an area that was not researched within their institution, and so may indicate a niche for further research in that specific context:

I’ve no idea, I can’t comment on that [is IELTS a useful indicator of subsequent academic performance?] as we don’t make a habit of correlating…we don’t have any mechanism for tracing that…we do get a reasonable spread of results, I don’t think there’s likely to be much of a correlation there. Int 6 (ac, new, pg, sci)

However, this was an area that some institutions saw value in investigating, as noted by Interviewee 7:

Generally speaking yes…ah that’s a very good question…we’re currently doing an exercise where we are correlating academic performance against English Language test attainment on entry…candidates coming from a certain small number of institutions are not performing as well as they should, and therefore what kind of entry level are they coming in on in terms of their IELTS or TOEFL. Int 7 (ac, old, pg, prof/voc)

Again it would be interesting to compare the outcomes of any such investigations to the wide body of research as outlined by Feast (2002) and Davies (2008), discussed in the literature review.

4.2.2 The process for deciding IELTS levels required for admission

In response to a question around the process for deciding IELTS levels required for acceptance onto programmes, the general picture emerging was one where departments had a degree of autonomy in deciding the appropriate level, as indicated in a number of responses:

The departmental committee - it’s been the same for the last 11 years …6.5 - this has been decided within the department through discussion among the research degrees committee and those responsible for postgraduate taught programmes. Int 4 (admin, old, pg, soc sci)

It was inherited from my predecessor and it more or less matched the university norms. Int 11 (ac, old, ug, soc sci)

However, there was an acknowledgement that such standards-setting processes are subject to university validation procedures for admissions at varying levels:

It’s partly set as equivalences through our university processes but then individual programme teams are able to, if they wish to, adjust the level up or down as we see fit… we do have some degree of autonomy, it’s part of the validation process obviously Int 6 (ac, new, pg, sci)

It’s a case of a mix between internal decisions and external university regulations.

Int 8 (ac, priv, pg, a&h)

The first academic director of the programme was insistent on having high quality students who could discurse (sic) very well in a sophisticated way in English right from the word go…it’s a decision which we make at departmental level but which has to be agreed by the university. Int 7 (ac, old, pg, prof/voc)

These standards-setting processes often seemed to be moderated by intermediary bodies, as indicated by Interviewee 3, who was an academic, currently working in administration in the international office of her institution, with responsibility for advising departments on admissions, pre-sessional and in-sessional English language support:

We work with the schools to decide what they need and we work back from that…it’s also with registry to make sure they’re happy with what’s going on and with our agents and institutions overseas to see what courses we need to put in place given the progress students are making over there and changes in development, developmental levels of students overseas etc. and in response to what students tell us here in feedback that the course is too intense or not intense enough or they wish they’d had further preparation… Int 3 (admin, new, ug/pg, gen)

In some instances, there was a more overt acknowledgement that students are expected to improve their English language skills while on the programmes, as a result of input from the programmes and in part due to an immersion into an English language socio-linguistic context:

They will be with me for two years part time … so a 5.5 might feasibly turn into a 6 or a 6.5 whilst they’re with us. Int 1 (ac, old, ug, soc sci)

4.2.3 Perceptions of appropriacy of required IELTS levels

This understanding was supported more generally in responses to questioning as to whether respondents felt the levels they required for admissions were appropriate, given that the questionnaire had indicated that 57% felt these levels should remain unchanged, while 43% felt they should be higher. In some instances there was evidence of a convergence with the status quo, and to a degree, an unquestioning acceptance of the institutional admissions policy:

It used to be 6 and they’ve changed it to 6.5 and it’s probably as good as it could be.

Int 4 (admin, old, pg, soc sci)

We hold to 6.5 - this has been decided within the department through discussion among the research degrees committee and those responsible for postgraduate taught programmes.

Int 9 (ac, old, pg, soc sci)

One respondent linked this issue with the need for supplementary support, an issue to be considered in more depth with the responses to a more specific question on additional needs:

We tried to implement something with the ELTC for all new postgraduates where they, if they had 6 or lower, they would go along to a couple of courses directed specifically at

them…chemical engineers.

Int 2 (ac, old, pg, sci)

Another category of response which emerged more fully at a later stage of the interview research was where interviewees linked this issue with the potential for, or result of, additional training or awareness-raising for those using IELTS in admissions:

My feeling is that Admissions tutors seeing what a 6.0 looks like would be more inclined to

actually want to up the entry requirement to a 6.5 or a 7…

Int 11 (ac, old, ug, soc sci)

Experience counts a lot - admissions tutors who actually understand what the numbers might mean in relation to a real student and how they might be able to achieve on a degree course… I’m really nervous of the very quantitative kind of approach… it’s not as simple as a 6.0 will succeed and a 5.5 won’t or whatever… Int 1 (ac, old, ug, soc sci)

One respondent linked this issue, again as highlighted by Edwards et al (2007), to the need to compromise between the competing pressures to maintain high academic standards and the economic imperative to recruit. This was further contextualised as a feature of the increasing comparative and performative nature of higher education in a neo-liberal world (Ball 2003) where performance accountability is situated in the contemporary wider public-sector reform movement, widely referred to as New Public Management (NPM) (Hood 1991), characterised by increasing marketisation, accountability, incentivisation and scrutiny:

We always wish we could have higher entry level scores but the reality of the situation is if …if we raise it from IELTS 6 to 6.5 for example then we just won’t be competitive with institutions which are higher up the league table and we won’t get any students coming to us, so we have to compromise a little bit. Int 3 (admin, new, ug/pg, gen)

4.2.4 Tensions between standards-setting and recruitment

This issue was taken up more comprehensively by interviewees in response to questioning around the presence or absence of a tension between setting language standards and recruitment, and, if present, attempts and strategies for the resolution of this tension.

One respondent took this issue of the marketisation of higher education and related it directly to the areas of recruitment and standards-setting:

Most definitely, academic staff would prefer higher standards as this would entail less work at tutorial level in comparison with native students. Both native English speaker students and non-native English speaker students need help and advice with some aspects of academic register but many non-native English speaker students require more specific technical help that academic staff either feel ill-equipped to deliver or feel this is not, or should not be, part of their tutorial role. However, there’s a growing pressure on course directors to meet recruitment targets and this institutional pressure means that sometimes we take on students who we feel might struggle due to their language proficiency as opposed to intellectual or conceptual capabilities. This creates both a practical and moral tension but I suppose the targets culture is a ‘sign of the times’ of a growing marketisation of HE and I’m not sure this is going to go away. Perhaps raising awareness at management levels that this is not always an effective approach due to the additional staff resources involved might be one way of addressing the issue but I fear the ‘bottom line’ will always be the prevailing factor.

Int 9 (ac, old, pg, soc sci)

Other respondents recognised the changing economic and management culture in HE but still highlighted their view that the maintenance of high standards and a responsibility to their students was paramount:

Of course there is, at least in principle, there’s always, particularly these days, always going to be a drive to get more in but the trouble is we know if they don’t have the standard then they’re going to fail. Int 6 (ac, new, pg, sci)

Even those who were in the fortunate position of easily meeting recruitment targets noted that the tension existed even if it did not affect them directly:

We’re talking about tens of students rather than as an undergraduate department you might be talking about seventy, eighty students so it’s less of an issue. Int 2 (ac, old, pg, sci)

Yes there can be but we’re in the fortunate position that we attract a high number of applications every year, so for example last year we had 668 applications and ended up with 88 students on the programme, so it’s not creating a dip in application numbers… We ask for an IELTS level of 8… we ask it because we can, it’s another way of filtering students for whom this course would not be suitable, who would not be able to cope because of their language skills and because we’re getting all these applications, we’re meeting targets in terms of our student numbers, it’s something we can do, but if we were not recruiting to target then that might be something we had to come back to and look at again.

Int 7 (ac, old, pg, prof/voc)

Interestingly, only one interviewee chose to ignore any tension and the increasingly marketised context and this was an administrative respondent, possibly indicating this was more of a tension for academic staff. She noted:

No, we could never take anyone below the minimum level purely and simply because we have at the moment students who have come in at that level and they are struggling so we would never go any lower. Int 4 (admin, old, pg, soc sci)

One interviewee noted that there was a degree of pressure to recruit students who did not have sufficient language skills and that this had an impact in demanding increased tutorial time, and as such a potential knock-on effect in terms of time for other activities:

I can only speak for my course but there is a pressure to recruit…the IELTS score is the bottom line…if they meet the minimum requirement I can’t reject people who seem to lack the communicative competence to be successful on the course…whilst we do provide in-house specialist support this is limited as it’s expensive and comes out of the course budget…the impact is tutors need to spend a lot of time working with international students on their technical English skills…it’s not very cost effective, they’re not experts in this field and so in some ways it’s a false economy as nobody bothers to think about the opportunity costs, the other things they could be doing with their time…I have some professors on my team and they are becoming very expensive EAP tutors! Int 9 (ac, old, pg, soc sci)

Another interviewee noted that there was sometimes a mismatch between the recommendations for entry between the department and central administration:

This is a departmental issue but indications are suggested centrally though these tend to be lower than departmental levels, an indication of the tension you described previously with the increased pressure on maximising fee income. Int 10 (admin, old, pg, soc sci)

There was some evidence to suggest though that, while the central administration offers advice and guidelines on admissions levels, in cases of dispute, the authority in decision-making remains with the department:

We had a candidate from Thailand who in fact we admitted because she had taken the test about three times and she’d achieved an 8 in the various components but just not all at the same time, and she ended up with a 7.5, and our International Office queried this with me and said how did you admit her on a 7.5 so I said if you look at the detailed scores she did achieve an 8 in the relevant components, just not all at the same time. Int 7 (ac, old, pg, prof/voc)

Despite the acknowledgement of potential tensions, it did not seem to be a consistent concern, although one interviewee did note that the tension in relation to profitability was having an impact in terms of appropriate teaching accommodation:

There is an institutional pressure to raise numbers but also we have an institutional pressure of space and our classrooms have been taken away from us so it’s a really fine line to tread between getting the students in and being able to teach them in adequate surroundings.

Int 3 (admin, new, ug/pg, gen)

4.2.5 Understandings of the content and process of IELTS testing

The vast majority of interviewees admitted that they did not really have a clear understanding of the content and process of the IELTS Test, as argued by Smith and Haslett (2007). This perhaps indicates a need for awareness-raising in this area and extends Rea-Dickins et al’s (2007) point that such a lack of awareness is primarily a feature of administrative staff understandings. This was evidenced in a range of comments:

If you’re saying would I be able to discriminate make judgements according to a…a spiky profile from speaking through writing through listening then then no I wouldn’t…

Int 1 (ac, old, ug, soc sci)

We don’t ever see the in-depth (certificate), at least as academics … we just see ‘Oh they’ve got 6.5’ and there’s apparently a certificate that somebody in admissions has had a look at.

Int 6 (ac, new, pg, sci)

I don’t think we do have a very detailed understanding of what the candidates are required to do and so in that sense our understanding of the test and what the process is quite superficial.

Int 7 (ac, old, pg, prof/voc)

All respondents admitted to making their selection judgments based on the aggregate banding, occasionally with specific requirements in individual papers. Even one of the three interviewees with a background in English for Academic Purposes admitted:

I have a background in ELT and EAP so am very familiar with the content of the IELTS examinations, but I still make a decision based on the global score. Int 9 (ac, old, pg, soc sci)

4.2.6 Potential for development around understandings of IELTS

On further probing as to whether interviewees felt that greater awareness of the content and process of the IELTS Test would be useful, several respondents were enthusiastic about such a development opportunity:

It would be extremely useful for admissions tutors to attend those kinds of sessions, yeah, and clearly with the increases in numbers of international students, yeah, it’s very important…

Int 1 (ac, old, ug, soc sci)

I think so as many of my colleagues are surprised at what work at the various levels of IELTS entails - the level is often much lower than they would ever have imagined so I reckon this kind of training or even just an awareness-raising opportunity would be really really useful.

Int 9 (ac, old, pg, soc sci)

I think it would be enormously helpful.

Int 6 (ac, new, pg, sci)

One respondent described the way in which exactly such a development opportunity had had a practical impact on her understanding and practice:

I didn’t even realise there was a, sort of, academic version and a non-academic version, so it’s only really through that (an awareness session run by the ELTC) that I’ve got an idea of what you can expect from students with certain levels. Int 2 (ac, old, pg, sci)

She went on to describe how previously she would have uncritically accepted the institutional admissions level:

Unless you’re so familiar with the process that you know that if they’ve got, say, a 6 in the reading then you know exactly what kind of qualification they’ve got then you’re just saying well the university say that 6 is OK so I’ll just accept 6. Int 2 (ac, old, pg, sci)

Two interviewees indicated scepticism as to the potential take-up of such development opportunities due to the increasing pressure of work, with one respondent indicating that the global reputation and visibility of IELTS might actually militate against the take-up of awareness-raising possibilities:

It may be…I know staff are very often so busy they prefer us to decide and to look at the IELTS exams and see that as our area of expertise. Int 3 (admin, new, ug/pg, gen)

I think we’re all the time-pressured that’s the problem. I think because we accept that the test is internationally recognised and is highly developed I think we trust the test and what it’s telling us even if we don’t know very much what lies behind it, I think we have that kind of measure which is accepted, I think it’s where tests are less globally recognised that you might query their validity. Int 10 (admin, old, pg, soc sci)

The former respondent in her role as an international officer also pointed out that there might be administrative practical implications with departments choosing very differing grades but, arguably, this demonstrates a concern more for ease of administration than a focus on the needs of the student and the requirements of the programme:

I could see that there could be difficulties in that departments might then decide that they would like to have different IELTS levels on different components of the IELTS course which would mean that we would have so many different variations, and we have a complex enough picture anyway, that that could be difficult for us with our agents and our overseas institutional partners, so we try because of the level that we are on the league table, because of our niche in the market, we try to keep it as general as possible, so that we can encourage the students to come…it’s a difficult one, it’s good to offer information is what I’d say.

Int 3 (admin, new, ug/pg, gen)

Opinion was divided over who would be best placed to provide such training, should it be deemed to be necessary and viable - either British Council/ IDP: IELTS Australia / the University of Cambridge ESOL Examinations or the HE institutions. Even those who suggested that it should be an internal responsibility appeared aware that the test providers/designers were ideally place to offer unique insights into these issues, and that this might indeed have some marketing value:

I guess the original designers would be… would imagine they could do a lot towards providing sort of threshold type levels of information about, if a student gets this grade then that means they can do this… and in conjunction if the ELTC showed you examples of stuff from different candidates than that would be reinforcing it. Int 2 (ac, old, pg, sci)

I think this should be the responsibility of the institution (in fact in my institution the English Language Teaching Centre run such sessions), but I think if IELTS were to offer such developmental programmes they would be warmly welcomed and prove very useful both for academic staff in terms of their understanding and potentially as a marketing strategy for IELTS. Int 9 (ac, old, pg, soc sci)

Well our quality enhancement department is probably quite well placed, positioned, to do that… many of us (academic colleagues) don’t have a detailed understanding of what IELTS is, we just know it’s just one of our criteria. Int 6 (ac, new, pg, sci)

These responses could be taken as evidence of a potential demand for awareness-raising activities, as noted by Smith and Haslett (2007). It might suggest the opportunity for British Council, IDP: IELTS Australia and the University of Cambridge ESOL Examinations to firstly canvass the degree of demand for such development and, if this proved to be considerable, to investigate ways in which these could be provided either intra- or extra-institutionally.

4.2.7 The need for post-admission additional language support

There seemed to be a consensus among the interviewees that for a number of students, though not all, meeting the minimum level still required additional language support, reflecting the evidence from the questionnaire where 74% of respondents felt that students did require additional post-entry support, while 26% did not, a position in line with that advanced by Read and Hayes (2003).

Some do, not all of them but some do …I know quite a few get their essays checked at the ELT Centre. Int 4 (admin, old, pg, soc sci)

Almost always - particularly in terms of writing skills. Int 10 (admin, old, pg, soc sci)

The exception came, unsurprisingly, with courses that required a very high entry level such as the comments from Interviewee 7, whose course had an entry requirement, in almost all circumstances, of IELTS 8:

No not at all, their English is almost always more than adequate. Int 7 (ac, old, pg, prof/voc)

Although she subsequently noted an exception to this:

We had a Chinese student last year, who came in on an 8, but she found it very hard to understand the heavily accented English of other overseas students, so for example the Indian students with Indian English accents, she could tune into what native speakers were saying but she couldn’t tune into what they were saying. Int 7 (ac, old, pg, prof/voc)

In querying whether such needs might be best met with generic language centre support or more course-specific support, possibly within the department, there seemed to be widespread support for the view that ideally subject-specific additional language courses were the preferred option, while recognising the value of independent dedicated language support units staffed by EAP specialists.

In an ideal scenario, the best place for it would be in the classroom, in the context of the subject-specific modules that students are studying, so in other words in departments rather than in some specialist centre elsewhere…having said that, there’s always going to be, in relation to second language students, a need for some kind of central specialised language classes as well. Int 12 (ac, old, pg, sci)

I think a bit of both really … but I do think that having more directed focused kind of support within the department is actually quite a good thing as well because then we can tailor it from year to year depending on who’s come in and what their needs are. Int 2 (ac, old, pg, sci)

It was also noted that pragmatically this could be financially difficult:

Simply as a result of resources it’s very much a case of trying to encourage students to seek help, in various ways, from the ELT centre … we don’t have the resources, we don’t attempt to provide specific additional English language, in terms of speaking and writing, support…

Int 1 (ac, old, ug, soc sci)

The other key issue raised here was the motivation of students to attend generic classes:

I feel that whilst EAP expertise is fundamental to this that more subject-specific support would be far more motivational and effective as the drop-out in generic classes at our English Language Teaching Centre would indicate. We tend to find that students sign up for the generic classes at the start of the year but very quickly as other pressures build up these are the first things they drop. Int 9 (ac, old, pg, soc sci)

This was also linked to resource issues:

What we find is the majority of students don’t really want those group classes, what they want is one-to-one support…we are revamping for next year and one of the things we are doing is having more subject-specific classes for schools where we have the larger numbers of students… we can do that based around core compulsory classes the students have to do but where we’ve got just one or two students a year we can’t to the same…it’s just not cost effective…with students like that we see them on a one-to-one basis…it’s a balance really.

Int 3 (admin, new, ug/pg, gen)

Ah, the issue is not so much whether the support itself is effective, it’s whether the student goes and gets it, and this seems to be the really, really big question, will they go and take advantage of it?…the take-up (of support sessions and clinics) tends to be very low…and so whether you provide it centrally or in departments is, I suspect, probably somewhat moot…what’s also of course important is if it’s provided at a central level, the centre pays the cost of the support whereas if it’s done by the faculty or the individual schools then we get hammered directly with the costs, given the pressures we’re all under financially and resource-wise. Int 6 (ac, new, pg, sci)

The challenge for institutions seems to be that there is a growing view amongst academic staff that students need to be supported by more focused subject-specific EAP assistance and development, while also harnessing the undoubted expertise provided by language support centres. Indeed perhaps the designation of these as ‘support’ units or student services, as opposed to valid academic departments, along with the attendant issues of pay and working conditions, is part of a discourse of marginalisation and deprofessionalisation. This could be addressed partly through the institutional investment of significant resources to enable language centres/units to work together more collaboratively with academic departments to provide a more unified and specified level of support and development for international students.

4.2.8 IELTS: fit for purpose?

On balance, the majority of interviewees felt that IELTS was a testing system that was ‘fit for purpose’ in the sense that it met their immediate needs. One interviewee pointed out that the issue of standards-setting is not fully understood by academic and administrative staff (Int 7 ac, old, pg, prof/voc), an issue hopefully to be addressed in part by the recommendations of this research and in part by the DVD IELTS Scores Explained. Another pointed out that issues of security were considerably important, and that IELTS met their needs in the sense that if the student actually has achieved themselves 6.5 or 7, yes, if they haven’t but they still pass it off as theirs, then obviously we have a problem (Int 6 ac, new, pg, sci). One other dissenting voice noted that IELTS did not meet their needs as it was used as a part of the pedagogic programme, but as the Test was designed for assessment and evaluatory as opposed to pedagogic and heuristic purposes, this lack of fitness for this particular purpose cannot be blamed on the designers/producer. This interviewee noted:

Staff were using IELTS tasks on the foundation programme…it was rather contrived, they were doing this in order to get students to pass IELTS when they didn’t feel they were really preparing them for undergraduate study. So we developed our own IELTS/EAP study skills classes instead of that to really prepare them for undergraduate study and then at the end of the course we said to them you can take IELTS if you’d like to, and we also had our internal tests that we developed, so we wanted to give students the flexibility that they could do IELTS if they wanted to, particularly if they wanted to go to another university, they were going to need that, but we wanted to make sure we were preparing them adequately for undergraduate study. Int 7 (ac, old, pg, prof/voc)

4.2.9 Potential for improvement of IELTS testing system

Interviewees were asked in what ways the IELTS testing system could be improved, and responses fell into five areas:

1. specialisation
2. internationalisation
3. comprehensibility of scoring
4. security
5. administration.

One respondent commented on the generic nature of the Test and questioned whether it would be feasible to tailor tests to more specific subject areas, while acknowledging the global remit of IELTS:

It would be nice if the IELTS tests could be tailored more to the individual subject areas that students are seeking to access…the kinds of questions are, they are quite formulaic, aren’t they, and there are particular ways of answering them that can be drilled in, this is the way to set out a decent answer to this one…but IELTS is of course a global industry, it needs to standardise, and an IELTS score in Hong Kong has to be the same as one in Canada…

Int 1 (ac, old, ug, soc sci)

Another respondent echoed some of these sentiments while offering a potential compromise solution to the tension described above:

I have often wondered about the way that the tests could be made more relevant to specific-subject areas whilst keeping the global brand recognition that makes IELTS so well known, even amongst those staff who don’t know much about EAP or the specifics of the tests. I’m not sure what’s the best way to do that…perhaps it could mirror the faculty structure, but but keeping a system where all the specific tests carried the same weight in terms of accreditation so that, say, an IELTS in Law would be worth the same as an IELTS in Engineering.

Int 9 (ac, old, pg, soc sci)

Clearly such a task would entail enormous restructuring and investment for the British Council, IDP: IELTS Australia and the University of Cambridge ESOL Examinations but perhaps there could be some merit in this research-oriented organisation investing in enquiry into the feasibility of such diversification.

One interviewee discussed the way that higher education was becoming more internationalised and how UK HE classrooms now regularly are sites of educational engagement where international students comprise a significant proportion of the student population. He noted that perhaps there was more scope for IELTS to acknowledge this in a more consistent use of international accents in the Listening test and that there could also be pedagogical washback from such a development:

I think there may be an argument for more international voices in the test, and I know there are some already, as students often take the qualification for entry into an increasingly internationalised world of HE. Students will often be in a position where they work with other international students so the test should not only be about how well they can understand native speakers. Int 9 (ac, old, pg, soc sci)

Such a move might begin to address the concerns expressed above by Interviewee 7 about problems of students’ lack of familiarity with other international English accents - both the inclusion of international L1 and L2 English accents - to help to familiarise students with the accents of potential classroom peers from other accent backgrounds, and with potential interlocutors. The issues of previous exposure, familiarity and future intentions become important here and such needs could be valuably addressed during preparation for testing, to ensure an appropriately context-specific pedagogic fit for all students. This is an issue that has been partly addressed by other testing systems in response to critiques. For example, in TOEIC, the Listening section utilises not only North American English speakers but now specifies a ratio of 25% each for American, Canadian, British and Australian/New Zealand pronunciation models. Perhaps more significantly this might represent one step in the direction of acknowledging and encouraging positive ways in which internationalisation could be fostered and recognised not simply as part of a marketised homogenisation of HE but as part of ‘a reclamation of the internationalisation agenda on the part of practitioners who are interested in creating culturally inclusive, fair and genuinely educational forms of multicultural HE teaching and assessment’ (De Vita and Case 2003, p396).

Other respondents had more practical concerns. One interviewee felt that she would benefit from a greater understanding of the relationship between differing band levels and language/communicative competence:

It would be useful I guess if when you got the score from the IELTS that they gave you an overall summary of what a student’s language capability would be…I would like more information…even if they just sent out a score sheet that wasn’t necessarily specific to each student but just gave an overview of what all the different categories mean and what the scores mean, that would be quite useful actually… I pretty much go on the global score; if the certificate is there then I’ll run my eyes down it, don’t go and look on the website.

Int 2 (ac, old, pg, sci)

Another participant, Interviewee 11 (ac, old, ug, soc sci), had similar concerns and suggested that it might be possible to include a brief explanation of the nine bands and their descriptive statements, as indicated on the IELTS website (http://www.ielts.org/candidates/afterthetestresults/article281.aspx), or perhaps a more specific set of descriptors geared towards an academic audience, on the back of the certificate, or as a separate handout, to facilitate the understanding of admissions officers. While this information is available via the IELTS website, there was no clear evidence that those responsible for admissions were aware of this, or used it. This points to a need to consider how to present this information in more user-friendly ways.

Other concerns centred around the issues of security, timing and the cost of examinations:

The only thing, and this is a common problem of distance type programmes and open-access examinations and so on, is impersonation, if you can fix that then we’d be all right…security issues… if we can solve that certainty, or uncertainty, around who took it then we’ve got a useable, fit-for-purpose system. Int 6 (ac, new, pg, sci)

Int 2 (ac, old, pg, sci)

I suppose one of the things that our candidates do tell us, one of the things they do have difficulty with is with, the availability of the test and I don’t know whether that is something that could be addressed by the organisation itself…times of year, timeliness, location is sometimes a problem, I think less so for IELTS than for some of the other tests…I think IELTS is more freely available, and cost is often prohibitively high candidates tell us…they do find it expensive so it’s in their interests to get the best score they can straightaway.

Int 7 (ac, old, pg, prof/voc)

Security is clearly an ongoing concern for the IELTS providers (Charge and Taylor 1997) and the issues of the cost and timing of exams have to be considered within the context of the location of candidates sitting the test, but it is worth noting that these issues are being raised by admissions officers.

The final aspect of the interview dealt with participants’ perceptions of the IELTS Scores Explained DVD and the degree to which it was felt to be a useful resource. As noted earlier, this element of the research was limited by the fact that only 17 questionnaire respondents requested a copy of the DVD, and of these respondents only six accepted the request to be interviewed. However, from this very limited sample, there was unanimous concurrence that the DVD was an extremely useful and informative resource. One participant noted it was a key element in her overall understanding of the IELTS system:

I’ve been on training at the ELT Centre then they’ve had their training for staff and I’ve picked up quite a lot on the job, and quite a bit has come off the DVD - very useful!

Int 4 (admin, old, pg, soc sci)

Other interviewees commented on its explanatory value and ways in which it could be used strategically by institutions, as well as making minor suggestions for improvements:

Very much so - it provided a clear indication of the system and its value as an indicator of acceptability onto courses. Perhaps more examples of student work at the various levels would be useful to serve the awareness-raising needs of academic staff I referred to earlier.

Int 9 (ac, old, pg, soc sci)

The DVD will be very useful to start to look, can we be cleverer, can we actually avoid wasting students’ money in a sense. Int 6 (ac, new, pg, sci)

However, we would argue that it might be valuable to conduct a more systematic and wide-scale evaluation of this resource, including suggestions for improvements, both in terms of potential for revision and modification of the DVD, and as a means of raising awareness of the existence and availability of the resource.

5. CONCLUSIONS

This project explored stakeholders’ perceptions of the role of the IELTS Test in the admissions processes of UK higher education institutions. It drew on two pieces of empirical study: a large scale questionnaire survey of those responsible for admissions’ decisions in a range of HE institutions in the pre- and post-1992 sectors; and a smaller scale interview-based qualitative study of a sub-set of these participants.

The aims of the research were to elucidate:

• what the entry requirements are for different programmes

• how consistent they are across universities

• how the standards are set

• whether or not there is a tension between standards and the need to recruit.

In summary, the findings indicate:

• that entry requirements are starkly different both across differing institutions and institutional sectors
• that there is no consistent relationship between entry requirements and programme across universities
• that standards are set in different ways, dependant on sector, marketability of programmes, institutional and administrative conventions and requirements, and individual admissions tutors’ perceptions of the degree to which an IELTS score indicates a candidate’s language capability, and likelihood of success on a given programme
• that in an increasingly competitive environment there is often considerable tension between standards and the need to recruit.

The research also established that there is a considerable lack of knowledge among admissions tutors about the IELTS Test and the researchers recommend (as did Rea-Dickins et al 2007) that IELTS needs to find ways to inform the stakeholders of the nature and value of the Test and how it differs from those of competitors, such as the TOEFL. The IELTS Scores Explained DVD was reported to be highly valued by those respondents who knew of it, but of 100 respondents only 17 asked to have a copy sent to them and only six of those agreed to be interviewed. This suggests that the DVD is an excellent resource but that it is only one possible approach of several to the problem of communicating with stakeholders.

To clarify and communicate these findings in more detail, we have addressed each of the research questions in light of the empirical data gathered and analysed.

5.1 Key findings in relation to the research questions

What IELTS level is the required minimum for student acceptance onto a range of programmes in various UK institutions? How consistent are these requirements in differing sectors of HE provision? What is the process for standards-setting in various HE institutions?

The research revealed that there was no clear pattern as to the relationship between particular faculties or departments and their entry requirements. Decisions were largely made on a case-by-case basis, the ultimate responsibility for standards-setting usually being in the hands of the admitting department, though often with general guidelines and advice from either a central registry or from a specific

department, part of whose role was to advise individual departments on issues surrounding English language proficiency. There was occasional evidence of stakeholders’ reflexive awareness of flaws in their procedures, although there was also evidence that departments are generally open to reviews in these processes.

General advice on the English language criteria for HE programmes is available from central sources, such as the band score search page of the IELTS website (http://bandscore.ielts.org/search.aspx) but participants in the research noted that departmental admission levels were often under review. So while such information is useful as an indicative guide to student language proficiency required, students would be best advised to check university department website or contact individual departments for up-to-date information.

While postgraduate requirements were generally higher than undergraduate requirements, which in turn were higher than foundation requirements, this pattern was not always the case. Contextual factors including subject matter, nature of pedagogic interaction, popularity of courses, and institutional status, all impacted on the levels set by individual course providers.

To what degree do stakeholders consider the IELTS Test a useful indicator of academic English proficiency appropriate for higher education study in the UK?

The vast majority of participant stakeholders felt that IELTS was a very useful indicator of academic English proficiency appropriate for higher education study in the UK. A minority of respondents commented that they felt the Test did not accurately predict test-takers’ competence in the academic activities expected for university study.

Interview data confirm these findings but highlighted some stakeholder reservations around the use of IELTS as an indicator of post-admission academic performance. Participants were generally happy with the use and validity of IELTS as part of admissions procedures but some concerns emerged around the qualitative value of judgments made on a global score, in that these did not always give a thick description of the context behind the band score. There is an argument that a reliance on the global score, and a disregard for the scores in individual skills’ tests, represents a waste of valuable information that could inform the admissions process. This might indicate the need for an increase of awareness of the nature and rationale of the Test and a desire for a more in-depth evaluatory picture, but also raised issues around the time available for non-EAP specialists involved in admissions to invest in such awareness-raising activities. However, there was an acceptance that the admission of students who are incorrectly assessed as suitable for a particular programme often led to major problems at later stages, and so time spent on a more comprehensive evaluation of appropriacy for admission was in the interests of both the student and the institution.

To what degree is there a tension between setting standards and the need to recruit?

A majority of those surveyed felt there was a significant tension between the setting of standards for admission and the economic institutional imperative to recruit, which we would argue are common and growing aspects of an increasingly marketised higher education context. Those who did not experience such a tension tended to be those who, due to the popularity and marketability of their programmes, did not face pressure to increase recruitment (recognising the tension but commenting it did not affect them) or those who indicated that they felt that the maintenance of high academic standards was paramount and had been successful in advocating this position within their institution. There were indications that sometimes students were recruited who might struggle due to their language proficiency in order to meet recruitment targets and that this could have an impact in terms of support needed for students, and the opportunity costs of this additional support may not be fully understood.

Almost three-quarters of respondents felt that students did require additional post-entry support, while almost two-thirds of these respondents indicated that the IELTS Test did not play a valuable diagnostic role indicating post-entry support needs, perhaps indicating that, while for some there may be diagnostic assessment potential, to assume this for all test-takers is beyond the remit of the IELTS Test as currently conceived.

What degree of additional post-entry EAP support do stakeholders find is necessary that is not indicated by IELTS levels? How do stakeholders respond to any additional identified EAP needs?

Almost three-quarters of those surveyed felt that students required additional post-entry English language support and, of those, almost two-thirds suggested that the IELTS Test did not indicate the need for such support. There seems to be a growing view among academic staff that students need to be supported by more specific EAP development which could be fostered by closer collaboration between academic departments and language support units.

What other English language qualifications do institutions accept as equivalent to IELTS?

All stakeholders surveyed revealed that their institutions accepted IELTS for admissions purposes underlining its position as a highly regarded and widely accepted means of assessing candidates’ preparedness to study in UK higher education. A range of other national and international English qualifications, or internal tests/interviews, were identified with institutions making individual decisions as to what they were prepared to recognise. This pre-eminence for IELTS seems to vindicate the value of a consistent research agenda concerning the IELTS Test, the relevance of investigating the perceptions of stakeholders and the value to the research sponsors (British Council, IDP: IELTS Australia and the University of Cambridge ESOL Examinations) of continuing efforts to raise awareness of, and improve the validity and reliability of, their testing system.

How aware are stakeholders of the process and content of the IELTS Test and what development needs does this reveal?

The vast majority of interviewees admitted that they did not really have a clear understanding of the content and process of the IELTS Test, indicating a need for awareness-raising in this area. This resonated with the findings of Smith and Haslett (2007). Stakeholders were often enthusiastic about such development, and those who had undertaken such development previously noted the impact this had had on their practice. However, the potential take-up for such development opportunities could be affected by the increasing workload and time pressures faced by staff in the contemporary higher education context. Opinion was divided as to whether such development would be better provided internally by the institution itself, or externally by the producers/developers of IELTS.

What are the implications of these understandings for the British Council, IDP: IELTS Australia and the University of Cambridge ESOL Examinations in terms of adjustments to, provision of, guidance for and the marketing of IELTS examinations?

The research has indicated a number of implications for the designers/producers of the IELTS Test, as indicated in the responses to the research questions above, but also in terms of a number of potential recommendations and directions for further future complementary research, which are discussed below.

6. RECOMMENDATIONS

On the basis of our analysis of the empirical data gathered through questionnaire and interview, we would recommend the following.

• The sponsors should consider publicity and development strategies for raising awareness of the different Test modules and the existence of the Academic module designed for those who wish to enter universities and other institutions of higher education. The term ‘module’ may lead to some confusion in that some stakeholders misunderstand this as entailing part of a test, as opposed to a test designed for a different audience.

• A lack of clarity on the part of a number of participants suggests that the sponsors need to improve publicity and development strategies to raise awareness of the rationale of the Test, in contrast to more psychometric types of test. Such strategies would serve to refute critiques that IELTS is not a true test of communicative competence, as suffered, for example, by the TOEIC exam.

• The British Council, IDP: IELTS Australia and the University of Cambridge ESOL Examinations should firstly canvass the degree of demand for development of academic staff’s understanding of the content and process of the IELTS Test. If this proved to be considerable, the test partners should investigate ways in which such developments could be provided either intra- or extra-institutionally, through training courses, seminars, online resources, DVD based resources etc, particularly given the significant internationalisation of UK HE in recent years. For the last three years there have been annual IELTS update seminars, organised by the British Council, held in London, Manchester and Dublin, designed to support educational institutions in the UK and Ireland setting minimum IELTS band score requirements and providing a general update on IELTS. Despite these, equivalents in Australia from IDP: IELTS Australia and information on the IELTS website, there is still insufficient awareness of these activities among stakeholders which suggests stakeholders need to engage in continuing and improving awareness-raising activities.

• There is a strong case for HE institutional investment of significant resources to enable language centres/units to work together more collaboratively with academic departments to provide a more unified and specified level of support and development for international students. While this issue might be seen as not being directly related to IELTS and the sponsors, we would argue that this is an example of feedback from research, the dissemination of which indicates the sponsors’ commitment to working collaboratively with HE institutions and demonstrating the symbiotic nature of the relationship between the sponsors and HE institutions. The way in which IELTS-sponsored research can inform institutional practices, in the same way that HE institutions sharing insights on the testing system, is indicative of how both parties can work in a reciprocal, collaborative manner to the benefit of students, institutions and the sponsors.

• The sponsors should re-visit the feasibility of tailoring tests to more specific subject areas, as sub-categories of the IELTS Academic module. As indicated in the literature review, it should also be noted that in the original ELTS and IELTS exams, there were subject-oriented papers and these were abandoned for good reasons - administrative difficulties in matching candidates to a subject-specific subtest, and research indicating that a single test did not discriminate for or against candidates of any one discipline area (Clapham 1996). The rejection of this was based on a lack of evidence that specific testing was advantageous, and the difficulty and complex nature of designing specific tests.

In relation to the reading element, Clapham noted ‘even if the texts are specific or highly specific it is not clear from my study how many students would profit or suffer from taking reading modules in different subject areas. It therefore seems advisable not to give academic students subject-specific reading modules, but to give them an [English for General Academic purposes] reading test instead’. (Clapham 1996, pp 200-201). The previous incarnation of subject-specific testing was also a tripartite one of dividing students into three groups - Business and Social Science, Physical Science and Technology, and Life and Medical Sciences. It is questionable as to the degree to which this taxonomy actually demarcated separate academic disciplines. Clapham’s critique was one which did not suggest that subject specific testing was invalid per se but that the assumed advantages were not supported by empirical evidence. The HE scenario is one which has changed significantly in both the UK and international contexts since 1996, and while Clapham’s critiques and conclusions might still prove valid, the current context of increased student voice and for courses more specifically designed for individual needs, the increased marketisation of HE and the increasing demand for accountability (Ball 2003, Hood 1991), perhaps the time is opportune, in the interests of ongoing enquiry and contextual consistency, to re-visit the merits and demerits of specificity and generality.

Clearly, with any return to a more specific notion of testing, there would be a trade-off between the advantages of diversity and the dangers of creating a plethora of specialised tests that could prove unmanageable. Also, as Davidson (1998) noted in reviewing Clapham’s (1996) work, reliance on general test tasks and topics is considerably cheaper and so the issue of economic viability would need to be considered. However, one potential approach could be to investigate the feasibility of creating a range of subtests in line with the general faculty structure of most HE institutions (both UK and internationally), which would incorporate the academic and vocational subject areas that attract most overseas students, for example subtests for Arts and Humanities, Social Sciences, Pure and Applied Sciences, Information and Communication Studies, Law, Medicine and Engineering. Such specialisation would have the advantage of indicating a candidate’s competence in the specific register of study and might have a spin-off marketing advantage for IELTS with departments opting for such specialised testing over other more general language proficiency testing. A feasibility study which incorporated both the impact and validity (face, internal, and consequential) of such specific testing together with more qualitative inquiry into the perceptions of candidates and institutional stakeholders would prove informative into the viability and desirability of such a major revision.

• IELTS already uses a variety of regional and international accents in the Speaking and Listening tests, reflecting its international nature, but perhaps there is scope to investigate the feasibility of including even more international English voices/accents (both L1 and L2) in the Listening test as a recognition of the increasing demand on international students to have to work collaboratively with other international students in the UK HE context, and to facilitate a degree of pedagogic washback from these tests in fostering the familiarity of international students with a range of international English accents and speech patterns.

• The analysis of the data indicates there is a case for the consideration of ways in which a greater understanding of the relationship between differing band levels and language and communicative competence can be provided in more user-friendly ways for admissions officers, eg provision of score sheets attached to certificates or band descriptors printed on the back of certificates.

7. FURTHER COMPLEMENTARY RESEARCH

As with any research project, while this inquiry answered a number of questions, it also raised others that might form the rationale behind further research projects. Specific areas for future enquiry raised include:

• a more systematic evaluation of the DVD, seeking to confirm this project’s initial findings that it appears to be viewed as a highly valuable resource and evaluating how it could be improved, perhaps with the inclusion of examples of student performance in a range of skills areas at different banding levels to enhance stakeholders’ understandings of what it means to be operating at a particular level
• more qualitative investigation into the reasoning behind individual faculties’ selection of particular entry requirements
• research into any correlations between IELTS and subsequent academic performance (taking forward the important research conducted on IELTS and predictive validity, and also on stakeholder perceptions in other countries, as discussed in the literature review) - if present, such correlations would strengthen the argument for the use of IELTS as it would offer a more substantial indicator when used as an admissions criterion
• a complementary research project investigating stakeholders’ perceptions of IELTS in tertiary education in other English-speaking countries which use IELTS as part of their admissions procedure, for comparative purposes.

It is to be hoped that this project has shed some light on stakeholders’ perceptions of IELTS as an entry requirement for higher education in the UK, and, in raising new questions and potential areas for further enquiry, it confirms the value of the sponsors’ continuing commitment to, and investment in, research informed development of the IELTS Test.

References

• Ball, SJ, 2003, ‘The teacher’s soul and the terrors of performativity’ in Journal of Education Policy, 18(2), pp 215-228
• Barkhuizen, G and Cooper, R, 2004, ‘Students’ perceptions of IELTS preparation: How helpful is it for tertiary study in English?’ in New Zealand Studies in Applied Linguistics, 10(1), pp 97-106
• Bayliss, A and Ingram, DE, 2006, ‘IELTS as a predictor of academic language performance’, Australian International Education Conference 2006, retrieved on 18 July 2007 from <www.idp.com/aiec>
• Bellingham, L, 1993, ‘The relationship of language proficiency to academic success for international students’ in New Zealand Journal of Educational Studies, 30 (2), pp 229-232
• Bool, H, Dunmore, D, Tonkyn, A, Schmitt, D and Ward-Goodbody, M, 2003, The BALEAP guidelines on English language proficiency levels for international applicants to UK universities, British Association of Lecturers in English for Academic Purposes, London
• Brown, A and Taylor, L, 2006, ‘A worldwide survey of examiners’ views and experience of the revised IELTS Speaking test’ in Research Notes (CambridgeESOL), Issue 26, November 2006
• Burger, JM and Krueger, M, 2003, ‘A balanced approach to high-stakes achievement testing: An analysis of the literature with policy implications’ in International Electronic Journal for Leadership in Learning, Volume 7, Number 4, retrieved 2 May 2007 from <www.ucalgary.ca/~iejll/volume7/burger.html>
• Camilli, G, 2003, ‘Comment on Cizek’s “More unintended consequences of high-stakes testing”’ in Educational Measurement: Issues and Practice 22 (1), pp 36-39
• Charge, N and Taylor, L, 1997, ‘Recent developments in IELTS’ in English Language Teaching Journal, 51/4, pp 374-380
• Cheng, L, Watanabe, Y and Curtis, A (eds), 2004, Washback in Language Testing: Research Contexts and Methods, Mahwah, NJ: Lawrence Erlbaum
• Cizek, GJ (ed), 2001a, Setting performance standards: Concepts, methods, and perspectives, Mahwah, NJ: Lawrence Erlbaum
• Cizek, GJ, 2001a, ‘Conjectures on the rise and call of standard setting: An introduction to context and practice’ in Setting Performance Standards: Concepts, Methods, and Perspectives, ed GJ Cizek, Mahwah, NJ: Lawrence Erlbaum Associates
• Cizek, GJ, 2001b, ‘More unintended consequences of high-stakes testing’ in Educational Measurement: Issues and Practice, 20(4), pp 19-27
• Clapham, C, 1993, ‘Is ESP testing justified?’ in A New Decade of Language Testing Research, eds D Douglas and C Chappelle, Alexandria, VA: TESOL
• Clapham, C, 1995, ‘What makes an ESP reading test appropriate for its candidates?’ in Validation in Language Testing, eds A Cumming and R Berwick, Clevedon: Multilingual Matters
• Clapham, C, 1996, The development of IELTS: A study of the effect of background knowledge on reading comprehension, (Volume 4, UCLES/CUP Studies in Language Testing series), Cambridge: Cambridge University Press
• Coleman, D, Starfield S and Hagan A, 2003, ‘The attitudes of IELTS stakeholders: student and staff perceptions of IELTS in Australian, UK and Chinese tertiary institutions’ in IELTS Research Reports, Volume 5, IELTS Australia, Canberra
• Cotton, F and Conrow, F, 1998, ‘An Investigation of the Predictive Validity of IELTS amongst a Group of International Students studying at the University of Tasmania’ in English Language Testing System Research Reports, 1, pp 72-115
• Criper, C and Davies, A, 1988, ELTS Validation Project Report: Research Report 1(i), The British Council/University of Cambridge Local Examinations Syndicate
• Davidson, F, 1998, ‘Book Review: The Development of IELTS: A study of the effect of background knowledge on reading comprehension’ in Language Testing, 15, pp 288-292
• Davies, A, 2008, Assessing Academic English Testing English Proficiency 1950-89: the IELTS Solution, (Volume 23, UCLES/CUP Studies in Language Testing series), Cambridge: Cambridge University Press
• De Vita G and Case P, 2003, ‘Rethinking the internationalisation agenda in UK higher education’ in Journal of Further and Higher Education, Vol 27, No 4, November 2003, pp 383-398
• Dooey, P, 1999, ‘An investigation into the predictive validity of the IELTS Test as an indicator of future academic success’ in Teaching in the Disciplines/Learning in Context, eds Martin, Stanley and Davison, proceedings of the 8th Annual Teaching Learning Forum, University of Western Australia, February 1999, pp 114-118
• Edwards, V, Ran A, and Li D, 2007, ‘Uneven playing field or falling standards?: Chinese students’ competence in English’ in Race Ethnicity and Education, Vol. 10, Issue 4, December 2007, pp 387-400
• Elder, C, 1993, ‘Language proficiency as a predictor of performance in teacher education’ in Melbourne Papers in Language Testing, 2 (1), pp 68-87
• Feast, V, 2002, ‘The impact of IELTS scores on performance at university’ in International Education Journal, Vol 3, No 4, pp 70-85
• Ferguson, G and White, E, 1993, ‘A small-scale study of predictive validity’ in Melbourne Papers in Language Testing, University of Edinburgh, pp 15-63
• Fiocco, M, 1987, ‘English Proficiency Levels of Students From a Non English Speaking Background: A Study of IELTS as an Indicator of Tertiary Success’, research report, Centre for International English, Curtin University of Technology
• Glesne, C and Peshkin, A, 1992, Becoming qualitative researchers: An introduction. London: Longman
• Graham, JG, 1987, ‘English Language Proficiency and the Prediction of Academic Success’ in TESOL Quarterly, 21 (3), pp 505-521
• Green, A, 2005, ‘EAP study recommendations and score gains on the IELTS Academic Writing test’ in Assessing Writing, Volume 10, Issue 1, pp 44-60
• Green, A, 2007, ‘IELTS washback in context: preparation for academic writing in higher education’ in Studies in Language Testing 25, Cambridge University Press and Cambridge ESOL
• Hawkey, R, 2006, Impact Theory and Practice: Studies of the IELTS Test and Progetto Lingue 2000, (Volume 24, UCLES/CUP Studies in Language Testing series), Cambridge: Cambridge University Press
• Hill, K, Storch, N and Lynch, B, 2000, ‘A comparison of IELTS and TOEFL as predictors of academic success’, IELTS Research Reports, Volume 3, IELTS Australia, Canberra
• Hood, C, 1991, ‘A Public Management for All Seasons?’ in Public Administration, Volume 69, No 1, pp 3-19
• Hughes, A, Porter, D and Weir, C, 1988, ELTS Validation Project Report: Proceedings of a conference held to consider the ELTS Validation Project Report - Research Report 1(iii), The British Council/University of Cambridge Local Examinations Syndicate
• Kerstjens, M and Nery, C, 2000, ‘Predictive validity in the IELTS test: A study of the relationship between IELTS scores and students’ subsequent academic performance’, IELTS Research Reports, Volume 3, IELTS Australia, Canberra
• Light, RL, Xu, M and Mossop, J, 1987, ‘English proficiency and academic performance of international students’ in TESOL Quarterly, 21 (2), pp 251-261
• Mehrens, WA and Cizek, GJ, 2001, ‘Standard setting and the public good: Benefits accrued and anticipated’ in Setting performance standards: Concepts, methods and perspectives, ed G. J Cizek, Mahwah, NJ: Lawrence Erlbaum, pp 477-485
• McDowell, C and Merrylees, B, 1998, ‘Survey of receiving institutions’ use and attitude to IELTS’, IELTS Research Reports, Volume 1, IELTS Australia, Canberra
• Miles, MB and Huberman, AM, 1994, Qualitative data analysis: an expanded sourcebook, second edition, London: Sage
• Mok, M, Parr, N, Lee, T and Wylie, E, 1998, ‘A comparative study of IELTS and ACCESS test’ IELTS Research Reports, Volume 1, IELTS Australia, Canberra
• Rea-Dickins, P, Kiely, R and Yu, GX, 2007, ‘Student identity, learning and progression: The affective and academic impact of IELTS on “successful” candidates’, IELTS Research Reports, Volume 7, IELTS Australia and British Council, Canberra, Australia
• Read, J and Hayes, B, 2003, ‘The impact of IELTS on preparation for academic study in New Zealand’, IELTS Research Reports, Volume 4, IELTS Australia, Canberra
• Smith, HA and Haslett, SJ, 2007, ‘Attitudes of tertiary key decision-makers towards English language tests in Aotearoa New Zealand: Report on the results of a national provider survey’, IELTS Research Reports, Volume 7, IELTS Australia and British Council, Canberra, Australia, pp 13-57
• Tahir, T, 2007, ‘Plan to cut entry mark scrapped’ in Times Higher Education Supplement, 7 December 2007
• Train, RW, 2002, ‘Foreign Language Standards, Standard Language and the Culture of Standardization: Some Implications for Foreign Language and Heritage Language Education’, UC Language Consortium Conference on Language Learning and Teaching: Theoretical and Pedagogical Perspectives, retrieved 10 April 2007 from <http://uccllt.ucdavis.edu/hli/papers/Culture%20of%20standard.htm>

APPENDIX 1: THE QUESTIONNAIRE

Investigating stakeholders’ perceptions of IELTS as an entry requirement for higher education in the UK

Please complete ALL questions in the questionnaire.

Your response to this questionnaire is vital for this research.

Please return this questionnaire by e-mail no later than

Friday, 7th September, 2007

Please return this questionnaire to:

d.hyatt@sheffield.ac.uk

Your responses to the questionnaire will be completely confidential to the project researchers. No-one from your institution will have access to this questionnaire.

This questionnaire should take about twenty minutes to complete.

THANK YOU FOR YOUR INVALUABLE HELP.

For further information please contact Dr David Hyatt at the above address.

Please complete the following questionnaire by writing your answers in the blank spaces provided (…) or by ticking appropriate answers from the options provided.

Respondent and course details

Please respond in relation to a specific course with which you are familiar. If you are would like to answer about more than one course please fill out a second form.

Questionnaire items

In return for completing this questionnaire, would you like to receive a complimentary copy of the IELTS Scores Explained DVD? If so please provide your name and address for delivery below.

Request for Interview

In order to supplement the information you have generously provided in this questionnaire, we intend to carry out a number of interviews with a cross-section of respondents.

If you would be willing and able to participate in a telephone interview at a later stage of the project, we would be most grateful if you could complete the section below.

I am interested in further participation in this study and I am willing to take part in an interview at a later stage of the project.

Please note that an offer to be interviewed is not binding.

Thank you for your help.

Please return this questionnaire to:

d.hyatt@sheffield.ac.uk

Dr David Hyatt, School of Education, University of Sheffield

388 Glossop Road, Sheffield S10 2JA

APPENDIX 2: THE INTERVIEW SCHEDULE

Investigating stakeholders’ perceptions of IELTS as an entry requirement for higher education in the UK

1. How does the IELTS testing system impact on your professional life? What connection do you have with this suite of tests?
2. Does IELTS give you a fair impression of a student’s language capability in an academic context? Is it a useful indicator of subsequent academic performance?
3. How is IELTS level required for acceptance onto your programme decided?
4. Is this an appropriate level? Why/why not?
5. Do you feel there is a tension between setting language standards and recruitment? If so, how do you seek to resolve this tension?
6. Is there any pressure to recruit students who you feel are might not have sufficient language capability?
7. Who ultimately decides - departmental/course admissions tutor or central admissions and recruitment office?
8. How familiar are you with the content of the IELTS suite of examinations? Do you make judgements’ about student suitability based on the global score or on a more detailed understanding of what the examinations involve?
9. Would greater awareness of the content and process of these exams be useful? Who should provide such development opportunities - IELTS or your institution?
10. Do students meeting the minimum level still require additional language support? In what ways?
11. Would any such needs be best met with generic language centre support or more course-specific support, possibly within the department?
12. On balance do you think that IELTS is a testing system that is ‘fit for purpose’, as a testing system that is designed to indicate whether or not a student has the language capability to be successful on a particular course, i.e. does the content of the IELTS examinations meet your needs?
13. How could the IELTS testing system be improved?
14. Did you find the DVD ‘IELTS Scores Explained’ a useful resource? In what ways?
15. Are there any other comments that you would like to make that might help to inform this research project?

Author biodata

David Hyatt

Dr David Hyatt is a lecturer at the School of Education, University of Sheffield. He directs a number of programmes including the taught doctorate in Language Learning and Teaching and the Singapore Distance Education programme. He is the departmental teaching quality director and the learning and teaching advocate. David is currently chairing a working group on assessment to investigate and disseminate good practices, including creative and innovative approaches to assessment and feedback. His research and publications cover areas such as critical literacy, academic literacy, English language teacher education and ELT assessment.

Greg Brooks

Professor Greg Brooks is Professor of Education at the School of Education, University of Sheffield, and Research Director of the Sheffield arm of the National Research and Development Centre for adult literacy and numeracy (funded by the Skills for Life Strategy Unit within the Department for Education and Skills - now the Department for Children, Schools and Families). Greg’s research interests include literacy (initial and adult), oracy, trends in standards of educational achievement over time, and research methodologies including randomised controlled trials. He has directed over 40 research projects in the fields of initial and adult literacy, assessment of oracy, reviews of adult basic skills, what works for children with literacy difficulties, and the phonics element of the National Literacy Strategy.