Investigating the English language proficiency gap: when university and industry standards are misaligned

Investigating the English language proficiency gap: when university and industry standards are misaligned

What if the English test score that got you into university wasn’t enough to get you a job in your field?

New research reveals inconsistencies between the English requirements for university degrees and the professions they lead to.

Professional bodies and universities may have different standards when using English language proficiency tests for recruitment.

This was the conclusion of researchers Dr Amanda Müller and Andrew Brenner, who analysed minimum English proficiency scores set by professional organisations and universities. Their research was focused on careers and courses in the areas of education, law, medicine, nursing, psychology, and social work. The results surprised them, revealing a general misalignment in current practices.

A university degree in a professional subject sets the foundations for an individual to enter that profession. A low English test score requirement for applicants to university could set students up for failure. If the student struggles to keep up with their course, they might not graduate - or worse, they may graduate but still not meet the professional language standards needed for their career.

Issues can also be caused if a professional body sets a score too low for a job that involves providing a service to the public. In an area such as medicine, insufficient communication skills can be a life-and-death issue. In legal settings, a poorly communicated argument can negatively affect outcomes too.

It is important that the content, quality and standards of each English proficiency test are clearly understood by applicants and organisations, to make sure a test meets their needs. It is also important that professional bodies and universities use the results of these different tests correctly - and that any comparisons made are robust.

For students to succeed, acceptable minimum scores set by universities must align with minimum scores set for professional registration - and the minimum scores of all these organisations should align with the scores recommended by test providers. This research aims to raise awareness among decision-makers about the implications of inconsistencies in score-setting.

It’s time for universities and professional bodies to work together to set clear, fair, and aligned English standards - so students don’t find themselves stranded between education and employment. We can help you do this, by providing you with support and advice on effective and best practice score-setting, language assessment literacy, research and expertise.

About the authors

Dr Amanda Müller is Associate Professor at Flinders University, Australia.

Andrew Brenner is a PhD student at Flinders University. He is also a Registered Nurse.

We were really shocked by how institutions set test scores that widely deviated from test maker recommendations, which are based on linguistic experience and evidence. We were most surprised by how different institutions varied so much in how they interpreted equivalency across English tests. The standardisation of equivalence scores is crucial.

Dr Amanda Müller

About the English proficiency tests

The data in the report covers the six most common English language proficiency tests used among the professions and universities included in the study:

• International English Language Testing System (IELTS) - Academic
• Occupational English Test (OET)
• Cambridge C1 Advanced
• Pearson Test of English (PTE) - Academic
• Test of English as a Foreign Language iBT (TOEFL)
• Duolingo English Test (DET).

To obtain score-setting data, the researchers analysed information available on the websites of these organisations and in the documents they provided.

3 key findings

Müller and Brenner’s report had three main findings:

1. The minimum scores of universities and associated professional bodies are not aligned.
2. There is a lack of a unified system for equating English proficiency scores across tests.
3. Universities and professional bodies are not always following test-developer recommendations.

1. Misalignment in minimum scores set by universities and associated professional bodies

Comparing the data in the report’s tables reveals that on average, minimum scores set for professional degree courses are lower than minimum scores required for registration to related professions. The range of scores set reveal that minimum scores required by different universities vary, as do minimum scores required by different professional bodies.

Table 1: Average minimum scores and ranges set by professional bodies

The blank cells indicate that no professional organisation in any country studied accepted the results of a particular test, and therefore there is no score available.

Table 2: Average minimum scores and ranges set by universities

Average scores lower for universities than professional organisations

Across all professions, an average of IELTS 7.0 was found for professional registration. Across degree courses for the same six professions, an average of IELTS 6.6 was found.

This means that universities are admitting students into degree pathways with a score lower than that required for professional registration.

This is despite the fact that there is often a clinical or practical component in university coursework. This component may involve a student carrying out supervised practice on the public.

Variation in range of minimum scores set by professional bodies and by universities

In terms of professional bodies, the biggest variation in minimum scores was for professional registration to education, ranging from IELTS 6.0 to IELTS 7.5.

In terms of minimum scores for entry to professional degree courses, education again showed the biggest variation and the range of scores was even greater, from IELTS 5.0 to IELTS 8.0.

For social work, there was no variation in the minimum score required for professional registration, which was IELTS 7.0.

For universities, minimum scores for social work degree courses varied by two whole band scores, ranging from IELTS 5.5 to IELTS 7.5. Scores for medicine, nursing and psychology also varied by two band scores (compared with one band score for professional registration). Scores for law varied by 2.5 band scores (compared with a variation of 0.5 for professional registration).

Variations were also found in minimum scores set for the other four English proficiency tests studied, as shown in the two tables.

Implications of different standards for professional registration and degree entry

A variation of half an IELTS band score is more significant than it may sound. Research has shown that for a student to improve by a 0.5 band score, they would need to study English for up to 6 months full-time because gains are slower at higher levels than lower levels.

Research has also shown that international students do not necessarily increase their IELTS score by more than a 0.5 band over an entire degree. Between 35 percent and 63 percent of students remain at the same level or worsen over their degree. This can be the case even when language and learning support services are available to them.

This reinforces the need for appropriate scores to be set. If an applicant enters university with a test score that is lower than a minimum score recommended by their target profession, they may not have the level of English they need to enter that profession when they finish university.

2. Lack of a unified approach by professions and universities for equating English proficiency scores across tests

Professional bodies and universities choose their own equivalence scores across different tests, and publish these on their websites and materials.

Müller and Brenner’s report found that in general, their equivalence scores cannot be trusted because they do not match the choices of other institutions. This means consumers cannot reliably convert the scores of the different tests across institutions. More importantly, since test-maker recommendations are not always followed, we cannot reliably know that the same skill level is being met when multiple tests are involved.

Illustrating the difficulties of comparing test scores

The report includes the recommended scores of different test providers, as shown in the table here. There is some level of agreement between test developers about what scores equate to each other on their different scales.

The graph shown here presents the range of TOEFL scores considered equivalent to each IELTS half-band score. While no two tests are the same, these two tests are similar in terms of purpose and structure. The average minimum scores and ranges set by professional bodies and universities are also plotted on the graph. This graph can be used to show how working backwards from a TOEFL score to an IELTS score is problematic.

Table 3: Equivalent test scores according to linguistic research and recommendations of test providers

Diagram 1: TOEFL - IELTS equivalent scores (range shown by line and average shown by shape)

The TOEFL example

If professional bodies and universities do not follow test provider guidance, the ranges of minimum scores set by organisations for TOEFL can result in the same TOEFL score being equivalent to more than one IELTS score.

In the case of professional bodies, a TOEFL score of 96, for example, may be equivalent to any score from IELTS 6.0 to IELTS 7.5. For scores set by universities, a TOEFL score of 86 may be equivalent to any score from IELTS 5.5 and IELTS 7.5, depending on the university.

Where organisations have not followed test provider guidance, it is therefore not possible to reliably equate a score set for TOEFL (or any of the other four tests) back to a single equivalent IELTS score.

Poor test score equivalency can produce poor results

If the minimum required scores set by a university for each test aren’t equivalent, this can mean it is easier for an applicant to get onto a course if they take one test rather than another. This may lead an applicant to go for what looks like the easier option.

If the minimum required scores for that test are set lower than they should be, the applicant may not have the appropriate English proficiency skills.

This can result in poor student well-being, higher dropout rates and, in the case of professions, employees who are ill-qualified to perform highly critical tasks.

3. Universities and professional bodies not meeting test-developer recommendations

The study found that the minimum required scores set by universities and professional bodies often don’t align with the recommendations of test developers. The scores of professional bodies were better aligned with test-developer recommendations than those of universities.

Range of scores set show many students need further English study

For starting a ‘linguistically demanding’ academic course like the ones featured in the study, an IELTS 7.5 band score is recommended, representing an ‘acceptable’ level of English proficiency. A band score of IELTS 7.0 is deemed ‘probably acceptable’.

The report’s data shows the average minimum score set by universities (see Table 2: Average minimum scores and ranges set by universities) was IELTS 6.6, deemed ‘English study needed’. Some universities set a minimum score as low as IELTS 5.0.

The average minimum score set for professional registration (see Table 1: Average minimum scores and ranges set by professional bodies) was IELTS 7.0, with some professional bodies setting scores as low as IELTS 6.0.

What does this research mean for me?

This report raises serious concerns about the negative impacts of setting inappropriate English proficiency scores for entry to university or for professional registration to the six professional areas investigated by the researchers.

Inappropriate score setting may mislead individuals into believing they have sufficient English language skills for academic and professional success. This can lead students to fail at university and not get the job they thought their course would set them up for.

If individuals manage to obtain a qualification in this concerning situation, and then fail to carry out their professional tasks due to a lack of English proficiency, this can also have serious implications.

Key takeaways for professional bodies and universities

To avoid these situations, we urge professional bodies and universities to:

• review equivalence scores and avoid automatically adopting scores from other organisations.
• review score-setting processes and collaborate with test providers on effective standards.
• align English language standards so students are set up for success and have the English proficiency skills they need to enter their chosen profession - leading them to have a positive impact on society too.

Müller and Brenner’s research was joint funded by the IELTS Partners: the British Council, IDP IELTS, and Cambridge University Press & Assessment.

Read the full report on Benchmarking English standards across professions and professional university degrees: IELTS.org/Benchmarking-English-Standards

British Council

1 Redman Place

Stratford

London

E20 1JQ

ielts@britishcouncil.org

IDP Education Ltd

Two Melbourne Quarter

Level 10, 697 Collins Street

Docklands

Victoria

Australia 3008

ielts@idp.com

Cambridge University Press & Assessment

Shaftesbury Road

Cambridge, CB2 8EA

United Kingdom

cambridgeenglish.org/

helpdesk