IELTS One Skill Retake: A Bias for Best Model

IELTS One Skill Retake: A Bias for Best Model

IELTS Research Group Position Statement (December 2022)

Dr Tony Clark, Head of IELTS Research (Cambridge University Press & Assessment)

Dr Emma Bruce, IELTS Research and Validation (British Council)

Dr Jamie Dunlea, Manager and Senior Researcher (British Council)

We recognise that test takers can underperform in assessment for any number of reasons. IELTS One Skill Retake improves fairness by providing an opportunity to retake a single section, if the test taker feels their original performance was not representative of their language proficiency level.

Restrictions are in place to ensure that the standard of the assessment is maintained. The decision to allow retakes has been based on academic research, test taker feedback and internal IELTS validation considerations.

Background

A transparent approach to communications and decision-making is central to policy adjustments such as this, in addition to involving stakeholders in the process (Jones & Saville, 2016; Chalhoub-Deville & O’Sullivan, 2020; Spiby & Clark, forthcoming 2023). This paper outlines some of the research and operational considerations around the introduction of IELTS One Skill Retake in December 2022 and documents our plans for ongoing validation work in the area. Both the implications of these changes and their relation to test consequences (Weir, 2005; Saville, 2009; Clark et al., forthcoming 2023) must be thoroughly investigated and recorded if they are to be understood by key stakeholders (Taylor, 2007).

1) Motivations for introducing One Skill Retake

The IELTS Partners have become increasingly aware of Recognising Organisations setting component-level requirements (e.g. Band 7 on each skill), which was not the case when the IELTS test was first launched. As a result, it is now possible for a test taker’s ability to be defined by their weakest skill, and for important decisions (e.g. the granting or not of a visa) to be taken based upon this.

In the light of such requirements from Recognising Organisations, the IELTS Research Group has been investigating ways to best ensure that the test continues to be fair for all test takers whilst maintaining its underlying validity. We would like to give test takers the best chance to demonstrate their language proficiency, and the flexibility afforded by IELTS One Skill Retake may now permit this.

Based on the perspectives of test takers, academic input and internally conducted validation research, candidates will now be permitted to retake a single component of the test if they believe that they did not perform to the best of their ability on that component. They will only be able to retake a single component once within 60 days of their original test.

2) Justification for implementing One Skill Retake in the short term

The underlying validity evidence of the test has not changed. The components remain the same and considering the validity of the scoring model, there are no issues which would critically militate against test takers.

A bias-for-best approach increases fairness for candidates.

Test taker characteristics are a component of test validity, and physical (e.g. short-term ailment), psychological (e.g. anxiety), or experiential characteristics (e.g. world knowledge) can potentially have an impact on test performance. A One Skill Retake model could mitigate the potential negative effect of test taker characteristics.

There is a growing body of evidence to support the technical qualities of such an approach to retakes.

Based on the above there are no critical reasons for not progressing with One Skill Retake at this stage.

3) Communication, ongoing monitoring and further research

Ongoing monitoring of scoring and implications of retakes for candidates is planned over the coming year (2023), as the policy is further rolled out (subject to operational considerations). This research will not be on scores alone, but all relevant aspects of the wider policy and the impact on candidates.

A paper outlining the validity implications for One Skill Retake in more detail is planned for the public sphere, building on the current document and incorporating some of the findings from the monitoring process described above. This will be done to demonstrate our ongoing commitment to quality and transparency, and to help score users understand more about what One Skill Retake will mean for them. This paper will lay out what the ongoing monitoring and research agenda needs to be to support the One Skill Retake initiative as a part of the wider IELTS test system.

Informing candidates of their choices around retakes and supporting their decision-making forms part of our responsibility. This will be supported with communications based on research insights of relevance for test takers.

References

• Chalhoub-Deville, M., & O’Sullivan, B. (2020). Validity: Theoretical development and integrated arguments. London: British Council Monographs.
• Clark, T., Holland, M., & Spiby, R. (forthcoming 2023). Seeking Empirical Evidence to Support Online Test Validation: Building on the Indicator Assessment Model. In Language Assessment at the Time of the COVID-19 Pandemic.
• Jones, N., & Saville. N. (2016). Learning Oriented Assessment: A Systemic Approach. Studies in Language Testing volume 45. Cambridge: UCLES/Cambridge University Press.
• Saville, N. (2009). Developing a Model for Investigating the Impact of Language Assessment within Educational Contexts by a Public Examination Provider. PhD thesis, University of Bedfordshire.
• Spiby, R., & Clark, T. (forthcoming 2023). Responding to a transformed assessment landscape: the example of IELTS. In Life After the Pandemic: ReVision and Reform in EAP Contexts.
• Taylor, L. (ed.) (2007). IELTS Collected Papers: Research in Speaking and Writing Assessment. Studies in Language Testing volume 19. Cambridge: UCLES/Cambridge University Press.
• Weir, C. J. (2005). Language testing and validation: An evidence-based approach. Basingstoke: Palgrave.