Investigating linkages between international students’ English language proficiency, social–contextual outcomes, and wellbeing in US universities

Abstract

Overview: Pre-enrolment IELTS scores of international students at U.S. universities linked with higher English confidence, higher social support from domestic students, and lower academic stress during students’ Spring 2021-Spring 2022 university experience.

This study examined the interplay between English language proficiency, English confidence, first language (L1) educational experience, academic stress, domestic student social support, social engagement, belonging, loneliness, and psychological wellbeing for 188 international students at 14 universities in the U.S. at three timepoints: Spring 2021, Fall 2021, and Spring 2022.

Longitudinal structural equation modeling revealed that English language proficiency (i.e., IELTS Overall Scores) predicted English confidence; higher IELTS scores were associated with higher English confidence. IELTS scores had a significant indirect effect on academic stress via English confidence, such that higher IELTS scores were linked with higher English confidence and thus lower academic stress across all three timepoints. IELTS scores approached significance with respect to indirect effect on domestic student social support via English confidence as well. We provide limitations and future directions for research.

Authors’ biodata

Nelson Brunsting

Nelson Brunsting, PhD, serves as the Executive Director of the Research on Abroad and International Student Engagement (RAISE) Center, and is a Research Associate Professor of International Studies at Wake Forest University (North Carolina, USA). His research focuses on understanding social-emotional outcomes of diverse populations in educational settings, specifically international students. Nelson serves as the Special Issues Editor for the Journal of International Students, and he has been Principal Investigator for research grants totalling over $183,000 from IDP: IELTS Australia and the Spencer Foundation, among others. Nelson received his MA in Classics at Victoria University of Wellington, New Zealand, and his PhD in Educational Psychology from University of North Carolina, Chapel Hill.

Qiaona Yu

Qiaona Yu, PhD, is an Associate Professor of East Asian Languages and Cultures at Wake Forest University, North Carolina, USA. Qiaona earned her PhD in Chinese Linguistics and Pedagogy at the University of Hawai’i at Mānoa, and her MA in Chinese Linguistics and Philology from Peking University. Qiaona’s research specialises in second language acquisition, task-based language education, language for specific purposes, and cognitive linguistics. She is particularly interested in defining and assessing translingual and transcultural competence. Qiaona has published first author manuscripts in Applied Linguistics and Global Business Languages. She reviews for Language Learning, System and the ACTFL Convention proposals.

Jon Smart

Jon Smart, PhD, is an Associate Teaching Professor in the Department of English at Wake Forest University, North Carolina, USA. Jon is the faculty lead for the Global Connections program in the Center for Global Programs and Studies at the university. Jon’s research interests are in data-driven learning and English for Academic Purposes. He is particularly interested in second language writing in academic contexts. He has an MA in Teaching English as a Second Language, and PhD in Applied Linguistics from Northern Arizona University. He has published research in ReCALL and Assessing Writing.

W. Patrick Bingham

W. Patrick Bingham, PhD, is a Research Coordinator for the Research on Abroad and International Student Engagement (RAISE) Center at Wake Forest University, North Carolina, USA. Patrick earned his MA and PhD from the University of East Anglia in research focused on Art, Media, and American Studies. He has published articles in Networking Knowledge (postgraduate journal by the Media, Communication, and Cultural Studies Association) and eJumpcut. He serves as a reviewer for the Journal of International Students.

Executive summary

A research team from Wake Forest University (Winston-Salem, North Carolina, USA) conducted a research study to understand whether the pre-enrolment IELTS scores of international students attending U.S. universities were linked with their social, emotional, and academic outcomes. Fourteen U.S. universities elected to participate, and the research team invited all international students at the universities to join the study. Of the 1035 international students who joined the study, 188 reported taking IELTS as part of the university admissions process. These 188 students were asked to complete an online survey at three timepoints: February-March 2021 (Spring in the U.S.), October-November 2021 (Fall in the U.S.), and February-March 2022. This longitudinal survey methodology provided the researchers with the ability to examine for relationships between variables across time, which provides stronger evidence toward making a causal claim (i.e., stating that something causes something rather than stating that two things co-occur).

Using a complex statistical analysis called longitudinal structural equation modeling, the researchers tested for relationships among a range of variables, including:

• English language proficiency (IELTS score)
• English confidence
• first language (L1) educational experience
• academic stress
• domestic student social support
• social engagement
• belonging
• loneliness
• psychological wellbeing.

Key findings included that English language proficiency (i.e., IELTS Overall Scores) predicted English confidence across time, such that higher IELTS scores pre-admission reported higher English confidence at each study timepoint. IELTS Scores also had a significant indirect effect on academic stress via English confidence. This means that individuals with higher IELTS scores reported lower academic stress across all three timepoints, and that the relationship is mediated by their higher English confidence. In essence, because students had higher English confidence due in part to their IELTS scores, they also had less academic stress. This is an important finding, as academic stress is linked in the research literature with other challenges including lower psychological wellbeing and academic achievement. Analysis of the data also revealed that IELTS scores may have in indirect effect of domestic student social support mediated by English confidence as well, though the results were almost but not quite significant in this study.

With respect to practitioners working with international students for whom English is not a first language, we share the following implications from the study. We note with caution-as further research replicating results is needed-that although IELTS scores are designed to assess academic readiness in an English-speaking context rather than predict international students’ social or emotional outcomes at U.S. universities, the current study provides initial evidence that IELTS scores predict social-emotional variables like English confidence and academic stress. If such findings were to be replicated and extended to other outcomes like loneliness or belonging, admissions officers at U.S. universities might consider additional benefits of IELTS scores for anticipating international student success on campus.

1 INTRODUCTION

International students’ cultural and social-emotional adjustment to U.S. universities can vary widely. Although 50% of international students (i.e., those studying with the purpose of obtaining a degree on student and scholar exchange visas) experience little challenge with culture shock or wellbeing throughout their first years at university (Wang et al., 2015), research indicates that international students often struggle obtaining social support from members of the host culture, gaining a sense of connectedness to their institution, and they feel lonely or isolated (Brunsting et al., 2018; Glass & Westmont, 2014). These and other social-emotional challenges are associated with health outcomes for international students including unhappiness, depression, and suicidal ideation (Du & Wei, 2015; Hirai, Frazier & Syed, 2015; Muyan & Chang, 2015). While social support appears an important predictor of social-emotional experiences (e.g., belonging, loneliness, wellbeing; Brunsting et al., 2021), the role of language competency in social interactions is not clear. A recent study revealed that international students’ perceived language competency, that is the degree to which students felt confident using the host country language, predicted a range of wellbeing outcomes (Luo et al., 2019).

Researchers, however, have yet to investigate fully the importance of language competence or proficiency, measured by language examinations such as IELTS, on international students’ social-emotional adjustment to colleges and universities in the U.S., using longitudinal approaches. IELTS is designed primarily to assess the readiness of students to study or train in an English-speaking academic context. Although researchers have investigated predictive validity of IELTS for academic achievement (e.g., Schoepp, 2018), the predictive validity of IELTS for social and emotional adjustment (e.g., belonging, social engagement, psychological wellbeing) remains unexplored, as it is not part of the test’s intended purpose. The current study addresses this gap with a large, multi-institutional longitudinal dataset.

A range of studies (e.g., Fraser, 2007; Yamashita, 2002) have investigated how L1 proficiency affects L2 development. However, researchers have yet to examine how students’ L1 educational experience prior to their international education experience may affect their social interactions and wellbeing while abroad. Here, consider the knowledge structure students built in their L1 which helps situate them in a globalised world with not only skills and proficiency to communicate but also the textual, historical, metacognitive, sociocultural, anthropological, ethnographical, and aesthetical analysis of how such content operates.

Prior research has demonstrated a range of linkages between important social-contextual processes and outcomes for international students at U.S. universities. A systematic literature review identified 30 quantitative studies of international students’ adjustment and social-emotional outcomes at U.S. universities from 2009 to 2017 (Brunsting et al., 2018). Key findings included that social support and social interaction were linked with international students’ belonging and wellbeing (e.g., Longerbeam et al., 2013; Van Horne et al., 2018). Recent cross-sectional research has identified perceived competence or confidence in English as linked with domestic student social support, acculturative stress, and wellbeing for international students (Koo et al., 2021; Luo et al., 2019). Similarly, Brunsting et al., (2021) in a semi-longitudinal study, found that domestic student social support was linked with international students’ belonging. In addition, Brunsting et al. (2021) identified loneliness as a predictor for psychological wellbeing and suggested that academic stress may have an impact on loneliness for students who forgo social interaction to focus on academic study.

However, these studies were conducted at a total of three universities and were either cross-sectional or half-longitudinal in nature. There is need to provide stronger evidence with a larger sample of students at more universities to build on and extend these findings. Most importantly, there is need to understand how English language proficiency supports international students in their English confidence, domestic student social support, social engagement, academic stress, belonging, loneliness, and wellbeing.

1.1 Theoretical framework

We draw on a systems framework (Tseng & Seidman, 2007), wherein individual factors (e.g., language proficiency) interact with resources (e.g., faculty) via social processes (e.g., social support) to influence social outcomes (e.g., belonging), which may impact individual outcomes in that setting (e.g., psychological wellbeing). Figure 1 below explains these relationships.

Figure 1: Theoretical framework for the current study

We integrate the systems framework with a sociocultural perspective of language learning which views language as a culturally-specific symbolic tool for humans to act on their interpersonal (social interaction) and intrapersonal (thinking) purpose (Vygotsky, 1978; Lantolf, 2000). Prior to their international education experience, students receive formal education in their local language at different lengths and depth. Given varied L1 educational backgrounds, we posit that international students are prepared with different degrees of control over a key factor for their cultural and social-emotional adjustment in the U.S. Once students embark on an international educational sojourn in the U.S., they need to do more than gain control of an additional L2 tool. Rather, just like the negotiation of their identity, the negotiation among the two languages (i.e., symbolic tools) and their context creates a “third place” (Bhabha, 1994; Kramsch, 2009). The two symbolic tools “encounter and transform each other” (Block, 2007, p.864). Such mastery of L1, L2, and the “third place” contributes to their situating themselves in a globalised world with not only skills and proficiency for interpersonal social interaction but also intrapersonal thinking, which both further contribute to their social engagement and wellbeing at the U.S. universities.

1.2 Research questions

This 1.5-year longitudinal study was designed to provide extensive data to answer the following research questions with a large multi-university sample of international students attending colleges and universities in the U.S.

1. Does English language proficiency as measured by IELTS predict increasing trajectories in perceived English language proficiency, social engagement, perceived social support, belonging, academic stress, and psychological wellbeing?
2. What are the unique contributions of English language proficiency as measured by IELTS score, perceived English language proficiency, and L1 educational experience for all other contextual or outcome variables?
3. Are relationships between English language proficiency, perceived proficiency, L1 educational experience and psychological wellbeing mediated by social engagement, perceptions of social support, belonging, loneliness, and academic stress?
4. Is there an interaction effect between English language proficiency and L1 educational experience for these mediating and outcomes variables across time?

2 METHOD

2.1 Procedure

The research team sought and received approval from the institutional review board of the authors’ institution prior to the beginning of the study. Our recruitment strategy was multi-layered. First, we sought institutional approval from universities. We generated a list of 236 U.S. universities by selecting the top 150 national universities as ranked by the U.S. News and World Report, and included top regional universities (i.e., regional universities focus on four areas of the U.S.: Midwest, Northeast, South, and West) as well as top universities for international students. Team members contacted personnel listed on university websites as overseeing international student and scholar services via email and phone calls in Fall of 2020. We explained the purpose of the study and the potential benefits to the universities, which included: (a) access to the research team for expertise and questions; and (b) reports at each timepoint with average scores of participants at their university and the national average across the study. Twenty-two universities responded, and 17 agreed to participate. Ultimately 14 universities joined the study. Participating institutions supported international student recruitment by either providing our team with a list of international students’ names and emails or by forwarding our recruitment email to all of their international students multiple times during the first timepoint of the study.

The study had three timepoints: T1 in Spring 2021, T2 in Fall 2021, and T3 in Spring 2022. We had two methods for recruiting student participants. For universities providing student email address, we asked that our university contact (e.g., Director of International Student and Scholar Services) email all international students three days prior to the start of the study to share their support for the study. The research team uploaded student information to Qualtrics and emailed the students the study information via Qualtrics. Students had the option to click to review the consent information and join the study. Students who approved that they were 18 years of age or older and consented to participate were then directed to the T1 survey. Non-responders received two more emails across the next two weeks. Non-responders at T1 were contacted at T2 and T3 via the same method with the opportunity to join the study at that time. Our second method of recruitment, for universities that elected not to share student contact information, was to ask the university contact to forward the recruitment email-which contained a link to the consent form on Qualtrics-to students three times over a three-week period.

Students who consented were directed to the survey, which also included an item to provide their email address so they could be contacted at T2 and T3 to continue to participate in the study. Participants were entered into draws to win one of five electronic $50 gift cards at each timepoint for which they completed a survey. A total of 1510 responded (response rate: 8.20%), of whom 1455 consented to participate. Of those who consented, 1035 students completed the survey at one or more timepoints, thus representing the study sample. This response rate is lower than in previous similar studies (e.g., Brunsting et al., 2021) and a previous wave of this study prior to COVID-19 (19% response rate). We attribute the decrease in large part due to COVID-19, which appears to have depressed response rates on longitudinal research nationally (McIllece, 2020). Of the 1035 participants in the larger study, 188 reported IELTS scores; thus, these 188 represent the focal participant sample of the study.

2.2 Participants

Table 1 sets out an overview of the participant demographic information. Although our participation rate is below standards for generalizability (as we note in limitations), we compared our sample to the total international student data from the 2020/2021 Open Doors report (IIE, 2021). With respect to region of origin, our sample is similar to the overall population, though we have a lower percentage of East Asian students (31% vs. 49%), Latin America and the Caribbean students (3% vs 8%), and Sub-Saharan African students (3% vs. 4%), as well as higher percentages of students from other regions: Europe and Central Asia (9% vs. 8%), Middle East and North Africa (15% vs. 6%), South Asia (39% vs. 22%). With respect to graduate level, the percentage of graduate students in the study was higher (59% vs. 47%) than the population. Thus, while the sample did not match perfectly onto the population, there were no concerning differences with respect to demographic information.

Table 1: Participant demographic information

Note. Not all percentages add up to 100% due to missing data or rounding. JD = Juris Doctorate; PhD = Doctorate.

2.3 Measures

2.3.1 Demographic variables

Demographic variables included country(ies) of citizenship (i.e., students could list two countries of citizenship to represent dual-citizens), race/ethnicity, gender, degree pursued, and years at college or university.

2.3.2 Predictor variables

2.3.2.1 L1 Educational experience

We operationalized L1 Educational Experience as the formal education students received in their native language by creating an ordinal variable with the following groupings:

• students who attended university in their native language prior to coming to the US
• students who completed high school using predominantly their native language
• students who attended English curriculum international high schools
• students who began formal education in US in high school
• students who began formal education in US prior to high school.

2.3.2.2 English language proficiency

Students selected which English language proficiency tests they submitted to their university and were asked to enter the score(s). Students’ self-reported overall score on IELTS was the measure of their English language proficiency. We operationalized IELTS scores as a metric specifically of academic English proficiency, as the receptive and productive language skills measured by the IELTS center on academic registers. Students’ social-emotional experiences related to belonging, loneliness, and wellbeing, however, may occur outside of conventional academic contexts or situations and are not within the target language use domains directly assessed by most university-oriented language proficiency tests.

2.3.2.3 English confidence

English confidence is a two-item measure adapted from a larger university academic confidence measure (Kohn & Frazer, 1986). Students responded on an 11-point sliding scale anchored at 0 not at all confident and 10 very confident for the items “speak English effectively” and “write English effectively.”

2.3.3 Setting social process variables

2.3.3.1 Social engagement

A single-item measure captures social engagement, or the frequency at which international students engage socially in their college environment (Brunsting et al., 2019). Students selected from five response options (0, 1-2, 3-4, 5-6, 7+) for the item: “how many times per week do you typically engage in social activities (e.g., group meals, intramural sports, parties, student organization meetings)?“

2.3.3.2 Domestic student social support

We examined the amount of social support students perceive from domestic students. Social support is a six-item measure adapted from Carver (2000). An example item is: “How much do the U.S. students at your university give you advice or information?” Response options range from: none at all to a great deal. The social support scale has demonstrated strong internal consistency across sources in recent studies (e.g., $\alpha = .91 - .93$ ; Brunsting et al., 2021).

2.3.3.3 Academic stress

Academic stress is a seven-item measure adapted from Kohn and Frazer (1986) to capture how stressful students perceive completing different academic tasks (e.g., take good class notes; manage time effectively, write course papers).

2.3.4 Setting social outcome variables

2.3.4.1 Belonging

Belonging is a four-item measure of the degree individuals feel valued and accepted by the people at their university. An example item is “people at my university make me feel included”, and response options indicate agreement on a six-point Likert-type scale ranging from strongly disagree to strongly agree. This measure has demonstrated strong reliability with international student samples ( $\alpha = .89$ ; Brunsting et al., 2021).

2.3.4.2 Loneliness

Loneliness is a four-item measure adapted from the UCLA Loneliness Scale, and has demonstrated high reliability in previous research ( $\alpha = .90$ ; Brunsting et al, 2021).

2.3.5 Outcome variable

2.3.5.1 Psychological wellbeing

Ryff’s (1989) Psychological Wellbeing Scales were included in the study. Forty-two items capture six subscales: autonomy, personal growth, self-acceptance, environmental mastery, relations with others, and purpose in life. The subscales have demonstrated strong reliability in previous international student samples ( $\alpha = .72 - .80$ ; Brunsting et al., 2021).

2.4 Data analysis

Prior to preparing data for longitudinal structural equation modeling, we tested the data for departures from univariate and multivariate normality. No items exceeded recommendations for skewness or kurtosis (+/- 2, and +/- 5, respectively; Bowen & Guo, 2012). No Mahalanobis values exceeded < .001; thus we identified no multivariate outliers. Because we used planned missing data design where surveys only contained some of the items to reduce respondent fatigue (Little, 2013), we anticipated structurally missing data. Thus, we focused on ensuring that covariance coverage of items or parcels exceeded the standard setting for our software, Mplus, > .10 (Muthén & Muthén, 2020). In preparation for structural modeling, we examined measurement invariance of latent constructs across time (see Appendix A for measurement invariance testing). All four constructs passed configural and weak invariance, suggesting the items load similarly on the constructs across time. Contrary to expectation, domestic student social support passed strong invariance, meaning that its means did not fluctuate much across time. We flagged domestic student social support and revisit it in the discussion.

Once measurement invariance was examined, we progressed to model fit progression for structural models: null model, measurement model, initial structural model with all parameters between constructs, and subsequent models in which we restricted non-significant parameters to obtain the best fitting model, termed the final structural model. We note the occasions in which the initial structural model was the best fitting model. We assessed the quality of model fit following Bentler and Hu’s (1999) recommendations: RMSEA < .06, CFI/TLI > .95, and SRMR < .08.

3 RESULTS

3.1 Descriptive statistics

Table 2 provides a review of the key study constructs which are measured in consecutive academic semesters at each timepoint in the study: T1 (Spring 2021), T2 (Fall 2021), and T3 (Spring 2022). We conducted repeated-samples t-tests to examine difference in means across timepoints.

Table 2: Means of key study variables at each study timepoint

Note. ^a Difference in means is significant at p < .05 as indicated by a repeated samples t-test. If we were to apply a Bonferroni-Holm correction for multiple comparisons, neither mean difference reported here as significant would remain significant.

Two differences were significant: (a) students’ reported social engagement increased significantly from Spring 2021 to Fall 2021; and (b) students’ psychological wellbeing increased significantly during the same timeframe. It is possible the differences may be due, in part, to the decreased COVID-19 restrictions at U.S. universities in Fall 2021 compared with Spring 2021. We also estimated Pearson’s correlations between study variables, including predictors of English language proficiency (i.e., IELTS score), L1 educational experience, and all constructs at T1. See Table 3 for the results. Unsurprisingly, IELTS score prior to admission was correlated significantly ( $r = .34$ , $p<.001$ ) with English language confidence.

Table 3: Pearson’s correlations between study variables at T1 (Spring 2021)

English language confidence, in turn, was significantly and positively correlated with domestic student social support (r = .18, p < .05) and significantly negatively correlated with academic stress (r = -.23, p < .01); thus the higher students’ English language confidence, the higher their domestic student social support and the lower their academic stress. Domestic student social support shared significant and positive correlations with social engagement (r = .34, p < .001) and belonging (r = .41, p < .001) and a significant negative correlation with loneliness (r = -.31, p < .001).

Thus, students reporting higher domestic student social support also reported higher social engagement and belonging and lower loneliness. The strength and number of correlations provided confidence that at least some of the relationships posited in the models in the hypotheses would emerge as significant during longitudinal structural equation modeling, but also provided caution that psychological wellbeing would share significant associations.

3.2 English proficiency and English confidence

To address the first two research questions, we developed structural models to examine the direct and indirect effects of English language proficiency (i.e., IELTS Overall Score), L1 educational experience, and English Confidence on students’ academic stress, social engagement, domestic student social support, loneliness, belonging, and psychological wellbeing. The model fit progression tables for the structural model testing for IELTS Score predicting English Confidence at T1 (Spring 2021) is available below (Table 4). The final structural model demonstrated good fit.

Table 4: Model fit progression examining relationships between IELTS score, L1 educational experience, and English confidence across time

Note. CFI = Confirmatory Factor Index; RMSEA = root mean square error of approximation. A lower RMSEA, particularly one < .06, indicates that the model fits well. Similarly, a higher CFI, ideally > .95, indicates better model fit.

The final structural model revealed that IELTS had both a significant $b = .41, p < .001$ effect on T1 English Confidence, predicting 16% of the variance in participants’ T1 English Confidence. Further, IELTS had significant indirect effects on T2 English Confidence $b = .23, p < .001, 5.29\%$ of the variance) and T3 English Confidence $b = .19, p < .001, 3.61\%$ of the variance). See Table 5 and Figure 2 for the final structural model paths.

Figure 2: Longitudinal structural model examining the relationship between English proficiency and English confidence over three semesters

Note. Solid lines represent direct effects, dashed lines represent indirect effects, and curved double-headed arrows represent covariances.

Thus, the data revealed that English language proficiency, as indicated by students’ total IELTS score, had both direct and indirect effects on international students’ English confidence across three semesters.

Table 5: Final structural model for direct and indirect effects on English confidence over time

Note. Eng Conf = English Confidence. *p < .05, **p < .01, ***p < .001.

3.3 English proficiency and outcomes

As we developed structural models, it became evident that students’ English language proficiency did not share direct effects on other outcomes (e.g., psychological wellbeing), and that testing for unique contributions of English language proficiency, English confidence, and L1 educational experience as planned in RQ2 was not the most salient approach. Instead, the structural models revealed-as we share in subsequent results-that the best approach was to examine the indirect effects of English language proficiency on outcomes through English confidence as a mediating variable and with L1 educational experience as a covariate rather than a predictor. Following this approach led to the models that best fit the data. After further testing a series of models with English confidence as a mediating variable, it became clear that testing the series of models proposed in RQ3 would provide the best test of the potential of English language proficiency as measured by IELTS Overall Score to have indirect effects on outcomes through English confidence and additional mediators (e.g., domestic student social support). We report below findings from the two series of models to yield good model fits and significant relationships between key variables.

3.3.1 English proficiency, English confidence, and domestic student social support

We tested a series of models to examine interrelations between English proficiency, English confidence, and domestic student social support. See Table 6 for model progression.

Table 6: Model fit progression examining relationships between English language proficiency, English confidence, and domestic student social support

Note. CFI = Confirmatory Factor Index; RMSEA = root mean square error of approximation.

The final structural model demonstrated good fit: $\chi^{2}(61)=95.74, p=.003;$

RMSEA = .055 (90% CI: .032, .075); CFI = .927. English proficiency had a significant direct effect (b = .42, p < .001) on T1 English confidence, and T1 English confidence had a significant direct effect on T2 domestic student social support (b = .27, p = .044; see Figure 3).

Figure 3: Longitudinal structural model examining the relationship between English proficiency, English confidence, and domestic student social support over three semesters

Thus, students who reported a higher IELTS score also reported higher English confidence in Spring 2021; IELTS score accounted for 18% of the variance in English confidence at T1. Participants reporting higher English confidence in Spring 2021 experienced higher domestic student social support in Fall 2021, accounting for 7.29% of the variance across time. These findings demonstrate directional and temporal associations between all three constructs; however, the indirect effect from IELTS on T2 domestic student social support was not significant, though it did approach significance (b = .11, p = .056). The final model paths are provided in Table 7.

Table 7: Final structural model for direct and indirect effects on domestic student social support

Note. DSSS = domestic student social support; Eng Conf = English Confidence. *p < .05, **p < .01, ***p < .001.

3.3.2 English proficiency, English confidence, and academic stress

We created a series of longitudinal structural models to examine the relationships of English proficiency, English confidence, L1 educational experience, and students’ perceived academic stress over time.

Table 8: Model fit progression examining relationships between English proficiency, English confidence, and academic stress

Note. CFI = Confirmatory Factor Index; RMSEA = root mean square error of approximation.

The final model fit was good: $\chi^{2}(39)=58.57, p=.023$ ; RMSEA = .052 (90% CI: .020, .078); CFI = .960. English proficiency had a significant direct effect (b = .36, p < .001) on T1 English confidence, and T1 English confidence had a significant direct effect on T1 academic stress (b = -.22, p = .012; see Figure 4).

Figure 4: Longitudinal structural model examining the relationship between English proficiency, English confidence, and academic stress over three semesters

Thus, students with higher IELTS scores reported higher English confidence in Spring 2021; students reporting higher English confidence in Spring 2021 reported lower academic stress in Spring 2021; English confidence accounted for $4.84\%$ of the variance in academic stress in Spring 2021. Interestingly, students’ English proficiency had a significant indirect effect $(b = -.07, p = .018)$ on academic stress at T3 (Spring 2022) via English confidence at T1, academic stress at T1, and academic stress at T2. Thus, students with higher IELTS scores reported higher English confidence, which in turn led to lower academic stress across all three semesters of the study (see Table 11).

Table 9: Final structural model for direct and indirect effects on academic stress over time

Note. Aca Str = Academic Stress; Eng Conf = English Confidence. *p < .05, **p < .01, ***p < .001.

3.4 English proficiency, English confidence, and belonging, loneliness, and psychological wellbeing

We tested a series of longitudinal models to examine relationships between English proficiency, English confidence, belonging, loneliness, and psychological wellbeing, as in Figure 5 below.

Figure 5: Example initial structural longitudinal equation model tested

There were linkages between belonging, loneliness, and psychological wellbeing, but no linkages were demonstrated between English proficiency, English confidence, and any of these key outcomes, even when including mediators such as social support or academic stress. Model fits were often poor (e.g., RMSEA > .10, CFI < .85) until English confidence and English proficiency were removed from the models; thus, we do not provide the final models as they include only, for instance, belonging and psychological wellbeing, which is not the focus of the current study.

4 DISCUSSION

We conducted a longitudinal study of the interplay between English language proficiency, L1 educational experience, and a range of social-contextual factors and outcomes for international students attending 14 universities across the U.S. Below we review our research questions, delineate limitations, and provide next steps for the field.

4.1 English language proficiency and outcomes

While investigating direct linkages between English language proficiency and outcomes, we identified a strong relationship between students’ pre-enrolment IELTS Overall Score and their English confidence across all three timepoints. Students with higher pre-enrolment IELTS scores also reported higher English confidence throughout their time at university. This is a key finding, as prior research has focused on English confidence or perceived English proficiency as a strength that propels international students to having better wellbeing, social engagement, and lower acculturative stress (Koo et al., 2021; Luo et al., 2019). However, in the current study, we did not identify direct linkages between pre-enrolment IELTS scores and students’ wellbeing, social engagement, or other outcomes. While we did identify indirect linkages of IELTS scores on domestic student social support and academic stress via English confidence (discussed further below), the lack of direct associations should not be surprising. Given that at least $69\%$ of the participants had been enrolled for four semesters or longer, they would have taken an IELTS test at minimum two years prior, and likely even longer ago. Over time at U.S. universities, students’ English proficiencies are likely to change at different rates and for different reasons-an important potential area of future inquiry.

4.2 English language proficiency, L1 educational experience, English confidence, and outcomes

We anticipated students’ social and university-contextual experiences and outcomes would be influenced not solely by their proficiency, but by their L1 educational experience and English confidence as well. As mentioned in the results, it became clear during our testing of longitudinal structural equation models that L1 educational experience best served as a covariate to be controlled for in this current sample, rather than a predictor. In essence, it was included in the model to share covariance with predictor variables rather than to test for direct effects (i.e., whether or not it influenced outcomes). Although we did not identify linkages with setting outcomes (e.g., belonging, loneliness) or psychological wellbeing, we did discover associations between English language proficiency, English confidence, and two setting social processes: academic stress and domestic student social support. With respect to academic stress, students’ pre-enrolment IELTS scores had an indirect effect on their Spring 2022 academic stress levels, which means that higher IELTS scores were associated with lower Spring 2022 academic stress by virtue of higher IELTS scores resulting in higher Spring 2021 English confidence. In turn, higher English confidence was associated with lower Spring 2021 academic stress, which was linked with lower academic stress in both Fall 2021 and Spring 2022.

This finding is unique with respect to the research literature, and provides evidence of predictive validity of IELTS for students’ adjustment years after the IELTS test was completed pre-enrolment and despite the unique language experiences each international student has at university. More research is needed to understand which key university processes and setting outcomes are impacted by international students’ academic stress.

Similarly, English language proficiency was linked with English confidence, which in turn was associated with domestic student social support. Although the indirect effect was approached but did not meet statistical significance (p = .056), we anticipate that future research might establish this linkage. Regardless, given findings of prior research linking domestic student social support with social engagement, belonging, and wellbeing (Brunsting et al., 2021; Luo et al., 2019 Van Horne et al., 2018), the potential for a multiple-years-old IELTS score to be linked with domestic student social support via English confidence provides another avenue for future studies of predictive validity of IELTS.

4.3 Social-contextual outcomes as linking mediator variables between English language proficiency and wellbeing

While we identified linkages between belonging, loneliness, and wellbeing, the data did not support relationships with setting process variables such as academic stress or domestic student social support. Although we anticipated English language proficiency to be associated with social engagement, it is possible that other factors (e.g., integrative motivation, language anxiety) are more salient (Gardner, 1985). Further, more theoretically distal relationships between English language proficiency and belonging or English confidence and wellbeing were not supported by the data.

It is also worth considering to what degree the language proficiency measured by the participant-reported test scores can predict language proficiency in the target language use domains of social engagement situations outside conventional academic encounters (e.g., building relationships with domestic students). The social situations that may affect students in areas like belonging and loneliness employ language registers that are markedly non-academic. Some elements of the IELTS tasks are related to informal, non-academic language use situations (e.g., Part 1 of the IELTS speaking test), and preparation for IELTS has been shown to have positive washback on students’ productive language use (e.g., Allen, 2016, Dang & Dang, 2021). Nevertheless, there are many areas of socialization and social emotional challenges where registers of language use differ meaningfully from academic language; proficiency in academic English may be a limited predictor of proficiency in other, interpersonal language use situations experienced by students.

When considering these outcomes, it is important to once again situate the findings within COVID-19 restrictions and general uncertainty. It is possible that, during this unique time, linkages previously identified in the research-such as English confidence and wellbeing (Luo et al., 2019)-were overwhelmed by the impact of COVID-19. Given the opportunity for the virus and subsequent restrictions to impact academic processes and daily interactions, which served as the mediating variables of social interaction, student social support, and academic stress, it is quite possible that these hypothesized relationships would be supported by a study less impacted by COVID-19 or other pandemics

4.4 Limitations and future directions

The current study is not without limitations. First, the response rate of participants did not reach the recommendation of 30% for survey studies (Dillman, 2007). COVID-19 may have reduced response rate, as other studies have documented lower response rates than usual during this time (McIllece, 2021). While our response rate was 8.2%, our sample data was not skewed and item averages were similar to past studies with 30% response rates (e.g., Brunsting et al., 2019).

Second, one of our key measures, psychological wellbeing did not demonstrate good model fit as it had in past studies (e.g., Brunsting et al., 2021). It is possible that psychological wellbeing was more complex during COVID-19, or it may be that items that were designed for U.S. citizens in the 1980s are not as accessible for today’s international students. Third, it would have been ideal to have a recent IELTS score to represent students’ English language proficiency rather than a self-report of their IELTS test that may have been taken four or more years prior to the study. As students’ experiences vary, their trajectories of proficiency also change-we encourage future researchers to consider how to obtain objective and current measures of proficiency (i.e., incentivising current students to take an IELTS test as part of the study) for more immediacy in prediction of other variables. Even with this limitation, the study results linking international students’ pre-enrolment IELTS scores with English confidence, domestic student social support, and academic stress during their time at university provides strong initial evidence for the predictive validity of IELTS, despite these outcomes not being part of its intended use. Fourth, it would have been ideal to have generated a more encompassing measure of English confidence to extend to more aspects of English use (e.g., listening). However, we maintained items from an existing and validated scale to ensure strong model fit. We recommend future research in this area to ensure English confidence is measured holistically.

As mentioned previously, COVID-19 may have depressed response rates; it is also likely that other factors which we were unable to control for may have influenced participant experiences. However, by Spring 2021, there were fewer city-wide restrictions and almost no city or state lockdowns (Funk et al., 2022). The majority of participants were taking in-person or hybrid classes by this time. In some ways, the study design protected against regional challenges from COVID-19, as we had a large sample across 14 universities and we used structural equation modeling which parcels out measurement error from the relationships examined.

With respect to next steps for research, we make three recommendations.

First, large-scale survey research is in need of rethinking to ensure higher response rates. Hopefully COVID-19 is a primary factor, but it may be that the increased press at U.S. universities for assessment has created permanent survey fatigue for students (Fass-Holmes, 2022). We encourage researchers to consider smaller-scale quantitative studies as well as qualitative research to replicate and extend findings linking IELTS scores with social and engagement outcomes.

Second, we recommend increased collaboration between research communities examining language learning, identity development, and social engagement of international students. International students experience their sojourn in a multiplicity of ways, and integrations of theories across disciplines are needed to provide more nuanced insights into the interconnected factors undergirding international students’ growth and development abroad.

Thirdly, to better understand the degree to which IELTS can serve as a predictor of students’ social engagement, it would be ideal to test students immediately before university entrance or to intentionally seek a sample of students who begin university with the minimum of time after receiving their IELTS scores. Further, it may be worth examining which IELTS subscores (i.e., listening, reading, speaking, writing) are most strongly linked with international students’ social-emotional outcomes at U.S. universities. And does this relationship hold or is it stronger for international students at universities in Australia, the United Kingdom, Canada, New Zealand, and other countries where English is the language of academic instruction and social life?

4.5 Conclusion and implications for practitioners

In a national study of international students at 14 universities across the U.S., the data revealed initial evidence of linkages between international students’ English language proficiency (e.g., IELTS Overall Score), their English confidence, academic stress, and domestic student social support such that students reporting higher pre-enrolment IELTS scores experienced higher English confidence and domestic student social support and lower academic stress.

Although IELTS is designed to assess readiness for university academic study in English, the findings from the current study reveal initial linkages with other important factors for university social engagement. If these results were to be replicated in future research, we would recommend that admissions offices consider adapting their process to include IELTS scores or other sources of information which have a demonstrated link with both university academic and social engagement outcomes.

References

• Allen, D. (2016). Investigating washback to the learner from the IELTS test in the Japanese tertiary context. Language Testing in Asia, 6(1), 7. https://doi.org/10.1186/s40468-016-0030-z
• Bhabha, H. K. (1994). The location of culture. Routledge.
• Block, D. (2007). The rise of identity in SLA research, post Firth and Wagner (1997). The Modern Language Journal, 91, 863-876.
• Brunsting, N. C., Zachry, C., & Takeuchi, R. (2018). Predictors of undergraduate international student psychosocial adjustment to U.S. universities: A systematic review from 2009-2018. International Journal of Intercultural Relations, 66, 22-33. https://doi.org/10.1016/j.ijintrel.2018.06.002
• Brunsting, N. C., Mischinski, M. M., Wu, W., Tevis, T., Takeuchi, R., He, Y., Zheng, Y., & Coverdell, T. (2019). International students’ social outcomes, educational status, and country of high school graduation. Journal of Studies in International Education, 23, 589-606. https://doi.org/10.1177/1028315318825362
• Brunsting, N. C., Zachry, C. E., Liu, J., Bryant, R., Fang, X., Wu, S., & Luo, X., (2021). Sources of perceived social support, social-emotional experiences, and psychological well-being of international students. Journal of Experimental Education, 89, 95-111. https://doi.org/10.1080/00220973.2019.1639598
• Brunsting, N. C., Katsumoto, S., Lee, H., Bingham, W. P. (2022). A (more) intersectional approach to quantitative research for international students: Gender, graduate level, and region of origin. Manuscript under review.
• Dang, C. N., & Dang, T. N. Y. (2021). The Predictive Validity of the IELTS Test and Contribution of IELTS Preparation Courses to International Students’ Subsequent Academic Study: Insights from Vietnamese International Students in the UK. RELC Journal. https://doi.org/10.1177/0033688220985533
• Du, Y., & Wei, M. (2015). Acculturation, enculturation, social connectedness, and subjective well-being among Chinese international students. The Counseling Psychologist, 43(2), 299-325.
• Fass-Holmes, B. (2022). Survey Fatigue-Literature Search and Analysis of Implications for Student Affairs Policies and Practices. Journal of Interdisciplinary Studies in Education, 11(1), 56-73. Retrieved from: www.ojed.org/index.php/jise/article/view/3262
• Fraser, C. A. (2007). Reading rate in L1 Mandarin Chinese and L2 English across five reading tasks. The Modern Language Journal, 91(3), 372-394.
• https://doi.org/10.1111/j.1540-4781.2007.00587.x
• Funk, C., Tyson, A., Pasquini, G., & Spencer, A. (2022). Americans reflect on nation’s COVID-19 response. Pew Research Center. https://www.pewresearch.org/science/2022/07/07/americans-reflect-on-nations-covid-19-response/
• Gardner, R. C. (1985). Social psychology and second language learning: The role of attitudes and motivation. Edward Arnold.
• Glass, C. R., & Westmont, C. M. (2014). Comparative effects of belongingness on the academic success and cross-cultural interactions of domestic and international students. International Journal of Intercultural Relations, 38, 106-119.
• Hirai, R., Frazier, P., & Syed, M. (2015). Psychological and sociocultural adjustment of first-year international students: Trajectories and predictors. Journal of Counseling Psychology, 62(3), 438.
• Hu, L. T., & Bentler, P. M. (1999). Cutoff criteria for fit indexes in covariance structure analysis: Conventional criteria versus new alternatives. Structural Equation Modeling: a Multidisciplinary Journal, 6(1), 1-55.
• Kiang, L., Brunsting, N. C., Zachry, C., He, Y., Takeuchi, R., & Tevis, T. (2021). Identity fusion of international students with different social groups and well-being outcomes: A longitudinal study. Journal of Studies in International Education, 25, 524-545. https://doi.org/10.1177/1028315320932320
• Kline, R. B. (2011). Convergence of structural equation modeling and multilevel modeling. Guilford.
• Kohn, J. P., & Frazer, G. H. (1986). An academic stress scale: Identification and rated importance of academic stressors. Psychological Reports, 59(2), 415-426.
• Koo, K., Baker, I., & Yoon, J. (2021). The First Year of Acculturation: A Longitudinal Study on Acculturative Stress and Adjustment Among First-Year International College Students. Journal of International Students, 11(2), 278-298. https://doi.org/10.32674/jis.v11i2.1726
• Kramsch, C. (2009). Third culture and language education in L. Wei & V. Cook (Eds.), Contemporary Applied Linguistics: Language Teaching and Learning, Vol. 1 (pp. 233-254). https://doi.org/10.5040/9781474211789.ch-011
• Lantolf, J. P. (Ed.). (2000). Sociocultural theory and second language learning (Vol. 78, No. 4). Oxford University Press.
• Little, T. D. (2013). Longitudinal structural equation modeling. Guilford.
• Longerbeam, S. D., DeStefano, T. J., & Lixin, Y. (2013). “We cannot reach them”: Chinese undergraduate student perceptions of the U.S. campus climate. Journal of Student Affairs Research and Practice, 50, 326-344.
• Luo, Z., Wu, S., Fang, X., & Brunsting, N. C. (2019). International students’ perceived language competence, domestic student support, and psychological well-being at a U.S. university. Journal of International Students, 9(4), 954-971.
• McIllece, J. J. (2020). COVID-19 and the Current Population Survey: Response Rates and Estimation Bias. US Bureau of Labor Statistics, 20.
• Muthén, L. K., & Muthén, B. O. (2020). Mplus user’s guide (9th ed.). Muthén & Muthén.
• Muyan, M., & Chang, E. C. (2015). Perfectionism as a predictor of suicidal risk in Turkish college students: Does loneliness contribute to further risk? Cognitive Therapy and Research, 39(6), 776-784.
• Ryff, C. D. (1989). Happiness is everything, or is it? Explorations on the meaning of psychological well-being. Journal of Personality and Social Psychology, 57(6), 1069.
• Schoepp, K. (2018). Predictive validity of the IELTS in an English as a medium of instruction environment. Higher Education Quarterly, 72(4), 271-285.
• Tseng, V., & Seidman, E. (2007). A systems framework for understanding social settings. American Journal of Community Psychology, 39(3), 217-228.
• Van Horne, S., Lin, S., Anson, M., & Jacobson, W. (2018). Engagement, satisfaction, and belonging of international undergraduates at U.S. research universities. Journal of International Students, 8, 351-374.
• Vygotsky, L. (1978). Interaction between learning and development. Readings on the Development of Children, 23(3), 34-41.
• Wang, K. T., Wei, M., & Chen, H. H. (2015). Social factors in cross-national adjustment: Subjective well-being trajectories among Chinese international students. The Counseling Psychologist, 43(2), 272-298.
• Yamashita, J. (2002). Mutual compensation between L1 reading ability and L2 language proficiency in L2 reading comprehension. Journal of Research in Reading, 25(1), 81-95.

Appendix A: Measurement invariance testing across time of outcomes variables

Note. CFI = Confirmatory Factor Index; RMSEA = root mean square error of approximation.