A comparative study of IELTS to ascertain its viability for the Malaysia private secondary school market

Abstract

This study sought to determine the suitability and viability of introducing the IELTS test to private secondary schools in Malaysia and to determine the most appropriate module for this cohort. The study was conducted with 151 randomly selected students from private colleges in the Klang Valley, Kuala Lumpur, Malaysia.

The results from the comparative study of the statistical data showed that secondary school students were able to perform when given either the Academic or General Training module, although it appeared that the candidates with high English proficiency had a tendency to do better in the Academic test. Analysis of the data also strongly suggested that the preparation for the national English language test, SPM 1119 English prepared and equipped the candidates with a good foundation in language skills to do either IELTS module.

1.0 Introduction to English as a foreign language in Malaysia

In the 1980s and early 1990s in Malaysia, school examinations offered by the University of Cambridge Local Examinations Syndicate (UCLES) ¹ had a firm hold in the annals of English language testing in Malaysia. Until it was phased out in 1996, the Cambridge O Level 1119 English exam successfully penetrated and monopolised the English language testing market completely. To illustrate, the upper secondary school students during the time span mentioned took the Cambridge O Level 1119 English exam automatically with the government achievement exam, Sijil Pelajaran Malaysia (SPM) - en masse. The Cambridge O Level 1119 English exam was conducted in a majority of schools throughout Malaysia. The exam was generally viewed by all quarters to be a requisite language certificate that not only enhanced one’s academic qualifications, but was also awarded by a respected UK organisation.

The implications behind O level 1119 English’s immense popularity with the Malaysian upper secondary school bracket lie mainly behind the Malaysian public’s reservations about the quality of the local English paper and how the standards have compared unfavourably with foreign English language certificates. The recent modification of the English paper resulting in the advent of SPM 1119 English in 1997 induced a perceptible downward shift in confidence in the local test. A paper presented at the May 2000 Malaysian English Language Teachers Association Conference, entitled ‘Making Washback Work - A Study of the Initial Impact of the SPM 1119 English Exam’, noted that whilst the new examination seemed to succeed in bringing about positive washback, other aspects such as the marking procedure and the lack of transparency concerning the assigning of two grades for one examination were directly responsible for the negative washback which will defeat the good intentions behind the introduction of the paper in 1997. The paper indicated there seemed to be a certain level of disenchantment with the SPM 1119 English paper.

In addition, the public perception is that attainment of language certificates awarded by international testing establishments with a history for excellence in English language testing would add a certain prestige to the students’ academic achievements and accomplishments. Therefore, with the withdrawal of the Cambridge O Level 1119 English, demand for a language test with a high level of international validation from institutions in both English speaking countries and non-English speaking countries had grown in magnitude. In fact, a host of other Cambridge-produced language proficiency tests have come to the fore, for example FCE and CAE (produced by Cambridge ESOL), in an attempt to fill the void left by Cambridge O Levels 1119 English as there appeared to be a market for language exams with international acceptability. IDP Education Australia Kuala Lumpur maintains that IELTS (produced by Cambridge ESOL in partnership with IDP Education Australia and the British Council) has the standards and also the flexibility to cater to this new market segment effectively and successfully.

2.0 Background

As illustrated by the increasing IELTS candidature world-wide and nation-wide, there appears to be a soaring demand for an internationally recognised and reliable English language test to gauge the proficiency of non-native speakers of English. In an attempt to maintain and supplement this increase, IDP Education Australia Kuala Lumpur hopes to introduce the IELTS test to a brand new market segment - the secondary school. Hence, the aims of the research were formulated to investigate the appropriacy of introducing the IELTS test to secondary schools across Malaysia and more importantly to determine which module, Academic or General Training, would be the most appropriate for candidates in this particular age group of 15-17 years.

In the interest of expanding the test to this relatively new user group, the linguistic competence of the cohort was measured using their performance in the SPM 1119 English paper, the national achievement language test. Their linguistic ability was indicated by their SPM 1119 English grade.

Given that this user group has the potential of becoming a significant presence in the IELTS test-taking population, the impact study to determine the most appropriate IELTS module (Academic or General Training) for this user group was considered imperative.

As the user group falls in the age range of 15 to 17, candidates have to make a choice whether to take the Academic or the General Training (GT) module. Generally, the Academic module is taken by candidates who want to further their studies at the tertiary level as undergraduate students, while the General Training module, is taken by candidates who are going to English-speaking countries to complete their secondary school education (The IELTS Handbook 2000). It is felt that the ambiguity and conflict resulting from the age factor of this user group made it difficult to make a definite recommendation on the module they should take as both descriptions given about the modules fit this user group precisely.

3.0 Aims

The primary aim of the research study was to investigate the viability of introducing IELTS to the upper secondary school market. The researchers felt that the general aims outlined in their original proposal had to be reworked as any analysis of the assessment criteria to investigate the issue of equivalence between the SPM 1119 English test and either module of the IELTS test was not possible due to the confidentiality restrictions of both tests. Therefore, the decision was made to refocus the general aims of the research to study the suitability of introducing IELTS to the target market segment.

These general aims are as follows.

• To investigate the suitability of introducing IELTS to the upper secondary school sector
• To determine the most appropriate module for use in the upper secondary school sector
• To investigate whether there is some common ground between SPM 1119 English and the IELTS modules.

4.0 Detailed research aims

To thoroughly investigate the questions posed, the aims were dissected to show comparisons in linguistic performance between SPM 1119 English and Academic and GT scores at various levels. The research needed to show the following.

• Whether there is any significant correlation between SPM 1119 English scores and Academic IELTS scores.
• Whether there is any significant correlation between SPM 1119 English scores and GT IELTS scores.
• Whether there is any significant difference between scores attained for Academic IELTS and GT IELTS.
• Whether there is any significant difference between scores attained for Academic IELTS and GT IELTS at various proficiency levels.
• Whether there is any significant difference between scores attained by candidates taking a single module compared to candidates taking both modules.
• Whether there is any significant difference in the Reading test scores attained by candidates handling Academic IELTS compared to those handling GT IELTS.
• Whether there is significant difference in the Reading test scores attained by candidates handling a single module compared to candidates handling both modules
• Whether there is any significant difference in the Writing test scores attained by candidates handling Academic IELTS compared to those handling GT IELTS.
• Whether there is significant difference in the Writing test scores attained by candidates handling a single module compared to candidates handling both modules.

5.0 Methodology

The sample population was constructed using the defining parameters of date of birth, school type, SPM 1119 English results and prior test-taking history inclusive of IELTS, TOEFL and the Malaysian University Entrance Test (MUET). Originally, it had been decided that random sampling was to take place to select the population. However, randomisation in the process of selecting students for the research was not possible as obtaining permission from the national schools and the Education Ministry to conduct the IELTS tests was extremely time-consuming and difficult due to school regulations, thus limiting the sampling size available from each school. In order to achieve the target population of 150, IDP extended an invitation to five major private colleges in Kuala Lumpur to also participate in the research.

The original criterion for inclusion in the sample population was that the candidate must have taken the national exam Sijil Pelajaran Malaysia (SPM) in 2000; a results slip bearing the date of the SPM test to be used as proof. Candidates who had just enrolled into matriculation programs in these colleges were selected. In addition, a second criterion was added requiring the students to attain a certain grade in the SPM 1119 English paper. Specifically, the grades of A1-C6 were needed. Therefore, the sample population represented predetermined groups that had satisfied the proficiency level requirements of A1-C6. By using these criteria, accuracy in constructing the desired sample population was ensured.

The sample population was drawn from candidates who did their IELTS tests in five major private institutions in March, April, May and June. Once the sample population had been constructed, it was analysed to establish a profile of the group according to their performance in the SPM 1119 English paper and the IELTS module or modules they were selected to do.

The population was divided into three proficiency groups based on their performance in their SPM 1119 English paper, namely, the High proficiency group of A1-A2, the Average proficiency group of B3-B4 and the Low proficiency group of C5-C6. The population was further divided into three groups:

• 50 candidates selected to undertake the General Training Module only
• 50 candidates selected to undertake the Academic module only
• 50 candidates selected to do both the General Training and Academic modules.

In addition to this analysis, the sample population was surveyed using a questionnaire to confirm the validity of the candidates through the provision of background information. The questionnaire was also designed to gauge the candidates’ perceptions of the SPM 1119 English paper and their level of preparation for the said paper. The questionnaire also briefly surveyed the candidates’ perceptions of the correlation between their preparation for the SPM 1119 English paper and their performance in the pre-selected IELTS module(s). The questionnaires were distributed on the day of the IELTS test after the candidates had completed all tests.

The analysis was augmented with a focus interview of the whole group clarifying views and attitudes towards the IELTS modules in terms of their familiarity with the modules and their perceptions of the national test SPM 1119 English.

The design of the questionnaire took into consideration the following issues.

• Verification of the information provided by the cohort.
• Design of the SPM 1119 English test.
• Candidates’ perception of preparation and difficulty in terms of the SPM 1119 English test.
• Candidates’ familiarity with IELTS GT and/or Academic modules after going through the SPM 1119 English curriculum.
• Candidates’ perception of the difficulty of the IELTS modules.
• Reasons for taking the IELTS test and the cohort’s possible future work and study destinations.
• Perceptions of the IELTS modules as seen from different levels of proficiency.
• Perceptions of the suitability of IELTS for the target market segment: the upper secondary school sector.

The initial phase of the project required a pilot test to be conducted on the three proficiency groups to test the validity and linguistic phrasing of the items asked. 28 candidates were selected for the pilot study. The questionnaire was given to them after the IELTS Speaking test. Once everyone had filled in the questionnaire, a focus interview about the questionnaire was done to ensure that honest reflection could result from a fresh recollection of the questionnaire.

6.0 The SPM 1119 English test

In order to analyse whether the preparation of the test takers for the SPM 1119 English test would have a washback effect that would benefit IELTS test takers, an overview of the test is necessary. The test consists of 3 papers and the format is briefly outlined in the following table (Dass 2000).

Table 1: SPM 1119 English Outline

7.0 Research findings

The findings of the research are dependent on the analysis and interpretation of two sources of data:

• the comparative analysis of SPM 1119 English scores and the various categories of IELTS test scores, and
• the questionnaire.

The findings are discussed in stages following the above sequence.

7.1 The comparative analysis of SPM 1119 English and IELTS test scores

The comparative analysis of SPM 1119 English and IELTS test scores is made in reference to identifying the better module for the secondary school sector. The analysis of the data collected follows an item for item structure according to the sequence suggested in Detailed Research Aims (Section 4.0). A range of statistical tests was used in the analysis of the SPM 1119 English test scores vis-à-vis the IELTS test scores.

7.1.1 Correlation between tests

Pearson’s Correlation Coefficients were used to determine the existence of any significant correlation between SPM 1119 English test scores and overall IELTS test scores as specified in the aims. These correlations were specifically conducted in two parts:

• a comparison of candidate scores for Academic IELTS and SPM 1119 English, and
• a comparison of candidate scores for GT IELTS and SPM 1119 English.

Table 2 shows that regardless of the cohort tested, the correlation between SPM 1119 English and IELTS overall scores are significant at -1% to 0%. This shows that there are linear relationships between SPM 1119 English and IELTS overall scores for all categories. As seen from Table 2, all correlations are negative because the IELTS test scores are in an ascending order while the SPM 1119 English test scores are in a descending order. In other words, an A1 in the SPM 1119 English shows

the best performance while a Band 1 in the IELTS test shows the worst performance. As the correlation between SPM 1119 English scores and IELTS scores for all groups are significant, it indicates that candidates with higher grades in SPM 1119 English perform better in the IELTS test.

Table 2: Correlation coefficients of SPM 1119 English and IELTS overall scores

7.1.2 Differences between IELTS module scores

A t-test was used to evaluate whether there was any significant difference between the scores attained by the cohort undertaking the Academic IELTS module vis-à-vis the GT module. Table 3 shows the probability value of the t-test as well as the mean and standard deviation of the two groups. The test was performed assuming that the two tests are equal in terms of grading and not influenced by human factors.

Although Table 3 shows the mean score for the Academic module to be higher than the mean score for the General Training module at 6.20 compared to 5.90, the difference in the means of the two tests was not of significance at the p-value of 0.125. This indicates that the candidates’ performances in either module would be similar and not significantly different.

Table 3: The mean and standard deviation of the IELTS overall scores

7.1.3 IELTS scores at different SPM 1119 proficiency levels

A t-test was used to evaluate whether there was any significant difference between the scores attained by the cohort undertaking the Academic IELTS module vis-à-vis the GT IELTS module at the 3 levels of proficiency: High (A1-A2), Average (B3-B4) and Low (C5-C6).

Table 4 shows the mean and standard deviation of the IELTS scores for the Academic and General Training module grouped according to language proficiency. When comparing the Academic scores with the General Training scores, the mean scores indicate that for candidates with average and low levels of proficiency (B3-C6), there is no significant difference in the performance of candidates who sat for either module. This is shown by the p-value of 0.518 for the B3-B4 cohort and 0.484 for the C5-C6 cohort.

However, there is significant difference between the mean scores for the Academic test and the General Training test in the highest proficiency level of A1-A2 at 7.15 and 6.38 respectively.

The available source PDF skips printed pages 94-95 after this point and resumes at section 7.1.7.

7.1.7 Writing test scores

The means, standard deviations and p-values of the Writing test are in Table 8, which compares the performances in the Academic module and the General Training module. The mean scores for the writing test do not differ significantly as the respective mean scores for candidates who did the Academic module when compared to those who did the General Training module stand at 5.63 to 5.54. The p-value of 0.614 further indicates that the difference in performance on either test is not of any significant concern. The choice of module would not yield a significantly different result.

Table 8: The mean and standard deviation of the IELTS Writing scores

7.1.8 Writing test scores for multiple modules

A t-test was employed to evaluate whether there was any significant difference between writing scores attained by candidates taking a single module compared to candidates taking both modules.

The means, standard deviations and p-values of the test conducted are in Table 9. The General Training Writing test results show that the comparative means for Writing of candidates doing a single module compared with those taking both modules stand at 5.44 and 5.61 respectively. However, as the t-test produced a p-value of 0.593, the difference is not considered significant.

Similarly the Academic test means produced by candidates doing a single module compared to candidates doing both modules stand at 5.63 and 5.61 respectively. As the p-value produced at 0.940 is higher than the fixed level of observed significance of .05, it can be concluded that there is no significant difference in the performances of candidates who do only one module compared to two modules.

In short, from the results shown on Table 9, we come to the conclusion that the differences in means for both cases (Academic and General Training) are not statistically different.

Table 9: The mean and standard deviation of the IELTS Writing scores for candidates taking one or both modules

7.2 Analysis of the questionnaire

The questionnaire aims to provide clarity to the following two issues:

• the viability of introducing IELTS to an upper secondary school market, and
• the existence of some common ground between SPM 1119 English and the IELTS modules.

The questions and corresponding responses are represented in a statistical format together with the researchers’ reflections of the responses.

7.2.1 Verification of the sample population

Four questions were included to verify the information provided at the onset of the selection procedure. They are as follows.

• Did you take the SPM exam in November 2000?
• Did you take the SPM 1119 English paper in November 2000?
• How old were you when you took the SPM exam?
• Have you taken any other language exams apart from SPM 1119 English?

7.2.2 Perceptions of candidates’ difficulty of the SPM 1119 English test

The SPM 1119 English paper consists of three parts and respondents were asked to rate the difficulty of each part. The exact prompt given was: “The SPM 1119 English paper consists of three parts and we would like to know your opinion of each part”. (Responses may be affected by the degree of English language proficiency among the candidates.)

The difficulty rating used were:

• very difficult
• difficult
• neutral
• fairly difficult
• not difficult at all.

To simplify the process of assessment, the levels of difficulty were further compressed into three categories, ie. difficult, neutral and not difficult.

Table 10 (following page) only shows percentages of candidates in the highest level of difficulty, ie difficult across the three categories of proficiency, namely High, Average and Low. The data depicts the cohorts general perception of how difficult they found the tasks in SPM 1119 English. Table 10 shows the kinds of tasks candidates were tested on specifically and their perceptions of difficulty of the tasks tested.

Perceived levels of difficulty are significantly higher for Paper 2: Summary and Paper 2: Continuous Writing, compared to the other sections for all proficiency levels. The respective p-values for Paper 2: Summary and Paper 2: Continuous Writing at 0.008 and 0.014 support this reflection.

For all three proficiency groups, the perceived levels of difficulty for all sections excluding Paper 2: Summary and Continuous Writing were not significant. The p-values recorded were higher than the 5% significant level. In other words, the levels of difficulty in the other sections did not appear to be independent of the cohort’s respective levels of proficiency. Therefore, we can surmise that there is no significant relationship between linguistic ability and perception of task difficulty for these sections. In other words, as the p-value shows no significance, the level of proficiency has no impact or effect on

the perceived difficulty of the task. To elaborate, proficient and weak users of English would have roughly the same perceptions of difficulty for these sections.

Table 10: Levels of difficulty in the SPM 1119 English paper

7.2.3 Perceptions of preparation and difficulty in terms of the SPM 1119 English test

Respondents were asked to rate their level of preparation compared to the grades attained in the SPM 1119 English. The exact question was: “How prepared were you to handle the tasks given in the SPM 1119 English paper?”.

The candidates were also asked about their degree of preparation in handling the tasks given in the SPM 1119 English paper. Graph 1(next page) shows the distribution of the levels of preparation; 72% of the candidates perceived that they were prepared to handle the tasks tested in the SPM 1119 English paper. Candidates widely believed that their level of preparation would normally correspond with the results acquired. For instance, as they attained good results in their SPM 1119 English, they must have prepared well. However, since this question is a perception question, then the answer given might not reflect the actual level of preparation of a student.

Graph 1: Distribution of the levels of preparation for SPM 1119 English paper

Further assessment of the correlation between the candidates with different levels of proficiency and their perceptions of preparation is depicted in Table 11. The table shows the level of preparation against the actual grade obtained in SPM 1119 English. It must be emphasised that the perceived level of preparation does not necessarily translate into the candidates’ actual level of preparation. The perceived level of preparation is divided into two categories:

• whether the candidates felt they were prepared to handle the tasks tested in SPM 1119 English
• whether the candidates felt they were unprepared to handle the tasks tested in SPM 1119 English.

The Chi-square test conducted produced a p-value of 0.001 indicating that the level of preparation does not appear to be independent of the language grades attained. In other words, the perception of being prepared would, to a significant extent, correspond with the grades attained.

In Table 11, the majority of candidates who perceived that they were prepared fell into the high and average proficiency groups. For instance, the percentages of 85.9% of High proficiency candidates and 56.1% of Average proficiency candidates represent a significant proportion of candidates whose perception of their level of preparation corresponded with the grade received. Conversely, there was a proportion of lower proficiency candidates, 73.3%, whose perception of their preparation did not translate into a good grade.

Table 11: Level of preparation among candidates

7.2.4 Perceptions of familiarity

Respondents were asked to indicate whether the question types in the IELTS modules (either GT or Academic) were familiar to them based on their SPM 1119 English preparation. The question was: “After completing the upper secondary school English curriculum (in preparation for the SPM 1119 English paper), how familiar were you with the question types used in the IELTS Academic/General Training modules?”.

Table 12 shows the percentage responses from 98 candidates who did Academic IELTS, and 103 who did General Training IELTS. The table shows the percentage of the candidates’ familiarity toward the four IELTS components of Listening, Reading, Writing and Speaking for the Academic and General Training modules. The respondents were asked to rate based on these familiarity ratings:

• very familiar
• quite familiar
• neutral
• not very familiar
• not familiar at all.

Table 12 aims to evaluate:

• the candidates’ perceived familiarity of the skills tested in the four IELTS Academic components after completing the upper secondary school English curriculum in preparation for SPM 1119 English.
• the candidates’ perceived familiarity of the skills tested in the four IELTS General Training components after completing the upper secondary school English curriculum in preparation for SPM 1119 English.
• the differences in the perception of familiarity between the Academic and General Training modules compared to SPM 1119 English.

As the Listening and Speaking components were identical for both modules, similar percentages of familiarity were obtained for both modules. The results show that in general, most candidates were least familiar with the Listening component with similar percentages of 37.8% and 37.9% claiming familiarity after doing the Academic test and the General Training test. Similarly, only 36.9% and 40.8% of the candidates who did the General Training and Academic modules respectively claimed that they were familiar with the Speaking module.

The main difference between the Academic and General Training modules lies in the Reading and Writing components. It was found that candidates who sat for the General Training module perceived that they were more familiar with the Reading and Writing tests compared to candidates who took Academic Reading and Writing. The respective percentages of 45.6% and 57.3% for GT Reading and Writing were marginally higher than the percentages garnered by the Academic cohort for Reading and Writing at 40.8% and 45.9%.

In short, from the results shown, we can conclude that less than 50% of candidates said they were familiar with the tests, except for the Writing test in IELTS General Training. However, the degree of familiarity of one module over the other was not so high as to warrant concern. In other words, the percentages of candidates who answered that they were familiar with the tests in either module are not significantly different.

Table 12: Percentage of candidates familiar with the IELTS tests

7.2.5 Perceptions of preparation, similarity and level of proficiency

Table 13 records the respondents’ perceptions on:

• whether the preparation for SPM 1119 English prepared them for the IELTS modules
• the similarity in skills and task types tested in IELTS General Training or Academic compared to SPM 1119 English, and
• the level of language proficiency required to do either IELTS module.

The table shows the percentage responses from all candidates. More than 70% of the candidates surveyed perceived that their preparation for SPM 1119 English prepared them to sit for either the Academic module or the General Training Module.

Respondents were asked to indicate whether the skills and task types tested in the IELTS General Training or Academic tests were similar to those tested in SPM 1119 English. The respondents were asked to rate based on these similarity ratings:

• yes
• no
• some similarities
• not sure.

73.5% of candidates who did the Academic module responded that there were similarities between the Academic IELTS and SPM 1119 English while 65.1% of candidates who took the General Training module perceived similarities between this module and SPM 1119 English.

Table 13 (next page) shows that a large number of candidates - 77.5% for Academic and 80.5% for General Training - believed that the IELTS test required a higher level of language proficiency than the SPM 1119 English. Although most candidates perceived that there were similarities between the two IELTS tests and SPM 1119 English, they believed that IELTS required a higher English proficiency level than their present levels as indicated by SPM 1119.

Table 13: Candidate perceptions of preparation, similarities and proficiency

7.2.6 Reasons for taking IELTS

Respondents were given a choice of the below seven popular reasons for taking the IELTS test in the Malaysian context, and asked to rate those that applied to them.

• The IELTS test would enhance and complement my existing language qualification.
• The IELTS test gives a more comprehensive evaluation of my communication skills.
• The IELTS test is prestigious.
• The IELTS test is internationally validated by institutions in both English speaking and non-English speaking countries.
• The IELTS test is accepted by the educational establishment of my choice.
• The IELTS test is a prerequisite as I intend to further my studies in an English-speaking country and a local language certificate does not meet the language requirements stated.
• Other reasons.

The most popular reason, with a percentage of 83.44, was that IELTS was widely recognised and had a high acceptance value among foreign English-speaking universities. Following closely at 82.78% was that IELTS provided a comprehensive evaluation of one’s communication skills. In addition to these two reasons 78.81% of the cohort chose IELTS because the test would enhance and complement the candidates’ existing language qualifications, making it the third most popular reason.

Graph 2: Reasons for taking IELTS

7.2.7 Perceptions of suitability

Respondents were asked whether they felt the IELTS exam was suitable for the upper-secondary school cohort. Table 14 shows the responses from 151 candidates.

Table 14: Perception of IELTS as a suitable test for upper secondary school students

50.9% responded positively to the IELTS test while 19.8% felt that the test was not suitable for them. 29.1% were neutral in their responses towards the test. Overall, the feedback provided by the candidates towards IELTS was encouraging.

7.2.8 Post-secondary school pathways

Respondents were asked to express what they were most likely to do after finishing their post-secondary school education. Six options were provided:

• full-time work
• study in a local government university
• study in a local private university
• study locally in a private college offering foreign tertiary programs
• study overseas in a foreign university in an English-speaking country
• matriculation in a college.

Table 15 (next page) shows what candidates are most likely to do after finishing their upper secondary school education. 61.5% will most likely study locally in a private college offering foreign tertiary programs, while 21.8% will study overseas in a foreign university and only 8.6% will do matriculation in a local private college. These figures indicate that most candidates will eventually need to sit for the IELTS test as it will be required by all these various institutions.

Table 15: Post secondary school destinations

8.0 Summary of the analysed data

The comparative analysis of SPM 1119 English and IELTS test scores illustrated that:

• there is correlation between SPM 1119 English scores and IELTS Academic scores.
• there is correlation between SPM 1119 English scores and IELTS GT scores.
• there is correlation between SPM 1119 English and IELTS Academic and GT scores.

The performance of candidates who did the IELTS Academic test compared to those who did the IELTS General Training test illustrated that:

• there is no significant difference in the scores
• there is significant difference in the scores of high proficiency candidates but not in the scores of candidates with other levels of proficiency.

The comparison between candidates who sat for a single IELTS module and those who completed both modules illustrated that:

• there is no significant difference between the scores attained by candidates who did the Academic module and candidates who did both the Academic and the GT module
• there is no significant difference between the scores attained by candidates who did the General Training module and candidates who did both the Academic and the GT module.

The comparison of performances between Academic Reading and General Training Reading illustrated that there is significant difference in the scores.

The comparison of reading scores between a single module and both modules illustrated that:

• there is no significant difference in the Academic Reading scores between candidates who took either a single module or both IELTS modules
• there is no significant difference in the General Training Reading scores between candidates who took either a single module or both IELTS modules.

The comparison of performances between Academic Writing and General Training Writing illustrated that there is no significant difference in the scores.

The comparison of writing scores between a single module and both modules illustrated that:

• there is no significant difference in the Academic Writing scores between candidates who took either a single module or both IELTS modules
• there is no significant difference in the General Training Writing scores between candidates who took either a single module or both IELTS modules.

9.0 Further analysis of the research findings

In our study we see that our cohort can complete either IELTS test as the SPM 1119 English curriculum provides a good basis and foundation for students. The SPM 1119 English could be described as a general purpose standardised assessment. Widdowson (1983) suggests that in general purpose language courses (and tests), a distinction is made between aims, the eventual target behaviours of the learners, and objectives, the pedagogical constructs which will enable the learners to achieve the behavioural targets. Hence, the goal of general purpose language courses (and tests) is to provide learners with an ability to solve independently the profusion of communication problems they will encounter when they leave the language learning classroom or when they face test taking situations. The impact the test has on the program of instruction is called washback. According to McNamara (1996), the washback effect is particularly evident in large-scale public tests which have become the focus of teaching programs.

Therefore, in analysing the SPM 1119 English test and the washback effect that it has on the curriculum, we would be able to view:

• the range of skills learnt in preparation for the SPM 1119 English test
• whether the range of skills employed and the tasks encountered give candidates adequate preparation to attempt IELTS, and
• whether the SPM 1119 English test was a good predictor of readiness and ability to do the IELTS test well.

The Ministry of Education establishes stages in learning by writing up different objectives for each level in the school curriculum. In Malaysia, they are expressed at two levels:

• terminal objectives
• unit or skills objectives.

As SPM 1119 English is modelled after traditional language tests which test skills in language, its curriculum offers a framework to build language knowledge and ability encouraging communicative competence (Hymes 1972). Communicative competence involves judgements about what is systemically possible (in other words what the grammar will allow), psycholinguistically feasible (what the mind will allow), socioculturally appropriate (what the society will allow), and about the probability of occurrence of a linguistic event and what is entailed in the actual accomplishment of it (Hymes 1972). Thus, Hymes’ concept of communicative competence is congruent with the goal of the Malaysian National Education Policy which seeks to educate students in the larger context of life - producing citizens who are knowledgeable, who have high moral standards and who are able to contribute to the betterment of the society and the nation (KBSM 1991). The correlation between SPM 1119 English and IELTS test scores could indicate that the high proficiency candidates have achieved communicative competence as a result of their SPM 1119 English preparation, their internalisation of the curriculum and their successful transfer of knowledge. ²

10.0 Recommendations

It appears that the SPM 1119 English candidates who performed well in the IELTS test possess communicative competence thanks to the KBSM English curriculum. According to Hymes, competence is more than knowledge: “Competence is dependent upon both [tacit] knowledge and [ability for] use” (Hymes 1972, p. 282; brackets and emphasis in original). The KBSM English curriculum, with its emphasis on teaching the sub-skills within the four basic language skills, equips candidates with language knowledge and strategic competence. Hence, candidates not only possess language knowledge, they also possess the cognitive skills needed to assess the test items, retrieve appropriate elements of language knowledge and apply it to the test items. The KBSM English curriculum, with its inclusion of sub-skills, becomes a thinking curriculum. The thinking curriculum specifies the types of higher-order mental processes that students engage in while learning (Resnick and Resnick 1991). The students are taught language knowledge together with cognitive skills which play an important role in the IELTS test. Cognitive skills help candidates make decisions on how to attempt the question, how to evaluate test items and how to sift through elements of language knowledge to select an appropriate response. It is this cognitive element in the KBSM curriculum that has enabled the candidates to perform in the IELTS test. Therefore, as the research has established that the skills taught in the KBSM curriculum approximate those tested in the IELTS test, this factor signifies that it is viable to promote IELTS to this cohort.

Although the standard of English in the IELTS test is higher than in the SPM 1119 English, candidates are able to do the test because they have knowledge of IELTS test taking strategies. They were inadvertently taught these strategies in the sub-skills. As a result, candidates who were able to apply them successfully to the test items would have achieved communicative competence and a good grade in the IELTS test. In addition, the lack of significant difference in performance between the candidates who sat for Academic IELTS and General Training IELTS is in part due to the strategies afforded by these sub-skills. Thus, it is felt that the education pathways the students choose should take precedence in the choice of module. As most potential test-takers of IELTS intend to further their studies to a tertiary level either locally or overseas, the researchers recommend that the Academic module be introduced to the secondary school sector.

Bibliography

• Anderson, Lorin W. and Sosniak, Lauren A. (1994). Bloom’s Taxonomy - A Forty-Year Retrospective (Part III). Chicago: NSSE.
• Anderson, R.C. (1984). Role of the Reader’s Schema in Comprehension, Learning and Memory, In R.C. Anderson, J. Osborn, and R.J. Tierney (Eds.), Learning to Read in American Schools. Hillsdale, NJ: Erlbaum.
• Bachman, L.F. and Palmer, A.S. (1996). Language Testing In Practice. Oxford: Oxford University Press.
• Bloom et al. (1956). Taxonomy of Educational Objectives, Handbook 1: Cognitive Domain. New York: David McKay.
• Bracewell, R.J. (1981). Writing As A Cognitive Activity. Visible Language XIV/4:400-22.
• Chapelle, C. and Douglas, D. (March, 1993). Interpreting Second Language Performance Data. Paper Presented at 2nd Language Research Forum. Pittsburgh, PA.
• Cotton, Julie. (1995). The Theory of Learning Strategies. London: Kogan Page.
• Cormier, Stephen M. and Hagman, Joseph D. (eds.) (1987). Transfer of Learning. San Diego: Academic Press Inc.
• Cummins, J. (1989). Empowering Minority Students. Sacramento, CA: California Association for Bilingual Education.
• Dass, Tania May J. (2000). Ujian Prestasi Optimum 2000 SPM Bahasa Inggeris. Pearsons Education Malaysia.
• Douglas, Dan (2000). Assessing Language For Specific Purposes. Cambridge: Cambridge University Press.
• Fielding, L.G. and Pearson, P.D. (1994). Reading Comprehension: What Works. Educational Leadership 51.
• Flower, L.S. and Hayes, J. (1981). A Cognitive Process Theory of Writing. College composition and Communication 32.
• Hillcocks, G. Jr. (1987). Synthesis of Research on Teaching Writing. Educational Leadership 44.
• Hymes, D. (1972). On Communicative Competence. In J.B. Pride and J. Holmes (eds.), Sociolinguistics. Harmondsworth: Penguin.
• Kilpatrick, W.H. (1979). A Project Curriculum. Curriculum Design. The Open University Press.
• Kurikulum Bersepadu Sekolah Menengah (KBSM): Huraian Sukatan Pelajaran Bahasa Inggeris Tingkatan 5. (1991). Dewan Bahasa dan Pustaka.
• Lee, Kin Siong and Wong, Fook Fei. (May, 2000). Making Washback Work - A Study Of Initial Impact Of The SPM 1119 English Exam. MELTA.
• Marzano et al. (1988). Dimensions Of Thinking: A Framework For Curriculum And Instruction. Alexandria, VA: Association for Supervision and Curriculum Development.
• Mayor, Barbara M. and Pugh, A.K. (1987). Language, Communication And Education. London: Croom Helm.
• McNamara, T.F. (1996). Measuring Second Language Performance. London: Longman.
• Mendelsohn, D.J. (1984). There ARE Strategies For Listening. TEAL Occasional Papers 8.
• Nord, J.R. (1981). Three Steps To Listening Fluency: A Beginning. In Winitz, H. (ed.), The Comprehension Approach to a Foreign Language Instruction. Rowley, MA: Newbury House.
• O’Malley, J. Michael and Pierce, Lorraine Valdez. (1996). Authentic Assessment For English Language Learners. Addison-Wesley Publishing Company.
• Pearson, P.D. (1985). The Comprehension Revolution: A Twenty-Year History of Process and Practice Related to Reading Comprehension. Reading Education Report No. 57. Urbana: University of Illinois Center for the Study of Reading.
• Reid, Joy M. (ed.). (1998). Understanding Learning Styles in the Second Language Classroom. NJ: Prentice Hall Regents.
• Fraenkel, Jack R. and Wallen, Norman E. (1990). How To Design And Evaluate Research in Education. New York: McGraw-Hill Inc.
• Tanner, Daniel and Tanner, Lauren N. (1975). Curriculum Development: Theory into Practice. Macmillan Publishers.
• The IELTS Handbook. (October, 2000) The Ministry of Education.
• Wallace, Michael J. (1998). Action Research for Language Teachers. Cambridge: Cambridge University Press.
• Widdowson, H. (1983). Learning Purpose and Language Use. Oxford: Oxford University Press.

Appendix 1: Malaysian English language curriculum

A1.0 Overview of the Malaysian English language curriculum

The curriculum is broadly defined as “the planned and guided experiences under the auspices of the school for the learner’s continuous and wilful growth in the personal and social competence.” (Tanner and Tanner, 1975). In a similar vein, Kilpatrick (1979) defined the curriculum as a series of guided experiences which are related to each other and that what is learnt in one serves to elevate and enrich the subsequent stream of experiences.

The Malaysian Education system follows the Integrated Curriculum for the Secondary Schools better known as KBSM (Kurrikulum Bersepadu Sekolah Menengah). The KBSM was implemented in 1988 and was greatly influenced by the National Philosophy of Education which is geared towards the development of a morally upright person who is intellectually, spiritually, emotionally and physically integrated (KBSM, 1991). SPM 1119 English is a core subject in KBSM requiring 200 minutes of contact time per week between student and teacher. This language program, which emphasises the teaching of both oracy (listening and speaking) and the literal (reading and writing) skills, has its beginnings in the lower secondary level. This program culminates in a final assessment exam where the candidates are expected to perform by pooling their resources and utilising their reservoir of cumulative curriculum knowledge.

The candidates are awarded two grades for the SPM 1119 English paper - the first grade according to the local examination standard and the second after the Cambridge O Level 1119 English assessment standard. For the research, we took into account only the former grade as we felt that that this was a more comprehensive assessment of the candidates’ level of proficiency.

In addition, we also wanted to measure if the candidates’ performance in this test was comparable when confronted with an international language test like IELTS. The second grade was ignored because the knowledge we had about the second assessment led us to believe that this grade reflected mostly candidates’ writing abilities.

A1.1 The four skills

It is traditional in language testing to categorise tests and sub-tests according to the four skills of listening, reading, writing and speaking. A sound knowledge of the KBSM curriculum would presuppose language knowledge and ability. The KBSM uses language knowledge ie.grammatical knowledge and textual knowledge to form the language contents required to teach these skills. Keeping in mind the time allocated for teaching English in schools, the teaching of the four skills and language knowledge tends to be integrated and taught as a whole. In the final examination the candidates’ communicative competence is manifested in their performance of tasks involving listening, reading, writing and speaking skills. Competence is realised in the performance of these tasks in a formally simulated test-taking situation.

A reformulation of Hymes’ (1972) theory of communicative competence by Bachman and Palmer (1996) postulates that the two components of communicative language ability are language knowledge and strategic competence. Bachman and Palmer have since extended Hymes’ notion of “ability for use” into what they term strategic competence. Strategic competence serves as a mediator between the internal traits of curriculum knowledge, language knowledge and the external context (ie. test-taking situation), controlling the interaction between them. The engagement of strategic competence presupposes cognitive skills which are responsible for assessing the characteristics of the language use situation, setting communicative goals, planning a response in the light of the goals and controlling the execution of the plan.

Many local curricula have begun to emphasise high-order thinking skills to match complex reasoning processes required by foreign and local professional institutions. Thinking skills are encouraged in the

KBSM curriculum as students are given the opportunity to perform test items that require comparing, contrasting, analysing and summarising information, ordering and classifying data, evaluating and justifying arguments. Core thinking skills are a means to a particular goal. They are essential to the functioning of other thinking activities and may be used in the service of metacognition, cognitive process, or critical and creative thinking (O’Malley and Pierce 1996).

Marzano et al. (1988) suggest that there are a number of core thinking skills that surpass the traditional analysis, synthesis and evaluation (Bloom et al. 1956). Representative core thinking skills include:

• focusing: defining problems and setting goals
• organising: comparing, classifying, ordering and representing information
• analysing: identifying attributes and components, relationships and patterns, main ideas, and errors
• generating: inferring, predicting and elaborating
• establishing criteria
• verifying conclusions.

The KBSM encourages teachers to ensure that core thinking skills are reflected throughout instruction and assessment. These skills can be used at any point in language learning and may be used repeatedly. The thinking skills can be taught effectively by integrating them with classroom tasks and curriculum concepts. These cognitive and thinking skills form part of the strategies required to do the IELTS test. Strategies and techniques play a crucial role in the IELTS test as they determine how candidates approach a test item and resolve the problem. They control the interaction between language knowledge and discourse context.

The KBSM curriculum is comprehensive and uses extensive teaching sub-skills (ie. discerning main ideas from supporting details, understanding sequences, inferring, etc.) to consolidate the four language skills. The KBSM teaches sub-skills in listening, reading, writing and speaking which encourage the cognitive process. These sub-skills are significant to candidates taking the IELTS test as they make up part of the IELTS test taking strategies. The ability of the candidates to perform successfully in the IELTS test stems not only from their language knowledge but from their ability to use the strategies taught in the sub-skills. Hence, we see that some students with moderate proficiency levels are able to do the test if they can understand and master these strategies.

Our analyses of the data indicates that the students perform at varying degrees of success depending on their performance in the SPM 1119 English test. This leads us to the opinion that the candidates’ success in performance is largely due to the training and preparation they receive in school under the English language curriculum. The objectives of the Malaysian English Language Curriculum define and identify types of knowledge and skills that are important in achieving success probably not just in the IELTS test but also in other English language proficiency tests. The objectives of the English Language Curriculum for upper secondary schools are as follows:

• listen to and understand spoken English in the school and in real life situations
• speak effectively on a variety of topics
• read and understand prose and poetry for information and enjoyment
• write effectively for different purposes (KBSM 1991).

Clear in its objectives to equip the candidates with the four basic language skills, the KBSM unwittingly focuses on all the various IELTS test components. The four language skills are taught within the background of language knowledge and strategic competence, ie. the accompanying sub-skills. Using Bachman and Palmer’s formulation of the components of language knowledge and Chapelle and Douglas’ (1993) reformulation of strategic competence, we can build a framework for communicative competence as shown in the Table 1.

Table 1: Components of communicative competence - language knowledge

Table 2: Components of strategic competence

It appears that the high proficiency candidates who absorbed or internalised the language and strategic components successfully in the SPM 1119 English curriculum were able to do the IELTS test. This is reinforced by the statistical analysis. Hence, a good grade in SPM 1119 English should guarantee and manifest a good grade in the IELTS test, whether the candidates choose to sit for Academic IELTS or General Training IELTS. The statistical analysis also shows that high proficiency students do better in the Academic module. In addition, the comparison of Reading test scores of the two modules illustrated that students doing the Academic module did better than those who did the GT module.

A2.0 KBSM’s approach to the four language skills

A2.1 Listening and speaking

In almost any setting involving native language, listening is the most frequently used skill, making up 45% of the time devoted to communicating. Speaking follows closely with 30%. Although listening is a fundamental language skill, it is often overlooked by second language teachers. While the other three language skills receive direct instructional attention, teachers frequently expect students to develop their listening capability by osmosis and without help (Mendelsohn, 1984).

Although listening is an important skill in the KBSM curriculum and has it’s own set of objectives, it is not formally tested like the other language skills. In this situation, we feel that a formal test would help secure the test a prime position. In addition, listening should be given clear focus in classroom instruction. In the survey, it was discovered that the candidates were least familiar with this skill and this could be the reason for that. Listening can no longer be relegated to the status of an enabling skill that simply helps students to speak (Nord, 1981).

The speaking test assesses a student’s ability to communicate for both basic communicative and academic purposes. Communicative or conversational skills involve face-to-face interaction where meaning can be negotiated and is supported by contextual cues, such as the situation itself, gestures, facial expressions, and intonation (Cummins, 1989). In the IELTS test, academic language proficiency is emphasised where students are asked to describe aspects of their environment in Part 1, speak about a topic in Part 2, and discuss and speculate on abstract issues in Part 3.

Listening is taught in unison with speaking skills in the KBSM curriculum. While there is no formal test of listening skills, the candidates’ oral skills are formally tested in the SPM 1119 English examination. This oral test lasts for 10 minutes and the candidates are judged on their ability to read aloud a passage and answer three questions based on the reading passage. This is followed by a conversation with the examiner where the candidate is required to discuss three questions based on visual stimulus.

The sub skills taught under these two components of the curriculum are important in IELTS test-taking strategies. The sub-skills of listening and speaking as taught in KBSM are listed in the table below.

Table 3: Listening and speaking sub-skills

A2.2 Reading

Although reading was once assumed to be a combination of decoding and oral language, it is now acknowledged that reading comprehension depends heavily on knowledge about the world as well as on knowledge of language and print (Fielding and Pearson 1994). Prior knowledge (in the form of schemata) influences our comprehension to a much greater degree than earlier research would have suggested (Pearson 1985). Anderson (1984) has summarised the influences that schemata play in our comprehension in these generalisations (these are close paraphrases of Anderson’s assertions), outlined below.

• Schemata provide ideational scaffolding for assimilating text information. Schemata have slots that readers expect to be filled with information in a text. Information that fills those slots is easily learned and remembered.

• Schemata facilitate the selective allocation of attention. It guides our search for what is important in a text, allowing us to separate the wheat from the chaff.

• Schemata enable inferential elaboration as no text is ever fully explicit. It allows us to make educated guesses about how certain slots must have been filled.

• Schemata allow for orderly searches of memory. For example, suppose a person is asked to remember what he did at a recent cocktail party. He can use his cocktail party schema, a specification of what usually happens at cocktail parties, to recall what he ate, what he drank, whom he talked to, and so on.

• Schemata facilitate editing and summarising. By definition, any schema possesses its own criteria of what is important. These can be used to create summaries of text that focus on important information.

• Schemata permit inferential reconstruction. If readers have a gap in their memory, they can use a schema, in conjunction with the information recalled, to generate hypotheses about missing information. If they can recall, for example, that the entrée was beef, they can infer that the beverage was likely to have been red wine.

The training and preparation in the KBSM curriculum and the successful internalisation of the program become schemata. The curriculum contributes to and builds on schemata. Schemata or prior knowledge is inherent after the candidates undergo KBSM or any other language preparation. Hence, schemata promotes a better understanding of IELTS test passages although these passages are definitely harder in terms of language and content than the ones encountered in the SPM 1119 English test. Although the candidates are second language learners possessing background knowledge which is often culture-bound and which may not match the schemata needed for a given reading text, the candidates are equipped with the reading strategies taught to them in the sub-skills section to help them cope with the foreign nature of the text. The SPM 1119 English exam evaluates how well the student can access the schemata required to handle the test items.

The sub-skills in the reading section are important in Academic and General Training Reading. Reading comprehension can be increased by teaching comprehension strategies directly (Fielding and Pearson 1994). These reading strategies are useful to candidates and can be crucial when doing the IELTS test because of the imposed time constraint.

The sub-skills taught in the KBSM are stated in the table following (KBSM, 1991).

Table 4: Reading sub-skills

Sub-skills of reading

The SPM 1119 English is similar to General Training Reading in terms of textual structure and content. However, the statistical data shows that the candidates performed better in the Academic Reading test. This could be explained by the fact that as most of the reading materials included in the KBSM curriculum have a cultural, national and regional bias, the contexts presented in IELTS General Training and Academic modules began to assume an alien aspect, thus neutralising the edge one group might have over the other. In other words, although the degree of unfamiliarity towards the contexts presented are not identical or level, the degree of unfamiliarity the students would have in attempting a foreign context in the Academic module is not significantly greater than that of the students doing the GT. This explains why the candidates who sat for GT Reading did not have an edge over the candidates who sat for the Academic Reading although GT Reading is perceived to be the easier test. In short, the students’ present schemata could not handle the unfamiliar contexts due to inadequate exposure in the curriculum. As a consequence, they had to rely more on the reading sub-skills to handle the test items. Those who did better in Academic Reading were possibly more able to activate the reading sub-skills taught in the KBSM curriculum.

A2.3 Writing

Writing effectively implies that there has been a successful fusion of thought and language to fit the rhetorical context ie. gearing the message to the audience. Here, there is a matching of content with form, ensuring that the writing has purpose and that its intended meaning is successfully conveyed to its intended reader. All writing activity is geared towards the goal of “putting together form and content of language to achieve extended sequences of text that are coherent” (Bracewell 1981, p. 412).

The analysis of the data shows that candidates doing Academic Writing found it to be less familiar at 45.9% while candidates doing the General Training Writing found this component to be more familiar with 57.3% claiming familiarity. Nevertheless, there is no difference in means between candidates doing Academic Writing and General Training Writing. It appears that with the Academic module, although the candidates have some knowledge of graphs taught in the Reading Section, they still would have found Writing Task 1 unfamiliar because they would have lacked the lexical resources and organisational skills required to illustrate and describe trends. As Writing Task 1 in the General Training test is letter-writing, the candidates were more familiar with it. Writing Task 2 would not have posed a great deal of difficulty to the candidates who sat for either module because they would be familiar with writing argumentative essays where they would need to give their opinions and persuade their audiences. Here, the candidates need to use cognitive skills such as analysing and evaluating information to argue a particular point of view in a convincing way.

The SPM 1119 English Writing test is more comparable to the General Training Writing test in terms of genre and syntactic forms. The sub-skills of writing as stated in the curriculum allow candidates to convey content and meaning in an organised form using proper genre and suitable conventions of writing. Hence, the candidates should be able to do Academic Writing as well.

Table 5: Writing sub-skills

In order to write well, candidates need to incorporate purpose into their own unique approach to writing by calling on four types of knowledge: knowledge of the content; procedural knowledge to organise the content; knowledge of conventions of writing; and procedural knowledge required to apply the three other types of knowledge in composing a written product (Hillcocks, 1987). These types of knowledge are inherent in any writing curriculum.

In expressing knowledge of the content, candidates use schemata, background knowledge and complex mental processes in developing new insights. Generating ideas is one of the important sub-processes that contributes to planning in writing (Flower and Hayes 1981). Brainstorming, making lists or semantic maps, collaborating with peers and elaborating on key ideas with personal information are some of the techniques candidates use to generate ideas. Next, candidates need procedural knowledge to organise the content, to group ideas in ways that match the purpose to the writing and convey coherent meaning.

The third type of knowledge candidates’ use in writing is knowledge of discourse structures, syntactic forms and conventions of writing. Discourse structures are evident in the way that various types of writing/genres are organised. Candidates must be familiar with the various ways of organising different genres and in expressing meaning through syntactic constructions and writing conventions.

The fourth type of knowledge students rely on is procedural knowledge for integrating all other types of knowledge. This constitutes the foundation of the writing process. Candidates need to use procedures that combine the three types of knowledge to convey purpose and meaning. Though it may be possible to probe certain superficial aspects of the activities which result in a finished text, the underlying procedures/processes of writing are ultimately too complex to define.

Appendix 2: Questionnaire

Appendix 2 questionnaire, page 1 of 7.

Appendix 2 questionnaire, page 2 of 7.

Appendix 2 questionnaire, page 3 of 7.

Appendix 2 questionnaire, page 4 of 7.

Appendix 2 questionnaire, page 5 of 7.

Appendix 2 questionnaire, page 6 of 7.

Appendix 2 questionnaire, page 7 of 7.