The effect of background disciplines on IELTS scores

Abstract

Three hundred and ninety-eight IELTS candidates were randomly selected from the IDP Education Australia offices in the Klang Valley of Malaysia to participate in an IELTS research project which was carried out to determine whether background discipline had any effect on the scores attained in the IELTS test. The results which were derived from the statistical test: the t-test and the p-value test of significance concluded that in most cases, where English language proficiency levels were high, background disciplines did not significantly influence the IELTS scores attained. However, IELTS scores attained by candidates with average and lower proficiency levels were significantly affected by their background disciplines. In addition, on the question of whether preparation taken will enhance the IELTS scores achieved by candidates with different background disciplines, it was found that the average unprepared candidate stood an equally good chance of attaining the same grade as the average prepared student.

1.0 Introduction

The intent of the research was to investigate the effect of the candidates’ previous disciplines on their IELTS scores. The question that the researchers studied was whether the candidates’ previous disciplines had any relationship with the level of achievement in the IELTS test. Another question was what the idea of appropriate difficulty was in relation to the candidates’ previous disciplines.

It is common knowledge that a revised IELTS test has been in use since April 1995. Prior to this date, the IELTS tests which were in use had been developed to be subject specific i.e. this meant that the modules were relatively closely related to the target discipline/subject of the candidates. Basically, the decision to sit for a particular module depended on the future educational options the students were to pursue. There were three main modules the candidates could opt for. There was, namely, Module A (Physical Sciences) for candidates opting for courses in Mathematics, Computing and Physics. Module B (Life Sciences) was for candidates pursuing a degree in Biology, Nursing and Medicine. Next, candidates who wanted to take up tertiary programmes such as Business, Economics, Journalism or Drama to name a few were advised to choose Module C (Humanities). All three modules shared the same listening and speaking sections. The differences among the three modules were found in the reading and writing sections which had been designed to feature and incorporate materials/text and tasks common to those areas.

It has been suggested in the research conducted by Alderson and Urquhart (1983) entitled ‘The Effect of Student Background Discipline on Comprehension: A Pilot Study’ that a candidate’s familiarity towards the subject matter tested is related to the candidate’s predicted performance in a language test. In other words, candidates performing in a familiar content area may perform better than candidates unfamiliar with the content matter of that field. ‘The latter, which could be argued, would lack familiarity not only with the content, but also with aspects such as genre effect, rhetorical organisation as well as linguistic and nonlinguistic relations’.⁴ In short, the assumption spells out that an engineering student’s linguistic competence would be much higher if tested on an engineering scope than on a general topic. The converse might also be true. Therefore, it could be assumed that the candidate’s performance on the reading and writing tasks could be coloured; influenced and affected - to varying degrees - by the factor of familiarity.

It can be said that the IELTS test conducted before April 1995 reflected an inclination towards specific purpose testing. In contrast, the IELTS tests conducted since April 1995 have focused on the candidate’s general competence. It requires all students who are pursuing higher academic qualifications at the tertiary level to sit for the same module, the Academic Module, regardless of disciplines. Therefore, the IELTS is the acknowledged standard form of measuring a candidate’s English language proficiency and is designed as a language test to ‘assess whether the candidates would be able to perform the language tasks required of them’. Traditionally, according to Alderson and Urquhart, in the field of Testing of English as a Foreign Language, a major criterion in text selection was that the subject matter should not be subject specific, but should be of general interest. The aim of that particular bent in English language testing was to avoid favouring any particular group and penalising others. It seems that test designers have returned to that traditional line of thought with several modifications.

The assumptions that underlie this approach are:-

• The existence of a general language ability which can be tested i.e. the common ground.
• The idea that a candidate who performs well on a general test will be able to cope adequately with subject-specific texts encountered in the course of his professional and academic studies. ⁴

2.0 Aims of the Research Study

The aims of the research were:-

1. To determine whether students’ preparation in the Science and Arts disciplines will affect the candidates’ performance in the IELTS test.
2. To determine if the students’ general level of proficiency can override the problem of lack of familiarity with the topics touched on in the Academic module in the IELTS test and whether it is reflected in the scores attained.
3. To determine if there is a difference in the scores received by Arts students and Science students and evaluate the significance of any difference.
4. To determine whether preparation can help candidates override the problem of lack of familiarity with the topics touched on in the Academic module in the IELTS test and whether it is reflected in the scores achieved.

2.1 Detailed Research Aims

In order to thoroughly research the questions posed, the aims are broken down into two main points. Please note that the national exam Sijil Pelajaran Malaysia (commonly known as SPM) includes the English language test English 322.

1. Whether or not previous disciplines (the respective Science and Arts Streams) have an effect on the IELTS scores attained.

a Whether scores attained by IELTS candidates from a Science background are significantly different from scores attained by IELTS candidates from an Arts background at a high level of proficiency.

b Whether scores attained by IELTS candidates from a Science background are significantly different from scores attained by IELTS candidates from an Arts background at an average level of proficiency.

c Whether scores attained by IELTS candidates from a Science background are significantly different from scores attained by IELTS candidates from an Arts background at a low level of proficiency.

2. Whether preparation would make a significant difference in the IELTS scores attained compared to no preparation at all.

a Whether preparation would make a difference in the IELTS scores attained by Arts/Science candidates with preparation compared to Arts/Science candidates without preparation. b Whether preparation would make a difference in the IELTS scores attained by Arts/Science candidates with preparation compared to Arts/Science candidates without preparation at a proficiency level of A1-A2 in the SPM English. c Whether preparation would make a difference in the IELTS scores attained by Arts/Science candidates with preparation compared to Arts/Science candidates without preparation at an average proficiency level of C3-C4 in the SPM English. d Whether preparation would make a difference in the IELTS scores attained by Arts/Science candidates with preparation compared to Arts/Science candidates without preparation at a low proficiency level of C5-C6 in the SPM English.

3.0 Hypothesis

The independent variable i.e. the discipline from which the IELTS candidates come from has no effect on the subject’s performance in the Academic module of the IELTS test.

4.0 Methodology

These procedures were taken in conducting the research.

Stage One:-

The research team examined past registration records of the months from 1 June 1996 to 31 July 1996 in order to define potential subjects who would fall within the research criteria. Potential test sites which served as feeders for the research were also identified for the research to ensure a random sample was obtained.

Stage Two:-

The questionnaire was designed. The pilot was trialled between 15 August 1996 and 31 August 1996. The questionnaire was given out with the IELTS registration form to all candidates tested by IDP Education Australia in the IDP offices in the Klang Valley.

Stage Three:-

The implementation stage took place between 1 September 1996 and 30 January 1997. By the end of 30 January 1997, the sample size included 530 subjects.

Stage Four:-

The analysis of the data began at the end of February 1997.

Stage Five:-

The production of the working paper began in May 1997.

5.0 Student/Candidate Background

Students who finish their secondary school education in Malaysia essentially come from two streams: the Science and the Arts. At the beginning of secondary school education in any government school, at the age of 13 to 15, the students will undergo the levels of Form 1 - 3 in the secondary school system during which they follow a common core group of subjects which is usually cross-disciplinary and is conducted in the fixed language of instruction: either Bahasa Malaysia or the Mandarin language. The subject English language is taught to all students over 180 to 210 minutes in their respective classes at every level every week. It is the only subject that is not taught in the school’s set medium of instruction.

At the end of their studies in Form 3, the students undergo a government achievement test called the PMR. The results from this test will be used to stream the students into two respective streams: Science and Arts. The decision regarding the streaming is based on the strengths and weaknesses of the students’ performances which were highlighted by the Bahasa Malaysia, History, Geography, Science, Kemahiran Hidup (Living Skills), Moral Studies and Mathematics. In order to get into the Science stream, the student must attain a grade of at least B and C for the Mathematics and Science combination. The student who has attained a C and C combination will automatically enter the Arts stream. The students from the Science stream may appeal to enter the Arts stream, but the converse situation is not allowed.

During these two years in the Science and Arts stream, the subjects learnt are streamlined and would generally set the foundation for any further studies the student will pursue i.e. in the scope narrowed by the subjects taught in the respective streams. During the two years in the Science and Arts stream, the students will go about their respective study programmes which will be conducted in Bahasa Malaysia. Therefore, from Form 1 to Form 5 (their formative school years), the students are exposed to formal English language learning for only about 180 to 210 minutes per week. The rest of their formal education is carried out in Bahasa Malaysia.

During these two years in their English classes, the students are prepared for the 322 English exam which is part of the national SPM exam. The 322 English exam is the Malaysian equivalent of the Cambridge 1119 English exam, but it is of a considerably lower standard by comparison. The 322 English exam is also used by students to apply for entry into the local private colleges.

6.0 Research Population

The population for the research was determined early during the onset of the research. It was decided that the population would consist of candidates tested in the IDP Education Australia offices in the Klang Valley and the pool of candidates from the private colleges in the Kuala Lumpur and Petaling Jaya areas. IDP Education Australia conducts the IELTS tests for candidates who are Australia, United Kingdom and New Zealand bound. The potential candidates were selected during the months of 1 September 1996 to 30 January 1997.

The prerequisites that were used to determine and isolate the candidates eligible for the research are:

• The candidates had completed an identical and standardised national exam called the Sijil Pelajaran Malaysia (commonly known as SPM) and had attained scores of between A1 - C6 in the English exam.
• The candidates had experienced their secondary school education in a medium other than English as their language of instruction. (For example, the candidates experienced their secondary school education in Bahasa Malaysia or Mandarin.)
• The candidates chosen had completed their secondary school years either in the Science stream or the Arts stream. Therefore, the students are separated into their respective groups according to their previous disciplines.
• The candidates were students who had never taken any other international English exam namely the TOEFL (Test of English as a Foreign Language) or the Cambridge English 1119.
• The candidates were taking the IELTS Academic Module for the first time.
• The candidates were students preparing to do an undergraduate program preferably overseas i.e. Australian tertiary institutions. Candidates who were doing IELTS in order to pursue a postgraduate program were not considered.

Once the population had reached the appropriate size, the process of random selection began in order to determine the subjects suitable for research purposes.

By the end of 30 January 1997, the population attained numbered 530. The sample size was first divided into two groups:-

a) candidates with an Arts background b) candidates with a Science background

These two groups were further divided into 3 groups according to proficiency levels. The level of proficiency prior to the IELTS test was measured by the grade acquired in the SPM 322 English exam and is as stated below:-

i) Candidates with a high proficiency level (A1-A2 in the 322 English) ii) Candidates with an average proficiency level (C3-C4 in the 322 English exam) iii) Candidates with a low proficiency level (C5-C6 in the 322 English exam)

Let it be noted that by 30 January 1997, it was decidedly difficult to get a sizeable sample size for Group (iii).

7.0 Research Instruments

Instruments utilised in the research consisted of:

a) IELTS scores b) Questionnaires

The questionnaire design took into account all the variables. Refer to Appendix 1 for the copy of the Questionnaire. The data obtained included:

• medium/language of instruction in school
• previous disciplines (i.e. either the Science or the Arts stream)
• SPM 322 English language qualifications
• whether it was the first time they had taken the IELTS Academic module
• whether the candidates had taken any other language exams like the TOEFL or the Cambridge English 1119
• whether the candidates were going to pursue an undergraduate program at a tertiary institution or a postgraduate program
• preparation for IELTS in the form of classes.

Please note that the IELTS preparation materials used in the private colleges were provided by IDP Education Australia. The books and cassettes which were the mainstay of the IELTS programmes in the local colleges were :

• Specimen Materials for the IELTS TEST, April 1995

• IELTS Practice Now by Carol Gibson, Wanda Rusek and Anne Swan, CALUSA 1996

• Prepare for IELTS - Academic Module by Vanessa Todd and Penny Cameron, University of Technology, Sydney 1996

• How to Prepare for IELTS by Ray deWitt, The British Council 1995.

Preparation classes in the local colleges were conducted over a period of 30 to 48 hours where skills and strategies to face the tasks presented in the four major components of the IELTS test were emphasised.

8.0 Sampling

Students for this study were selected from the population of IELTS candidates who sat for IELTS exams conducted by IDP Education Australia at the approved test sites. The method used to gather the sample from the target population was stratified sampling because it allowed researchers to compare the performance of different subgroups.

The steps in stratified sampling began with the identifying and defining of the population. The size of the population was decided by the number of IELTS candidates who had taken the test between 1 September 1996 and 30 January 1997. The population size gathered by the end of 30 January 1997 was 530. The desired sample size was 398, approximately 75% of the population size.

The variable of interest is the previous disciplines of the IELTS candidates of which there are 2 main groups:

a) The IELTS candidates with an Arts stream background b) The IELTS candidates with a Science stream background.

These two main groups are then further classified into 3 subgroups:-

• students with scores of A1-A2 in the standardised English test called SPM English 322.
• students with scores of C3-C4 in the standardised English test called SPM English 322.
• students with scores of C5-C6 in the standardised English test called SPM English 322.

As we wanted proportional representation, 75% of each subgroup was randomly selected using a table of random numbers. At the completion of this process, we had a probable representative sample of the subgroups. Stratified sampling has succeeded in narrowing the sampling for each group to:

• A1-A2: Science : Arts = 124:124 (out of 330)
• C3-C4: Science : Arts = 60:60 (out of 160) C5-C6: Science : Arts = 15:15 (out of 40).

9.0 Analysis and Interpretation of the Data

9.1 Statistical Tests

In order to test if the hypothesis is correct, the statistical tests, the t-test and the p-value, were conducted. The t-test was conducted to determine whether we could reject the null hypothesis whereas the p-value is a test of significance. The test results which were derived from the t-test are reported to be significant at the 5% significance level. The p-value is used here to view whether the scores attained by the candidates are in any way significantly influenced by their streams: the Science and Arts. The p-value represents the probability of observing a sample outcome more contradictory to $H_{0}$ than the observed sample result if, in fact, $H_{0}$ is true. The smaller the value of this probability, the heavier is the weight of the sample evidence for rejecting $H_{0}$ . Therefore, the statistical tests with an observed level of significance of below .05 have more evidence for the rejection of $H_{0}$ .

Table 1 Differences between Arts and Science candidates

A: Mean score for Arts candidates; S: Mean score for Science candidates.

Table 2 Differences between Arts and Science candidates, with and without preparation.

P: Mean score for candidates who have taken an IELTS preparation course; U: Mean score for candidates who have not taken an IELTS preparation course.

A general overview of the results show:

• Highly proficient students from the Science and Arts Streams scored comparatively similar results in the IELTS test.
• For average and low proficiency students, the Science students did significantly better than the Arts students. As the proficiency level was considered as the constant variable, the statistical tests showed background disciplines could have an effect on the IELTS scores.
• Using the discipline(s) as a constant variable, there was no significant difference in the results scored by preparation students and non-preparation students at all proficiency levels. Therefore, preparation did not make a significant contribution to the attainment of a better IELTS score.

In short, preparation did not make a significant contribution on IELTS scores at all while background disciplines had a significant effect on IELTS results for students with average to low proficiency levels.

9.2 Effects of the Malaysian Education System

In order to analyse the rationale behind the significant difference in the IELTS scores between the students of different disciplines at average and low proficiency, the research would take a closer look at the possible roles of certain aspects of the Malaysian Education system such as:

• The Curriculum
• Schema
• Transfer of Learning
• Learning Styles

It is felt that these aspects have played a deciding role in helping the Science students attain a significantly better grade in the IELTS test than the Arts students.

9.2.1 The Malaysian Curriculum

Tanner and Tanner (1974) broadly defined curriculum as ‘the planned and guided experiences under the auspices of the school for the learner’s continuous and wilful growth in the personal and social competence.’ ⁸ Collins and Kilpatrick (1979) further defined the curriculum as a series of guided experiences which are related to each other and that what is learnt in one serves to elevate and enrich the subsequent stream of experiences. ⁹ The Malaysian Education system follows the Integrated Curriculum for the Secondary Schools (Kurikulum Bersepadu Sekolah Menengah) which is better known as KBSM. The KBSM was implemented in 1988 and was greatly influenced by the National Philosophy of Education.

The KBSM in their effort to produce intellectually, spiritually, emotionally and physically balanced individuals try to provide an educational framework wherein the subjects learnt are:

• linked in some way to one another
• guides towards the formation of the learning styles needed to meet the realities of the nation’s need for an educated and highly moral workforce. ¹⁰

Most importantly, according to Denton (1992), the curriculum is categorised into subjects to enable the learner’s brain to handle the concepts taught and to think effectively. The process of learning requires organisation and interpretation of experiences. ¹¹ Organisation is usually done through the use of conceptual frameworks from different subject areas. Different disciplines in the KBSM are seen to have their own frameworks to organise the concepts learnt. At the upper secondary level, the KBSM categorised the subjects into two major categories: the core subjects and the electives.

The list of core subjects for the upper secondary level is :-

• Bahasa Melayu
• English
• Islamic Studies/ Moral Education
• Mathematics
• History
• Physical Education
• Science(this subject is not taught to the Science stream students as they have three Science options already).

The Electives available under the Arts discipline are:

• Malay Literature
• English Literature
• Art
• Accounts
• Economics
• Geography.

The Electives available under the Science discipline are:

• Additional Mathematics
• Physics
• Chemistry
• Biology.

At the upper secondary school level (Form 4 and Form 5), it is compulsory for the students to study the core subjects and a minimum of 2 or a maximum of 4 elective subjects. It must be noted that the core education received was directed at developing a foundation of similar cognitive styles and learning strategies upon which the subsequent cognitive styles and learning strategies advocated by the respective disciplines were further developed. ¹²

Hudson (1968) states that by allowing the upper secondary students to actively specialise or choose a category of subjects, they were essentially choosing a chosen style of thinking. ¹³ In sum, different disciplines utilise different schemas and hence promote different learning styles. Thus, it is logical to assume that students from different disciplines will, after prolonged conditioning, have different methods of interpreting the demands of a task as well as different methods of arriving at the solution.

9.2.2 Previously Learnt Schemas

Schemas are important as they impose an (a known) order or organisation in an otherwise unfamiliar task or concept. According to Dansereau in Teaching for Transfer, a structural schema provides a ready made organisation for capturing and cataloguing new ideas no matter how they are presented. The schema is used to organise the information presented and draw parallels between a new task item and one that is more familiar to the student. This point is particularly important during the IELTS exam as one’s structural schema would facilitate the organisation of the task items provided and students would find it easier to draw parallels between the task items found in the KBSM English language syllabus and the IELTS exam and perform accordingly. Further research by Dansereau supports this point as he concluded that students felt that schemas help them decide what they needed to know and assisted them in recalling strategies required and organised the information clearly for tests and papers. ¹⁴

9.2.3 Successful Transfer of Learning

Broadly defined, transfer involves prior learning affecting new learning or performance. The latter performance can differ from the original learning in terms of the task involved (McKeogh, Lupart and Marini). Successful transfer is achieved when the learner is able to readily access required resources when the transfer opportunity presents itself (i.e. during exams) and recognise (automatically or consciously) appropriate transfer situations (Prawat). ¹⁵

In the IELTS test, the students who had effected a successful transfer of learning have accessed the language skills and required resources as well as the problem-solving approaches characteristic of the disciplines the students originate from. For instance, high proficiency students of both streams with grades of A1-A2 in the SPM English achieved similarly high IELTS scores indicating a successful transfer had taken place. In contrast, a learner, who cannot easily access the resources required for a successful transfer and does not recognise the relevance of prior learning to the task at hand as the task was presented differently from the original context, will have achieved an average to poor grade depending on the seriousness of the learner’s inability to activate his knowledge. For instance, average to poor grades are achieved in the IELTS score when the students are unable to, in varying degrees of seriousness, rely on the combination of language skills and methods which are idiosyncratic to their respective disciplines. Due to poorer language skills, the students would in fact have a greater reliance on the problem-solving tactics reflective of the cognitive styles and strategies learnt in the respective streams to compensate for the lack of language skills.

9.2.4 Learning Styles Promoted By The Arts and Science Disciplines

The effect of background disciplines in moulding the learning styles of students cannot be emphasised enough. It is felt that the KBSM curriculum in its mission to create versatile learners who can flexibly vary problem solving strategies according to the task presented, has encouraged two kinds of learners with different learning styles as shaped by their respective disciplines. It is felt that the Science students have a tendency to use the serialist learning styles whereas the Arts students have an inclination towards the holist learning styles. The National Association of Secondary School Principals defines learning styles as ‘the composite characteristic of the cognitive, affective and physiological factors that serve as relatively stable indicators of how learners perceive, interact and respond to the learning environment’. ¹⁶ Therefore, it is important to view the nature of the methods developed by the nature of the Science and Arts disciplines.

9.2.4 (i) The Nature of the Science Syllabus

According to Tan and Loo science is commonly seen as positivistic in nature, a way of knowing that proceeds from observable evidence to accurate prediction. Science teachers in Malaysian schools would immediately recognise the hypothetico-deductive nature of the standard scientific method as comprising:

• Identifying the Problem

• Selecting the Hypothesis

• Conducting an Experiment

• Collecting Data

• Testing the Hypothesis

• Rejecting or Accepting the Hypothesis

The Malaysian Science Curriculum gives emphasis to the mastery of the scientific method. The Inquiry-Discovery Approach and the Constructivism Approach are deeply entrenched into the science curriculum. In the Science syllabus for Secondary Schools (Ministry of Education, 1988), it is stated that :

Science process skills that are systematic, creative, critical, analytical and rational in terms of thinking include :-

• Collecting information through objective and meaningful observation to develop or test a hypothesis or inference

• Collecting information from various sources according to the ability levels of pupils

• Recording observations accurately and organising data to make it more useful

• Organising data and information through comparison and classification as well as showing the relationship among various sets of data

• Analysing and interpreting data to form conclusions. ¹⁷

Science students use formal analogies in their attempts to organise and systematise regularities observed in the real world. The science students are rarely afforded the opportunity of appealing directly to previous personal experience. This step by step method is particular to the serialist style of thinking.

According to Kolb, Science stream students would normally become convergent learners who rely on abstract conceptualisation and active experimentation. Refer to the table below to view the strengths and weaknesses of this learner type. ¹⁸

Table 3 Science Student Characteristics

The serialist cognitive style or operations approach to learning adopted by the Science students concentrates more narrowly on details before conceptualising an overall picture. Serialists combine information in a linear sequence and focus on the problem in a deductive manner. In addition, according to Tilema, the serialists work step-by-step, concentrating on well-defined and sequentially ordered details related through simple links. Serialists examine links between concepts to develop an objective logical argument. They find it easier to recognise and select the important information from its surrounding field/task. Their analytic approach to the task at hand leads them to be more experimental as well as have tendencies toward generating structure and their own hypotheses. ¹⁹ Witkin and Goodenough state that Science students have a greater ability to analyse and structure disorganised fields of information, a skill that is essential to problem solving. These students are more adept at imposing their own cognitive structures on situations encountered. ²⁰ Therefore, having a step by step approach to problem-solving helps the Science student to systematically analyse and single out the best or correct answers in the tasks presented such as in the IELTS test.

9.2.4 (ii) The Nature of the Arts Syllabus

The Arts curriculum gives more emphasis to the interaction between personal analogies and the information presented in order to form a direct connection between abstract principles and previous understanding. The method employed by the Arts students is reminiscent of the holist style of thinking. The Arts discipline encourages the creation of a symbol system that functions as a means for both the conceptualisation of ideas about aspects of reality and as a means for conveying one’s interpretation of or reflection on the task. In sum, the Arts curriculum is concerned with the capacity to respond intellectually, creatively and aesthetically to experience. It involves the exploration and understanding of personal analogies in relation to the task and connecting that to the process of making, composing and inventing solutions. ²¹

The Arts syllabus promotes a number of skills such as:-

• Collecting information through subjective and meaningful observation to develop /test an inference or a theme
• Using personal analogies to interpret the task
• Conceptualising the ideas/themes inherent in the task
• Selecting from a range of alternatives
• Exploring and elaborating on the chosen alternative/solution.

According to Kolb, the Arts Curriculum will produce two kinds of learners depending on their chosen electives. ²² Arts students selecting Arts electives such as Geography and Economics would normally become assimilative learners who rely on abstract conceptualisation and reflective observation. The weaknesses and strengths of this learner type are shown in the following table.

Table 4 Arts Student Characteristics - Assimilative Learner

Arts students selecting electives such as English and Malay Literature and Art would normally become divergent learners who rely on concrete experience and reflective observation. The strengths and weaknesses are as outlined.

Table 5 Arts Student Characteristics - Divergent Learner

The electives in the KBSM Arts Curriculum compels the students to develop a holist cognitive style that can be described as a global thematic approach to learning by concentrating first on building broad descriptions. The holist learner typically focuses on several aspects of the subjects at the same time. Interconnections between the theoretical, practical and personal aspects of a topic are made through the use of analogies, illustrations and anecdotes. ²³ Not focusing enough on detail is characteristic of the holist learning style. In addition, Arts students are more field dependent which means that they are more likely to accept and encode the information as it is presented. The Arts discipline trains its students to adopt a more global approach that allows them to experience and interpret a new concept within the presented format and context. Arts students rely more readily on external cues of an experience to point the way to understanding. For instance, Malay and English Literature encourage students to interpret literary tasks such as passages in short stories and novels and poems in order to find themes through interaction with the task and interpreting it using personal analogies (Witkin and Goodenough). ²⁴

In short, student learning would begin to differ because the students’ thinking processes differ depending on what the student is trying to learn (Jonassen and Grabowski). The nature of thinking and learning processes varies with the task. The learning outcomes of the different electives pursued required students to think in different ways. Jonassen and Grabowski related further that learning outcomes may be fostered or taught in many ways through the use of different micro level and macro level strategies. Learning outcomes are also affected by the form of instruction. ²⁵ Thus, different instructional activities will differentially affect learning outcomes. Learning outcomes, and the nature of thinking and learning processes related to them, are reinforced through standardised exams such as the PMR and the SPM. Science and Arts students also bring with them their respective sets of prior knowledge, prior structural knowledge and prior achievement. Due to the different emphasis of the respective disciplines, individuals differ in their abilities to process information, construct meaning from it and apply it to new situations i.e. approach employed in problem-solving.

As a conclusion, the individuals from the Science and Arts streams would differ in their response and interpretation of task items in the IELTS test due to their respective background disciplines. This aspect becomes important when the language skills the students possess are not adequate in helping them solve the tasks presented in the IELTS test, thus, compelling them to turn to alternative support i.e. their background disciplines. Therefore, students with average to low proficiency levels do significantly rely on the familiar problem-solving tactics learnt from their background disciplines which are relevant to the IELTS tasks as the statistical results have shown. It is probably due to the Science discipline advocating a more systematic step-by-step approach to getting the answer that the average to weak proficiency students in the Science stream had a greater number of correct answers than the Arts students of the same proficiency levels. Furthermore, it could also be due to the global approach that the Arts students rely on to derive the external cues in order to interpret the task items. This yields too many alternatives which leads to a higher probability of incorrect decisions.

Source note: The source PDF skips printed pages 54-55 at this point. The surrounding text is left as it appears in the available PDF.

background disciplines. From the mean score, it can be seen that the average candidate with an Arts background with a mean score of 5.5333 will not do as well as a candidate from a Science background who has an average score of 5.9667.

In summary, a candidate with weaker proficiency levels will rely more on skills learnt from their respective backgrounds to make up for the deficiency in language proficiency as well as the problems experienced due to unfamiliar IELTS topics. The results at the average level of proficiency also signified a difference. Therefore the situation suggested that as levels of proficiency decrease, the tendency for candidates from a Science background to do better than their Arts counterparts increases. At this level of proficiency, the scores attained by the two different sets of candidates suggest the edge one’s previous disciplines give over another candidate from a different background discipline.

The situation can be explained by:

• the policies implemented in the Education system which favour the Science students, as well as
• the influence and the effects of the learning styles advocated by the different disciplines on the IELTS test.

10.2 The effects of preparation classes on IELTS test results

The fourth aim is:

To determine whether preparation can help candidates to override the problem of lack of familiarity with the topics touched on in the Academic module in the IELTS and whether it is reflected in the scores.

2. As the IELTS test is a proficiency test, preparation would not make a difference in scores attained compared to no preparation at all.

On the whole, the t-test conducted produced the t-value of .69. Since the value falls outside the rejection region (-1.96 < t < 1.96), the hypothesis that students do equally well regardless of preparation cannot be rejected. In addition, the p-value which is derived from the two-tail significance test further supports the hypothesis. The p-value of .489 which is greater than .05 indicates that there is no significant difference between students who had gone for preparation and those who had not had any preparation at all. The mean score achieved by an average prepared student is 6.7800 whereas an average unprepared student achieved 6.7265. As can be seen, there isn’t much difference in the scores attained by the groups. This is probably due to several factors. For one, candidates who do not attend preparation classes would normally have a very high level of proficiency. Secondly, IELTS is a test of proficiency. Thus, it can be suggested that the knowledge of IELTS test-taking techniques is not a good enough substitute for fluency. It can, of course, aid the candidates in getting a better grade but the improvement in the grade attained is not large enough to be significant.

2.a.i For candidates with an Arts background, preparation will not make a difference in scores attained compared to candidates with no preparation at all.

The t-test conducted produced a t-value of .73. This value falls outside the rejection region -1.96 < t < 1.96. Therefore, for candidates from an Arts background, preparation will not make a difference in the score attained. Candidates from both groups have an equal chance of achieving a similar score. The p-value of .465 (a value which is larger than 0.05) derived from the two-tail significance test also supports the supposition that the hypothesis cannot be rejected as there is no significant difference in the scores achieved by average candidates from both groups. The mean scores achieved by average candidates from each group further attest to this. The average prepared candidate achieved a score of 6.7500 while the average unprepared candidate achieved a mean score of 6.6644. This lends weight to the hypothesis that the IELTS is a proficiency test. Therefore, preparation or no preparation would not make any significant difference to the scores attained. As both groups underwent education in the same discipline, the focus on the role played by IELTS preparation instead of the candidate’s background discipline is examined and can thereby be concluded to provide no material improvement to the scores.

2.a.ii For candidates with a Science background, preparation would make no difference in the scores attained by the prepared candidates as opposed to those candidates who have had no preparation at all.

The t-test conducted resulted in a t-value of .21 which fell outside the rejection region -1.96 < t < 1.96. The hypothesis cannot be rejected. Therefore, this result can be interpreted to mean that the preparation taken by an candidate from a Science discipline will not result in a significantly higher score. An unprepared candidate would have an equal opportunity to achieve a similar grade. The p-value of .832 achieved also shows that there is no significant difference in the scores attained by candidates from the two groups. The mean score further supports this hypothesis, as the mean score achieved by a prepared candidate is 6.8100 compared to the 6.7886 achieved by a non-prepared candidate.

2.b.i. At a high level of proficiency: A1 - A2 as measured by the SPM English 322 exam, the hypothesis states that preparation would make no difference in the scores attained by prepared candidates and unprepared candidates from the Arts discipline.

The test statistic conducted produced a t-value of -.70 that fell outside the rejection region of -1.96 and 1.96. Thus, the hypothesis cannot be rejected. The p-value achieved further supports the hypothesis, as the value of 0.486 can be interpreted to mean that there is no significant difference in the scores achieved by an average prepared candidate (the mean score achieved is 6.9571) from the scores achieved by an averaged unprepared candidate (the mean score achieved is 7.0393). At a high level of proficiency, the role that preparation plays is minute.

2.b.ii At a high level of proficiency, A1 - A2 as measured by the SPM English 322 exam, the hypothesis states that preparation would make no difference in the scores attained by prepared candidates and unprepared candidates from the Science discipline.

The test statistic conducted produced a t-value of -.41 that fell outside the rejection region of -1.96 and 1.96. Thus the hypothesis cannot be rejected. The p-value achieved further supports the hypothesis as the value of 0.680 can be interpreted to mean that there is no significant difference in the scores achieved by an average prepared candidate (the mean score achieved is 7.000) from the scores achieved by an averaged unprepared candidate (the mean score achieved is 7.0449). At a high level of proficiency, the role that preparation plays is small.

2.c.i At an average level of proficiency, C3 - C4 as measured by the SPM English 322 exam, the hypotheses states that preparation would make no difference in the scores attained by prepared candidates and unprepared candidates from the Arts discipline.

The test statistic conducted produced a t-value of -.23 that fell outside the rejection region of -1.96 and 1.96. Thus, the hypothesis cannot be rejected. The p-value achieved further supports the hypothesis as the value of 0.843 can be interpreted to mean that there is no significant difference in the scores achieved by an average prepared candidate (the mean score achieved is 6.2667) from the scores achieved by an averaged unprepared candidate (the mean score achieved is 6.3000). At an average level of proficiency, the role of preparation is also insignificant.

2.c.ii At an average level of proficiency: C3 - C4 as measured by the SPM English 322 exam, the hypothesis states that preparation would make no difference in the scores attained by prepared candidates and unprepared candidates from the Science discipline.

The test statistic conducted produced a t-value of -1.30 that fell outside the rejection region. Thus, as the t-value is found within the boundaries of -1.96 and 1.96, the hypothesis cannot be rejected. The p-value achieved further supports the hypothesis as the value of 0.200 can be interpreted to mean that there is no significant difference in the scores achieved by an average prepared candidate (the mean score achieved is 6.3667) from the scores achieved by an averaged unprepared candidate (the mean score achieved is 6.5556). At an average level of proficiency, the role of preparation is also insignificant.

2.d.i and 2.d.ii At a low level of proficiency, C5 - C6 as measured by the SPM English 322 exam, the hypothesis states that preparation would make no difference in the scores attained by prepared candidates and unprepared candidates from Arts/Science discipline.

No analysis was possible as one of the groups was empty.

In summary, the effects of IELTS preparation on students’ performance as reflected by their IELTS scores is insignificant. With the knowledge that preparation classes revolve around the techniques required to perform well in the IELTS test, the results still show that there is no significant difference between the results of students who went for preparation and those who did not.

The reason could be due to the brief duration of the preparation classes if compared to the duration of the disciplines the students had undergone i.e. two years. The transfer of learning had probably not taken place successfully in the former situation. While the students may have internalised the cognitive styles inherent in the subjects characteristic of the disciplines (due to the continuous reinforcements brought about by the education system which strongly advocates exams as an important evaluative tool), short programmes do not allow the students enough time nor the opportunity to actually internalise the methods enough for them to be of any significant use. In addition, it is also probable that the students could not adequately acquaint themselves with the topics touched on in the IELTS test to produce a significantly improved set of results. In short, by maintaining the Arts/Science curriculum as a constant, the use of preparation as a tool shows that there is no significant difference in the scores attained by the respective Arts/Science students at all levels with or without preparation.

A brief summary of the effects of preparation is shown:

• Students with strong language skills - language proficiency results of A1 - A2 in the SPM English had already had access to the strategies required to attempt the tasks successfully. Therefore, for students with the same level of proficiency, the preparation classes merely consolidated the skills they were to use during the IELTS test. The preparation classes probably also served to mentally prepare the student i.e. imbue them with a sense of confidence. In any case, the skills used by the prepared and unprepared candidates were probably similar if not identical.
• Students with average language skills - the C3 and C4 proficiency results in the SPM English had some access to the strategies required to attempt the test. Unfortunately, a C3 or C4 in the SPM English exam indicates that they could not easily access the language strategies and resources required. Thus, an average grade in SPM English has, as the statistics have shown, translated into an average grade in the IELTS test. Evidently, even a maximum of 48 hours of preparation classes was not sufficient in upgrading the IELTS scores significantly. Therefore, it is probable to assume that a short period of preparation classes is not enough to change, enhance or mould the learning styles of the candidates sufficiently for students to be able to bridge a taught concept or technique to new and similar instances or task items.
• Students with poor language skills - the C5 and C6 proficiency results in the SPM English had limited access to the strategies required to attempt the test successfully. Due to a poor structural foundation in language skills as indicated by the poor results, the students in this category opted for preparation classes. Unfortunately, the preparation programmes had only succeeded in introducing them to the techniques as they are unable to access the required resources when the transfer opportunity presented itself. In addition, they lack the ability to recognise appropriate transfer situations although the task was presented in a parallel format/situation.

11.0 Conclusion

Several conclusions can be drawn from the context in which the IELTS test is run in Malaysia. Firstly, it can be concluded that in the Malaysian context, background discipline does not generally affect one’s performance in the IELTS test. When this conclusion is scrutinised to a finer degree, it is found that candidates with a high proficiency in English do as well in the IELTS test regardless of their background disciplines. However, this situation is not mirrored in the performances achieved by candidates from different streams with average and low proficiency in the English language. It is found that Science students score better than their Arts counterparts in the IELTS test. This situation indicated that the Science students probably do have better language skills than the Arts students. This factor has been readily explained in the Section 5.0 Student Candidate Background. The streaming in the government schools has contributed to a situation in which Science students have a slightly higher level of proficiency than their Arts peers. In addition, the problem-solving skills taught in the Science streams probably had a more significant effect on IELTS scores than the Arts discipline. This factor has been explained in Section 9.2 Effects of the Malaysian Education System. Thirdly, it is also found that preparation does not change the IELTS score attained significantly. As can be concluded from the results, the average unprepared Science student has an equal chance of achieving the same IELTS score as the average prepared Science student. This scenario is similar to the situation faced by the average prepared Arts Student. Thus, a short IELTS course or program does not allow students enough time or the opportunity to actually internalise the methods enough for them to be of any significant use.

Bibliography

• Alderson, J.C. and Urquhart, A. H. (1983). Current Developments in Language Testing. The Effect of Student Background Discipline on Comprehension: a Pilot Study. Academic Press.
• Denton, H.G. (1992). Towards Maximising Pupil Endeavour: An Enquiry into a Learning Approach Centred on Team Work and Simulation in The Context of Technology Education. Loughborough University.
• Entwhistle, Noel(1988). Styles of Learning and Teaching: An Integrated Outline of Educational Psychology for Students, Teachers and Lectures. David Fulton Publishers.
• Hudson, L. (1968). Frames of Mind. Methuen Press.
• Jonassen, David H. and Grabowski, Barbara L. (1993). Handbook of Individual Differences, Learning and Instruction. Lawrence Erlbaum Associates.
• Keefe, J.W. (1979). Learning Styles: An Overview in NASSP’s Student Learning Styles: Diagnosing and Prescribing Programs. NASSP Press.
• Kilpatrick, W.H. .(1979) A Project Curriculum. Curriculum Design. The Open University Press.
• Language Testing in the 1990s(1991). Bands and Scores. Modern English Publications and The British Council.
• Lebar, Mohd Salleh (1996). Kurikulum Pendidikan Malaysia. Berita Publishing Sdn. Bhd.
• Loo, Seng Piew and Ismail, Maznah (Ed.s) (1994). Innovations in Education: Significance for Teaching and Learning. USM Press.
• McKeogh, Anne, Lupart, Judy and Marini, Anthony (Ed.s) (1995). Teaching for Transfer. Lawrence Erlbaum Associates.
• Medenhall, William, Reinmuth, James E. and Beaver, Robert J. (1993). Statistics for Management and Economics. Duxbury Press.
• N. Ford, (1985). Learning Styles and Strategies of Postgraduate Students. British Journal of Educational Technology 16(1).
• Tanner, Daniel and Tanner, Lauren N. (1975). Curriculum Development : Theory into Practice (1975). Macmillan Publishers.
• Testing English for University Study (1988). Testing English for specific purposes: How specific can we get? Modern English Publications & The British Council. The National Philosophy of Education.
• Tillema, H. (1982). Sequencing of Text Material in Relation to Information-processing Strategies. British Journal of Educational Psychology 32.
• Witkin, H.A. and Goodenough, D.R. (1976a). Field Dependence and Interpersonal Behaviour. Educational Testing Services.
• Veres, J.C., Sims, R.R. and Locklear, T.S. (1991). Improving the Reliability of Kolb’s Revised Learning Style Inventory. Educational and Psychological Measurement 51.