Text analysis and the assessment of Academic Writing

Abstract

This study was undertaken to investigate candidates’ interpretation of prompts and to compare the written responses of English background speakers with those of IELTS intending candidates, who were not English background speakers. Subjects, both native and non-native speakers, were interviewed during a pre-writing session as they prepared for a practice writing test. They then took the writing test under examination conditions. The interviews were analysed in order to gain insights into the subjects’ test preparation. The detailed analysis of the essays followed a discourse approach in order to add to the understanding of the characteristics of native speaker and non-native speaker candidates’ essays through the comparison of the essays. The findings suggest implications for item writers and for teachers preparing candidates for IELTS examinations.

Introduction

How test takers respond to writing assessment tasks is a neglected area of research (Green 1998, Hamp-Lyons 1990). Hamp-Lyons (1990) comments: “We need to collect process data as writers prepare themselves for a writing test, encounter it, and make their response, if we are to understand why they arrive at particular products…” (82). The study of writers taking tests is referred to as response validity. Alderson, Clapham and Wall (1995) describe response validation in the following way:

”… an increasingly common aspect of test validation is to gather information on how individuals respond to test items. The processes they go through, the reasoning they engage in when responding, are important indications of what the test is testing, at least for those individuals. Hence there is considerable current interest in gathering accounts from learners/test takers on their test-taking behaviour and thoughts”. (176)

The study reports on an investigation into the composition of texts by candidates preparing for the IELTS Academic Writing Subtest. In a previous study, Mickan, Slater and Gibson (2000) investigated the capacity of non-English speaking background students to demonstrate their writing ability on the IELTS academic writing subtest. In that study the researchers analysed data from the verbal protocols of intending IELTS candidates and from post-test interviews in order to explore aspects of response validity of the IELTS Academic Writing Test. The researchers identified a number of influences on candidates’ composition of responses to academic writing task prompts. Candidates’ comprehension of task prompts was influenced by the socio-cultural information embedded in the task prompt. This contributed to candidates’ uncertainty about their interpretations of the prompt. This in turn influenced the ability of candidates to produce appropriate responses to the prompt. Candidates used information in the prompt, as well as their internalised social knowledge about the purposes and structures of texts, including the lexicogrammatical resources for the realisation of their knowledge in the texts, for the creation of their responses. The study recommended further investigation into candidates’ interpretation of prompts through documentation of verbal responses during the initial reading of the prompt. In addition the study suggested the functional analysis of candidates’ written responses in order to identify how candidates’ interpretation of prompts guides the textualisation of meanings for the composition of appropriate responses.

This study addresses factors identified as influential determinants of response validity:

1. candidates’ capacity to interpret test prompts appropriately, and
2. candidates’ ability to act on their understandings of prompts through the textualisation of meanings in conventional, written responses.

The general purpose of the researchers in this study is to enhance test candidates’ demonstration of their writing ability. The study therefore has a practical purpose - to inform test designers, so that test prompts generate valid responses, as well as to provide insights into test design factors, which have implications for teachers, candidates and raters.

The study focuses on Task Two, which reflects concerns informally expressed by raters that a number of candidates do not complete Task Two, which suggests that they are not assessed on their level of actual performance. Some candidates appear to write without attending to the organisation of their written responses, which may be due to poor allocation of time on the part of candidates. It may also indicate candidates’ insufficient familiarity with the discourse resources necessary for the composition of their responses. It may be that Task Two places greater textual demands on candidates than Task One. The problem is that as Task Two is given greater weight in the evaluation of writers’ performance, the actual writing ability of candidates’ may not be exhibited in the responses. It seems therefore important to identify factors in the testing context which affect the performance of writers.

1.0 Context Of The Study

Candidates’ ability to show their writing ability is influenced by their interpretation of the task and their knowledge of the text type called for in the prompt. Crucial variables include the wording of the task and the representation of the task to the candidates as a distinguishable text type. The nature of the text type is significant, as raters’ scoring is influenced by candidates’ framing or structuring of their responses according to expectations of the particular text type being assessed (Tedick and Mathison 1995). These issues are examined in the following sections.

1.1 Candidates’ interpretation of the task

The nature of the examination task significantly influences candidates’ capacity to demonstrate their writing ability. In a discussion of the design of writing prompts Kroll and Reid (1994) assert candidates’ need for equal access to the task content. They explain that ‘Sometimes the difference between a well developed prompt and an inadequate or inappropriate one can reside in a single word or phrase that might render an otherwise opaque prompt transparent’ (248). The first part of this investigation was designed to gain insight into those features of prompts, which may influence candidates’ understanding of the task.

Various features in prompts impact on candidates’ understanding. A critical variable for non-native speaker (NNS) assessment is the extent to which tasks are culturally accessible, especially when tests are administered to candidates from different cultural backgrounds (Kroll 1998). Hamp-Lyons (1996) points out the need to avoid cultural bias by assuming shared cultural background knowledge. Candidates’ topic knowledge impacts on their comprehension of tasks. In addition, Mickan, Slater and Gibson (2000) describe how lexicogrammar influences the readability of prompts.

In order to investigate which aspects of a prompt candidates find problematic, we interviewed students individually during a ten minute period allocated for reading the prompt. The intention was to distinguish factors in the selected prompt which influenced comprehension and which had an impact on preparation for writing the response.

1.2 Knowledge of text conventions

A second factor, which significantly influences writers’ performances, is the nature or type of written text signified in a task. The authoring of texts is a socially contextualised activity. Social functions or purposes determine genres or text types, which are realised through socially sanctioned selections of discourse resources. Kobayashi and Rinnert (1996) point out in a study of cross-linguistic influences on discourse level composition that a lack of familiarity with English writing conventions may be detrimental to NNS’ display of writing competence. Mickan, Slater and Gibson (2000) note the differences in the ability of European candidates to compose responses compared with candidates from non-European cultures. European candidates display a wider range of discourse-semantic resources for the construction of their texts. The ability to access the textual resources of a known genre, which is invoked by a particular assessment task, enables participants to allocate attention to the planning of what to say (Skehan and Foster 1999) and to the formulation of meaning (Cumming 1989, Mickan 2000).

Research in this area suggests that the specification of a particular text type or genre determines candidates’ selection of textual or discourse elements for the composition of their responses. Kroll and Reid (1994) observe that there is a ‘problem when the presumed rhetorical style of a desirable response is outside of the cultural frame of reference for the test candidates’ (241). Reid (1990) reports that the nature of the writing task affects syntactic complexity, use of cohesive elements and the percentage of abstract words. Wallace (1994) comments that ‘The term text conventions is the most difficult for students to understand’ (56). Wallace also notes that experienced writers talk as though they have ‘an extensive repertoire of alternative conventions they could use’ (56), whereas less experienced writers appear to have a smaller repertoire. Kobayashi and Rinnert (1996) show that composing texts involves a multiplicity of factors including topic, coherence and language use, and rhetorical pattern.

Such studies suggest the importance of the discourse or rhetorical structure of written texts for the composition of responses. An implication for the design of prompts is the clarity with which a prompt articulates the type of text or genre expected in the response. The responses of NNS will be determined at least to some extent by their familiarity with the text conventions signalled in the task.

1.3 The Genre of Academic Writing Task Two

In the previous section we argued that the kind of written task set in the prompt is a significant influence on the way writers compose their texts. It influences candidates’ selection of rhetorical structure and the choice of lexico-grammar for the creation of particular types of texts. In addition, the organisation of the text influences raters’ expectations associated with the genre called for in the test prompt (Tedick and Mathison 1995). Therefore an initial question to be addressed is: What kind of text does the IELTS Task Two prompt require?

Moore and Morton (1999) claim ‘clearly IELTS Task 2 items most resemble the format of the university essay’ (85). They point out that the ‘written argument’ nomenclature does not correspond exactly to any of the genre terms identified in the university corpus they analysed for authenticity of task type. Their analysis of the Task Two, which has also been used in the present study, allocates it to the genre ‘written argument’ or ‘case’ with rhetorical functions of hortation and evaluation. Moore and Morton (1999) have noted that there is disagreement about the classification of genres. They have therefore used the concept of genre in an ‘unproblematic, self-referential way’ (72). However, given the significance of genre specification for authoring of texts, it is suggested that a more precise identification of genre may be necessary. The problem with a general classification is that it leaves open a range of text options, only some of which may be acceptable in an evaluation procedure. Dudley-Evans (1988) has pointed out the difficulty with instructions which lack specification of text type or genre in the prompt. In his analysis of the term ‘discuss’ in examination questions he points out that a general term like ‘discuss’ suggests at least three different text types. A general instruction therefore fails to give a writer direction for the selection of discourse semantic and lexicogrammatical elements for the composition of a text. It also fails to specify distinguishing textual criteria for raters’ evaluation of texts.

The instructions in the IELTS Handbook (1999: 11) state the following:

In Task 2 candidates are presented with a point of view or argument or problem.

Candidates are assessed on their ability to:

• provide general factual information
• outline a problem and present a solution
• present and possibly justify an opinion, assessment or hypothesis
• present and possibly evaluate and challenge ideas, evidence and argument.

Appropriate responses are personal semi-formal or formal correspondence (Task 1) and short essays or general reports, addressed to course tutors or examiners (Task 2). (11)

The instructions suggest that candidates will be required to produce an argument type text, which puts a point of view. The instructions also outline features of genre with the reference to register, rhetorical organisation, style and content. These are conventional features characteristic of texts types or genres.

The prompt used in this study requires candidates to write ‘an argument or case’. Analysts of genres consider ‘argument’ as belonging to a class of genre termed ‘exposition’ (Martin 1985), of which there are various subtypes. Gerot and Wignell (1994) describe ‘analytical exposition’ as a genre which has the social function ‘to persuade the reader or listener that something is the case’ (197). They also describe an ‘hortatory exposition’, also referred to as argument or persuasion, the social function of which is ‘to persuade the reader or listener that something should or should not be the case’ (209). They differentiate the two genres as follows:

analytical exposition argues that $\mathbf{X}$ is the case

hortatory exposition argues that X ought or ought not to be or should or should not be the case; exhorts someone to take or to desist in some action.

Gerot and Wignell’s (1994) analysis suggests that Moore and Morton’s (1999) classification of the rhetorical function of this particular task as ‘hortation and evaluation’ (80) implies different genres. The instructions to candidates for the Task Two used in this study match Gerot and Wignell’s (1994) description of the genre of ‘analytical exposition’. This description provides a focus for the linguistic analysis of the subjects’ essays in terms of genre, and provides a framework for the comparison of the native English speaker (NS) and NNS essays.

2.0 Methodology

The general purpose of the research was to explicate factors which impact on the capacity of non-English speaking candidates of IELTS to demonstrate their writing ability on the IELTS Academic Writing Task Two. We selected qualitative research procedures previously used with success in the investigation of written activity - the recording of verbal protocols and the text-linguistic analysis of subjects’ responses. During the pre-writing stage of the assessment procedure, we audio-recorded subjects’ verbal accounts of their actions as they read the task prompt (Mickan, Slater and Gibson 2000, Smagorinsky 1994). We analysed subjects’ essays from a systemic functional linguistic perspective (Gerot and Wignell 1994, Halliday 1985, Mickan 2000).

As a previous study (Mickan, Slater and Gibson 2000) had raised the question of cultural influences on candidates’ understanding of prompts as well as on the nature of their responses, we selected two cohorts of subjects - native English speakers and speakers with a background in languages and cultures other than English.

The aims of the study were:

• to analyse native and non-native speaker students’ interpretations of selected IELTS writing task prompts and their stated intentions for their written response texts;
• to analyse the resultant written products of the native and non-native speaker subjects in order to uncover their use of knowledge about text genre and topic knowledge in the composition of their responses;
• to compare the results from the native and non-native subjects in order to describe specific textual and knowledge factors which influence test candidates’ ability to compose appropriate texts in response to item tasks.

2.1 Procedure

The researchers selected subjects for the study from one institution which educated native as well as non-native students. Negotiations for the study began with the English as a Second Language (ESL) teacher, who provided essential assistance throughout the project. Initial communication with the teacher involved describing the project and discussing the feasibility of working in the school. The ESL teacher negotiated approval with the School Director. We prepared letters and an information brochure about the project for students and their parents/guardians and invited participation as well as consent for participation. This was obtained for all students who took part in the project.

2.1.1 Site and subjects

The subjects in this study attended senior level classes in a metropolitan secondary school. In order to achieve a degree of similarity in terms of participants’ educational contexts across the native and non-native cohorts, it was decided to invite twelve students from the same institution to participate in the study. They were studying similar subjects in senior school. Students’ school assessments ranged from pass to distinction level. All subjects were female students intending to go to university. The NNS came from the following first language backgrounds: 1 Japanese (aged 19); 1 Indonesian (aged 17); 4 Chinese (aged 17). The native speakers were all Year 11 students aged 16. It was felt that by selecting the NS at the lowest recommended age for IELTS we would to some extent reduce their advantage in terms of experience with academic writing in English.

The NNS were taught by an ESL teacher in one class. The NNS subjects had not taken an IELTS examination at the time of data collection, but they were intending to take the IELTS examination in the next school term. The NNS had been practising for the IELTS examination once a week over a period of three months. The NNS students were studying their other academic subjects in an English medium. The NNS had studied English for different lengths of time and had studied in a variety of programs before coming to school in Australia. The types of programs ranged from traditional grammar to English as medium of instruction. Instead of attempting to achieve experiential equivalence in terms of the English amongst the NNS, the researchers chose instead to select students enrolled in the same year level.

The NS students had no prior knowledge of IELTS tests, and had received no preparation for the IELTS writing subtest.

Students who initially indicated an interest in taking part in the project attended an information session in which we gave a description of the project and answered questions. The researchers then gave an outline demonstration of the planned interview process, prior to students’ rehearsal of the same process with a partner. This was followed by questions and further discussion of the procedure for data collection. A date was set for all subjects to participate in the data collection at the same time.

2.1.2 The interviews

The interviews with each subject were conducted for a ten minute period. The series of questions used as an interview resource and as the main basis for prompting are shown below.

(You can) Write notes (to help you) if you like.

Tell me about the question.

What is the question about?

What does the question ask you to do?

How will you answer the question?

Tell me about the topic.

Tell me about what you are going to write.

Figure 2.1: Interview prompts

The researchers conducted the interviews. They attempted to keep the interviews relaxed. They prompted subjects gently when they fell silent for a while. The aim was to maximise subjects’ talk about their actions in a relevant way. The researchers did not offer comment when subjects sought explanations from them about the actual content of the task prompt. The interviews were audio-recorded for later analysis of the content.

2.1.3 The written test

The same task was administered to all subjects (Figure 2.2). The task selected for use was available in IELTS Specimen Materials, therefore it was not subject to the confidentiality restrictions of non-live IELTS prompts. The researchers considered the topic was of international interest and relevance and therefore accessible to both NS and NNS subjects.

WRITING TASK 2 (taken from IELTS Specimen Materials 1995)

You should spend about 40 minutes on this task.

Present a written argument or case to an educated non-specialist audience on the following topic.

The first car appeared on British roads in 1888. By the year 2000 there may be as many as 29 million vehicles on British roads.

Should alternative forms of transport be encouraged and international laws introduced to control car ownership and use?

You should write at least 250 words.

You should use your own ideas, knowledge and experience and support your arguments with examples and relevant evidence.

Figure 2.2: Task Two used in the study

The subjects took the written test directly after the interview. They went into another room where they were supervised by the ESL teacher and given thirty minutes to write their responses. They were able to take into the examination any notes written in the 10-minute interview. At the end of the 30 minutes, all materials were collected and given to the researchers.

2.1.4 Analysis of data

We collected two sets of data - recorded verbal protocols and subjects’ responses to Task Two. The two sets of data required different analytical procedures. The recorded data from the interviews were transcribed and read critically by both researchers in order to identify issues raised by subjects. The researchers undertook this independently in order to reach agreement on significant factors reported by subjects as they read and prepared to do the writing task. The data from the NS and NNS were compared.

The essay scripts were analysed from a systemic functional linguistic perspective (Halliday 1985). The focus was on distinguishing linguistic features of the essays. The mistakes and errors in the use of English were not included in the analysis. The linguistic approach to the analysis of the essays enabled specific identification of characteristics in the essays, which appeared to be features of successful argument texts.

3.0 Interpretation of Prompts and Preparation for Response

The first aim of the project was to analyse native and non-native speaker students’ spoken interpretations of an IELTS Writing Task Two prompt, and their preparatory activity for composing a response. The interviews with students revealed a number of features of the task which proved challenging for the NNS subjects. The factors identified and discussed in the following sections are selected for the prominence accorded to them by the NNS. The selections focus on the general perceptions and accounts, which illustrate differences between NS’ and NNS’ task preparation.

3.1 Understanding the task

The interviews with native speakers (NS) revealed that they grasped the meaning of the topic quickly and easily. During the ten minutes of the interview they commented on aspects of the topic and they began to formulate written responses. They did not have to spend a lot of time thinking about the meaning of the prompt, giving them more time to prepare their responses.

For non-native speakers (NNS) this was not the case. They had difficulty with some individual words as well as with grasping what the question was asking for. Three students found the meaning of ‘alternative’ problematic, which contributed to their uncertainty about the topic. One NNS was unsure whether ‘British roads’ meant roads in Britain or roads built by the British, presumably during colonial hegemony.

In addition the purpose of the task was not clear to some of the NNS. One student found it confusing:

Her comment is one example of the difficulty NNS experienced when seeking the meaning of the task. Similarly NNS 11 said:

Actually ‘erm I’m not really sure what the questions are asking. [NNS 11]

The prompt is in fact quite complex: it is ‘double-barreled’, requiring candidates to evaluate alternative forms of transport and international laws as a means of controlling car ownership and use.

Compared to the NS, the NNS had to devote comparatively more effort to configuring the meaning and purpose of the task and therefore had less opportunity to prepare and to organise concepts for their responses.

3.2 Topic knowledge

Topic knowledge embedded in the task presented some of the NNS with difficulties. An example was the term ‘international laws’. NNS 8 for example said:

Also NNS 7 stated that:

Another NNS student seemed to assume that there was information, with which she should have been familiar, whereas the particular angle of the question was a specific prompt for a more general issue.

I got surprise when I read it sentence ‘The first cars appear on British roads in 1888’. I didn’t know that actually Sorry!; I think I read some articles before but I forgot already otherwise I think it would help in my essay for idea.; [NNS 11]

It seems that the NNS had an expectation that there was a particular body of knowledge with which they were expected to be familiar. The NS on the other hand didn’t interpret the task as a test of knowledge or embodying a request to show knowledge previously learnt.

The NS did not need to wrestle with the meaning of the task. Their focus during the ten minutes of prewriting was on the generation of ideas and to a lesser extent on the organisation of their responses. The NS generated a range of ideas while brainstorming options for their responses. They evaluated and cross-referenced different ideas.

3.3 Determining the Task

In 3.1 above we mentioned that some of the NNS subjects expressed problems with grasping the purpose of the task. The NNS required more of the 10 minutes of time than the NS students to decide on the nature of an appropriate response to the prompt. They searched for contextual understanding and appropriateness as they thought about the topic:

I don’t know how serious the topic is, I mean about needing alternative transportation… [NNS8]

I want to know if British roads is in a country and then in only one country. [NNS11]

There is evidence of a lack of confidence on the part of the NNS in the interpretation of what to do:

Researcher: Can you understand what to do?

Erm…not really; I can think for ‘against’ but I can’t think idea for ‘for’, [NNS7]

Actually not quite sure what’s the difference between argument and case [NNS10]

I don’t really understand actually [NNS12]

Because each like this question have like two questions it should be like the answer must be balanced but I don’t think I could balance the quest..answer; [NNS7]

The subjects’ decisions about what the prompt required in terms of text type set the parameters for the structuring and wording of their essays. The determination of the nature of the task was closely related to another issue, which was highlighted in the interviews - the adoption of a point of view.

3.4 Adoption of a point of view

The prompt requires writers to adopt a point of view. The NS tended to express with some precision their intentions for the written response. For example

Even when a NS expressed a lack of knowledge about cars she was nevertheless confident about how to tackle the question:

Okay I don’t know much about cars much - but yes I agree with it and I think that other forms of transport should be used [NS1]

This statement of NS1 contrasts with the following, in which NNS8 also expressed lack of knowledge of an aspect of the topic:

yeah I think erm public transports perhaps should be encouraged but erm international laws I don’t know much..too much about laws or things like that how it works… [NNS8]

In general the NNS are more tentative in outlining their intentions. They give the impression of less certainty in the representation of their perspective. NNS7 for example says:

I think I can put something like we should like alternative forms erm because erm I can think something like greenhouse effect [NNS 7]

3.5 Comment

The interviews with subjects as they prepared for writing their responses revealed specific aspects of the prompt which have an effect on candidates’ capacity to demonstrate their written competence. The production of a text under constraints of time and the pressure to perform add to the complex interplay of task purpose, topic knowledge and point of view which comprises the authorship of texts. A candidate has more time to devote to preparation for composition of a response if she can determine with relative ease the purpose and topic of a prompt. Attention can be devoted to the decisions required for the composition of the response.

For the NS it was unnecessary to explicate the meaning of the task, which NNS needed to do. During the preparation time the NS combined the generation of ideas with the formulation of responses. They accessed technological terminology related to the topic. They also expressed their ideas in a coherent way.

The NNS spent time reviewing the prompt in order to determine what to do and how to do it. They needed to reassure themselves that they had understood the task. Intermingled with this they searched for key concepts and deliberated on the organisation of their responses.

4.0 Analysis of Essays

The second aim of the study was to analyse the essays written by the subjects in order to look at their realisation of the task and then to compare the main factors which influenced candidates’ capacities to compose appropriate responses.

The following analysis focuses on differences observable in the language of the texts. We have not specifically examined errors or mistakes in the essays.

4.1 A general comparison of NS and NNS texts

The topic of the chosen task was relevant for the subjects and all of them were able to address aspects of the topic in their responses. It met the broad requirements of a well-designed task in that the topic was ‘of general interest’ and required “no specialist knowledge” (IELTS Handbook 1999: 11). The NS essays were not rated. The non-native speakers’ essays were rated at an average level of five by an accredited IELTS assessor. This suggests that NNS subjects grasped the general meaning of the task and were able to generate sufficient relevant content for the authoring of satisfactory responses

When we examined the surface features of the essays, such as the number of words and the length of sentences, it appeared that differences between the texts were not significant. Therefore an accurate differentiation needed a closer analysis of the linguistic features which made up the essays of the students.

In the next section the essays of the NS and NNS cohorts are compared. In the subsequent section the differences are examined through a detailed linguistic analysis of two representative essays, which illustrate differences between native speaker and a non-native speaker essays.

4.2 Elements of text - a linguistic analysis

The following analysis is based on systemic functional grammar (Eggins 1994, Gerot 1995, Gerot and Wignell 1994, Halliday 1985, Halliday and Hasan 1985). We have described the essays as argument texts belonging to a genre group called analytical exposition (Gerot and Wignell 1994), which is concerned with the analysis and interpretation and evaluation of events in the world around us. The social purpose is to persuade someone to a point of view - to take a position and to justify it. An argument is built up through language patterns of causality and logic and the presentation of point of view. This is created through a writer’s selection of lexical and syntactical resources. Thus the social purpose is achieved through particular linguistic choices which include:

• to refer to generic or non-human participants;
• to nominalise and use abstract technical terms in the attempt to establish objectivity;
• to develop an argument and persuade readers through the organisation of coherent text, for example, relational processes and conjunctions to stage the argument;
• to compress information and be concise through, for example, nominalisation.

The following analysis focuses on selected linguistic elements in the essays, which contribute to objectivity, coherence and conciseness. Words crossed out have been signalled through the use of brackets [ deletions ], and words inserted into the essays are indicated by arrows > inserted text <.

4.2.1 The organisation of the essays

The social function or purpose of a text is realised on a whole text level through the organisation or staging of the text. Eggins (1994: 89) calls this ‘generic coherence’. The schematic structure of a text moves the reader through the stages of a text in order to achieve its purpose or goal. The different stages of a text use different resources of the language in order to achieve the purpose or goal of the writing task.

The organisational features of the essays of the two research cohorts do not conform in a narrow sense to a particular genre, and there is considerable variation in the texts of the two cohorts. In general the native speakers have structured their texts more closely to the staging of an analytical exposition. In their opening paragraphs they present a thesis or take a position or outline the issue. The NS writers have begun their essays in different ways. Here are two contrasting essay openings:

I think that, in today’s world, improvement [are] is encouraged and seen as an important aspect of society. However, there is a difference between deciding and thinking something is a good idea, and actually carrying out that idea. I agree that alternative forms of transport be >should< encourage and international laws should be introduced to control car ownership and use.’ [NS4]

The rising number of cars on our roads is certainly alarmingly high, given the dangers they present to the environment, however, their use as a major form of transport in today’s society should not necessarily be discouraged nor controlled by the introduction of international laws [NS3]

The first extract personalises a point of view, whereas the second extract is abstract and impersonal. However, both writers use the opening paragraph as a foundation for articulating their positions, which they subsequently elaborate in building up their arguments. Similarly the NS explicitly write summary statements in their final paragraphs which restate their point of view. They signal this linguistically in the following ways: ‘in closing’ [2 students]; ‘in conclusion’ [2 students].

As with the NS’ essays, the organisation of the NNS’ essays show considerable variation. In the following essay openings the writers establish the topics which they elaborate upon in their essays.

I think today there are to many cars everywhere. it causes more traffic jams and more accidents everywhere all over the world. When the first car appeared on British in 1888, there were only a number of cars. But by the year 2000 there may be as many as 29 million vehicles on British roads [NNS 9].

Cars are important transportation in today’s world. Without cars, we will have trouble travelling around [NNS 10]

Other essays of the NNS’ are less explicit in articulating arguments in their opening paragraphs. Here are two examples:

The growth of technology have become very convenient to us all. Cars [has been] is one of a >very < big issue presented to [us] > the world we’re living in<. the transportation of cars brought convenience to many people, but the technology has now seem to lead to other problems such as [popl] [pup] pollution and over populated with vehicle in certain area’ [NNS 8].

[As past > time has< gone past quietly >and< everything changed [no] not only we are] [The] Many countries have been developed as past have gone past. Our living standard have been improved a lot too. People [for example] > can [get] buy< more things that they wanted and people have longer life as the medicine. population actually increased[d]. From 100 years ago only one car, there will be 29 million vehicles by the year 2000. there are two sides of [arguments] > points views< whether [we] > people< should [* illegible] control car ownership and use as there will be more > people driving on the road<

Some people think that [we should have] > driving their < own is better [NNS 11]

The NNS writers raise issues in the opening sections of their essays but they do not articulate a definite point of view. In her opening paragraph NNS 11 writes:

there are two sides of [arguments] >points of views< whether [we] >people< should [*illegible] control car ownership and use as there will be more >people driving on the road<. [NNS 11]

In the next paragraph she writes ‘Some people think ..’. She opens her third paragraph with: ‘On the other hand’. In the final paragraph the writer adopts a position on the topic: ‘Personally, I think..’. The essay of NNS 11 is organised as a discussion. The organisation of the essay includes the preparation of the reader for the presentation of alternative viewpoints in the body of the essay, with the writer finally asserting her perspective in the final paragraph.

Whereas the NS conclude their arguments in their final paragraphs by restating their positions, this was not typical of the NNS. NNS 12 finishes her essay as follows:

Owning a car maybe can make easier for your life, but [use] if you use more [trans] public transport will make more health for the young generation. [NNS 12]

NNS 8 writes at the end:

[I thi] Uses of cars should indeed be reduce for the sake of a better environment and we all live in an area that we don’t have to feel [indecipherable word] to our own surroundings [NNS 8]

The texts are rounded off with a final paragraph but without drawing a conclusion.

The NS essays reflect the social purpose of argument texts: ‘to persuade the reader or listener that something is the case’ (Gerot and Wignell 1994: 197). The native speakers use resources for the organisation of their arguments so that there is transparency in the point of view being presented.

In the NNS’ essays there is less transparency. The writers raise a range of issues and discuss them in their texts. Discussion texts have the social purpose ‘to present (at least) two points of view about an issue’ (Gerot and Wignell 1994: 214). The essays are generally not structured technically as argument texts, but rather as discussion texts, in which different ideas are discussed rather than arguments developed.

4.2.2 The lexicogrammar of the essays

The following analysis highlights in more specific terms differences between the texts of the two cohorts. The analysis of the lexicogrammar provides a basis for the differentiation of the NS and NNS essays.

4.2.2.1 General participants

The nouns that refer to what is going on in a sentence are referred to in functional grammar as participants (Halliday 1985). The analysis of participants in a text reveals who or what is taking part in whatever is happening in the text. Written argument texts assume academic features and the stating of an argument in objective rather than personal or emotive discourse. Argument texts utilise impersonal participants, achievable through the use of the passive, or general participants, as for example ‘people’, in order to make a convincing case.

The NS writers have used both the passive and general participants in their texts. Only two NS use I/we: NS 4 uses it to organise her argument. She personalises her argument beginning with the statement of thesis - ‘I think’ - then follows with the expression of her point of view- ‘I agree that alternative forms of transport >should< be encouraged’ and ‘Although, I must clarify my opinion’. However the usual pattern for native speakers is to use either generic or non-human participants.

The NNS use personalised and general participants. The passive form of the verb is used less by the NNS and the prose resembles the spoken language of a discussion. NNS 10 ends her essay in the following way:

Imagine that, if it really happens. What will our future world be like? with all these cars all over the place? Will they even have room for the pedestrians to walk on?

[NNS 10]

The personal form of address is not as convincing as the use of generic participants.

4.2.2.2 Nominalisation

One of the markers of linguistic maturity is the compacting of information in texts with abstract and technical wordings (Derewianka 1999). Gerot (1995) points out that ‘Heavy nominalisation makes a text sound prestigious, abstract and formal, authoritative and impersonal’ (76). Through nominalisation, information or meanings are first flagged and then built up through a text.

A strategy the NS use to make their texts sound authoritative is the use of nominalisation. Apart from one of the NS texts this is used effectively to concentrate information in their texts. NS utilise the information-packing capacity of nominalisation and nominal groups as normal components of their essays. NS 3 for example begins her text

The rising number of cars on our roads is certainly alarmingly high. [NS 3]

By way of comparison NNS 9 the essay begins her essay as follows:

I think today there are to many cars everywhere.

[NNS 9]

Although the two openings express plainly comparable viewpoints, the nominal group (underlined) in NS 3’s text creates an impression of mature writing.

4.2.3 Cohesion

Cohesion refers to the linguistic resources a writer uses to link together parts of a text. It describes the linguistic resources for providing continuities in a text (Halliday and Hasan 1985) and examines meaning-making at the level of text. The meaning relations between the parts of a text are constitutive of the texture of the text (Halliday and Hasan 1985: 71). The relationships between the parts of the text are realised through the lexico-grammatical patterns in the text.

In this section we have selected three kinds of relationships which go to create continuity in text. These are of particular interest in the comparison of NS and NNS essays because a feature which differentiates their essays is the use of cohesive ties creating internal unity in texts.

4.2.3.1 Reference

Reference refers to the resources in English which are used to follow the identity of participants through a text. Participants may be people, places or things. After a participant has been introduced into a text a reader keeps track of the participant through the system of reference in the language. A pronoun refers back to the mention of the particular participant in a previous part of the text, so that, for example, when a reader encounters the pronoun the reader retrieves or recalls the participant which the pronoun refers to in previous sections of the text. Reference items include the definite article, demonstrative pronouns such as ‘that’, ‘these’, and pronouns such as ‘he’ ‘she’ and ‘it’. When a reader has difficulty in the retrieval of a referent, this makes a reader’s understanding of the text difficult.

Learners of English have trouble with the system of reference when learning to write, because the systems of reference are not the same across languages (Gerot and Wignell 1994). Patterns of reference present difficulties for learners of written English. When the system of reference is not used with clear links between referents and their reference items, the reader experiences difficulty with the interpretation of the meaning of the parts of a text.

The comparison of two excerpts illustrate how reference contributes to the impression of continuity in the essays.

There are many alternative forms of transport available in the world today. Busses, trains and car pools should be encouraged to be used by making the public more aware of the availability of them. This could be done by showing people that less cars means a better environment and cleaner air, in the future. Possibly making these services more convenient would encourage more people to use them. [NS 6]

Alternative form of transport should be encouraged, so that the air will be less polluted. We should use more of the public transport, if they are there, why not use it? Also, international laws should be introduced to control car ownership as well, so that it might lessen the number of people >to< own cars. There are already many cars on the road now and cars also cause traffic congestion. [NNS 10]

[Note: words crossed out by the writer have not been included in this copy]

The instances of reference using the demonstrative pronoun in the first text above, which have been underlined, are easy to follow. Also the list of means of transport in the second sentence refer without explicit explanation to alternative forms in the previous sentence. The pronouns underlined in the second text are not so easily traced to items which have already appeared in the text. The reader must stop to trace the antecedents in the text, which is made more difficult by the mixing of plural and singular pronouns. Across the essays of the two cohorts the NS use reference with more assurance than the NNS.

4.2.3.2 Lexical cohesion

Lexical cohesion describes the relationships between content words in a text. It establishes through the selection of specialised vocabulary the topic or topics of a text. Lexical cohesion works together with Reference to establish continuity in a text. The choice of words creates connections through repetition, through similar meanings and through collocations, in which words tend to occur together (Halliday 1985: 289). Although differences in lexical cohesion between texts of the two cohorts of writers are not definitive, there are some tendencies which suggest this as an area which marks out successful text construction. NS tend to use synonymy and collocation as a means of building lexical cohesion through their texts, whereas the texts of NNS incorporate more repetition of content words.

In the essay segments above, both writers use lexical cohesion to build information about transport in their texts. The lexical strings related to transport have been italicised. The topic of transport accumulates meaning with the use of a variety of lexical terms associated with transport:

alternative forms of transport; Busses, trains and car pool cars; cars; services

[NS6]

alternative forms of transport; public transport; car ownership; cars; cars; cars;

traffic congestion

[NNS10]

This is a linguistic means of linking content through the text, which supports understanding of the text and the making of meaning throughout the text. The repetition of ‘cars’ in the second excerpt [NNS10] is a feature of other NNS texts, which generally utilise fewer lexical options for assembling the topics in their essays.

4.2.3.3 Conjunctions

Conjunctions link together information in texts. They join clauses and also sentences. A text is structured using, for example, temporal conjunctions. The use of temporal conjunctions creates steps in the argument of a writer. The NS use causal-conditional conjunctions for linking points in their argument. They use external temporal conjunctions to stage their arguments e.g. ‘first’, ‘second’. They also express their reasoning or argument through causal conditional conjunctions, thereby linking points in their arguments. They also use comparative conjunctions to introduce counterpoints - ‘however’, ‘on the other hand’ - and to draw conclusions or to finalise a point - ‘therefore’, ‘consequently’. The NS manage this aspect of text so that the reader keeps track of the argument and point of view of the writer while reading through the essay. In general these resources have been used more effectively by the NS writers than by the NNS.

In this section we have introduced the analysis of selected textual features of the essays of both the NNS and the NS. The features, which contribute to differences between the texts of the two cohorts, are examined in more detail in the next section through the analysis of two texts.

5.0 Analysis of Texts

The linguistic analysis of a NS and a NNS text demonstrate features distinguishing the texts. The focus is on selected, textual differences between the texts, rather than on discourse features which are common to both. The selection of the two texts was based on the exhibition of features representative of the NS and NNS texts generally. The two selected texts were of exactly the same length.

5.1 Analysis of native speaker text

The essay written by NS 2, a native speaker, is reproduced in Figure 5.1.

Figure 5.1: Essay of NS 2

5.1.1 The schematic structure of the text

This text is structured to carry out the social function of building a persuasive case. In Figure 5.1 the right hand column summarises the schematic structure of the essay. It shows a clearly structured text in which NS 2 foregrounds her thesis in the opening sentence, stating her point of view. The position she takes on the argument is indicated through the use of modality to express opinion - ‘should be encouraged’. The essay develops the thesis through the presentation of three main ideas with supporting points and elaboration. The first idea focuses on ‘alternative transport’, which is elaborated through a series of points. The second idea puts a case for encouraging the use of alternative forms of transport and the reasons for this. The third idea takes up the topic of international laws and proceeds with the stating of a series of points. The final paragraph clearly signals the end of the argument with the words ‘In conclusion’ and restates the writer’s position. This schematic structure provides the reader with recognisable linguistic signals, which aid the interpretation of the text. The organisation of the text meets the expectations of a reader thus strengthening the capacity of the text to persuade the reader to the writer’s point of view.

5.1.2 Participants

NS 2 does not personalise the participants. She uses instead abstract nominal groups such as ‘public transport’ and she also uses impersonal participants such as ‘people’. The use of non-human, abstract and impersonal participants establishes a voice, which comes across as academic and authoritative.

5.1.3 Nominalisation

NS 2 utilises nominalisation to compact information in her text. She has used the prompt as a source for the nominalisation ‘Alternative forms of transport’, which establishes the topic of her essay. This abstract summary also gives the text a technical and academic voice. After developing the theme with an explanation of alternative forms, NS 2 develops the idea further with the nominalisation ‘Public transport’. Because she has established the theme from the beginning, this becomes assumed information as the text progresses, so incrementally she develops the argument. Later in the same paragraph she signals a new theme with the abstraction ‘harmful pollutants’ and develops a new argument, which is expanded over two sentences.

The same strategy is used in the next paragraph which begins with ‘International laws’. The topic ‘international laws’ is foregrounded in the theme position which is built upon in the rheme position of the clause, ‘to control car ownership’, another instance of densely packed information in the noun-like form. The subsequent sections of the essay refer back to the previous information, while moving the argument forward. The second to last sentence makes reference to the previously nominated topic of ‘harmful pollution’: ‘This would reduce pollution for one day at least’.

Nominalisation enables NS 2 to pack information densely into each clause. It compacts the information in an abstract and authoritative style. The use of nominal groups also compacts information into a clause in examples such as ‘a limit on how many cars each household can own’; ‘the number of cars on British roads’; ‘the amount of pollution released into the atmosphere’. Nominalisation also helps to structure the text through topic nomination and elaboration and through subsequent reiteration. Ideas which have been recorded in early sections of the text are established as points of reference later in the text and permit thematic progression. This is part of the patterning system in the essay, which the writer exploits for maintaining continuity in the essay.

5.1.4 Cohesion

NS 2 uses a variety of textual resources for the creation of cohesion in her text. We have already seen how she does this through nominalisation, where the meaning of a theme is picked and carried through lexical cohesion. In this section the contribution of Reference and Lexical Cohesion to the creation of cohesion in the essay are examined.

5.1.4.1 Reference

Reference describes the language resources which introduce and track the identity of participants through texts (Gerot and Wignell 1994). An example is the use of anaphora - ‘this’, ‘they’ - which refers to previous ideas, persons or segments within a text. In the following extract from the essay of NS 2 the reference items are in bold with their referents underlined.

However, walking or riding a bike is even more of an advantage as they are environmentally friendly. This can also be a disadvantage because it takes longer to travel from A to B. A way to encourage people to use these alternative forms of transport, can be done through school and the media which includes, the radio, T.V., newspapers and magazines.

This example illustrates the chain of reference to one participant group - ‘walking or riding a bike. The referents are not consistent as in the use of This and these. Nevertheless the reader is able to retrieve the reference items, without losing the meaning of this section of the text.

The insertion of ‘this’ in the next example illustrates how anaphora is used to refer to a segment of the text:

‘Maybe there should be a law introduced where there is a car free day, one day for the week. This would reduce pollution for one day at least.’

‘This’ here refers back to the previous section of the text and not just to one participant.

5.1.4.2 Lexical cohesion

The writer has used words from the prompt and also chosen vocabulary appropriate to the topic and technical nature of the text to establish topic links in the essay. The following extract (paragraph 2 - NS2) contains a number of lexical strings. The clauses are numbered.

1] Alternative forms of transport include, buses, trains, car pools, walking and riding a bike. 2] Public transport means 3] that there is less people driving cars 4] which reduces the number of cars. 5] However, walking or riding a bike is even more of an advantage 6] as they are environmentally friendly. 7] This can also be a disadvantage 8] because it takes longer to travel from A to B. 9] A way to encourage people to use these alternative forms of transport, can be done through school and the media 10] which includes, the radio, T.V., newspapers and magazines. 11] First of all, for people to be encouraged, 12] they must be educated in school about the environmental impacts on the environment 13] if the number of cars on British roads increases. 14] Harmful pollutants are released from cars 15] which causes global warming and smog for instance, not to mention the problems with acid rain. 16] People have to be encouraged to use alternative forms of transport if possible, 17] and they have to be educated about the environmental impacts on the environment.

Figure 5.2: Lexical chains in extract of essay of NS 2

This diagram displays the sequence of the main lexical chains on three main topics. The clusters of words around certain sections of the text, such as those to do with the environment from clauses 12 to 15, indicate that the writer is focusing on a particular theme at that point. This theme is embedded in the main argument on alternative transport, which continues through this extract from clause 1 to clause 16. This keeps the main argument on track. The lexical strings show the use of technical words for the development of the topic. The use of limited repetition contributes towards the continuity in the text. It shows the general semantic relationship between words and contributes to the cohesion or continuity in the text.

NS 2 has used a variety of resources for the creation of textuality, showing flexibility in the selection of appropriate ways of conveying meaning and creating continuity through the text. In addition she has chosen linguistic resources which mark the text as academic and authoritative.

5.2 Analysis of non-native speaker text

The text in Figure 5.3 was written by one of the NNS. The following analysis does not focus on specific errors in the text, but rather concentrates on discourse features and on the linguistic resources used by NNS 7 to compose the essay.

Figure 5.3: Essay of NNS 7

5.2.1 The schematic structure of the text

On a first reading of the essay it is not easy to discern the type or genre of text produced by the writer. This can be explained by the lack of schematic structuring of the text according to the conventional organisation of an argument or case (see 1.3 above). This is apparent in the analysis of the schematic structure in Figure 5.3. The writer has divided the text into three paragraphs, but otherwise there is no discernible marking of an argument text beginning with a preview as introduction, followed by points of argument which are elaborated, and finishing with a summary of the case or argument. This deprives the reader of the familiar staging of an argument text, which, we suggest, supports understanding of a text.

It could be argued that the organisation of the text does not follow the instructions in the prompt “to present a written argument or case”. There is no introduction presenting the thesis. The subsequent presentation of ideas is structured as a discussion text with the presentation of different points of view, or arguments and counter arguments. NNS 7 only states her point of view in the final paragraph. The final statement reads as an apology as though the writer recognises that this aspect of the prompt has not been adequately dealt with. The sentence reads like a spoken statement, rather than a summing up of argument. This is more typical of a spoken discussion.

A reader who expects an argument text is not guided through the text by its structural elements. The lack of staging makes it difficult to follow the meaning of the text.

5.2.2 Participants

The writer has used the general participant ‘people’ as the main means for the expression of agency in her essay. Its repetition does not create variety in the essay. In her final paragraph she uses the first person to express her point of view. This may be less convincing than the use of impersonal agents as it comes across as a personal viewpoint rather than as an authoritative point of view or perspective.

5.2.3 Nominalisation

NNS 7 does not normally signal topics through nominalisation. This is apparent in the opening sentences of the essay:

In British use of car is increasing rapidly. It was introduced in 1888. Now, most of families have at least one car. As a result, it is thought by the year 2000 there will be about 29 million vehicles in British. Some people concern about it because of air pollution. Other people think it wouldn’t have much differences. There is a idea >that is< to introduce international laws to control it.

NNS 7 has opened her essay with a clause which is not as condensed and abstract as ‘The rapid increase in British car use…’ Later, in the same paragraph, by way of contrast, NNS7 nominalises the topic of the clause:

The benefits of alternative ways are there will be less polluted air. the one alternative way is using solar energy cars. These cars will not produce any poisoness gas. This poisones gas [ is ] causes the greenhouse effect. People who use car everyday have different idea.

Here the summarising statement ‘The benefits of alternative ways’ establishes a topic, which the writer subsequently builds upon as assumed knowledge.

Infrequent use of nominalisation limits a writer’s development of argument by not marking a theme as a stepping stone for subsequent ideas. This does not mean that the writer has not presented or established a series of arguments through the essay. The second paragraph shows how the writer constructs an argument around the notion that it is more comfortable for people to drive their own cars and makes a series of points to support this view. The points however are loosely strung together without building upon a nominated topic. Without nominalisation it is not so easy to demarcate the content territory of an essay for a reader. Instead of signalling key concepts for the reader concisely, the reader has to search for the main arguments or ideas.

The limited use of nominalisation together with the use of human participants and processes creates an impression in the reader of a text, which is not highly academic and objective. It therefore does not come across as convincing. By comparison NS 2 has used nominalisation to foreground thematically her arguments and then to elaborate points on the topic. NNS 7 uses processes for the expression of ideas, rather than packaging them conceptually.

5.2.4 Cohesion

An analysis of cohesion in the essay shows how the use of cohesive devices are related to the building of texture and unity in the text. In this section Reference and Lexical Cohesion are examined.

5.2.4.1 Reference

The NNS writer has some difficulty with the system of Reference. This is already apparent in the first section of the essay.

In British use of car is increasing rapidly. It was introduced in 1888. Now, most of families have at least one car. As a result, it is thought by the year 2000 there will be about 29 million vehicles in British. Some people concern about it because of air pollution. Other people think it wouldn’t have much differences. There is a idea >that is< to introduce international laws to control it.

The pronominal referents are underlined in the extract. From the opening words of the essay the reader faces problems with tracking the identity of participants. The pronominal referents are not clearly linked to the reference item or items. It is not clear what the pronominal reference ‘It’ in the second sentence refers to. The third sentence does not directly relate to previously stated topics or ideas. This makes hard work for the reader, who has to conjure up the textual connections in order to follow the writer’s line of argument. The continuities in the text are not easy to follow or to retrieve. The writer does not use the opportunity to build incrementally the meanings in the text through language resources such as synonymy. Additionally the use of ‘it’ fifteen times in this essay increases the work a reader needs to do to comprehend meanings. Also it has been suggested that strings of pronouns make uninteresting writing (Knapp and Watkins 1994).

5.2.4.2 Lexical cohesion

The analysis of the writer’s lexical items shows a low use of content words. This diminishes the composition of continuities and relationships through the text. We have examined this as lexical chains in the following extract. The numbering refers to the sequence of the clauses.

1] In British use of car is increasing rapidly. 2] It was introduced in 1888. 3] Now, most of families have at least one car. 4] As a result, it is thought 5] by the year 2000 there will be about 29 million vehicles in British. 6] Some people concern about it because of air pollution. 7] Other people think 8] it wouldn’t have much differences. 9] There is a idea >that is< to introduce international laws to control it. 10] People think 11] we should think alternative way or introduce to transplate people without using people. 12] The benefits of alternative ways are 13] there will be less polluted air. 14] the one alternative way is using solar energy cars. 15] These cars will not produce any poisoness gas. 16] This poisons gas [is] causes the greenhouse effect. 17] People who use car everyday have different idea. 18] People use cars 19] because it is very comfortable. 20] Many people know 21] what are the problems 22] but they can’t stop using them. 23] If they can use cars, 24] nobody need to look time for bus or tram. 25] Sometimes it will make more time to play. 26] It also give some private out side of houses.

In the next Figure the lexical chains are presented diagrammatically.

Figure 5.4: Lexical chains in NNS7 essay

In Figure 5.4 there is a word identity chain built with the repetition of ‘car’ and with the use of synonymy, ‘vehicles’. As with the use of “it” referred to earlier, the excessive use of repetition reduces conceptual enrichment and variation. Consequently the writer does not use the opportunity to build meanings in the text through the addition of other technical or topical words connected with transport. The use of different keywords is a technique for writers to expand on a topic under discussion. The writer does this with the words connected with pollution: ‘solar energy’, ‘greenhouse effect’, ‘air pollution’. The use of key, technical vocabulary gives the text a knowledgeable air and suggests that the writer possesses topic expertise.

5.3 Comments

The comparison of two essays depicts some of the specific textual differences evident in the essays of the two cohorts. What appears to be particularly crucial for the composition of appropriate texts are the discourse resources of genres or text types. Although this analysis has focused on only some aspects of the textuality of the essays, nevertheless the features selected for analysis differentiate qualitatively the texts of native speakers from non-native speakers. The implications of this for the design of test prompts, for the evaluation of candidates’ responses and for teaching are discussed in the concluding section.

6.0 Discussion

The linguistic analysis in this study has implications for the design of tasks, for the rating of written texts and for teaching students preparing for the IELTS examination.

6.1 Task design

The impact of the wording of prompts on candidates’ responses has been demonstrated in other studies (Hamp-Lyons 1996, Mickan, Slater and Gibson 2000). Based on the analysis of the texts of the writers in this study there is at least some ambiguity about the nature of the type of text to be produced. Writers’ texts correspond in general to two different text types. What is of particular relevance is that the NS’ text types correspond more closely to an analytical exposition with hallmarks of mature writing such as transparent organisation of the text, academic objectivity and impersonal voice. The NNS on the other hand have written texts which correspond to discussion texts with less transparent structure and more personal, spoken features.

This is of significance for response validation in that candidates are instructed in the IELTS Handbook (1999) as follows: ‘Part of the task realisation is to respond appropriately in terms of register, rhetorical organisation, style and content’ (11). For Task Two in this study candidates were instructed to ‘Present a written argument or case to an educated non-specialist audience …’. The linguistic analysis of texts suggests that ‘argument’ covers a class of genres (Gerot and Wignell 1994). The instruction to ‘present a written argument’ is a non-specific reference. A further complication for writers is the additional instruction to candidates in the IELTS Handbook: ‘Appropriate responses are short essays or general reports, addressed to tutors or examiners’. This instruction widens the possible text types a writer might attempt to compose. There are many different kinds of essays and the social purpose of a general report is normally not the same as the social function of an essay. This is evident in the wording of texts.

What does a non-native writer make of the instructions? In the present data the subjects comprehended the task instructions, but it appears that the specification was not sufficient to direct their decision-making in terms of the appropriate rhetorical and lexical selections which constitute a particular argument genre. However, it may be that the NNS in this study lacked the discourse knowledge and resources to undertake the task. In both cases the nature of the prompt influences the outcome. Test prompts direct writers in the kind of texts they are expected to compose. They determine the lexicogrammatical choices writers make for the construction of their texts. The clarity or precision of the prompt regarding the type of text or genre may be crucial for the wording of candidates’ responses. If this is the case then there are implications for task design, for instructions given to candidates and for the rating of responses.

6.2 Text factors in the rating of essays

The significance of text types for assessment is compounded with the findings of research which shows that raters respond to the rhetorical organisation of texts (Tedick and Mathison 1995). IELTS raters evaluate written work according to predetermined criteria. They are trained in the evaluation of texts. The three profile criteria for assessing Task 2 are:

• Arguments, ideas and evidence
• Communicative quality
• Vocabulary and sentence structure

These criteria may be interpreted in different ways. The use of general criteria such as ‘communicative quality’ allows for a wide range of interpretations and variations in evaluating the quality of the essays. The separate consideration for vocabulary and sentence structure distracts from the quality of text and from an evaluation of meaning or messages in the text as a whole. What is significant in the composition of written texts is the assembled whole - how the lexicogrammar and the discourse semantic resources are used to achieve textual unity and specific meanings. Highlighting vocabulary and sentence structure attracts separate attention to discrete elements of a text rather than to the discourse as a whole. Readers read texts. They have expectations about the nature of texts. The expectations are based on experience of texts with conventions which perform particular social functions.

What is central in academic writing is the social function of the task, which directs the nature of composition - how the text is assembled. Given the importance of text conventions in academic writing, it may be useful to consider the description of criteria in terms of discourse conventions at text level. Because the criteria of an argument text are not specifically defined for IELTS examiners it is to be expected that there is some latitude in what might constitute mature examples of a text. The specification of a particular genre based on the linguistic analysis of text features might contribute to the development of more consistent rating tools for the assessment of written competence as a measure of writing ability on particular academic tasks.

6.3 Implications for teaching

This study suggests a number of areas in which teachers could prepare IELTS candidates for the effective demonstration of their written ability. One proposal is the identification of the linguistic resources writers utilise in the composition of text types invoked by a prompt. A second proposal is for the selective teaching of those discourse features of texts, which constitute successful compositions. Teaching activities could include the differentiation between text types with explicit reference to the social purposes of texts, and their realisation through lexicogrammatical and discourse semantic choices. The analysis of genres enables teachers to specify the linguistic resources which typify texts, so that the preparation of candidates provides a sound basis for the evaluation of written competence on specified tasks.

6.4 Further investigation

The study suggests further research into the assessment of candidates’ written responses using text-based criteria for differentiating levels of performance. Academic writing involves the composition of different genres. The linguistic analysis of genres and their social purposes could provide data for the development of evaluation criteria which are specific to task types. This research could be connected with a study of raters engaging in rating essays, using concurrent verbal protocols. Such investigations are needed in order to explore further what criteria raters depend upon for the determination of scores.

Note

We wish to acknowledge the invaluable assistance of the staff and students who allowed us to work with them and who provided us with fascinating insights into assessment processes. We were very privileged to work with you! Thank you!

We thank IELTS Australia for enabling us to undertake the research. The financial support is essential for undertaking such activity.

Bibliography

• Alderson, J.C., C. Clapham, & D. Wall. (1995). Language Test Construction and Evaluation. Cambridge: Cambridge University Press.
• Cumming, Alistair. (1989). Writing expertise and second-language proficiency. Language Learning 39,1: 81-141
• Derewianka, Bev. (1999). An editorial footnote. The Australian Journal of Language and Literacy 22,1: 22-25
• Dudley-Evans, Tony. (1988). A consideration of the meaning of ‘Discuss’ in examination questions. In Pauline Robinson. Academic writing: process and product. Hong Kong: Modern English Publications in association with the British Council: 47-52
• Eggins, Suzanne. (1994). An introduction to systemic functional linguistics. London: Pinter Publishers
• Gerot, L. (1995). Making Sense of Text. Cammeray, NSW: Antipodean Educational Enterprises.
• Gerot, Linda and Peter Wignell. (1994). Making sense of functional grammar. Cammeray, NSW: Antipodean Educational Enterprises.
• Green, A. (1998). Verbal protocol analysis in language testing research. Cambridge: Cambridge University Press
• Halliday, M. (1985). An introduction to functional grammar. London: Edward Arnold.
• Halliday, M.A. and R. Hasan. (1985). Language, context, and text: Aspects of language in a social-semiotic perspective. Deakin University: Victoria.
• Hamp-Lyons, Liz. (1990). Second language writing: assessment issues. In Barbara Kroll (ed). Second Language Writing. Cambridge: Cambridge University Press: 69-87
• Hamp-Lyons, Liz. (1996). The challenges of second-language writing assessment. In Edward White, Willian Lutz and Sandra Kamusikiri (eds). Assessment of writing: politics, policies, practices. New York: The Modern Language Association of America: 226-240
• International English Language Testing System. The IELTS Handbook (1999). Cambridge: University of Cambridge Local Examinations Syndicate.
• Kobayashi, H. and C. Rinnert. (1996). Factors affecting composition evaluation in an EFL context: cultural rhetorical pattern and readers’ background. Language Learning 46, 3: 397-437
• Kroll, B. (1998). Assessing writing abilities. Annual Review of Applied Linguistics 18: 219-240
• Kroll, B. and J. Reid. (1994). Guidelines for designing writing prompts: clarifications, caveats, and cautions. Journal of Second Language Writing 3,3: 231-255
• Martin, J. (1985). Factual writing: exploring and challenging social reality. Victoria: Deakin University
• Mickan, P. (2000). Textualising meanings: second language writers in action. Unpublished Ph.D. thesis, Macquarie University
• Mickan, P., S. Slater and C. Gibson. (2000). A study of the response validity of the IELTS writing subtest. In R. Tulloh (ed). IELTS International English Language Testing System Research Reports 2000 Volume Three. Canberra: IELTS Australia: 29-48
• Moore, T. and J. Morton. (1999). Authenticity in the IELTS academic module writing test: a comparative study of Task 2 items and university assignments. Research report. Canberra: IELTS Australia
• Reid, J. (1990). Responding to different topic types: a quantitative analysis from a contrastive rhetoric perspective. In B. Kroll (ed.). Second language writing: research insights for the classroom. Cambridge: Cambridge University Press: 191-210
• Skehan, P. and P. Foster. (1999). The influence of task structure and processing conditions on narrative retellings. Language Learning 49,1: 93-120
• Smagorinsky P. (ed.) (1994). Speaking About Writing. London. Sage Publications.
• Tedick, D and M. Mathison. (1995). Holistic scoring in ESL writing assessment: what does an analysis of rhetorical features reveal? In D. Belcher and G. Braine (eds). Academic writing in a second language. Norwood NJ: Ablex Publishing Corporation: 205-230
• Wallace, D. (1994). Teaching collaborative planning: creating a social context for writing. In L. Flower, D. Wallace, L. Norris and R. Burnett (eds). Making thinking visible: writing, collaborative planning, and classroom inquiry. Urbana, Illinois: National Council of Teachers of English: 48-66.