- * Nous remercions Steve Brown pour la relecture des textes en anglais, ainsi que Jacques Delaforge po (...)
1This paper is concerned with the use of high stakes tests and examinations. It focuses in particular on their use as a means of raising standards in schools and for the purposes of accountability. The contexts within which such testing systems have been introduced vary. Whilst our main focus is on the market reforms in school-based education in England, given that the increase in high stakes testing systems is an « international phenomenon » linked to widespread changes in educational systems (Whetton, 2004: 2), we also draw on literature from the US. In so doing, we seek to answer the following questions. First, high stakes testing by definition has “serious consequences”; what are these consequences in different jurisdictions? Second, high stakes testing is frequently accompanied by dysfunctional consequences; what does the research evidence tell us about these? Third, can we be confident that improvements in high stakes tests are authentic? Fourth, what are the consequences of high stakes testing on education policy?
2The following section outlines the major reforms that have been introduced in England in school-based education. The next section discusses the concept of high stakes testing, and its serious and dysfunctional consequences as well as other risks associated with high stakes testing. This is followed by an exploration as to whether the improved scores on high stakes tests are genuine and an examination of the consequences of high stakes testing on education policy. The final section concludes the paper.
3Education policy in England over the past two decades has been strongly influenced by both the neo-liberal and neo-conservative approaches of the “New Right” (Chitty, 1989). Under the Conservative administrations between 1979 and 1997, market-oriented reforms were introduced into the school system. The election of a Labour government in 1997 did not change the trajectory of the education reforms. Indeed the market-oriented philosophy has been reinforced with the introduction, on the one hand, of “Public Services Agreements”, which the Government uses to evaluate the performance of individual Ministries, and the introduction of targets for schools to meet in tests and public examinations.
- 1 Except in the case of academically selective (“grammar”) schools and religious schools.
4The education reforms have had a profound effect on the state (publicly-funded) school system in England and how it functions. The legislation commenced with the 1980 Education Act and culminated in the 1988 Education Reform Act. The reforms comprised a number of interlocking elements. The 1980 Education Act gave parental choice of school a higher priority, whilst the 1988 Education Reform Act changed the method of financing schools, with funding being largely determined on the basis of the number of pupils enrolled; schools were required to admit pupils up to the physical capacity of the school1 and from 1992 public examination results were made available by the Government and published in the national press as “league tables” (Le Grand & Bartlett, 1993; West & Pennell, 1997, 2000, 2002).
5A “quasi-market” was established in school-based education; this term is used to describe changes to a system whereby a monopolistic state provider is replaced with independent competitive providers; such a system is called a “quasi” market as it differs from a conventional market; in particular, there is no financial transaction between the consumer (in this case the parent acting on behalf of the child) and the provider (school) (Le Grand & Bartlett, 1993).
- 2 Originally €150,000 under the Conservatives ; reduced to €75,000 under Labour.
6One of the pre-requisites of a market is for a diversity of providers to exist to enable “choice” to be exercised. There was some diversity in the state school system prior to the reforms that took place in the 1980s; in particular, around 15% of all state-maintained secondary schools (voluntary-aided schools) had a religious affiliation (e.g. Church of England, Roman Catholic) (West & Pennell, 2003). However, the 1988 Education Reform Act heralded major reforms. The Act allowed for city technology colleges to be established. These were a hybrid type of school, officially classified as “independent”, with capital costs being met by private sponsors and running costs met by the state. Very few such schools were set up (only 15 across the whole of England). The Education Reform Act also allowed for state funded schools to choose to “opt out” of local authorities and become “grant-maintained”, following a ballot of parents. Once grant-maintained, the school was no longer funded by the local authority, but by central government (via a funding agency) at a higher level than previously; one of the responsibilities it acquired was that of admissions to the school (although schools that were voluntary-aided already had this responsibility). “Specialist schools” were also set up; the specialist schools programme was designed to help state secondary schools to “specialise” in technology, science and mathematics or modern foreign languages. Specialist schools are those that have been successful in the competitive bidding process associated with the programme. They receive additional capital and revenue funding to develop the subject specialism; initially schools were also required to raise € 150,000 from private or charitable sponsors (Bell & West, 2003)2.
7Together with the market-oriented, neo-liberal approach to education, the Conservative Government also introduced a national curriculum, a centralising, “neo-conservative” element. However, the national testing programme introduced alongside the national curriculum together with the widespread publication of “school performance tables” (“league tables”) formed another plank of the quasi-market reforms: the results were designed to provide information to enable parents to make informed choices (in reality to state preferences) about the school their child might attend (although there is evidence to suggest that not all parents use or understand the information provided, cf. West & Pennell, 2000).
8The English national curriculum consists of a number of “key stages” which are assessed through a national system of assessment comprising national tests at the ages of 7 (the end of “key stage 1”), 11 (“key stage 2”) and 14 (“key stage 3”), along with assessments made by teachers. The results of the tests in English, mathematics and science at the age of 11 are published in the national press as school performance tables (“league tables”) and are available on the Internet, including on the web-site of the Department for Education and Skills. In addition to the national tests, there are also public examinations taken by the majority of school pupils at the end of compulsory education, when pupils are around 16 years of age (these are the General Certificate of Secondary Education examinations (GCSES); these are taken in individual subjects (e.g. English language, English literature, mathematics, science, French, geography, history). At the end of upper secondary education, when pupils are aged around 18 years of age, those who have followed an academic route take the General Certificate of Education Advanced (GCEA) level examinations generally in a small number of subjects. The results of these examinations are also published in the form of “league tables”. One of the aims of the tests and examinations is to inform parents about the performance of schools for their child. In this context it is important to note that the focus is on “raw” examination results; no account is taken of socio-economic background although there is a measure of the “value added” by the school which uses prior attainment.
9The final part of the reforms was the establishment, in 1992, of an inspection body, the Office for Standards in Education (OFSTED) to inspect the quality of education provided by schools. By these various means the Government introduced a competitive system that had as its overarching aim a desire to increase standards (i.e. academic results) in education.
10The quasi-market, national curriculum and national testing programmes were introduced by Conservative administrations. The Labour Government when it was elected into office in 1997, did not change the fundamental principles of the quasi-market. However, it did abolish grant-maintained schools (by this time around 20% of secondary schools had become grant-maintained (West & Pennell, 1997). Following the 1998 School Standards and Framework Act, schools were designated community, foundation, voluntary-aided or voluntary-controlled. Grant-maintained schools, in the main became foundation schools, or in the case of those that were previously voluntary, reverted to voluntary status. There are important differences between these categories of schools. In community schools, the local education authority is responsible for pupil admissions. In foundation and voluntary-aided schools the governing body employs the school staff and is responsible for pupil admissions. In voluntary-controlled schools, the local education authority is responsible for pupil admissions (For foundation and voluntary schools, the school’s land and buildings are normally owned by a charitable foundation – DfES, 2004a.)
11The Labour Government has embraced the concept of diversity and parental choice in a similar way to its Conservative predecessors. It has developed and expanded the number and types of specialist schools (Bell & West, 2003) and introduced a further new type of school: academies. Academies are sited in areas of disadvantage and are intended to raise standards. These schools are defined as “independent schools in the state sector” (DfES, 2004b); they are sponsored by business, faith or voluntary groups by up to € 3.5m and offer a specialist subject(s) as do specialist schools. They have more freedom than other maintained schools over what they teach and are, like foundation and voluntary-aided schools, responsible for admissions. In January 2005, there were 17 academies but the government proposes that there should be 200 by 2010 (DfES, 2004b). It also has a target for 95% of secondary schools to be specialist schools (or academies) by 2008. It will also be made easier for schools to become foundation schools and gain the significant amount of additional independence that this status brings (DfES, 2004b).
12As well as carrying forward the Conservative Government’s agenda, a distinct policy move was made by the Labour Government, namely, to introduce “targets” with the aim of increasing the overall levels of achievement in England. Amongst these were specific targets to increase the proportion of those aged 11 reaching the “standard” of literacy and of numeracy to 80% and 75% respectively by 2002; and for an increase in the proportion of those aged 16 who achieve five or more passes in the General Certificate of Secondary Examination at grades starred A(A*), A, B or C to 50% by 2002 (Treasury, 1998).
13In general terms a high stakes testing system can be considered as one that is used to determine – or help to determine – the future of pupils, teachers or schools on the basis of test or examination scores. In contrast, a low stakes system could be characterised as one that does not have a direct bearing on the future of individual pupils, their teachers or their schools (OECD, 1997). However, this definition can be contested as some seemingly “low stakes” tests (such as those set by schools themselves) can in some circumstances have consequences for individuals (e.g. pupil progression from one year to the next). In the current context, the distinguishing feature of a high stakes test is that it can have serious consequences not only for individuals, but also for institutions. Our main focus, then, is primarily on the consequences associated with tests used for the purposes of accountability, although they may also determine the future of individuals.
- 3 A high stakes testing system was introduced following the publication by the National Commission on (...)
14In England, high stakes testing is at the heart of the reforms in school-based education initiated by the Conservative government of Margaret Thatcher in the 1980s and continued by subsequent Conservative and Labour governments. In the US context, Amrein and Berliner note that « high-stakes, or serious consequences, were attached to tests in order to hold schools, administrators, teachers, and students [pupils] accountable for meeting the newly imposed high standards »3 (Amrein & Berliner, 2002: 4). They note that in making tests high stakes:
« policymakers borrowed principles from the business sector and attached incentives to learning and sanctions to poor performance on tests. High performing schools would be rewarded. Under performing schools would be penalized, and to avoid further penalties, would improve themselves. Accordingly, students [pupils] would be motivated to learn, school personnel would be forced to do their jobs, and the condition of education would inevitably improve, without much effort and without too great a cost per state. What made sense, in theory, gained widespread attention and eventually increased in popularity as a method for school reform » (Amrein & Berliner, 2002: 4).
15In England there are a number of serious consequences that are associated with the use of high stakes testing including: reductions in school funding and the risk of school closure; headteachers’ pay; and various impacts on pupils and their teachers. There is also a strong and in some ways perverse impact on educational performance and policy as schools strive to improve their league table position or meet other high stakes targets.
16Both the national tests taken at the age of 11 years and the public examinations taken at the ages of 16 and 18 years, can be considered to be “high stakes”. They are high stakes to the schools in question as they are published widely in the national and local press and via the Internet; in the media they are often published in “rank order”. Low scores in the national tests or public examinations can also result in the school being at risk of closure. They can affect not only the school’s overall budget (determined largely on the basis of the number of pupils enrolled in the school) but also who chooses to apply to a particular school. If more middle-class parents with high achieving children apply to a particular school the chances are, given the link between higher socio-economic status and higher attainment, that there will be a positive effect on the school’s league table position and hence future demand for places from parents with higher attaining pupils. The national test results are also high stakes in that they are used to inform the inspections of schools that are carried out by the Office for Standards in Education, which is responsible for making judgments about school quality; the inspection reports produced can result in schools being labelled as “failing” schools, with consequences for their viability. The reports produced by this body can, like league tables, also affect the school “choices” made by parents. The national test results at the age of 11, are not, however, high stakes for the pupils concerned.
17By way of contrast, the public examinations taken by pupils at the age of 16 years (the end of compulsory schooling), the General Certificate of Secondary Education (GCSE), and those taken at the age of 18 years, General Certificate of Education Advanced (GCE A) levels, are high stakes both for schools and their pupils. For schools, the results of the GCSE examinations are high stakes, with a heavy focus being on the percentage of pupils achieving “high grade” passes (A*, A, B and C). The higher the percentage of pupils achieving five or more high grade GCSEs the more popular the school is likely to become and the greater its budget. Schools that perform well in the “league tables” are also more likely to be successful in their bid to become specialist schools and so obtain additional capital and revenue funding (see Bell & West, 2003). The public examinations are also high stakes for the pupils who take these examinations as they are important in determining their future educational or employment trajectory.
18Results in the GCSE examinations are likely to determine if a pupil will be able to study GCE A levels. GCE A levels are the normal requirement for university degree programmes (unlike in France, there is no right to enter university even if GCE A levels have been gained – cf. Vaniscotte et al., 2003). Pupils with higher GCE A level results are more likely to gain admission to more prestigious universities as the higher education system in England is stratified with some universities requiring higher grades than others. For pupils who choose to leave school at the age of 16 years of age GCSE results are high stakes as they have value in the labour market. GCE A level results also have a value in the labour market.
19Both national test results and public examination results are also important for the Government and could also be construed as being high stakes for individual Ministers. Public Services Agreements are set by the Treasury (the Ministry of Finance) for individual Government Departments (Ministries) and the performance of Ministries is then evaluated against these agreements. The performance of the Government will be judged by the electorate at election time and one element of the assessment could be its success in meeting the range of targets it has set itself, some based on high stakes tests.
20Turning to the US, Amrein and Berliner (2002) carried out a major study on high stakes testing. They reported that 20 states provided financial rewards to “successful” schools, and 19 to “improved” schools. Interestingly, negative sanctions were twice as common as rewards: 45 states publish school or district report cards (league tables). Of these 27 used rating and ranking mechanisms; 14 had the power to close, reconstitute of take over poorly performing schools and 16 had the authority to replace teachers or administrators. Teachers with low average class scores may be prevented from receiving salary increases and in 16 states low scores may result in dismissal. High average class scores may also have financial consequences in terms of bonuses or pay increases.
21In addition to serious personal, educational and institutional consequences arising from high stakes tests, there may be also « distorting and corrupting effects » (Amrein & Berliner, 2002: 5) or « highly dysfunctional consequences » (Goldstein, 2004: 10). In this section, we examine some of these consequences – relating to distortion of the curriculum, narrowing the curriculum, teaching to the test, coaching for the tests, inappropriate practices to improve test results, pupil stress and demotivation, diversion of resources, creaming and effects on teacher recruitment.
22Concern has been expressed about the distortion of the curriculum as a result of over-concentration on the areas of the curriculum subject to testing (Goldstein, 2004). The narrowing of the curriculum to improve test scores has also been identified (Klein et al., 2000). Concerns have also been raised over pupils being coached on how to answer the types of questions used in tests (e.g. Klein et al., 2000). And in England the Qualifications and Curriculum Authority expressed concerns that too much time was being devoted to revision for tests at the expense of other areas of the curriculum (Times Educational Supplement, 2004).
23The pressure on teachers in schools has resulted in an apparent increase in inappropriate practices to improve test and examination performance. In England, in 2002, 479 cases of alleged malpractice or maladministration in the national testing programme were reported to the Qualifications and Curriculum Authority « leading to the annulment of the results of seven schools » (Hansard, 2002, column 909W). In the US, Klein et al. (2000) reported an increase in prevalence of activities that reduce the validity of the test scores.
24Specific effects on pupils of high stakes testing have been identified including demotivation and pupil stress particularly among low achievers (Goldstein, 2004). It is also thought that the over concentration on those aspects of the curriculum subject to testing at the expense of others may have negative effects on pupils’ performance in subsequent stages of education (Wiggins & Tymms, 2002). In the case of teachers, it has been suggested that it undermines their professionalism and reduces the opportunities for innovative teaching (Goldstein, 2004).
25In England, there is growing evidence that high stakes testing has caused a diversion of resources from certain groups of pupils to others as schools seek to increase their league table position. This effect has been found in the national tests for 11 year olds and, in particular, at the General Certificate of Secondary Education (GCSE) level. The key indicator in the latter test is five or more high grade passes (grades starred A(A*), A, B and C); thus, pupils who are perceived to be on the borderline between grade C and grade D are targeted for additional teaching support with a view to increasing their grades from, say 5 GCSE D grades to 5 GCSE C grades. “Performance maximising behaviour” has included non-teaching staff being used to support pupils on the C/D borderline (Golden et al., 2002) and the use of volunteers to release teachers’ time so that they can devote more time to pupils on the borderline of achieving five or more high grade GCSEs (West & Pennell, 2000).
26The incentive structure is thus likely to benefit certain pupils, for example, the GCSE borderline pupils or those at the borderline of the “expected level” in the national tests for 11 year olds. On the other hand, it will not benefit those who are either performing above or below the borderline. It could be argued, however, on altruistic and pragmatic grounds, that the major losers will be those performing well below the target levels; they are the group most in need of additional help in order to increase their personal opportunities and to minimise the risk of failure, disaffection and the consequent costs to society (West & Pennell, 2003).
27It is interesting to note the findings of a comparative study of primary school testing in England and Scotland (Wiggins & Tymms, 2002). In both countries the assessments used are similar but in England they are high stakes and in Scotland they are not (primary school test results are not published). The researchers found fewer dysfunctional effects arising from the use of the tests in Scotland and more evidence in England of teachers concentrating on improving the performance of borderline pupils.
28In England another consequence is the tendency for some schools to select pupils. Whilst parents are able to state preferences for the school(s) of their choice, if there are more applicants for a school than there are places available, the school’s published admissions criteria are used to determine who should be offered places: parental choice in these circumstances in effect becomes school choice. One of the consequences of high stakes testing, alongside schools being placed in competition with one another to recruit pupils who are likely to enhance their league table position, is that there are incentives for schools to “cream skim”. However, only some schools are in a position to do this – those that are both oversubscribed and in charge of their own admissions – namely the autonomous voluntary-aided and foundation schools. The evidence indicates that these schools are more likely than others to have overtly or covertly selective admissions criteria (West et al., 2004). On average, autonomous schools, which tend to have more selective admissions criteria, obtain higher results in public examinations and they also have lower levels of pupils with special educational needs (West et al., 2005).
29There also appears to be an impact on the distribution of teachers between schools. Noden (2001) as part of a major survey of teachers employed in schools in disadvantaged local education authorities asked teachers whether, if they were thinking of applying for a job at another school, which particular school characteristics would make them more likely or less likely to apply for a job. It was found that 69% of respondents reported that good examination (“league table”) results would make them more likely to apply for a job at that school.
30In England, the national tests (taken at the ages of 7, 11 and 14) are organised by a national agency, the Qualifications and Curriculum Authority. The Government wishes to see standards increase, not only because it wishes to raise overall achievement levels, but also because the success or failure of individual Departments and indeed Ministers is evaluated on the basis of the test results for which there are Public Services Agreements. There is a risk that pressure could be put upon advisers working at the national agency to make test papers easier or to ensure that tests or examinations are marked more leniently in order to meet Government Public Services Agreements.
31The General Certificate of Secondary Education and General Certificate of Education Advanced (A) level examinations in England are set and managed by privately-run examination boards. However, even here there is a risk that pressure could also be put on the examination boards. In 2002, following reform of the GCE A level examination (Tomlinson, 2002), there was particular concern, expressed predominantly in the media, that the standards of the GCE A level would fall. When the examination results were made public, headteachers expressed concern that the marks awarded to some pupils had been “downgraded” – that is, that they were lower than expected. They accused the Qualifications and Curriculum Authority (QCA) of pressurising the examination boards into lowering results in order to counter claims that the examinations had become easier. The concern became so widespread that the head of the QCA was dismissed by the most senior education Minister (the Secretary of State for Education). Moreover, an inquiry into GCE A level standards was commissioned by the Government. This sought to « provide assurances that the A level standard is being maintained year on year » (Tomlinson, 2002: 2). It was noted that a range of views had been received « on the relationship between [Government] Ministers and the QCA. Many of these turn on whether QCA is, and is seen to be, sufficiently independent of Ministers » (2002: 29). The report was also explicit in stating that Ministers « should not intervene in the ongoing operation of the regulatory framework nor in the details of the QCA’s relationship with the awarding bodies » (Tomlinson, 2002: 29).
32It is also important to bear in mind that the GCSE and GCE A level examination boards are in competition with each other for candidates to enter their examinations; schools select examination syllabuses from examination boards, purchase examination papers from them and return scripts to them for marking; the boards then mark the papers and return results to schools. Given that the boards are in competition with one another, there is an incentive for them to try and maximise the numbers of students who sit their examination papers. Schools in turn are keen to raise their own position in published examination performance tables and so are likely to select examination boards that they feel will maximise the performance of their pupils. Thus, in theory there are incentives for relevant bodies to seek to maximise apparent pupil performance.
33There are other apparent consequences arising from the testing programme concerning the publication of league tables and government targets. For example, there has been an increase in the number of top grades (Grade A) in GCE A level examinations in recent years and a concomitant concern about the extent to which these examinations still serve a “selection” function particularly in relation to university entrance, which in the UK is academically selective (Vaniscotte et al., 2003; West et al., 2001). A recent report commissioned by the QCA has argued that « results in [General Certificate of Education Advanced] A levels at the top end need to be more discriminating » (Mc Gaw et al., 2004: 31).
34Another consequence relates to the way in which public examination data are used to produce supranational indicators, published by bodies such as the OECD. West (2003) has argued that there are problems with the allocation to international levels as the judgements that are made are not equivalent across countries. The quasi-market has created additional difficulties, as incentives exist in England, which do not exist elsewhere, for results to improve. For example, in France, policy is oriented towards 80% of young people succeeding in the Baccalauréat. Whilst this could be seen to be a similar policy “target” to the English public service agreements for education, there is no quasi-market in operation with the incentives outlined earlier.
35Given our previous discussion, the question arises as to the authenticity of high stakes test results: « The distortions and corruptions that accompany high-stakes tests make inferences about the meanings of the scores on those tests uncertain. If there is uncertainty about the meaning of a test score, the test may not be valid » (Amrein & Berliner, 2002: 5). In this section, therefore, we examine the extent of improvements in test and examination results in England.
36The criterion referenced national tests take place at the ages of 7, 11 and 14, and at this stage, pupils are allocated to one of several different levels of performance. The Government has set a “standard”, or “expected level” for these tests. Focusing on the tests for 11 year olds, there has been an increase in the percentage of pupils reaching the expected levels in English and in mathematics (DfEE, 1999, 2000; DfES, 2003). The percentage reaching this level in English reached 75% in 2000 (where it remained until 2003) an increase of 12 percentage points since 1997 (when the Labour Government was elected into office); and the percentage reaching the expected level in mathematics reached 73% in 2003, an increase of 11 percentage points since 1997. However, there is a suggestion that the national test results do not in fact reflect a genuine improvement in achievement. Tymms (2004) concluded that although there had been an increase in performance in mathematics and English between 1995 and 2000, this was not as great as official data suggested; interestingly, he also found between 2000 and 2003, both official data and data he used concurred in finding little increase in the level of performance.
37GCSE examinations are taken in individual subjects at around 16 years of age. Since their introduction in 1988, marked improvements in achievement have been recorded (West & Pennell, 2003) in the percentage of 15 year olds achieving five or more GCSE examination passes at grades starred A to C (A*, A, B and C); the percentage of young people obtaining five or more GCSE grades at A* to C increased from 45.1% in 1996/97 when the Labour Government came into office to 52.9% in 2002/3. There has thus been an increase of 7.8 percentage points in absolute terms (DfES, 2001, 2004c). Moreover, the Government met its own Public Services Agreement of 50% in 2000/01. However, there is no evidence to support or otherwise the view that these results represent a genuine improvement. There is no common “objective” measure, so it is not possible to determine whether a change in tests scores is due to a change in performance, a change in the difficulty of the test both (Goldstein, 2001).
38Turning to the US, there is a low stakes test available, namely, the National Assessment of Educational Progress (NAEP). This enables comparisons to be made with other testing regimes. Klein et al. (2000) found that the gains on the high stakes test the Texas Assessment of Academic Skills (TAAS) were greater than they were on the NAEP in relation to the gap in average scores between white pupils and pupils from minority ethnic groups. According to the NAEP the gap was large and increasing slightly, whilst according to the TAAS, the gap was smaller and decreasing markedly.
39Amrein and Berliner (2002) compared the achievement in high stakes tests with those in a number of tests including the National Assessment of Educational Progress (NAEP). In general, for the mathematics and reading scores, high-stakes testing policies did not usually improve the performance of students in reading or in mathematics. They concluded that « At the present time, there is no compelling evidence from a set of states with high-stakes testing policies that those policies result in transfer to the broader domains of knowledge and skill for which highstakes test scores must be indicators. Because of this, the high-stakes tests being used today do not, as a general rule, appear valid as indicators of genuine learning… » (2002: 49).
40High stakes test and examination results are used by the Government to evaluate the education system. However, the education system is influenced by factors outside the control of any individual school. Nevertheless, the focus on schools as the main locus for government policy is reinforced by the use of test and examination results. From a policy perspective, this is, perhaps, not surprising. However, it is important to stress that the educational research literature in the UK and elsewhere contains strong evidence of differences in outcomes of pupils from different social backgrounds, with lower outcomes being associated with pupils from more disadvantaged backgrounds (e.g. Bramley, 1989; West & Pennell, 2003). Indeed, where attempts have been made to determine the relative weight of school and non-school factors in pupil outcomes, non-school factors have been found to account for most of the variation in educational attainment at school. Reynolds et al. (1997) concluded, in a review of school effectiveness research, that schools appear to have an independent effect on attainment of between 8 to 12% of the total variance. Further, Mortimore et al. (1988) demonstrated that no school reversed the usual pattern within schools of advantaged pupils performing better than the disadvantaged. The issue of compositional (or peer) effects is of particular relevance in this context. On average, the higher the proportion of pupils known to be eligible for free school meals, the less progress pupils make (e.g. Strand, 1997). Pupils attending schools that have more advantaged intakes are likely to achieve higher results in part because they are being educated with more advantaged students. Likewise students attending schools with more disadvantaged intakes are likely to achieve poorer results.
41This question of school mix is of fundamental importance and brings us back to the schools quasi-market in England. Schools that serve disadvantaged communities tend to obtain lower results in the high stakes tests largely as a result of these compositional effects. The Government’s Public Services Agreement “floor targets” set all schools the target that 25% of their pupils should obtain five GCSEs at grades starred A to C by 2006 (Treasury, 2005). Thus, schools with disadvantaged pupil intakes are likely to be penalised in the high stakes assessment environment as they are less likely to perform well in the published league tables – moreover, even if the focus is on the value added by schools, such schools are likely to be at a disadvantage as pupils in disadvantaged schools tend to make less progress than those in advantaged schools.
42This paper set out to answer four questions. First, high stakes testing, by definition, has “serious consequences”; what are these consequences in different jurisdictions? Second, high stakes testing is frequently accompanied by dysfunctional consequences; what does the research evidence tell us about these? Third, can we be confident that improvements in high stakes tests are “authentic”? Fourth, what are the consequences of high stakes testing on education policy?
43The impact of high stakes testing is wide ranging when such tests are used as an accountability tool as is the case in England. Both Conservative and Labour Governments have used the examination and national test results as a way of providing judgemental data on schools in the form of league tables, which form a central part of the quasi-market. High stakes testing can affect a school’s budget, its ability to recruit high quality teaching staff and result in risk of closure. The performance in meeting targets based on high stakes testing can also affect the future of Government Ministers and possibly even the Government itself. A range of dysfunctional and distorting consequences have been identified, including teaching to the test and narrowing the curriculum.
44In England, one of the aims of high stakes testing is to evaluate the education system. The difficulty here is that in England we cannot be confident that the improvements observed in national test and examination results in England reflect authentic improvements in achievement. There is an assumption that the tests are “objective” as they are set, marked and controlled by an external organisation and not the school and are sufficiently “reliable” to provide the basis of comparison year on year. However, given that there is no common scale for the different tests there is no way of knowing whether any change in scores are genuine or not. Finally, there are a number of consequences for education policy of high stakes testing. The focus on results of tests and examinations and pupils’ education progress, does appear to have resulted in a focus on internal school processes as the key means of improving learning and knowledge, rather than, for example, addressing structural issues to do with pupil mix or structural inequalities within society. It could be argued that policy makers use a deficit model of schools – arguing that the fault is that of the schools and not of factors to do with school composition or other factors, that are within the realm of Government to address.
- 4 In England there was formerly a similar body called the Assessment of Performance Unit.
45In order to make valid judgements about changes in “standards” over time a national testing programme that is not high stakes is needed. This could be an carried out by independent body on the lines of the NAEP in the USA.4 A major reason why a nationally representative “low stakes” test of educational achievement is needed to evaluate an education system is the existence of incentives for different stakeholders to maximise performance with high stakes testing programmes. With a low stakes test only a sample of pupils is tested, thus there is not much of an incentive for teachers to teach to the test in a way that could invalidate the measure of performance (Mc Gaw et al., 2004).
46Even if a low stakes system is used the problems we have highlighted cannot all be easily overcome. A recent study examining weak and strong accountability systems in US states found that in states with strong accountability (which includes high stakes testing) there was an increase in test performance on the low stakes NAEP compared with states with weaker accountability systems. However, there were no positive effects on pupils progressing from one year to the next or on completion of secondary school. The authors considered various explanations for this finding including two that are of particular interest in the current context, first, that the tests may be improving test-taking skills and second that the NAEP may not measure what the authors call “real” learning (Carnoy & Loeb, 2002).
47To conclude, it does appear that simply replacing high stakes tests with low stakes tests is not sufficient if pupils simply learn how to take the tests so that all that is being tested is pupils’ ability to take tests. What is required is more difficult to put into place; a more sophisticated assessment is needed that endeavours to measure the genuine learning that has taken place, rather than results on tests that appear not to measure “real” or “authentic” learning.