“Some schools are dull, depressing, even terrifying places, while others
are lively, comfortable and reassuring. If we think of school life
as an end in itself, rather than as a means to some other end,
such differences are enormously important”
(Jencks et al., 1972: 256). |
1. INTRODUCTION
Commenting on the development of research on school effectiveness over the last twenty years, David Hargreaves has remarked:
“Since 1979 (the date of Rutter’s study Fifteen Thousand Hours) the outcomes that specify the effective school have been progressively narrowed and in many studies are reduced to test results of academic knowledge. These are important outcomes of schooling but not the only outcomes that matter” (Hargreaves, 2001: 488, my insertion).
|
A variety of factors have doubtless influenced the development of the research in this way, not least the ready availability of data on academic outcomes and the relative paucity of evidence on others. However, another justification for this comparatively narrow focus can also be adduced, namely that the factors influencing schools’ general ‘effectiveness’ stem from similar and related sets of features of schools’ functioning. Summarising the evidence on his four outcomes of schooling, Rutter concluded, “on the whole, schools which have high levels of attendance and good behaviour also tend to have high levels of exam success” (Rutter et al., 1979:92). Gray and colleagues begged to differ, on the basis of their research in Scottish secondary schools, claiming that the “correlations between different outcomes were sufficiently low to suggest that researchers... should remain open to the possibility that different outcomes may be influenced by different school variables (Gray et al., 1983:289).
Since that time the assumption that the factors influencing schools’ overall ‘effectiveness’ constitute a bundle of related factors has tended to prevail and has, indeed, subsequently acquired a certain orthodoxy. The International Handbook of School Effectiveness Research, for example, draws the following conclusions:
“... few studies have examined (the consistency) of both cognitive and affective/social outcomes.Of those that have, primary studies suggest that schools’ effects on the two domains are weakly positively related and may be independent. At the secondary level results suggest that effects on academic and certain affective/social outcomes may be more closely linked, particularly for examination results and attendance and behaviour” (Teddlie, Reynolds and Sammons, 2000).
|
Despite the centrality of this issue to the conduct and interpretation of research on school effectiveness, the issue has been largely neglected since the debates some twenty years ago. Furthermore, the total number of studies on which any conclusions about the relationships between academic and affective/social outcomes could be based remains very small indeed. Perhaps the Handbook’s most important conclusion is that more studies are called for. At the same time, one might add, more needs to be understood about how this sort of work might be undertaken and facilitated.
This review takes up some of these neglected challenges. What do we know about differences between schools in terms of other pupil-related outcomes of schooling beyond academic results? What factors need to be taken into account in interpreting such differences? How powerful are the explanatory models? And to what extent do schools which do well in terms of one set of outcomes do well in terms of others? Clearly these questions are of both substantive and methodological interest.
2. Selection of studies
In selecting studies for closer consideration I restricted my interest to work which had been conducted within the United Kingdom on secondary schools since the assumption of consistency across different areas does not appear to have taken hold in the primary sector to anything like the same extent. There were two further important criteria for selecting studies for closer examination. First, and most importantly, the research should have included at least one other (‘non-cognitive’) outcome. Second, it should have explicitly undertaken some analysis of differences in outcome(s) between schools. The effect of these two criteria, in combination, was essentially to restrict the field to studies conducted within the school effectiveness tradition although a number of other possibilities were initially considered. In reality, very few studies conducted over the past three decades turned out to fit the bill. It should, perhaps, be stressed at this point that I do not consider here several more qualitatively-based studies which suggest between-school differences because of the difficulties of comparing them systematically with more quantitatively-oriented approaches. At a later stage in the argument, however, I do point to some of the opportunities such research can present.
3. Liking for school: what do students feel?
Do some students ‘like school’ more than others? ‘Liking school’ in this context is a broad-brush construct, which I distinguish from studies of pupils’ attitudes towards particular subjects, which are sometimes undertaken as an adjunct to work on academic outcomes. The question is easy enough to pose but whilst each of the studies discussed below is pioneering in some way or other, hardly any asked exactly the same questions. One of the best known is by Keys and Fernandes (1993) for the National Commission on Education. This cross-sectional study was based on around 1800 pupils divided fairly equally between Years 7 and 9 from 83 English secondary schools. Students were asked to agree/disagree with the statement ‘On the whole I like being at school’. 12% of Year 9s ‘strongly agreed’ with this statement, 64% ‘agreed’, 11% were ‘not sure’, 10% ‘disagreed’ and 4% ‘strongly disagreed’. Keys and Fernandes also constructed a ‘Positive Attitudes to School’ scale in which the ‘liking school’ item was included as one of ten items. Generally, girls were more positive than boys as were pupils who reported that their parents were interested in how they were doing. Pupils’ attitudes also became less positive from Year 7 to Year 9.
Thomas and colleagues (2000) studied over 3,500 Scottish pupils in 36 secondary schools. Just 3% of their sample in S4 (the final year of compulsory schooling) ‘always liked going to school’ but 71% ‘usually liked school’; 22% ‘hardly ever liked school’ while only 4% ‘never liked school’. There was tendency for the older pupils to like school less than the younger ones but no clear correlation between pupils’ overall levels of attainment and their ‘liking for school’. This item was incorporated into a four-item ‘engagement with school’ factor. Previously reported levels of ‘engagement’ in S2 turned out to be a good predictor of levels of ‘engagement’ in S4 as did gender, with girls again responding more positively than boys. Pupils who had a ‘Record of Need’ (an indicator of special needs) were also more positive.
Daly and Defty (2001) used data on about 30,000 students in 255 English secondary schools, drawn from the archives of the YELLIS project at Durham University. They too constructed an ‘Attitudes to School’ scale containing six items including, amongst others, ‘I really like school’. Students’ attitudes to school were assessed at two time points (first in the fourth year of secondary school and then again in the fifth). The mean score of the sample was just over 20 points (out of a possible total of 30) indicating that, overall, the sample had very positive attitudes towards school; girls were again slightly more positive than boys. There was a positive but low correlation with a measure of ‘developed ability’. The means for the sample at both time-points were very similar and individual pupils’ attitudes were also fairly stable over time (correlation of 0.74) suggesting that changes in attitudes might have some validity as an outcome measure.
As part of a major study of Scottish schooling Gray, McPherson and Raffe (1983) focused on 69 ‘uncreamed’ comprehensive secondary schools. Students were asked retrospectively after they had left school whether ‘on the whole, (their) last year at school was worthwhile’, essentially a measure of pupil ‘satisfaction’. The strongest correlation was with overall levels of pupil attainment. Over seven out of ten boys and girls achieving at the highest levels said ‘yes’ compared with just under a quarter of the lowest attainers. Responses were also negatively correlated with social background, the more advantaged proving on balance more favourably disposed towards their schooling. There were few differences in terms of gender on this question. There were similar findings for a related question asking whether ‘on the whole, (they) would say (they) had enjoyed their last year at school’, although the differences in terms of academic attainment were not quite so marked; girls were slightly more positive than boys.
One other study deserves mention. Smith and Tomlinson (1989) studied 19 multi-racial comprehensive schools. They asked pupils in Years 8 and 9 about their ‘enthusiasm’ for school through the prompt: ‘Imagine that you are lying in bed and you start to wake up, and you think to yourself – ‘It’s the first day of term. I’m going to school today?’ How do you feel? 19% said they would feel ‘excited and happy’, 39% ‘quite happy’, 29% ‘a bit gloomy’ and 13% ‘really fed up’. 67% of girls said they would be ‘happy’ or ‘quite happy’ compared with 50% of boys (Smith and Tomlinson, 1989:90-91). Pupils of South Asian and West Indian origin generally gave more positive replies.
It seems safe to conclude from these various studies that the greater majority of pupils say they like their schools. However, up to one in five don’t. There is some decrease in enthusiasm as pupils get older but only rather modest gender differences (depending on the question) with girls being slightly more favourably disposed towards school than boys. Differences in attitude tend to be linked to levels of individual attainment, with higher attainers being more positive but the strength of this relationship varies across studies and does not seem, for most stages of compulsory schooling, to be particularly strong although negative attitudes towards school do crystallise during the final years. There were also some differences in terms of social class backgrounds, with the most socially disadvantaged being least positive.
4. Some initial evidence of ‘school differences
Do students in some schools ‘like school’ more than ‘similar’ students in others? An important consideration for the selection of the studies just discussed was that they had something to say about the extent of ‘school differences’. However, the ways in which such differences were established varied considerably. In the Keys and Fernandes study (1993) around 9% of the total variance on the Positive Attitudes to Schooling scale lay between schools with 91% lying between students within schools. Whilst the study draws attention to the importance of within-school differences it nonetheless supports the view that there is variation between schools. Daly and Hefty (2001) report that just over five per cent of the variance on their Attitudes to School scale was at the school level, a finding which they refer to as ‘a selectively small estimate but in keeping with comparable findings in the research literature’. Smith and Tomlinson (1989) have reported similar results, also finding that the variations between schools were ‘rather small’ ranging from 2.86 for the highest to 2.32 for the lowest with a mean of around 2.5.
Gray and colleagues (1983) meanwhile, using a statistically less sophisticated approach than those now available, report variations between schools before and after controlling for background differences. In the median school in their study 47% of the pupils were ‘satisfied’ with a range from just 28% in the lowest to 80% in the highest. After controlling for background factors, the size of the differences between schools was roughly halved, a finding which echoes the research on differences in terms of cognitive measures.
The study by Thomas and colleagues reinforces some of these conclusions but also suggests some caveats are needed about others. The total variance between schools in terms of attitude items at one point in time varied from 1% to 3% on four attitude scales (of which ‘pupil engagement’ was one). But when significant explanatory factors were allowed to account for differences between schools, the remaining variation between schools was reduced to less than 1% for three of the scales (including ‘pupil engagement’) and was just 2% for the fourth (‘teacher support’). Thomas and colleagues report that on only 3 items out of 42 examined was ‘the percentage of total variance in pupils’ item scores attributable to schools greater than 3% (having already controlled for their previous item responses in 1995).’ In short, evidence of school differences in this study was very limited. Furthermore, Thomas and colleagues report that comparisons with another, very similar data-base from the English county of Lancashire LEA largely confirmed these estimates with, if anything, a slightly smaller range of differences across schools.
I conclude from these studies that there are some between school differences to be explained in terms of the proportions of their pupils ‘liking school’. However, the amounts vary between studies and generally seem lower than comparable estimates for academic outcomes. A median estimate for these latter studies might be 10%; in the case of attitude items this would seem to represent an upper limit. Within-school differences seem much more important.
5. The underlying explanatory models
What are some of the key variables which need to be built into studies of attitudes? In most school effectiveness studies some measure of prior attainment is almost always the best predictor of subsequent performance. Intuitively the same logic would appear to apply to the study of attitude variables. This impression is confirmed by the evidence from Daly and Hefty (2001) who report a strong correlation (0.74) between attitudes to school at two time-points; however, the gap over which change was monitored in this case was just one year of secondary school.
Thomas and colleagues’ results, by contrast, do not support these assumptions of stability. In general they report much lower correlations across a two-year period. The correlation, for example, between ‘engagement at school’ scores in S2 and S4 was as low as 0.22 whilst for ‘pupil culture’ it was 0.50. ‘Nevertheless’, as they state, ‘pupils’ prior attitudes were by far the most important variables to control for in the multilevel analyses of attitude outcomes – explaining between 5-26% of the total variance and between 3-69% of the variation attributable to schools’ (Thomas et al., 2000:293). Prior ‘pupil engagement’ at Time 1, along with self-reported ‘behaviour’ at Time 1, both made significant contributions to the explanation of differences in levels of ‘pupil engagement’ at Time 2.
Unfortunately, with the possible exception of the Thomas study, there is an essentially ad hoc element to the problem of what to control for by way of background factors. In the study by Gray and colleagues, for example, no prior attainment measures were available. The main ‘controls’ employed were nonetheless important ones: social class, parental education and gender. All made sizeable contributions to the reduction of initial school differences. Daly and Hefty, meanwhile, used measures of ‘academic aptitude’ (administered at around the same time as the other data were collected), home language, socio-economic status, free school meals and gender; the effects of these, once prior attitudes were controlled for, were modest but mainly on the borders of statistical significance.
By comparison the ‘controls’ used by Thomas and colleagues were more extensive. They included: gender, age, learning support, Record of Need, mobility (years the student had attended current school) and entitlement to free school meals. None of these factors, in fact, contributed significantly to the explanation of differences in ‘pupil engagement’. Gender and free school meals, along with a measure of whether a pupil was ‘younger than normal for the cohort’, did however feature in some of the analyses of other attitudinal outcomes. Gender and ‘parental interest’ were the only background factors reaching statistical significance in Keys and Fernandes’ predictions of Positive Attitudes to School.
With the exception of Thomas’ research, none of the studies were in a position to control properly at the individual level for pupils’ prior attainment in terms of academic attainment. The evidence from this her study suggests that it is probably advantageous to do so.
To summarise, some measure of prior attitudes in the same area(s) as those serving as outcome measures seems highly desirable. Background controls for gender and social disadvantage, along with measures of prior attainment, also appear useful although the explanatory power of these variables has usually been much weaker. The most salient feature of all of them, however, is their relatively limited effect on the reduction and explanation of between-school differences. The contrast with the models typically developed to explain academic outcomes is striking. The correlations between ‘key’ variables are often on the low if positive side, as is their combined explanatory power.
6. Truancy and Attendance
The discussion now turns to other outcomes. Amongst these ‘truancy’ is the only other area in which several studies have been conducted. The Scottish study by Gray and colleagues (1983) shows marked differences between comprehensive schools. At the extremes 37% of the pupils in one school reported that during their last year at school they truanted for ‘several days’ or ‘weeks at a time’; in another school none said they did so. The median school had 12% reporting this level with an inter-quartile range of 11% but introducing various controls for background factors (see above) reduced the inter-quartile range considerably from 3% more truancy than predicted to 4% less.
The levels reported in other studies support the case that there are variations in truancy between schools. In the Keys and Fernandes study (1992), for example, 23% of Year 9’s replied ‘yes’ to a question about whether they had ‘ever played truant during the last year’. Meanwhile in the study by Thomas and colleagues (2000) 4% of S4 pupils reported that they had ‘regularly missed a week or more’ whilst a further 30% said they had ‘missed a day here or there’. Unfortunately in neither study was the exploration of between-school differences per se pursued.
The Rutter study took a different but related tack, focusing on attendance registers for first, third and fifth year students across 12 secondary schools. Differences between schools in attendance increased with age (Rutter et al., 1979, Figure 5.1). By the fifth year rates varied rather more, ranging from around 85% to as low as 65%. A variety of background controls for differences between schools were considered including verbal reasoning group and parental occupation as well as other measures of school mix. However, the overall explanatory power of these measures, even in combination, remained relatively low (multiple r of 0.42, op cit., Table 9.7). Meanwhile, Reynolds (1976) reports fairly stable differences in attendance rates across schools over a period of seven years in 9 Welsh secondary modern schools; rates ranged from 75% to 88% with an average of around 80%.
In sum, it seems clear from these various studies that ‘similar’ pupils attending different secondary schools report different levels of ‘truancy’ and attendance.
7. Further outcomes
The number of other outcome measures considered in any of the studies dwindles rapidly at this point. Rutter reports that delinquency rates between schools in his study varied for boys from 16% to 44% and for girls from 1% to 11% whilst Reynolds (1976) found rates varying from below 4% to above 10% across 9 schools. In neither study did traditional background factors (including prior attainment) play much part in explaining between-school differences.
Perhaps the most interesting contribution to the evidence base in recent years comes from a Scottish study of ‘health-related’ behaviours by West and colleagues (in press). They researched the incidence of smoking, drinking, drugs and ‘unhealthy’ diets in 43 West of Scotland secondary schools; a longitudinal study, most of their analyses are based on samples of some 2,500 pupils. Differences between schools on some of these measures are quite striking. The range across schools for S4 pupils reporting that they were ‘currently’ smoking varied from 0% to 50% with a mean of 25%. For ‘current’ drinking the figures ranged from 33% to 84% with a mean of 63%. The incidence of having ‘ever’ taken drugs was seemingly high with a mean of 40% in S4 and a range from 17% to 79% whilst the proportions experiencing ‘unhealthy’ diets varied from 27% to 88% with a mean of 61%.
For each outcome the ‘controls’ included a measure from the primary school (P7) of the same behaviour along with gender, social class, parental and other background factors. Amongst these the previous measure of the same behaviour always appeared important. Nonetheless, perhaps the researchers’ most important conclusion, in line with other studies reported here, is that the ‘proportion of total variance between individuals attributable to the school level appeared small (2%-9%)’. “With one or two exceptions, the relationship between individuals’ perceptions of school life and each of the health behaviours are similar in any school. Smoking, drinking, drug use and (less so) unhealthy diet are more likely among pupils who are disengaged from education and who get on with fewer teachers, these in turn being associated with lower levels of school involvement and poorer ratings”. West and colleagues also report that the evidence for a ‘school effect’ varied across the outcomes. There was less evidence of such an effect for drugs use and diet (which were both more susceptible to ‘community’ influences) than for reported levels of smoking and drinking.
Some citizenship issues have been considered in a study by Schagen (2002), which was conducted as part of a larger international IEA study of civics. Perhaps the most striking thing about this study is the way the same background factors explained different proportions of the between-school variance for different measures. In the case of ‘knowledge of civics’, for example, background factors explained large amounts of the variance between schools but for ‘attitudes towards immigrants’ only 12%. Whether there is such a thing as a ‘civics-promoting’ school (and, if there is, how one would identify it) seems doubtful.
Finally, the Smith and Tomlinson study (1989) looked at more direct measures of pupils’ ‘engagement’ by considering pupils’ ‘participation in school activities’. By ‘participation’ they meant such things as “playing in school teams, involvement in a school play or concert or special evening, etc, going on school trips and visits, doing something special in an assembly or in a meeting of your year or house”. (1989:114). Again rates across schools varied with levels of participation in some double those in others.
The overall conclusion again seems clear - schools vary, to some extent, on these ‘other outcomes’.
8. Beyond the conventional
The discussion, to this point, has been confined to consideration of what might be viewed as some of the more conventional outcomes of schooling. Suffice it to say that most work on the ways in which schools might differ has been confined to studies of aspects of academic achievement. Over the years a number of researchers have pointed to the potential power of the conceptual frameworks employed within this tradition but bemoaned the essentially conservative nature of the outcomes deemed suitable for exploration.
There is, of course, no necessary reason why this ‘bias’ should exist. As Wrigley (2003) has recently reminded us, in reality society’s aspirations for schools extend considerably beyond the narrowly conventional. Furthermore, different constituencies expect different things. Some seek greater inclusion and social justice from schools, others a stronger contribution to the development of students’ creativity’, whilst still others are concerned about their democratic awareness. Such ‘outcomes’ might usefully be considered from a ‘school differences’ perspective. Suffice it to say that within the United Kingdom this has yet to happen on any significant scale – the evidence reviewed here scarcely amounts to a commitment. There are signs, however, that such considerations are being embraced more strongly elsewhere although still less than wholeheartedly. In Australia, for example, Ainley and colleagues (1998) looked at several ‘social objectives’ for schools which included: how students related to others; their commitment to community well-being; conformity to rules and conventions; their interest in learning; their self-confidence; and their optimism about the future.
9. The consistency debate
It is now possible to assess the question of whether school which do well on one outcome tend to do equally well on others and, in particular, Rutter’s ‘on the whole’ claim. Rutter reported that the correlation between delinquency and attendance was 0.77; for delinquency and academic behaviour 0.68; and for delinquency and behaviour 0.72 (Rutter et al., 1979:92-93). All of these are reasonably high correlations. Unfortunately other lower correlations were not reported directly in the text but can be calculated from the information provided. Between attendance and behaviour, for example, the correlation was only 0.44 whilst for academic attainment and behaviour it was 0.55. If all these figures had been reported at the same time, then more questions might have been raised about the assumption of consistency. The correlations between outcomes in the Welsh study (Reynolds, 1976) were up to 0.60 and, prima facie, seem to offer support for Rutter. On the other hand, none of the correlations in the Scottish study (Gray et al., 1983, Table 15.4) were above 0.30 which would seem to run counter to the Rutter position.
It is of interest, therefore, that over the intervening decades just two studies, those by Thomas and colleagues(2000) and Smith and Tomlinson (1989), appear to have anything to add to the debate. Of the two the former is the more substantial contribution. Thomas and colleagues concluded that the relationship between ‘value added residuals in terms of cognitive and attitude scale outcomes appears to be weak (the correlations range from 0.31 to -0.37) .....These findings suggest that in Scottish schools separate dimensions of effectiveness exist in the two areas of the cognitive and affective’ (2000:297).{1}
Smith and Tomlinson reached similar conclusions about the relationships between levels of pupil ‘participation’ and academic outcomes. They remarked that whilst there were “important differences between schools in the level of participation.......no significant correlation has been demonstrated in this small sample of schools between participation and progress in attainment at the school level.....the schools that do well and badly in terms of participation are different from those that do well and badly in terms of progress in attainment” (1989:177).
In sum, more than two decades later, the evidence relating to this key assumption about the consistency of school effects across outcomes remains meagre but the pendulum appears to have swung. In broad terms, the links between cognitive and other outcomes appear weak or non-existent.
These conclusions about ‘consistency’ across outcomes are somewhat less surprising if the limited evidence from primary school studies is factored into the debate. Mortimore and colleagues (1988), for example, noted that there were some schools which performed well in both the cognitive and affective areas but nonetheless concluded that: “...the effects of school membership on non-cognitive outcomes were not highly related to those on cognitive areas. It seems that the two dimensions were largely independent of each other” (1988: 204). A study of Dutch primary schools by Knuver and Brandsma (1993) is also frequently quoted in this connection. It found that the correlations between school ‘well-being’, ‘language achievement’ and ‘arithmetic achievement’ were low and concluded that “on the whole one can interpret the two domains as relatively independent but not in any way contradictory to one another......only a small minority of schools is highly effective or ineffective in both domains simultaneously” (1993:201-202). Both studies support the view that ‘success’ in one area is no guarantee of ‘success’ in others.{2}
Figure 1a attempts to capture the conventional wisdom with respect to schools’ consistency across cognitive outcomes. The general assumption, in line with much of the research, is that schools which do well on one cognitive outcome also tend to do well on others. Most schools, in other words, are believed to be distributed in the upper right and lower left segments of Figure 1a. Again the assumption is that schools have an underlying ‘capacity’ to generate differing degrees of academic success across a range of academically-oriented indicators. Research suggests that this view needs to be qualified, however, in several respects. Crucially, within the British system, it does not seem safe to assume that because a school ‘performs well’ in on subject area it will necessarily perform well in others – there is considerable variability.{3}
FIGURE 1A
Cognitive 1 (high)
I
I
I
(low) Cognitive 2------I-----Cognitive 2 (high)
I
I
I
Cognitive 1 (low)
The evidence reviewed here suggests that similar assumptions cannot be made about the consistency of schools with respect to relationships between their cognitive and affective/social outcomes. Schools would appear to be distributed (to a greater or lesser extent depending on the outcome measures) across all four quadrants of Figure 1b.
FIGURE 1B
Cognitive 1 (high)
I
I
I
(low) Affective 1------I-----Affective 1 (high)
I
I
I
Cognitive 1 (low)
The accumulated research evidence, contrary to some earlier assessments, now seems to suggest that the cognitive and affective/social outcomes of schooling are probably independent of each other. In short, it cannot be safely assumed that schools which do well in one area do equally well in others.
10. Discussion
Researchers and critics of ‘school effectiveness’ research have differed over the years regarding the extent to which the factors which contribute to school’s ‘effectiveness’ cluster and cohere in ways which constitute some kind of ‘capacity’ or ‘engine room’ for raising pupil performance. On the ‘coherence’ side of the debate Murphy, for example, has argued that:
“One of the most powerful and enduring lessons from all the research on effective schools is that the better schools are more tightly linked – structurally, symbolically and culturally – than the less effective ones.....There is a great deal of consistency within and across the major components of the organisation, especially those of the production function – the teaching/learning process” (1992:96).
Others, by contrast, have claimed that they can discern little or no coherence, suggesting that lists of ‘effectiveness’ factors are simply variables that have emerged in one or more studies as being associated with enhanced cognitive outcomes – in many cases, they argue, there is no necessary causal connection. Lauder and colleagues (1998:57), for example, refer dismissively to: “lists of ingredients of what makes an effective school.....identified in an empiricist way”. To talk of schools’ having ‘capacity’ in such circumstances would be inappropriate.
Both ‘sides’ may be guilty of overstating their case but the research reviewed here suggests some intriguing and, hitherto, largely unexplored possibilities. If one supposes, for a moment, that there is a strong (and probably causal) link between the traditional ‘effectiveness’ factors and pupils’ academic results, it would be convenient to assume that these same factors boosted the other outcomes of schooling as well. This assumption has often been made. Unfortunately, the evidence reviewed here does not support this interpretation. ‘Strong’ versions of theories about the links between outcomes and processes would demand that additional sets of factors were brought into play to explain other outcomes of schooling. These might or might not be aligned with each other. Suffice it to say that previous research has not explored this issue in any depth.
‘Weaker’ interpretations of the same evidence suggest other possibilities. Theoretically, it would be possible for a single set of ‘effectiveness’ factors to be related to academic outcomes and, at the same time, for the same set of factors to be related to other outcomes, provided that we accepted that the correlations in each case would be relatively low. This conceptualisation has the obvious advantage that it fits the evidence but there is also an equally obvious casualty – factors which are only ‘weakly linked’ to overall ‘effectiveness’ almost certainly challenge assumptions about causality.
A third possibility turns the assertion of a single underlying ‘effectiveness’ component on its head. This approach starts with the assumption that there is likely, on a priori grounds,to be a separate ‘capacity’ or ‘engine room’ for every outcome area. It would then become a matter for empirical investigation if these turned out to be related and conceptually linked. In the metaphor offered by Weick (1976), secondary schools may turn out to be more ‘loosely-coupled’ organisations than has previously been admitted. Certainly work on variations in ‘departmental effectiveness’ across subjects and over time would seem to point in this direction (Sammons et al., 1997:56).
Some lateral support for this view comes from the major review of American studies of ‘effective’ secondary school organisation undertaken by Lee and colleagues (1993) a decade ago. Reviewing the products of different organisational forms they remark:
“From the bureaucratic perspective, schools are seen as ‘formal organisations’ characterised by a functional division of adult labour into specialised tasks; teaching roles are defined b y subject matter and types of students; there is an emphasis on social interactions that are rule-governed, are affectively neutral and have limited individual discretion; and a form of authority that is attached to the role within the organisation rather than the person occupying the role. The ‘communitarian’ perspective (by contrast) views schools as ‘small societies’, organisations that emphasise informal and enduring social relationships and which are driven by a common ethos” (1993:173).
In the process, they develop ‘a heuristic model for the organisation of secondary schools’ which is sensitive to a range of levels contributing to outcomes for both students and, interestingly, teachers. Teacher outcomes include ‘satisfaction and efficacy’ and ‘commitment of effort’. With respect to ‘student outcomes’ two major possible outcomes are identified – ‘engagement’ and ‘achievement’. Importantly, in terms of this discussion, three potential features of the ‘internal organisation’ of schools which can enhance performance are highlighted. These relate to the ‘organisation of authority’, the ‘organisation of work’ and the ‘social organisation’ of the school. The analysis and partitioning of a school’s functioning into these three separate but related components lends credence to the findings reported here.
11. Conclusion
At the time of the original debates some two decades ago, there were too few British studies around to make any definitive judgements about how the different outcomes of schooling might be linked. With the considerable benefit of hindsight it is now clear that this was a key debate and one that has structured much of what we have subsequently come to term research on ‘school effectiveness’. It may have been a convenient assumption to make at that time that different outcomes were closely linked and that the factors that contributed to a school’s academic functioning facilitated other outcomes as well. If the evidence reviewed here had been available then, however, it might have tipped the research in a different direction.
What this review has shown is that there are differences between schools with respect to affective, social and other non-cognitive outcomes. Generally speaking, schools’ ‘effects’ in these areas do not seem as large as those relating to academic ones; nonetheless, there is fairly clear evidence that there are differences. It seems that there is no single underlying dimension of school ‘effectiveness’. Schools which do well in one area may do well or badly in others. Furthermore, the factors which have been used to explain differences between schools in terms of academic progress would seem to have limited explanatory power with respect to these other outcomes. Schools probably have several sources of ‘effectiveness’ which differ according to the outcome being considered. The need to retain young people’s commitment to education (and, as a corollary, to make schools places in which they want to spend more of their lives) remains pressing. It may be time, therefore, to focus attention more systematically on the task of teasing out what makes a difference in these neglected areas.
Refencias Bibliográficas
Ainley, J., Batten, M., Collins, C. y Withers, G. (1998). Schools and the Social Development of Young Australians. Camberwell: the Australian Council for Educational Research.
Daly, P. y Defty, N. (2001). Investigating variation across high schools in students’ attitudes to school, in their attitudes to science and in their attainment in science. Paper presented to the Annual Conference of the American Educational Research Association, Seattle.
Gray, J., McPherson, A. y Raffe, D. (1983). Reconstructions of Secondary Education: Theory, Myth and Practice Since the War. London: Routledge and Kegan Paul.
Hargreaves, D.H. (2001). A capital theory of school effectiveness and improvement. British Educational Research Journal, 27(4), pp. 487-503.
Jencks, C., Smith, M., Acland, H., Bane, M., Cohen, D., Gintis, H., Heyns, B. y Michelson, S. (1972). Inequality: A Reassessment of the Effects of Family and Schooling in America. London: Penguin Books.
Keys, W. and Fernandes, C. (1993). What Do Students Think About School? Research into the factors associated with positive and negative attitudes towards school and education, A Report to the National Commission on Education, Slough: nfer.
Knuver, J. y Brandsma, H. (1993). Cognitive and affective outcomes in school effectiveness research. School Effectiveness and School Improvement, 4(3), pp.189-204.
Lauder, H., Jamieson, I. y Wikeley, F. (1998). Models of effective schools: limits and capabilities. En Slee, R., Weiner, G. y Tomlinson, S. (eds.) (1998) School Effectiveness for Whom? Challenges to the School Effectiveness and School Improvement Movements. London: Falmer Press.
Lee, V., Bryk, A. y Smith, J. (1993). The organisation of effective high schools. Review of Educational Research, 19, pp.171-267.
Murphy, J. (1992). School effectiveness and school restructuring: contributions to educational improvement. School Effectiveness and School Improvement, 3(2), pp. 90-109.
Reynolds, D. (1976). The delinquent school. En M. Hammersley y P. Woods (eds.) The Process of Schooling. London: Routledge and Kegan Paul.
Rutter, M., Maughan, B., Mortimore, P. y Ouston, J. with Smith, A. (1979) Fifteen Thousand Hours: secondary schools and their effects on children. London: Open Books.
Sammons, P. (1999). School Effectiveness: Coming of Age in the Twenty-First Century. Lisse: Swets and Zeitlinger.
Sammons, P., Thomas, S. y Mortimore, P. (1997). Forging Links: Effective Schools and Effective Department. London: Paul Chapman.
Schagen, I. (2002). Attitudes to citizenship in England: multilevel statistical analysis of the IEA civics data. Research Papers in Education, 17(3), pp. 229-259.
Smith, D.J. y Tomlinson, S. (1989). The School Effect: A Study of Multi-Racial Comprehensives. London: Policy Studies Institute.
Teddlie, C., Reynolds, D. y Sammons, S. (2000). The methodology and scientific properties of school effectiveness research. En Teddlie, C. and Reynolds, D. (eds.) International Handbook of School Effectiveness Research. London: Falmer Press.
Thomas, S. Smees, R., MacBeath, J., Robertson, P. and Boyd, B. (2000). Valuing pupils’ views in Scottish schools. Educational Research and Evaluation, 6(4), pp. 281-316.
Tymms, P. (2001). A test of the big fish in a little pond hypothesis: an investigation into the feelings of seven year old pupils in school. School Effectiveness and School Improvement, 12(2), pp. 161-181.
Weick, K. (1976). Educational organisations as loosely coupled systems. Administrative Science Quarterly, 21, pp. 1-19.
West, P., Sweeting, H. y Leyland, A. (in press). School effects on pupils’ health behaviours: evidence in support of the health promoting school. Research Papers in Education.
Wrigley, T. (2003). Schools of Hope: a new agenda for school improvement. Stoke-on-Trent: Trentham Books.
{1} Some stronger correlations were found with individual attitude items such as ‘liking school’, ‘interest in school work’, ‘level of work’ and ‘truancy’ (see Thomas et al., 2000, Table 5). In these few cases the correlations were 0.3 or greater.
{2} A study by Tymms (2001) suggests that there may be weak but low correlations. However, perhaps the most interesting finding to emerge from this particular study is the relatively low proportion of the variance at the school level for any of three attitude items.
{3} There is a lengthy discussion of these assumptions in Teddlie et al. (2000:116-120) which suggests a number of qualifications need to be made to the overall proposition. In general they suggest composite school effectiveness indicators yield greater evidence of consistency than component ones (related to separate subjects).
|