Chris here: Because U.S. academia is so decentralized and stratified, few new regimes have a national scope and can be hard to see coming. The opposite is the case in the U.K., where the influence of Westminster can be felt at the same time on campuses across all four constituent countries, with the partial exception of Scotland. External assessment increasingly defines and drives peer review in British universities. In the hope of getting more U.S. academics thinking about the effects of external assessment. we're cross posting this piece from the UK's Sociological Review. It offers a primer on the Research Excellence Framework and argues that it has had a negative impact on the depth and ambition of research in sociology--and by analogy, in other fields.
by Les Back, Professor of Sociology, Goldsmiths College London.
Twenty years ago public sociology was something you did in your spare
time. Even writing for newspapers or magazines was thought of as an
extra-curricula and extra mural pursuit. That all changed as a result of
the debate stimulated by Michael Burawoy’s influential Presidential
address to the American Sociological Association address in 2004 that
called for a public sociology.
Burawoy’s spirited appeal to revive sociology’s public mission licensed
a wide range of productive arguments and unruly activities. I have
always held the view that intellectual life is nothing if it is not
addressed to a wider public.
More recently, and significantly in the UK, the emergence of the
‘impact agenda’ in British universities has forced academic researchers
to evidence our influence on society and ultimately to justify our
worth. In austere times universities are required to show that they are
worth their salt. Indeed, this was legislated by the Treasury as part of
the spending review in order to secure continued investment in
university research. It is embarrassing to remember that some of us - at
least initially - thought that ‘impact’ promised the possibility of
institutional recognition for public sociology. Might the emphasis on
relevance and engagement create a ‘public agora’ for sociological ideas
of the kind described by
Helga Nowotny and her colleagues?
Another President, this time of the British Sociological Association, had a very different view. John Holmwood
warned in 2011 that
it was “naïve” to think that the turn to impact would lead to an
enhanced public sociology. Rather, he suggested in contrast that UK
research would likely be “diverted into a pathway to mediocrity." Surely
not, I felt when I first read this piece: John, you are being overly
pessimistic! How right he has been proved to be.
Academic research in the UK is evaluated and scored by periodic
reviews of the research of every sociology department. This has taken
different forms and the latest version is called the Research Excellence
Framework. Sociologists are required to nominate four publication for
review by a panel of senior academics within each field. Impact was
institutionalized in the recent REF2014. 20% of the scores were
measured through ‘impact case studies.’
The impact case studies were
required to narrate and evidence the detail of the impact that the
underpinning research had on society and they either focused on
individual staff members or clusters of academics. These impact case
studies were scored by ‘practitioners’ working in applied areas that
were added to the overall judgment of the REF panel. In case readers
need to be reminded,
HEFCE defined public value of impact along the following lines:
Impact includes, but is not limited to, an effect on, change or benefit to:
• the activity, attitude, awareness, behaviour, capacity, opportunity, performance, policy, practice, process or understanding
• of an audience, beneficiary, community, constituency, organisation or individuals
• in any geographic location whether locally, regionally, nationally or internationally.
Also, the number of impact case studies required for each unit of
assessment varied according to the number of staff submitted for the
assessment. Small submissions of up to 15 members of staff required just
two case studies, while large submission of over 35 members of staff
required five cases studies.
It seems clear that impact is here to stay. At present it is likely
to increase to 25% of the grade for each unit of assessment in the next
REF2020. A whole infrastructure is emerging to
assess the assessment,
where consultants act as ‘Impact Tsars’ and offer advice and software
designers are developing digital tools to trace and matriculate the
public imprint of our research endeavors.
The shame of the Research Excellence Framework is it secrecy: all the
data on the process of the decision-making was destroyed. There is no
mechanism for appealing or questioning these judgments. While the list
of panel members is public, the specific reviewers are anonymous and
therefore individual department's sociologists do not know who is
judging them and whether or not they are qualified (see
Derek Sayer's analysis).
The level of feedback is insubstantial, while the results have profound
consequences for each unit of assessment in terms of their income and
academic reputation.
In large part the "impact agenda" has licensed an arrogant,
self-crediting, boastful and narrow disciplinary version of sociology in
public. This is impact through "big research stars" that are scripted –
probably by the editors of the impact case studies rather than
themselves – as impact "super heroes" advising cabinet ministers and
giving evidence to parliamentary select committees. This version of
public intervention is by definition narrowly concerned with evidencing
its own claims. It is aligned with providing a kind of reformist
“empirical intelligence” that nudges at the edges of policy and
political influence. Reviewing the 96 REF2014 impact case for sociology,
80% of them can be categorised in this way.
That isn’t the whole story and 20% of those impact case studies
entered showed radical ambition. What was inspiring in the best of the
impact case studies is how they also point towards a different kind of
model of public engagement by challenging campus sexism though
collaboration with Students Unions or creating archives of political
activism. In the most appealing and compelling cases, clusters of
scholars worked together to try and shift the public agenda through
evidence and critical enquiry that challenged conventional thinking be
it around race and segregation or casual forms of class stigma and
hatred. These examples offer an alternative way to think about how to
hold to a public commitment within the current climate.
In 1967 Howard S Becker wrote an influential essay called ‘Whose Side Are We On?”. This
essay is often understood simplistically as a sociological call to
simply to align with the underdog. It is important to note that Becker’s
argument is critical of sentimentality that also can be blinding while
posing in colours of radicalism. Rather, Becker says it is not possible
for a sociologists to stand outside the issue of value and values: “the
question is not whether we should take sides, since we inevitably will,
but rather whose side we are on."
For all of our radical affections and promises, a close look at the
public portrayals of sociology in REF2014 show the ‘impact agenda’ to be
tinkering with minor reforms. In the final analysis, this agenda puts
us on the side of the political elite, Ministers of State, Job Centre
Managers, Immigration Officers and the apparatchiks of prevailing
government policy. Bluntly, it puts us on the side of the powerful.
Is that version of sociology worth its salt? Is this a compromise too
far for the discipline? Some will say, “well this is ‘just a
neo-liberal game” and this “isn’t all of sociology.” They are right. Not
every sociologist in Britain has to write an impact case study--yet.
They might also console themselves with the idea that this is just a
language game: we need to play and not take it too seriously. I would
suggest that these patterns amount to more than that. The ‘impact
agenda’ is coming to constitute our self-understanding, guide our
decisions around job appointments, and I would go further to suggest it
limits the public ambition of our discourse. Remember that next time
someone says: “I think that might make a really good impact case study.”
This preoccupation is acting as a filter for our sociological
attention.
It is a reminder to those of us who feel that sociology has or had
radical potential that ‘the public’ is not necessarily populated by
incipient transformative forces and potentials. Burawoy’s conception was
limited by leftist assumptions regarding the radical potential of civil
society. What we are seeing is something much closer to
Antonio Gramsci’s characterization of ‘organic intellectuals’ who
are tied to the interests of institutions and a narrow set of functions
that are “organisational and connective”. Edward Said, commenting on
Gramsci’s prescience, wrote that, as a result,
“organic intellectuals are always on the move, on the make”.
This is reminiscent of today’s “impact agenda” and the opportunistic
way we have been steered to think about sociology in public, where the
bottom line becomes “can I make this into a impact case study?” We are
required – by our institutional commitments and responsibilities – to be
on the move and on the make.
Today sociology’s radical ambition is being dwarfed by a conservative
and timid version of the discipline. This is what we are seeing in the
Research Excellence Framework, which itself produces an academic
performance of self that is in keeping with its own definition of public
value. There are other ways to think of public sociology that return us
to Buroway’s intervention of more than a decade ago. His vision of
sociology in public might be usefully supplemented with the educational
ethos that is steeped in the tradition of extra-mural studies led by
Richard Hoggart, Raymond Williams and Stuart Hall and out of which
cultural studies emerged. This was a communal vision of higher learning
or what the Worker’s Education Association called a collective highway.
A sociology with and for the public, is, I would argue, one that is
humble, collective, dialogic, inventive, artful and trans-disciplinary.
Here sociological ideas can offer a navigation device or a compass and a
way of attending to what is before us but also to determine our
direction of travel. That would be a future sociology worth its salt. But it is not one supported by the REF assessment system.