Some thoughts on monitoring, evaluating and researching culture

Last week, I participated in a round table discussion organised by the Cultural Value Scoping Project at Tate Modern. The initiative is a collaboration between the Arts and Humanities Research Council, the Paul Hamlyn Foundation,  Arts Council England and King’s College London and King’s College London. It aims to explore how research, evaluation, evidence-building and analysis into cultural value might best be supported in the future. Those present included people working in cultural institutions, academic researchers and others involved in the field. About 15 speakers were invited to contribute a 10 minute reflection during the day, with the rest of the time given over to some fruitful discussion. My contribution  – the notes are below – addressed the question:

In what concrete ways could the new platform dedicated to research and analysis into cultural value support the cultural sector?

Value is subjective

There is no universal agreement about what is valuable beyond a small number of essentials, such as life, shelter, education and so on. Because there is agreement about these things, they are described as rights, not values. But there aren’t many: the Universal Declaration of Human Rights has just 30 articles.

Everything beyond that (and nowadays, even some of that) is subjective, a matter of choice, including culture. Human beings do not all value the same things. We sometimes find what others value ridiculous or even offensive, including culture. But in a society where human rights are respected, we tolerate other people’s choices.

Voltaire said, ‘Dissension is the great evil of mankind and tolerance its only remedy’. And politics is the means by which we negotiate our disagreements about value and the limits of our tolerance.

Value is comparative

Value is also comparative. Even absolute value, such as the right to life, is evident only when compared with alternatives: being alive is better than being dead. Disagreements about value are also understood comparatively. A democratic society is better than an autocratic one. A kidney dialysis machine is worth more than a public sculpture. Both of these statements are choices that reflect comparisons in the context of values held.

One of tasks of politics is to negotiate the relative value that we give to different things, expressed in the abstract system of money. How we gain and spend money – individually and collectively – is an expression of values, choices made on the basis of comparison. It’s messy, sometimes ugly and often unfair, but we haven’t got a better way of preventing our disagreements about value being resolved by violence.

Cultural value is political

It is many years since I began trying  to understand the value of participation in the arts, first in my own work and then in the work of others. Often that has involved working with people who want to prove the value of art or culture. I understand that desire. If your work depends on comparative choices made by other people, it is natural to want to persuade them that what you do has relatively high value.

The problem is that value, which is subjective, cannot be proven because proof is an objective concept used in the natural sciences. We can prove that the melting point of iron is 1,538 degrees Celsius. Once proven, it is no longer a matter for debate, or an ‘alternative fact’. We cannot prove the value of culture. We cannot even agree a definition of culture.

Acting politically

Those who want culture to have a greater importance in society, public policy and budgets, must act politically, not scientifically. They must engage others – and particularly those who disagree with them about culture’s value – in persuasive debate. Data, evidence and knowledge may all help in that task, but so will the ancient tools of politics: rhetoric, argument, emotion and the rest.

Trying to prove cultural value is like hunting the Snark, and while we are occupied with that fruitless search, we fail both to engage effectively in political debate about culture’s value and to learn from the knowledge that our research does produce.

The purpose of cultural research

If we separate debating value from understanding value we enter into more fruitful territory.  Instead of trying to find proofs that can persuade others to change their beliefs – which, insofar as it can be achieved at all, is a political task best undertaken with political means – we can look for knowledge that can influence the actions of those who already hold certain beliefs about the value of culture. In this context, there is practical value in testing individual subjectivity to find the common ground for shared judgements.

The proper purpose of cultural research, I believe, is not persuasion but improvement.  Whether that research is undertaken independently by academics, commissioned by a cultural actor, required by a funding agreement or simply done by a cultural organisation as part of its everyday operations, its purpose should be to increase knowledge and understanding of cultural activity.

How and why

As such, its questions are concerned less with whether than with how and why. Whether something happens is important, of course, especially to funders and in political discussion. How and why it happens are much deeper and more powerful questions because they have the potential to challenge the cultural actor’s own beliefs and assumptions. They have the potential to influence the way art is created, managed and presented, the programmes offered and the ways in which people are able to engage with them, the interpretation and meaning of culture itself.

In my experience, however, it is not only politicians who are uninterested in how and why cultural experiences produce value for people. The cultural profession – with exceptions – is not very interested either, especially if the insights research produces might point to ways in which they could productively change. As far as cultural policy and management goes, it seems all but impossible to get beyond ‘whether’ and the mirage of proving value.

What might be done?

As we all know, the first thing to do when you’re in a hole is to stop digging. And I believe that research and evaluation into public cultural investment is in a deep hole. Let me clarify that I’m concerned here only with what is done within, and required by, the public cultural sector, whoever is doing it. I’m not referring to the growing body of independent, mostly academic research into the theory, policy, work, management or practice of culture.

The principal reason, as I’ve suggested, is that professional engagement with cultural value  is geared towards an unavailable proof, but over the years a host of subsidiary reasons – such as the adoption of inappropriate public management concepts and practices – have also contributed.

When I say, stop digging, I mean let’s take a pause and look around. The Cultural Value Scoping Project is an opportunity to do that, to ask what we are trying to understand and why. Then we might review what we are currently doing and apply a simple test to it: Does the use to which it is put justify the cost of producing it?

If it were possible to back out of the hole we’re currently in, I’d suggest a different approach to monitoring, evaluation and research, which would begin by distinguishing clearly between the three kinds of activity.

Monitoring

The foundation of knowledge about cultural life must be factual and at least where public spending is concerned, there is a great deal that can be known simply through the process of making grants and monitoring the resulting activity. It should be possible, especially now that the grant-making process is computerised, to design a system that can produce reliable data about the nature and extent of activity supported, say, by Arts Council England.

Accountability in grant making should be achieved mainly through monitoring, not evaluation. If the activity was assessed as having public value at application stage, and monitoring shows it to have been completed as expected, then it should follow that its value was delivered in terms of policy and spending decisions.

Refining the application and reporting requirements so that they are factual, consistent over time and used in across the sector, would enable ACE and independent researchers to analyse the resulting data from different perspectives and produce reports that highlight patterns or inconsistencies.

Evaluation

With monitoring processes that are capable both of demonstrating the public value of grants and providing big data that can be investigated for trends, the evaluation process can be freed from the need to justify spending decisions. It can then become a truly open-minded inquiry into how and why results have been produced from which things can be learned and changes to practice made.

In this context, the Arts Council’s recent focus on self-evaluation is very welcome. Evaluation is a critical part of creative work which artists and arts organisations should be doing naturally and in order to improve their work. It will be very difficult to change a culture that sees evaluation principally as a tool of self-justification. But with time, training support and a greater use of monitoring in assessments, some progress could be made.

At the same time, self-evaluation needs to be reported in a form that can be shared internally and with peers. Again, the collection and cataloguing of this material would provide a huge body of data that could be investigated by researchers.

Research

With more reliable data produced by reformed monitoring and evaluation activity, a body such as the Arts Council would be in a position to commission a range of external and independent studies of cultural activity. Some of that might test the self-evaluation process by looking at a representative sample. Some of it might use the data in order to answer specific questions about sub sectors or kinds of practice. This approach would allow the regular publication of good quality thematic research reports that can contribute to debate within the sector and at the Arts Council and thus lead to changes in policy or practice.

In conclusion

This is no more than a sketch of some ways in which research into cultural value might change to become more influential than it has been to date. This may not be the best approach and, to be honest, even if it were I’m doubtful whether it is achievable given the  entanglement of judgements about the value of individuals and of practices with financial decisions. Better research can give us better knowledge but in the end, politics will determine the decisions we make.

•

Postscript

I’ve been writing about cultural value in one way or another for a long time: some of my previous posts on the subject can be found here, though I’m aware that there’s a certain amount of saying the same things again and again:

• On ‘the very idea of measuring cultural value’ (January 2012)
• Quality in community art (November 2013)
• A Three Pipe Problem (March 2014)
• Both sides of the coin, art and health (March 2016)