A Different Heartbeat (MCV 3)

The gold standard in evaluation

Twenty years ago, when I began thinking about how to improve the evaluation of community arts programmes, I turned to practice in health and medicine. I had been working with other artists in hospitals, care centres and other institutions for several years. Two large projects had been funded by the Department of Health, the Social Services Inspectorate and the Joseph Rowntree Foundation and required substantial evaluation.

But more importantly, perhaps, I shared the widespread perception that the health profession owned the gold standard in outcome evaluation: the randomised controlled trial (RCT). My assumption then was that policy makers would consider the value of participation in arts programmes only if the outcomes could be down in research that compared with that medical benchmark.

Like many people educated in the arts, I was willing to defer to scientific methods and epistemologies, without knowing exactly to what I was conceding authority.

The limits of medical research

But the more I read about health and medicine, about policy formation, evidence and evaluation, the more questionable that deference came to seem. In 1996, I published Defining Values, a working paper with some preliminary ideas (as part of the research eventually described in Use or Ornament?). In it I noted some gaps and inconsistencies in health evaluation, including the fact that RCTs are required only for drugs and not for procedures, which are much more unevenly evaluated. I also found that some drugs established before RCTs became universal have never been tested in this way: since they are known to be effective, such a trial would require denying them to patients who would benefit from them.

Most strikingly, I came to see how health policy – including major changes such as NHS reorganisation – was rarely based on RCTs, clinical evidence, or even piloting. It was principally driven by the beliefs of politicians and their allies.

In itself, that is not wrong. How can we act, except on our beliefs? But if we deceive ourselves about the evidence base on which those beliefs are based, if we claim evidential authority as a way to quash objections or stifle argument, we step onto a very slippery path indeed.

Looking for alternatives

In subsequent years, more experience in research and further reading have made me increasingly wary of the misapplication of scientific – and specifically medical – evaluation models to the complex experiences that people have through the arts. I value those methods in themselves. I applaud the efforts of people such as Ben Goldacre and Mark Henderson to defend the integrity of scientific method from misuse, ignorance and misrepresentation. But simplistic attempts to transfer them to art and culture are unwise and misleading.

So in recent years I have looked for alternative ways to account for (a concept I prefer to ‘evaluate’) the outcomes (preferred to ‘impact’) of people’s experiences of the arts. An account describes rather than judges; it is modest in its nature, allowing the reader space to form a response. The approach also recognises and values, though not uncritically, the particular ways in which textual and visual language make sense of human experience.

It has also encouraged me to test existing research norms and practices, something that may be easier to do from the position of a writer or an artist than that of a researcher bound by the procedures of her or his institution; that, however, is for another time.

A Different Heartbeat

The images that illustrate this post are taken from A Different Heartbeat, an account of a residency by musician Patrick Dineen at the Kidney Dialysis Unit, Royal Liverpool University Hospital, in spring 2011. They were done by Mik Godley, with whom I’ve since been working on a project about artists in old age, during the several visits we made together to the ward to speak to patients, staff and others involved in the residency.

A Different Heartbeat describes an intimate, small scale arts in health project, and places it into a wider context of questions about chronic illness, well being and the nature of benefits. It is an essay, a reflection on particular experience, and so about as far from a randomized controlled trial as it could be. But perhaps in that difference is something of value also.

In Defining Values I quoted the medical sociologist, Clive Seale, writing that:

The results of research may often influence policy makers in a more diffuse way, seeping into their thinking, in a manner that eventually provides enlightenment.

In the unavoidable absence of certainty, enlightenment is good enough.

Download 2012 A Different Heartbeat (PDF 940kb)