Resetting the relationship between local and national government. Read our Local Government White Paper
Ben Walmsley and Anna Kime from the Centre for Cultural Value explore people-centred evaluation in the cultural sector.
Times are changing in the world of cultural policy. With both main political parties committed to further devolution in England, we are likely to see increased funding and decision-making at the local and regional levels. With local government’s core spending power having reduced in real terms by 24 per cent since 2010, this change is broadly welcome. Yet, it also raises concerns. After all, devolution will occur in the context of non-statutory local funding for culture and patchy, inequitable cultural infrastructure across the country.
Regardless of the challenges involved in distributing and allocating cultural funding at the local and regional levels, this shift in policy has significant implications for data, research and evaluation.
As cultural policy moves closer to people and their communities, we will need to rethink the established approaches to data collection and analysis. This will include challenging Treasury orthodoxy surrounding econometric methods of capturing value, which lack the nuance or humanity to reflect people’s lived experiences of cultural production and engagement.
Data is messier and often harder to capture and interpret at the local and hyperlocal levels. It requires a nuanced understanding of how people transverse a city or even a neighbourhood. It demands combining quantitative and qualitative data to capture both breadth and depth of impact. It also needs people-centred research based on anthropological methods such as participant observation and deep hanging out, as well as more traditional qualitative methods such as focus groups and interviews.
At the Centre for Cultural Value, we have co-created a bespoke set of principles to support evaluation in the cultural sector. Our four key principles outline how evaluation should be connected, beneficial, people-centred and robust. These core principles are all relevant when capturing cultural value at a more local level. Still, the notions of connected and people-centred evaluation, in particular, become increasingly vital the closer that policy shifts to people.
Connected evaluation means ensuring that evaluation practices are transparent, aware and shared. We must act ethically, be conscious of context and effectively communicate our findings while being mindful of our position and privilege as evaluators. Although this is easier at the local level (at least in theory), it requires different skills and more human resource than large-scale quantitative evaluation.
People-centred evaluation needs to be empathetic, many-voiced and socially engaged. It is hard for large-scale evaluations to lean into individual’s specific needs and interests, and usually impossible for them to co-create explorations of cultural value with local and sometimes marginalised groups. However, smaller-scale and more localised evaluation can explore the impacts of cultural policy effectively with local communities. In turn, this promotes direct representation and citizen engagement.
Over the past year, we have been testing out the Centre for Cultural Value’s Evaluation Principles through a series of living case studies and via dedicated policy placements with Leeds City Council (LCC) and West Yorkshire Combined Authority (WYCA).
Our placement with Leeds City Council’s Culture Programmes team last year focussed on how to evaluate the impact of their cultural investment funding programme. We also explored the framework within which the team collected cultural data.
Using the Centre for Cultural Value’s Wheel of Change Framework and Evaluation Principles, we proposed a cyclical and iterative approach. Funding priorities informed a route through to defined ambitions and a recommended accumulation of baseline data during the first year of delivery. Critically, our methodology was both pragmatic and tailored to the Culture Programmes team resource and expertise.
Our placement came during a period of significant change and provoked considerable thought about the role and experience of fund managers as policymakers: they are a vital part of the sector ecology, committed champions of culture, and often hidden within institutions or mired by the challenging decisions they’re required to make. We worked with the team to help highlight the integral role they played in the likely success of the investment programme.
Taking a people-centred approach to our own work, we helped the team acknowledge their expertise and experience in supporting the sector across the city. This also meant adopting an evaluation approach comprising a deeper, more rounded methodology. We helped identify points for reflection throughout the grant-making process, encouraging curiosity and a spirit of exploration about the data that led to the team feeling confident in their skills and insight and instrumental in developing the sector.
The Culture Programmes team was also placed at the heart of the developing cultural data strategy. The co-development of guiding principles for data collection encouraged the team to empathise with end users, such as freelancers and cultural venues tasked with collecting and returning the data. The Evaluation Principles illustrate an ambition for data collection that is respectful, equitable and transparent. The critical dimension here is to consider plural points of view over a dominating stakeholder requirement.
This year, the Centre for Cultural Value has embarked on a new policy partnership with West Yorkshire Combined Authority (WYCA), focusing on cultural evaluation and data collection across the region. The collaboration comes at a pivotal time for West Yorkshire, which will have hosted no fewer than five dedicated years of culture from 2023 to 2025. These include LEEDS 2023 and Kirklees Year of Music last year, plus Wakefield and Calderdale are both hosting year-long cultural celebrations in 2024. Meanwhile, momentum builds towards Bradford UK City of Culture 2025.
Mayor Tracy Brabin and the five district leaders have taken the unique opportunity presented by the years of culture to unify approaches to evaluation and the collection of cultural data. In developing our partnership with WYCA, the Centre will be part of a new cultural data and evaluation working group. This group will bring together representatives from Bradford, Calderdale, Kirklees, Wakefield and Leeds councils alongside Bradford 2025 and LEEDS 2023 to develop shared evaluation and data collection practices across the region.
The Centre will bring to the new partnership specific insight from our 2021 national research project Making Data Work. Our research concluded that there are significant strengths in current data resources, with substantial benefits anticipated if these can be linked and made more accessible. However, most cultural organisations lack the resources to employ data scientists to benefit from existing data. It is also the case that the core driver for evaluation data is often to meet funding requirements, leaving creative practitioners less able to develop skills or processes to support robust qualitative or mixed-methods data gathering and evaluation. This means that current methods and evaluation approaches often fail to convey the social impact of cultural activity in any compelling way, leaving it even more vulnerable to funding cuts.
As the Royal Society of Arts (RSA) sets out: “Currently, it is rare for local authorities to cover economic, social and natural systems in a single strategy. If future efforts are to be more successful, this needs to change. What is required is a single, long-term integrated strategic plan to deliver prosperity to city residents and beyond through the regeneration of its economy, society and environment.”
We argue for strategic leadership and long-term, shared solutions to address these issues. Our partnerships in West Yorkshire offer us invaluable opportunities to continue embedding our Evaluation Principles and to learn more about how to fund and evaluate place-based cultural activity ethically and effectively.
We know from our constituents that the brokering, convening and synthesising role the Centre plays between research, practice and policy is both welcomed and highly valuable. The opportunities to collaborate with local and regional authorities promise to enrich our insights even further. As partners, we will continue to champion a holistic approach to data and evidence gathering that is robust, connected, beneficial and people-centred.
After all, faced with difficult funding and resourcing challenges, we must be clear-sighted about what kinds of cultural activities create value, for whom, and in which conditions and contexts. This knowledge will help ensure cultural policy can build more equitable, vibrant and stronger communities for everyone.