Evaluating Public Library Impact and Outcomes

Searching for meaningful and measurable impact and outcome measures has become something of a Holy Grail for public libraries.

The approach to date has been an attempt to replace quantitative statistical indicators with qualitative outcome measures. But maybe there is another way forward which can translate some of our traditional performance indices – visits, circulation, membership – into proximity measurements for much larger societal impacts and outcomes. This would eliminate the need for each public library to devise its own unique set of outcomes.

Instead, all public libraries could focus on their individual contributions to big picture agendas which have a high local, provincial and national profile.

Meaningful Impact and Outcome Measures

Creating meaningful impact and outcome measures can be challenging and it is often difficult to distinguish between cause and effect. For example, what, if any, is the relationship between library use, equality, happiness and well-being? And, if there is a relationship, do public libraries make people feel more equal, happier and improve their well-being; or are people who feel more equal, happier and who have a greater sense of well-being more likely to use the library?

Library statistics such as visits, circulation and membership have traditionally been used to measure public library inputs and outputs. But these statistics could also be used to measure the library contribution to impacts and outcomes such as equality, happiness and well being. These metrics may indicate contribution, not attribution. In other words, they measure whether the library is one of the causes of improvements in the lives of library users, not whether (or how much) the library is directly or solely responsible.

Establishing the costs of public libraries is relatively straightforward, but calculating the benefits can be more challenging. So that we can demonstrate whether or not an intervention creates a net benefit, we would ideally like to express benefits in monetary units thus allowing direct comparison with costs. In standard cost-benefit analysis, monetary values are estimated by making inferences about people’s willingness to pay from market data or from asking them directly for their willingness to pay in a Contingent Valuation study. So, if we wanted to value the use of public libraries, we would typically look for market data and/or we could ask people hypothetical questions about their willingness to pay for particular activities and benefits.

Key Studies

A Spanish study used Contingent Valuation to estimate The Economic and Social Value of Information Services: libraries – report of findings (Yanez, 2014) to public library users ($309 pa) and non-users ($46 pa).

A UK study has estimated the Economic Value of Library Services (Archives, Library and Museums Alliance, 2012) to be between $43 and $49 for each visit. That is between 5.5 and 7.5 times greater than the actual cost of provision. The study considered five measures of value: user investment, community benefits, cost of alternatives, employment effects and supply chain effects.

So Much More: the economic impact of the Toronto Public Library on the City of Toronto (TPL, 2013) was the first Canadian public library study to measure in concrete economic terms the Return on Investment for library service. For every dollar invested in TPL, Torontonians receive $5.63. This benefit results from the market value of services delivered, or direct tangible benefits and the stimulus to Toronto’s economy from direct spending and re-spending (indirect tangible benefits). The total direct benefit of being a library member has been estimated at $502 pa.

The methods used by these studies are not without their problems, however. Revealed preferences are often not available and, where they are, it is often questionable whether they capture the true impact on ‘utility’ of a good or service. One of the main problems is that our preferences are often ill-informed and influenced heavily by context and sometimes by irrelevant cues and framing. There is now an alternative approach to valuation which shows considerable promise and which is increasingly being used in the public policy context.

The Well-Being Valuation Approach

The Well-being Valuation approach looks at the impact of a range of factors on Subjective Well-being (SWB). If we also gather data on income, we can look at the effect on SWB of a change in income alongside the effect of a library intervention (or expected benefit from that intervention). In so doing, we are able to estimate the income required to bring about the same impact on SWB as the library intervention, thus enabling us to express the value of the intervention in monetary units. So, if we want to value library engagement, we can now look for the impact on subjective well-being that particular activities have and compare that to the impact from income. The derived values show the increase in income that would be required to result in the same well-being increase.

This methodology was used in a government sponsored study in the UK, Quantifying and Valuing the Wellbeing Impacts of Culture and Sport (Fujiwara, 2014), which identified the well-being impacts of cultural engagement and sport participation. A significant association was found between frequent library use and reported well-being. Using libraries frequently was valued at $2,446 per person per year for library users, or $203 per person per month. Sport participation was also found to be associated with higher well-being. This increase was valued at $2,028 per person, per year, or $169 per person per month. Arts engagement was also found to be associated with higher well-being. This was valued at $1,951 per person per year, or $162 per person per month.

The monetary well-being value is based on individuals’ own perceived value to themselves of engagement rather than a wide value to society. This individual value (primary benefits) should be considered in light of the wider social impacts (secondary benefits) in order to provide a more holistic consideration of the full non-economic impacts of library engagement.

John Pateman is the CEO / Chief Librarian of the Thunder Bay Public Library. The Open for All? column explores the nature of libraries and their commitment to openness.