Joan Mackintosh

Background: In this methodological paper we document the interpretation of a mixed methods study and outline an approach to dealing with apparent discrepancies between qualitative and quantitative research data in a pilot study evaluating whether welfare rights advice has an impact on health and social outcomes among a population aged 60 and over. Methods: Quantitative and qualitative data were collected contemporaneously. Quantitative data were collected from 126 men and women aged over 60 within a randomised controlled trial. Participants received a full welfare benefits assessment which successfully identified additional financial and non-financial resources for 60% of them. A range of demographic, health and social outcome measures were assessed at baseline, 6, 12 and 24 month follow up. Qualitative data were collected from a sub-sample of 25 participants purposively selected to take part in individual interviews to examine the perceived impact of welfare rights advice. Results: Separate analysis of the quantitative and qualitative data revealed discrepant findings. The quantitative data showed little evidence of significant differences of a size that would be of practical or clinical interest, suggesting that the intervention had no impact on these outcome measures. The qualitative data suggested wide-ranging impacts, indicating that the intervention had a positive effect. Six ways of further exploring these data were considered: (i) treating the methods as fundamentally different; (ii) exploring the methodological rigour of each component; (iii) exploring dataset comparability; (iv) collecting further data and making further comparisons; (v) exploring the process of the intervention; and (vi) exploring whether the outcomes of the two components match. Conclusion: The study demonstrates how using mixed methods can lead to different and sometimes conflicting accounts and, using this six step approach, how such discrepancies can be harnessed to interrogate each dataset more fully. Not only does this enhance the robustness of the study, it may lead to different conclusions from those that would have been drawn through relying on one method alone and demonstrates the value of collecting both types of data within a single study. More widespread use of mixed methods in trials of complex interventions is likely to enhance the overall quality of the evidence base.

Socio-economic variations in health, including variations in health according to wealth and income, have been widely reported. A potential method of improving the health of the most deprived groups is to increase their income. State funded welfare programmes of financial benefits and benefits in kind are common in developed countries. However, there is evidence of widespread under claiming of welfare benefits by those eligible for them. One method of exploring the health effects of income supplementation is, therefore, to measure the health effects of welfare benefit maximisation programmes. We conducted a systematic review of the health, social and financial impacts of welfare rights advice delivered in healthcare settings. Published and unpublished literature was accessed through searches of electronic databases, websites and an internet search engine; hand searches of journals; suggestions from experts; and reference lists of relevant publications. Data on the intervention delivered, evaluation performed, and outcome data on health, social and economic measures were abstracted and assessed by pairs of independent reviewers. Results are reported in narrative form. 55 studies were included in the review. Only seven studies included a comparison or control group. There was evidence that welfare rights advice delivered in healthcare settings results in financial benefits. There was little evidence that the advice resulted in measurable health or social benefits. This is primarily due to lack of good quality evidence, rather than evidence of an absence of effect. There are good theoretical reasons why income supplementation should improve health, but currently little evidence of adequate robustness and quality to indicate that the impact goes beyond increasing income.