Using “low-end” evidence in social policy: Case from Guangzhou, China
Governments value evidence-based policy; but are policy makers using all possible evidence to inform their decisions? Dr. Anna N. Li, Postdoctoral Fellow at UNSW Canberra argues that "soft, qualitative, practice-based evidence can be used to better inform decision making by providing frontline, implementation information, which can increase the chance of policy success.
This blog post is based on Dr. Li's paper that was presented at the 5th Public Governance Forum in Greater China: "Evidence-based Policy Making in Greater China and Australia," hosted by Fudan University in Shanghai, China on 21-22 October, 2017.
Evidence-based policy-making is by no means new to any administration nowadays. This includes the Australian Government, which recognises that “it is as important that [the Government has] a rigorous, evidence-based approach to public policy in Australia today as at any time in our history (Banks 2009).”
The enthusiasm of evidence has also been stimulated and encouraged in developing economies like China. The country seems to be in the midst of a global trend to promote evidence-based policy making, or a call for “scientific and democratic decision-making” since the Hu-Wen Administration. When the reliance on evidence and knowledge in policy making is increasing globally, two classic questions remain - what is meant by the term “evidence”, and what determines the uptake of evidence.
Evidence could be from various sources, but they are collected through a “systematic process of critical investigation and evaluation, theory building, data collection, analysis and codification related to development policy and practice (Sutcliffe and Court 2005)”. Systematic research or scientific knowledge has provided an important contribution to policy making, but it is only one of the inputs for evidence-based policy. The larger world of policy and program debate comprises of several other types of knowledge and expertise that have legitimate voices in a democratic society (Head 2009). Head (2009) summarises four types of knowledge related to evidence-based policy making: political knowledge, scientific rigorous knowledge, professional-managerial knowledge, and client and stakeholder knowledge.
What’s more remarkable, however, is that different kinds of evidence or knowledge share different levels of importance, relevance or weighting, or what we call the “hierarchy of evidence.” Government organisations and agents tend to make hierarchical judgements in choosing what evidence to use and how, and these decisions are often deeply embedded in the value system and the power structure of the organisation.
In reality, evidence shows that consumers rely more on the “top-end” evidence from high-level evidence producers, which normally refers to scientific, rigorous knowledge produced by academics and research institutions. This also creates ignorance of practice-based knowledge is low in the hierarchy and grassroots knowledge such as professional-managerial knowledge as well as client and stakeholder knowledge.
In addition, evidence consumers are in favour of “hard” evidence such as primary quantitative data collected by researchers from experiments, secondary quantitative social and epidemiological data collected by government agencies, clinical trials, and interview or questionnaire-based social surveys. In contrast, “soft” evidence from the lower parts of the information hierarchy, which is viewed as consisting of qualitative data such as ethnographic accounts and autobiographical materials, receive little attention from policy makers (Marston and Watts 2003). This ignorance could limit potential policy solutions, which in turn, may increase the chance of policy failure. Therefore, it is important to acknowledge the considerable diversity of evidence, determine how the practice-based evidence and knowledge flow between the producers and users, as well as establish what determines the application of this kind of evidence.
China’s Guangzhou Municipal Government has an interesting case for illustrating how this ‘low-end’ evidence can be used effectively. The massive growth of the aging population has given rise to the widespread concern about the urgent need for long-term aged care in Guangzhou. Meanwhile, intensive inter-city competition in social policy innovation also pushed the local government to be active in regulating non-state owned old age homes.
The government officials, in this sense, have shown their great interest in taking evidence-based policy making to avoid or minimized policy failures caused by the mismatch between their expectation and actual situation on this major policy issue. When the municipality’s Bureau of Civil Affairs, which oversees social welfare, had to decide whether they should abandon the double-track registration for old-age homes, they sought knowledge from the Guangzhou Social Welfare Association rather than from academic institutes. This industry association derives its legitimacy from providing advice as well as connecting government departments and practitioners. Its staff are former government officials who have acquired years of practical knowledge rather than deriving information from academic research. In the absence of market information, the government department was in the great need of specialist expertise to address the problem and make the right choice. The staff of the Association collected opinions from the practitioners through a small-scale informal questionnaire survey.
The results, which were submitted to the Bureau, clearly indicated that the problem laid not with the registration status of old age homes, but rather the lack of financial support from the government. The Bureau has since adopted the opinion from the industry association, and maintained the double-track registration system, which has since become the most innovative feature in the social policy framework of Guangzhou.
The above case shows that evidence and knowledge from the lower rungs on the hierarchical ladder can impact policy processes, with this type of “soft” evidence being increasingly used in two instances: 1) Solution-driven governmental departments are more open-minded to practice-based knowledge and evidence produced by both academics and practitioners as they are likely to require frontline information on potential implementation challenges; and 2) In order to solve complex social policy issues that involve a great number of stakeholders, policy makers will seek professional and stakeholder knowledge to map the terrain and legitimise the policy making process.
While “low-end” information is being integrated into policy making, some policy makers still hesitate to use “soft” evidence. Academic research has long been legitimised by its systematic and scientific process of investigation and evaluation; but in contrast, the legitimacy of “low-end” knowledge is much harder to obtain as it relies on the trust between the knowledge provider and the consumer, a more fragile foundation for evidence acquisition. Furthermore, how can we guarantee the reliability of this kind of evidence and the independence of “low-end” expertise? Until we develop a way to legitimise and validate “low-end” evidence, the rigorous and continued use of “soft” data in evidence-based policy making may remain elusive.