PART ONE: UNDERSTANDING AND USING ‘EVIDENCE’
3. Hierarchy of Evidence
The evidence that informs our practice can come from a variety of sources. It can include academic research, statistics and data, expert experience, organisational experience, consumer feedback, client outcomes, consumer demographics, government papers, public surveys, and even magazines and news media (NSW Government, 2010). These sources, however, are not always considered to carry the same weight. Their significance is determined through a framework called the “hierarchy of evidence” (Evans, 2002).
The hierarchy of evidence is most helpful in interpreting and understanding the value of research evidence. In academic research, different types of research are considered more valid or reliable, and considered to be stronger forms of evidence. In general, the more clearly research can prove that a particular approach or intervention is responsible for any changes seen, the more weight it carries as evidence (Evans, 2002). The hierarchy of evidence framework is shown below.
*Adapted from NSW Government (2010) What is Evidence? Retrieved from: https://www.facs.nsw.gov.au/__data/assets/pdf_file/0007/789163/What-is-an-Evidence-Hierarchy.pdf
When considering the layers of evidence, it is important to understand that the hierarchy of evidence relates to research evidence in a broad range of fields. In social research, proving that an approach or intervention is responsible for the change caused can be difficult, as removing or controlling a range of social influences is not always possible. For this reason, the highest level of evidence is not always as accessible in social research. This does not mean we should not look for all types of evidence, but anticipate that we might have more access to other evidence.
We also need to look at the context of research findings to help determine their applicability to the people with whom we work. An intervention proven to work with a particular population group might not be a good fit for other population groups, or need some adaptation in consultation with the people or setting we are using that evidence in. For example, research with families who are predominantly Euro-Australian, will likely reflect particular cultural values and ideas about families. The findings of that research may not be as relevant to families from other cultures.
In social services we also put higher weight behind lived experience knowledge, and research where people share their experiences, than the hierarchy suggests. This reflects the subjective nature of social service work and a range of professional values that highlight people’s ability to know their lives best and what will and will not suit their circumstances.
How is the Hierarchy of Evidence relevant to you?
Understanding the Hierarchy of Evidence can help you interpret what you are reading, as you search for and identify research evidence informing the work you do. It can also help you to understand why certain types of evidence might be more prominent and the value of this research in informing social service provision.
Highest Level of Evidence
Evidence from the top of the hierarchy is considered the most rigorous and reliable. This kind of research evidence can show whether a certain intervention works or not (NSW Government, 2010). Systematic reviews are considered the most thorough form of research. These are research papers that look at the results of many studies on a topic together and summarise all the information to reach conclusions. Meta-analyses are research papers that examine results from many studies on an intervention, rather than just one experiment, to draw conclusions about how effective a particular intervention is.
A randomised control trial (RCT) is a research experiment where interventions are tested on a “test” group, and the researchers also monitor a “control” group, who have not had the intervention. If the intervention is effective on the experimental group but not the control group, it is likely that the intervention is effective. They are seen as rigorous because they allow a clear comparison between the group that was tested and the control group (Evans, 2002).
It is important to note that there are many ethical considerations when conducting RCTs to make sure that participants can take part voluntarily and safely. For example, RCTs generally go through a rigorous, formal ethics approval process, by a Human Research Ethics Committee (HREC) before they can even begin. RCTs are conducted by experienced researchers, and while they are a useful resource for services, there are generally not carried out by service providers.
A note about ethics and research conducted by services: Conducting research with the people who use your service requires in-depth consideration of a range of ethical issues including: use of power, choice to participate, confidentiality of data, clarity around research as opposed to practice as usual, and managing subjectivities and conflicts of interest during research design, data analysis and reporting – to name a few. This is not to say that services cannot conduct meaningful and important research, but that this requires time to plan, seek ethical and research advice, and to implement.
Second Level of Evidence
The second level of evidence also generally includes research studies, however, results from studies of this type are not considered as strong an indication that a particular intervention works. They describe the use of an intervention, but often without a control group for comparison. Examples include a cohort study, which looks at the experiences of a group of people over time to draw conclusions, or a case study of an intervention. These studies still provide useful information about how effective an intervention might be, but they are not as reliable for definitively saying whether an intervention worked or caused change (Drisko & Grady, 2019). In these studies, it can be hard to definitely determine the point of change.
These studies are some of the most common types of evidence in human services research. They are relatively practical to carry out with human participants, as they often involve observing change over time or describing a case study of something that has already happened, rather than creating tightly controlled experimental conditions. They also do not carry the same ethical considerations as RCTs (although still require ethical approval), as they tend to be lower risk to participants, and do not withhold or delay providing an intervention to one group of people.
Third Level of Evidence
This category contains a wide variety of evidence types, which are generally not those from the highest-standard academic research conditions. They do, however, contain valuable information that can be used to guide practice and decision-making, as well as indicating what might work in different circumstances. These types of evidence often inform developments in other types of evidence. They include case studies of what was effective with particular client groups, surveys/studies with groups before/after receiving an intervention, and the opinions of experts in the field who have extensive practice knowledge (Drisko & Grady, 2019).
To inform and justify the ways we approach our work and show the value of the work we do we can draw on all levels of evidence.
Take a moment to explore what kinds of evidence supports the work you are doing:
Pick one element of your service that you are intentional about. For example: taking time to build rapport with young people, providing families with a consistent case worker, or matching the focus of your work to the developmental stage of children and young people.
Open Google Scholar (which provides search hits more specific to academic research – books, journal articles, thesis etc, than Google) and search for research related to that area of practice.
You can play around with your search until you see headings that look relevant. For example, searching for ‘time to build rapport with young people’ delivers the following sources:
Spend a moment opening up any titles that look relevant to your work and pick the three abstracts/summaries that look most useful. For each of the three sources, note:
- What kind of research it draws on – e.g. RCTs, surveys, interviews?
- What level of the hierarchy of evidence that research belongs to?
- The context for the research – national, international, specific population groups?
- How you could relate those findings to the work that you do.
