PART ONE: UNDERSTANDING AND USING ‘EVIDENCE’
4. Using Evidence in Service Design, Planning and Practice
While the ‘Hierarchy’ sets out what evidence types are considered most valid, there are several ways these different information types are used in human services. Even the “gold standard” research evidence is not enough to be the only information used in decision making in diverse practice contexts with human beings (Plath, 2013). For example, RCTs for social interventions are complex to undertake with human participants because humans have complex lives with many variables.
It can be difficult to determine which aspect of an intervention makes a difference for any given person. Organisations need to be creative and flexible in the way they seek out and use evidence (Plath, 2013). All three types of evidence (research, contextual and experiential) have validity and use in human services, and can help describe how a service is being informed by evidence.
Research Evidence
The ‘Hierarchy’ provides a framework for understanding how research evidence can be used by your service. When applying research evidence, it is also important to think about how and in what ways the research findings are applicable to your setting. For example:
- Where was the research conducted?
- Who are the people who participated in the study, e.g. gender, age, cultural background?
- How similar are the research context and participants, to your organisational context and the people with whom you work?
- How many people participated in the study?
- Do the authors claim the results are relevant to other settings?
- Are there detailed insights from people accessing services or experiencing similar challenges to the people with whom you work?
- Are there any limitations of the research disclosed within the article?
- Is the research in a peer-reviewed journal? (This indicates a rigorous review process prior to publication that can help you feel confident in the research processes and quality)
Research findings do not have to be a perfect fit to your context to make them relevant. The closer the fit, the more confidently we can say that the approach we are using or intend to use has been found to be effective with similar population groups. When there are some differences between our population group and that of the research, we might note the context/circumstances where this intervention has been proven effective, highlight population overlaps that might indicate potential in your setting, and then note the ways you intend to consult about and evaluate its implementation in your setting.
You might already have other types of evidence that indicate it is effective in your context – for example, worker insights, feedback from those accessing your service, consistent referrals to your service, and/or service access metrics about whether people return and how many sessions they attend for. This kind of evidence is detailed further below.
A note about working with available research: In some instances, services may be drawing on established programs which can clearly present the evidence informing them (for example, Triple-P Parenting, which is freely available through the Commonwealth Government). However, many services are integrating a range of evidence to reflect the diversity of the people with whom they work, the variety of services they offer, or emerging research about creative practices. There are good reasons for thoughtfully integrating a range of approaches. This can require a little more thinking time at the point of service development and planning, to consider all the parts of the service you offer or plan to offer, and the various elements you can evidence.
Spend a moment to document all the parts of your practice:
Thinking about stages of practice can be helpful with this. What are you intentional about at each of the points of practice outlined below? What kinds of evidence does or could inform this – how do you know that work is effective?
Service access: How people learn about and come into contact with your service, and how your service meets community needs (this might include local population data, percentages of various wellbeing and welfare issues, and evidence of effective approaches for this population)
Engagement: Welcoming people to the service and building rapport, and the ways in which this is responsive to the diversity of people using the service.
Assessment: Working out what is happening in people’s lives.
Activities / Interventions: The work you do with people.
Modes of delivery: Such as home-based, drop-in, outreach, groups, and consistent workers.
Capacity and skill-building: Helping people to identify and develop skills and social networks that will support them in the longer term.
Underlying frameworks and practice principles: For example, being strengths-based, trauma informed, rights based, culturally responsive, anti-racist, or child and family centred (this language is used a lot, so make sure to include what these approaches entail, the evidence that supports their value, and the ways your practice reflects them).
Exit / Discharge: When and how people conclude their involvement with your service, and any referral or transition processes.
Evaluation: How you evaluate the effectiveness of the work and people’s experiences of your service.
Workers: Any training, professional development, support and evaluation of staff.
When we consider the range of elements of practice listed above, we can appreciate how difficult it can be to find specific research findings that talk to all elements of our service, approach, and desired outcomes. Most often, we need to look for evidence that relates to each of these elements, and then articulate how they are thoughtfully combined to meet the needs of people accessing your service.
Once we have drawn these areas of evidence together it can be helpful to have someone external to our program, review what we have written. When we are familiar with our service offering, we sometimes miss opportunities to make clear what we are doing and why, as we can sub-consciously fill in the gaps, that others may not be able to do (for example, those assessing our grant applications!).
When we review our work, it is important that we do not only look for evidence that supports what we are currently doing but consider more recent or emerging evidence that could inform us in additional ways. For example, if your service intentionally engages caregivers at the beginning of work with children and young people, we might look for broader research about ways to engage families when working with children and young people. This can help us evidence, review and further develop our practice.
Contextual Evidence
Organisations hold knowledge about the reality of implementing particular practices in specific contexts; information that is not easily captured in research studies (NSW Government, 2022). Contextual evidence might include:
- Demographic data on the people using the service, e.g. knowing the age, disability status, gender, cultural background of who is accessing services can help to identify what programs are needed.
- Data on the local population.
- Client feedback on how useful the service was.
- Contextual knowledge on the needs of the local community, e.g. high rates of homelessness, low local access to education or employment
- Knowledge of the social factors likely to impact how a service is delivered, e.g. knowledge of factors like public transport that impact how people access services
- Knowledge of what else is provided in the local context, and therefore what service gaps exist in the community
- Knowledge of what the service has previously tried and what has or has not been useful.
- Service evaluations that identify strengths and needs of the service.
Experiential Evidence
Many organisations use experiential evidence across their service delivery (Drisko & Grady, 2019). This might include:
- Case studies that capture practitioner knowledge of what works with different client groups
- Anecdotal evidence across the service about trends in who is accessing services, or what is working with clients
- Describing the theories or ethical principles that inform the work of practitioners in human services, as these theories are usually supported by research evidence.
Hypothetical Case Study
Below we provide a hypothetical case study on how a range of evidence can be used and combined to inform a service and highlight its value:
An organisation aims to provide tailored, early support to families experiencing difficulties, to prevent challenges from escalating and to minimise the need for more invasive intervention in the future. The program is responding to the need to reduce adverse outcomes for children and families, such as statutory involvement, risk of youth homelessness, and youth mental ill-health.
Through reviewing and considering the contextual, experiential and research evidence to inform their service design and practice, the organisation opts to develop a place-based program that includes prosocial and skill-building activities for children and young people; case management, coordination and brokerage for families; and a group parenting program.
The desired program outcomes include: (a) improved child and adolescent sense of belonging, self-worth and social cohesion; (b) improved system navigation and access to ongoing supports for families; and (c) improved parenting strategies and capability, to build family functioning.
Below, we outline potential evidence in each of the evidence categories that can inform this service model and be drawn together to highlight the value of the service in meeting current local needs in an informed way.
Examples of research evidence that may inform the service model:
- Reviewing existing parenting group programs, to determine which have a stronger evidence-base (and would also be appropriate to implement within the local context). For example:
- Parenting Research Centre – Review of parenting interventions in Australia
- Preventing child maltreatment – evidence informed programs
- Journal articles which report on the evidence for parenting programs and adaptions for specific contexts such as Promoting relationship building and connection: Adapting an evidence-based parenting program for families involved in the child welfare system
- Reviewing theories, principles and approaches for working with children and young people, to support evidence-informed practice.
For example:
Examples of contextual evidence that may inform the service model:
- Demographic data about families in the local region.
For example: AEDC data demonstrating high levels of vulnerability; census-data regarding household income and composition - Demonstrated increases in service demand over previous years from families seeking support.
For example: More families seeking support related to cost of living and barriers to supports (e.g. NDIS); contributing to increased family stressors. Limited other place-based support services for families either within the local region, or easily accessible elsewhere.
Drawing on other local surveys, consultations and resources that provide context to the identified need or gap, or which could inform how the service will be designed.
For example: The findings of the Missing Middle consultation report, which describe the informal and formal barriers for children, young people and families to access services in the ACT.
Examples of experiential evidence that may inform the service model:
- Learning from lived experience. For example:
- Anecdotal reports from families that it is difficult to access supports outside the immediate region when juggling multiple family priorities, and hearing from families about what they need/want in local services.
- Drawing upon the findings of the Our Say Youth Lived Experience FAQ’s Resource, to better understand young people’s perspectives on service delivery
- Worker knowledge and experience regarding principles and approaches to working with children and young people (also building on research evidence).
For example: ACT Youth Work Code of Ethical Practice
- Service feedback from clients regarding what has worked well or not worked well in their experience of the service.
For example: Previous clients reporting that they need for childcare for children, to enable them to access group programs.
As the program commences, mechanisms and processes are also embedded to ensure that the program can monitor and review its practice and delivery. This ensures that it continues to learn from contextual and experiential evidence, in order to effectively respond to evolving community need. For example, client feedback and satisfaction processes, and outcome measures.
