“What? So what? Now what?”
Reflections from ADAPT’s Learning Community on Evaluation
ADAPT, the training and technical assistance division for substance use prevention programming in High Intensity Drug Trafficking Areas (HIDTAs), offers a variety of services to meet the training and technical assistance needs of HIDTA communities. Learning communities are one of the more in depth services provided to support competence and skill development by providing a “space and a structure for people to align around a shared goal.” Effective learning communities “enable knowledge sharing and connect people that are eager to learn and work across boundaries, while adhering to a common agenda, metrics, and outcomes”.[1]
When ADAPT successfully wrapped its first Evaluation Learning Community recently, the gains reported by participants far exceeded their investment. Like all of the products and services offered by ADAPT, the Evaluation Learning Community was aligned with an identified need within HIDTA communities across the United States. Over the past two years, ADAPT has received a number of technical assistance (TA) requests related to evaluating substance use prevention strategies. The purpose of the Evaluation Learning Community was to strengthen program evaluation skills through training from subject matter experts and case-based learning. Participants in the Evaluation Learning Community brought their own case examples into interactive discussions with peers and content experts, who answered important questions and provided supportive feedback.
“I took statistics in undergrad and again in grad school but that was a long time ago! This was a great review and extremely helpful. I have a better understanding that I need to do this and am able to discuss this with partners”. – Public Health Analyst participant
ADAPT invited HIDTA prevention and Overdose Response Strategy (ORS) representatives from six HIDTAs to participate in the pilot Evaluation Learning Community. A total of 17 individuals joined seven virtual sessions to enhance and apply their understanding of evaluation fundamentals.
Facilitated by Dr. Robert LaChausse, Professor at the California Baptist University where he teaches a variety of courses related to prevention science and evaluation (Health Behavior, Disease Prevention, Statistics, Research Methods, and Program Evaluation), the Evaluation Learning Community guided participants through a systematic process of thinking through the core components of evaluation and using them to determine if a program is effective. ADAPT and Dr. LaChausse planned the series with didactic learning coupled with hands-on application components, where participants individually or in pairs prepared pieces of an evaluation plan and discussed each step in the design.
Results from the post session surveys administered to participants demonstrated that, on average:
- 95% found the content relatable,
- 92% found the content digestible,
- 68% reported at least 50% or more of the content taught was new knowledge, and
- 100% of participants expressed their intention to share the new knowledge with others.
Comments also suggested there is a clear demand for more services, including learning communities, on this topic.
“The visuals made it easier to understand the terminology and concepts being explained. I also found the open questions and conversations very useful. I’m a little older and new to this work but I found the information very useful and not as intimidating as I thought it would be. These are extremely informative so please continue”.
-Drug Intelligence Officer participant
ADAPT’s planning of each piece of the Evaluation Learning Community was purposeful and intentional, including the use of visuals and relatable examples, in order to help participants use their evaluation skills in relevant ways. For example, participants learned about the Behavior-Determinant-Intervention (BDI) logic model which is created by working backwards in a stepwise fashion to demonstrate the evidence-based relationships among interventions, determinants of behaviors, the behaviors themselves, and the ultimate health goals. Since participants came to the table with varied levels of skills and learning needs, the course was flexible enough to meet all participants where they were at. Each of the seven sessions was followed by an exercise posted on a discussion board, and resource materials were made available to the participants throughout the duration of the series. A summary of important points from the Evaluation Learning Community can be found here.
ADAPT offers a variety of services and products to meet the substance use prevention technical assistance needs of HIDTA communities. Moving forward, ADAPT will continue to support competence and skill development on this topic and has plans to offer another Evaluation Learning Community in the spring for HIDTAs and their communities. To be notified of upcoming webinars, products, and events, subscribe here!
[1] (Key Concepts: Learning Communities. Harvard University, 2022)