Stage 3: Implementation

Effectively implementing your EBPs involves anticipating or identifying problems and challenges and responding to them before they affect your desired outcomes.

Below are some concrete action steps compiled from best practices and from interviews with current and former SS/HS project staff that may be helpful as you implement your EBP.

Action steps for successful program implementation

1. Monitor Implementation

Use the monitoring mechanism you created in the preparation stage to conduct a process evaluation. This will help you readily identify and address needs and challenges as they occur. See the EBP Implementation Checklist.

Make sure that implementers stick to the core elements of the EBP to ensure that your programs are implemented with fidelity.
If available, use the developer’s EBP fidelity checklists, survey instruments, and teacher implementation logs.
Peer coaching may be necessary in order to ensure users are implementing EBPs with fidelity (see Basics of Coaching brief).

[A] part of the evaluation was to have them do a check-off regularly of what curriculum was taught . . . we built in accountability . . . check and cross-check."

—Boston, Massachusetts

Compare the process data to the targets set in your logic model.
Depending on the results of your process evaluation, you may need to provide additional preparation or professional development and other staff support, or consider selecting a different EBP.

At one site, the administrators found out from the data they were collecting and billing from events that one teacher was not implementing with fidelity because [the teacher wasn’t] adequately trained."

—Black Oak Mine, California

For more information on evaluation needs and available strategies at this stage, see Working with Your Local Evaluator and Evaluation.

The Eudora (Kansas) Public Schools SS/HS Initiative has established first-tier universal systems by training and staffing coaches and PBIS teams. A district leadership team meets quarterly to monitor implementation and address district‐wide needs. To ensure that PBIS is implemented with fidelity and that decisions are driven by the data, additional training is required. To meet these needs, a small group of staff have been trained and certified in the School-Wide Information System—a behavior collection system used in conjunction with PBIS—as well as the School Evaluation Tool, which evaluates the integrity of universal-level implementation of PBIS, and the Teaching Pyramid Observation Tool, which checks the fidelity of practices.

Monitoring implementation fidelity and effectiveness helps the district identify ongoing needs for staff development, for example:

Staff development for principals to help them understand the need to link students to mental health assessment, services, and/or skill-building when behavioral issues are severe or chronic.
Training to help staff use additional data sources to target individual school issues and update PBIS matrices. For example, bullying is an issue that often surfaces in the school surveys but not as much in the behavior incident report data. Since bullying is often hidden from adults, factoring in a variety of data sources is necessary.

2. Provide Ongoing Professional Development

Professional development is not a one-time event. To be effective, implementers need ongoing professional development and support, such as:

Coaching and mentoring by skilled implementers
EBP booster sessions
Demonstration lessons by skilled implementers
Regular opportunities to share challenges and successes

Building ample time for professional development into the schedule will demonstrate the importance of the program to staff and that the administration is behind it.

We hold a monthly core team meeting at each school site. And then we hold a quarterly core team meeting with all the districts so people can share strategies, successes, ideas, those kinds of things, to keep people motivated and interested. It’s easy to start [an EBP] and then kind of lose sight of it. We want people to stay on top of it to continue to recognize good behavior, to promote it, to model it, and to make sure that everybody is following the rules consistently."

—Vail, Arizona

Our trainer for Second Step is someone who has a great deal of experience with Second Step and has made herself available above and beyond the day of the training. She’s gone back to the schools additional times with staff to talk with them about problems they might be having."

—Rome, New York

To train new staff, build EBP training into orientation for new staff so that all implementers are properly prepared.

The biggest challenge will be—because we’re a growing district and new staff come onboard—to maintain training of that [new] staff and make sure they understand how PBIS works. We do have a brief training on it that’s been built into our new teacher induction, so that will be helpful."

—Vail, Arizona

To read about challenges to overcome and questions to consider when planning for professional development, see Providing Professional Development.

Strategies

Providing Professional Development

Key Resources

Basics of Coaching
The Impact of Fidelity of Implementation in Effective Standards-Based Instruction

3. Utilize Key Supports

The developer will often provide technical assistance and guidelines on how to implement the EBP with fidelity while adapting it to meet the particular cultural and linguistic needs of your community. For more information, see Working with the EBP Developer.

If one of the implementers had an issue or problem or didn’t understand something, they picked up the phone and called the Mendez Foundation [developer of Too Good for Drugs and Too Good for Violence], and [the Foundation] would tell them what to do, or how to implement this, or how they could change or modify this—or if they needed to change the program and stretch it out over two weeks as opposed to one week, [the Foundation] would give them some corrective ways to make those changes."

—Wheeler, Georgia

The grant itself provides many built-in supports for implementation, through the Technical Assistance Specialists, the Federal Project Officer, the National Evaluation Team, and the Communications and Social Marketing Team. For more information on SS/HS support around implementation, see Utilizing Grant-Provided Support.

[Our TAS] was previously a project director, so she’s been really, really helpful from a project director perspective, but also for technical assistance with everything. [She’s] guiding us through what her experiences have been, and what might work, and what might not."

— Anaheim, California

[Our TAS] knows us very well. I think it’s important for [the TAS] to really know your program, know how they function . . . and [be able to] support you with fidelity and information. She’s helped us with mapping out our needs and come on-site for visits, which I strongly encourage people to do."

—Allamakee County School District, Waukon, Iowa

Strategies

Working with the EBP Developer
Utilizing Grant-Provided Support

4. Collect Data

SS/HS grantees cite good outcomes as key to spreading excitement about a program and increasing the willingness of teachers to implement programs. Throughout this stage, the evaluator, partners, and program staff must provide the project director with process and outcome data about the following:

If the EBP is being implemented with fidelity
If the EBP is an appropriate fit for our community
If there is adequate buy-in from staff and partners
If any adaptations compromise the EBP’s core elements
If the challenges and needs identified by implementers are being successfully addressed by ongoing professional development and support
If the site is effectively using grant-provided supports, such as the TASs and FPOs
If the program outcome data are meeting the objectives identified in the logic model
If processes are being put in place to sustain the program, such as dealing with staff turnover, building staff training capacity, and ensuring that policymakers are willing to integrate effective programs into the system

Provide regular communication about positive outcomes to implementers, school staff, and community members. Where possible, link EBPs to academic outcomes and to state standards.

If desired outcomes are not being achieved, make adjustments. To understand how data can help make program improvements during this stage, see Evaluation.

The Tigard-Tualatin (Oregon) School District Check In/Check Out program has been implemented as a second-tier intervention for the improvement of school attendance and school engagement for selected middle school students with attendance or early behavior problems. Key features of this program are frequent feedback to students about their behavior, in order to prevent future problem behavior; positive reinforcement to students for meeting daily goals; data tracking of positive behavior in addition to disciplinary issues; and the acknowledgement of expected behavior through a reward system. There is solid evidence that this approach is working. For example, in 2010–2011, there was a 34 percent decrease in office disciplinary referrals and a 54 percent decrease in suspensions among students participating in Check In/Check Out.

In Newport-Mesa (California) Unified School District, a large and diverse district in Orange County, tracking their data and communicating the success of their programs for parents of older children has allowed the SS/HS Initiative to expand their parenting programs to parents of elementary school and pre-school children, addressing behavioral issues before they become more critical and interfere with children’s academic success. In addition, the district implemented a software system that tracks students’ progress in real time; it monitors student behavior, attendance, and academic outcomes to quickly identify students and families who need support. As a result, instead of intervening to address problems that exist, the district focuses on promoting positive behaviors and preventing risk behaviors in a culturally competent manner.

5. Build Sustainability

You can promote sustainability by creating a supportive infrastructure for your highest-priority EBPs:

Build the capacity of key implementers by having them attend training-of-trainer sessions provided by the developer.
Continue to involve school leadership to build buy-in for your program.
Show how the EBPs complement and coordinate with existing programs.
Use your Communications Specialist and the resources of the grant-provided Communications and Social Marketing Team to communicate your program successes and to build broad support in your community. (For further information, see Utilizing Grant-Provided Support.)
Work with school leadership and partners to create and institutionalize policies, such as anti-bully and early screening for at-risk youth, that support the work of your most effective EBPs.
Find ways to leverage your partnerships and to link the functions and outcomes of effective EBPs to school-related goals in order to obtain sustainable funding beyond the grant.

For strategies for building EBP sustainability, see Leaving a Legacy: Six Strategies for Sustainability. The six strategies discussed in this tool include financing, evaluation, partnerships and collaboration, implementation, communications/marketing, and leadership for change.

For the Chicago Public School, District #299, implementing programming to support students at all levels of the pyramid required a number of key infrastructural supports:

Comprehensive professional development, including training and coaching to support the staff implementing the programs
A screening and referral process that teachers and administrators understood and could readily use to identify student needs and refer them to the appropriate services
Coordination among departments and school programs and services
A data system to track behaviors and services (e.g., the School-Wide Information System), monitor the referral process and program implementation, and collect outcome data (e.g., discipline referrals, school attendance)

With the development of a supportive infrastructure, the ability to deliver EBPs increased significantly. Positive outcomes included an increase in average daily attendance, an increase in the number of students who received appropriate referrals for services, and a majority (84 percent) of SS/HS students in grades 6–12 stating that they perceived positive adult support in school.

We had a strong communication relationship with the local media. They did a radio station program two days a week where they’d talk about the programs, how the lessons were going, how the kids were adapting to it, etc."

—Wheeler, Georgia

We built a community coalition out of this initiative. This should be useful for sustaining in the future, since having a community nonprofit will allow them to go for a lot of funding that isn’t offered to schools."

—Black Oak Mine, California

Strategies

Utilizing Grant-Provided Support

The Adams County Youth Initiative (ACYI) (CO), has taken numerous steps to sustain their highly successful Incredible Years program and to make it an integrated part of the community. Prior to implementation, a diverse group of stakeholders from across Adams County met to ensure that there was a need for the program and to develop a work plan, which included an evaluation plan to measure program and implementation outcomes. Since program implementation in 2007, program partners have worked to ensure the program continues beyond SS/HS funding. These partners have met 6 times per year for several years to coordinate implementation, and have developed a sustainability timeline that is being implemented in all five districts. Recommendations involve embedding expertise and peer coaching capacity at the local level, so that Adams County can sustain program implementation, while reducing reliance on the trainer, Invest in Kids.

The ACYI build the infrastructure necessary to support a long-term collective impact. The initiative is aimed at transforming the lives of Adams County children and young people by coordinating, leveraging, and integrating significant resources to support children through their developmental years. As ACYI/SSHS prepared for no-cost extension and sustainability, they established as a 501C3 with non-profit status, and continue to support early childhood programming including Incredible Years.