Program Evaluation (2010-2012)


Evaluation of Horizon's Integrated AODA Treatment/HIV
Services for Justice-Involved Women

Nena Messina, Ph.D., Principal Investigator (nmessina@ucla.edu)
 Kira Jeter, M.P.H., Project Director

The purpose of this SAMHSA-funded project was to expand Community Advocates - Horizons Campus’ program to increase access to integrated outpatient AODA treatment and HIV services for a minimum of 600 justice-involved women, who are less likely to enter treatment through formal systems. Objectives were to: (1) increase the number of women who participate in integrated outpatient AODA/HIV services from 25 to 125 women per year; (2) enhance Community Advocates-Horizons Campus’ program with research-based AODA/HIV curricula that engaged hard-to-reach women, their children, and their partners; and (3) increase the number of women and partners who receive rapid HIV testing from 90 to 200 per year. ISAP conducted the evaluation, which (1) included process and outcome components to monitor the implementation of the planned intervention and (2) measure observed changes in participants from before to after their participation in the project in the following domains of functioning: (a) alcohol and drug use; (b) parenting behaviors and family functioning; (c) quality of life and social functioning; (d) criminal behavior and legal status; (e) mental health and physical health status; (f) HIV/AIDS risk reduction behaviors; and (g) treatment and services received. The program worked with parole and probation departments in the Milwaukee area to recruit participants.  Horizons’ has been successful in completing follow-up interviews. Once engaged, participants continued to stay well connected to the staff and/or services offered by the program.

Evaluation of Community Advocates - Horizons Campus Integrated AODA Treatment/HIV Services for Justice-Involved Women was funded by the Substance Abuse and Mental Health Services Administration, grant TI18592 to Community Advocates-Horizons Campus, from October 2007 to September 2012.

Top of Page


Evaluation of People in Progress’s
New Elements Treatment for Homeless Program

Richard A. Rawson, Ph.D., Principal Investigator (rrawson@mednet.ucla.edu)
Anne Lee, M.S.W., Project Director

ISAP conducted the process and outcome evaluations of the New Elements Treatment program, which is an intervention project provided by People in Progress. The project was funded by a 5-year grant from the Substance Abuse and Mental Health Services Administration, Center for Substance Abuse Treatment. New Elements provides comprehensive substance abuse treatment and case management services to homeless individuals. The evaluation study tracked the characteristics of project participants and examined the delivery of project-specific services, adherence to the treatment models, barriers encountered in project implementation, and solutions devised to address those barriers. Focus groups with project participants and interviews with project staff were conducted in order to monitor their respective perceptions and experiences of the project; feedback was provided from the evaluation in order to refine the project implementation. Findings from client focus groups conducted during the project consistently revealed that NECT participants felt positive about their experience in NECT in regard to the services received and their interaction with the director, case managers, and staff.  At follow-up, a greater proportion of NECT clients reported higher rates of abstinence from alcohol or illegal drugs, fewer arrests, higher rates of employment/education, increased social connectedness, and stable housing.  Additionally, a greater proportion of NECT clients had stable housing compared to the proportion at intake, and they exhibited less psychological distress over time.

Evaluation of People in Progress New Elements Treatment for Homeless Program was funded by People in Progress, Contract 20071053, from November 2006 to October 2011.

Top of Page


Evaluation of the Bridges to Housing Project

Christine E. Grella, Ph.D., Principal Investigator (cgrella@mednet.ucla.edu)
Kira Jeter, M.A., Project Director

The Bridges to Housing project aims to assist individuals with a history of chronic homelessness and co-occurring mental disorders with transitioning to stable housing and securing benefits that will promote increased stability of functioning.  The project, led by the CLARE Foundation in Santa Monica, is based on a collaborative community partnership model and the Housing First approach to eradicating homelessness among individuals who are most in need.  In addition to comprehensive case management services, the intervention will incorporate several evidence-based practices, including integrated care for co-occurring disorders, motivational interviewing, cognitive behavioral therapy, and assertive case management.  The project will recruit 30 participants each year who will receive collaborative case management through the community partnership.  ISAP is conducting the process and outcome evaluation of Bridges to Housing and will focus on (1) successful implementation of  the community collaboration model and evidence-based practices, recruitment of the target population, and staff attitudes and beliefs, and (2) outcomes of the 6-month follow-up regarding participants’ substance use, psychosocial functioning, housing stability, and benefits status.

Evaluation of the Bridges to Housing Project was funded by the CLARE Foundation, contract 20114621, from September 2011 to September 2014.

Top of Page


Evaluation of the Community Bridges Project

Christine E. Grella, Ph.D., Principal Investigator (grella@ucla.edu)
Kira Jeter, M.A., Project Director

The purpose of the Community Bridges Project was to strengthen and enhance the comprehensive treatment system in Santa Monica, California, for chronically homeless persons who have co-occurring mental illness and substance use disorders. Led by the CLARE Foundation, the project was based on a model of services collaboration among community-based providers of substance abuse treatment, mental health treatment, health services, shelter and drop-in services, outreach and engagement, and transitional and permanent housing. The integrated treatment model incorporated several evidence-based practices, including motivational interviewing and relapse prevention, with the aim of (1) increasing access to treatment, (2) increasing continuity of care and services integration, (3) improving quality of life and increasing self-determination, and (4) improving levels of functioning among participants.  In addition, the project aimed to increase integration of services across the participating providers, to eliminate barriers to system entry, to improve awareness of service needs for this population within the broader community, and to improve staff competencies for delivering services to this population.  A total of 247 participants were recruited over 5 years.  A majority of participants were cross-referred across multiple partner agencies for services, demonstrating that the project achieved its goal of implementing a collaborative community partnership model. Three outcome clusters that varied in degree of stability of functioning at follow-up were determined: (1) not stable (33.3%): clients who reported any alcohol or other drug (AOD) use in the past 30 days regardless of criminal justice involvement, housing status, or mental health status; (2) moderately stable (49.2%): clients who reported no AOD use, but who were mentally unstable and not housed, and/or had criminal justice involvement; and (3) highly stable (17.5%): clients who reported no AOD use and were either housed or mentally stable, regardless of criminal justice status.

Evaluation of Community Bridges Project was funded by the CLARE Foundation (Grant 20063816), through a grant from the Substance Abuse and Mental Health Services Administration (SAMHSA) from September 2006 to September 2011).

Top of Page


Evaluation of the Homeless Co-Occurring Treatment Program

Christine Grella, Ph.D., Principal Investigator (cgrella@mednet.ucla.edu)
Luz Rodriguez, Project Director

The Homeless Co-occurring Treatment Program (HCOTP) is a 4–6 month intensive outpatient treatment program that integrates substance abuse and mental health services for individuals who are in transition to permanent housing.  An Assertive Community Treatment (ACT) model is used to provide integrated and comprehensive services within the context of intensive case management, referral to wrap-around and recovery support services, and motivational interviewing to increase treatment engagement.  Targeted outcomes include reductions in substance use and mental health symptoms, increases in social support, and sustained residential stability.  The project will serve 38 individuals per year for 5 years, for a total of 190 individuals.  ISAP is conducting the process and outcome evaluation of this project, which includes client-level assessments at baseline, discharge, and 6-month follow-up; tracking of services provided to participants and their satisfaction with services; focus groups with staff and participants; and ongoing observations of the implementation process. 

Evaluation of the Homeless Co-Occurring Treatment Program was funded by Special Service for Groups, contract 20104695, from September 2010 to September 2015.

Top of Page


Evaluation of the Liberating our Families
from Drugs and Incarceration Program

Christine E. Grella, Principal Investigator (grella@ucla.edu)
Luz Rodriguez, Project Director

ISAP conducted the evaluation of the SAMHSA-funded project “Liberating our Families from Drugs and Incarceration (LOFFDI),” which was part of the Pregnant and Parenting Women’s Treatment Initiative at the Center for Substance Abuse Treatment.  The LOFFDI program was provided at the Walden House program in El Monte, CA, and the target population was women parolees who had a history of substance use problems and who were pregnant or had young children.  The program’s objectives were to improve the mental and physical health status, employment status, and parenting skills of participants so that they could successfully reintegrate into the community following their release from prison.  Intensive case management services were provided within the context of residential substance abuse treatment, which included children’s services, family counseling, parenting skills training, vocational services, mental health services, trauma-related services, and referral for medical services for the women and their children. The evaluation study showed that approximately 60% of participants (N = 110) completed the LOFFDI treatment, compared with an average of 39% for participants in other aftercare programs for female offenders in California.  Moreover, LOFFDI participants demonstrated significant improvements in their alcohol and drug use, psychological status, family functioning, employment, and quality of life at the 6-month post-discharge follow-up.  Participants had a high level of satisfaction with the services received, and staff improved their competency in delivering parenting and other services over the course of project implementation. Overall, the evaluation demonstrated that LOFFDI participants improved in targeted domains that address the multiple and complex problems faced by female offenders and their families as they re-integrate into the community following parole.

Evaluation of the Liberating our Families from Drugs and Incarceration Program was funded by Walden House, Inc., contract 20064370, from October 2006 to September 2010.

Top of Page


Evaluation of Trauma Informed Substance Abuse Treatment (TI-SAT)

Christine E. Grella, Ph.D., Principal Investigator (cgrella@mednet.ucla.edu)
Nena Messina, Ph.D., Co-Investigator
Luz Rodriguez, B.A., Project Director

The Trauma-Informed Substance Abuse Treatment (TI-SAT) program was established at the Leo Chesney Community Correctional Facility in 2008 to provide gender-responsive and trauma-informed substance abuse treatment for women offenders.  The treatment approach recognizes that childhood and adult trauma exposure is pervasive among women offenders and intricately related with their substance abuse problems.  ISAP conducted an evaluation of the TI-SAT to (1) determine whether the trauma-informed treatment components were implemented as planned; (2) determine whether obstacles were encountered in the implementation process, and if so, to determine the strategies used to address these obstacles; and (3) determine the characteristics of participants, their treatment retention and aftercare participation, and their outcomes on parole.  A total of 106 participants were recruited to participate in the outcome study; interviews were conducted at study intake and at 6-month follow-up.  Outcome analyses examined the effects of program participation on status and functioning of participants, including their relapse to substance use, criminal behavior involvement, employment, parenting and family relationships, and overall mental health and psychosocial functioning.  In addition, surveys with program staff and observational site visits were conducted to determine whether the gender-responsive and trauma-informed program components were successfully implemented. Findings demonstrated that the program components were successfully implemented and that participants in the TI-SAT had higher levels of treatment retention and aftercare participation than women in other prison-based treatment programs.

Evaluation of Trauma Informed Substance Abuse Treatment (TI-SAT) was funded by the State of California Department of Corrections and Rehabilitation, contract C08.106, from October 2008 to March 2011.

Top of Page


Evaluation Services to Enhance the
Data Management System in California

Richard A. Rawson, Ph.D., Principal Investigator (rrawson@mednet.ucla.edu)
Valerie P. Antonini, MPH, Project Director (vpearce@mednet.ucla.edu)

A top priority of the California Department of Alcohol and Drug Programs (ADP) is to improve the accountability of the substance use disorder (SUD) treatment system in California in terms of ensuring quality services and effective client outcomes.  Within this contract, ADP requested a 3-year work plan for 2009–2012 to continue evaluation work and concepts, as defined within the previous Continuum of Services System Reengineering (COSSR) state contract (07-00176), in support of establishing a more effective SUD system in California.

Within Year 1, the work plan was focused around the following objectives:

Objective 1: Examine the California Outcomes Measurement System for SUD Treatment (CalOMS-Tx) data to enhance SUD treatment services and client outcomes in California.

Objective 2: Enhance the CalOMS-Tx system to include performance measurement/management data.

Objective 3: Enhance the CalOMS-Tx system to include performance and outcome measurement in support of the COSSR model.

Objective 4: Increase the capacity of SUD stakeholders (state ADP, county administrators, and providers) to understand how to apply performance and outcome data to improving the quality of treatment services.
Within Years 2 and 3, the work plan was amended to address new priorities established following the passing of H.R. 3590 – the federal Patient Protection and Affordable Care Act, to assist the State of California and its 58 counties in their preparation for health care reform (HCR). It is anticipated that HCR will result in (among other things) modifications in how services will be funded, the types of services delivered, the venues where they are delivered, the individuals who will receive the services, the work force that delivers the services, how services are measured, and how service benefits are evaluated.   The major focus of the revised work plan was to conduct data collection, data analysis, and related activities to determine models of service delivery that may facilitate this new system, and disseminate this knowledge to policy makers and practitioners across California.  The tasks and activities were focused around the following topic areas:

  1. Data Systems Improvements
  2. Substance Abuse, Mental Health, and Health Care Integration
  3. Performance Measurement/Dashboards
  4. Development of Recovery Measurement and Resources
  5. Planning for Prevention
  6. Workforce Development

Reports

  1. 2009-2010 Report: Evaluation Services to Enhance the Data Management System in California; Richard A. Rawson, Ph.D., Darren Urada, Ph.D., Valerie Pearce, M.P.H., Desirée Crèvecoeur-MacPhail, Ph.D., Allison Ober, Ph.D., Rachel Gonzales, Ph.D., Joy Yang, M.P.P., Sarah J. Cousins, B.S., B.A, and Stella M. Lee, B.A.
  2. 2010-2011 Report: Evaluation Services to Enhance the Data Management System in California; Richard A. Rawson, Ph.D., Darren Urada, Ph.D., Valerie Pearce, M.P.H., Elizabeth Nelson, B.A., Desiree Crevecoeur-MacPhail, Ph.D, Howard Padwa, Ph.D., Umme Warda, M.S., Suzanne Spear, M.S., Allison Ober, Ph.D., and Stella M. Lee, B.A.
  3. Substance Use Disorder Treatment Services of the Future: Technical Assistance Plan to Prepare the Workforce for Healthcare Reform, October 2011; Thomas E. Freese, Ph.D., Richard A. Rawson, Ph.D., Darren Urada, Ph.D.
  4. Considerations for Reorganization: California’s Departments of Mental Health and Alcohol and Drug Programs, August 2011; Richard A. Rawson, Ph.D. and Stella M. Lee, B.A.

Publications

Crèvecoeur-MacPhail, D. et al, Inside the Black Box: Measuring Addiction Treatment Services and Their Relation to Outcomes, Journal of Psychoactive Drugs, SARC Supplement 6, September 2010.

Ford, L., Zarate, P., Closing the Gaps: The Impact of Inpatient Detoxification and Continuity of Care on Client Outcomes, Journal of Psychoactive Drugs, SARC Supplement 6, September 2010.

Harris, W., Ryan, J., Indicated Prevention: Bridging the Gap, One Person at a Time, Journal of Psychoactive Drugs, SARC Supplement 6, September 2010.

Herbeck, D. et al., Performance Improvement in Addiction Treatment: Efforts in California, Journal of Psychoactive Drugs, SARC Supplement 6, September 2010.

King, B et al., A Field Experiment in Capitated Payment Systems and Recovery Management: The Women’s Recovery Association Pilot Study, Journal of Psychoactive Drugs, SARC Supplement 6, September 2010.

Rawson et al., Improving the Accountability of California’s Public Substance Abuse Treatment System through the Implementation of Performance Models, Journal of Psychoactive Drugs, SARC Supplement 6, September 2010.

Stanford, M. et al., Chronic Care and Addictions Treatment: A Feasibility Study on the Implementation of Posttreatment Continuing Recovery Monitoring, Journal of Psychoactive Drugs, SARC Supplement 6, September 2010.

Evaluation Services to Enhance the Data Management System in California was funded by the State of California Department of Alcohol and Drug Programs, contract 09-00115, from July 2009 to June 2012.

Top of Page


Kern County Project CARE Evaluation

Richard A. Rawson, Ph.D., Principal Investigator (rrawson@mednet.ucla.edu)
Darren Urada, Ph.D., Co-Investigator
Valerie Antonini, M.P.H., Project Director

In 2011, California’s Kern County contracted with UCLA ISAP to provide evaluation, training, and technical assistance services for Project Care, which aims to integrate substance use and mental health services into primary care settings.  In 2011–2012, UCLA’s work with Kern County’s Project Care fell into two broad categories: (1) General Integration Activities, Training, and Technical Assistance, and (2) Measures of Integration.  Trainings to date have been well received and initial measures indicate increasing levels of integration at the participating sites.

Kern County Project CARE Evaluation was funded by Kern County, contracts 241-2011 and 566-2012, from May 2011 to June 2013.

Top of Page


Los Angeles County Department of Mental Health
Performance Outcomes Survey

Christine E. Grella, Ph.D., Principal Investigator (cgrella@mednet.ucla.edu)

UCLA ISAP collaborated with the Los Angeles County Department of Mental Health (DMH) Quality Improvement Division, Program Support Bureau, in conducting a countywide survey of consumers of outpatient mental health services. The survey is intended to meet the goals of the County Strategic Plan, including Performance Based Contracting.  ISAP staff participated in countywide trainings of mental health service providers regarding the administration of the survey to all consumers of outpatient services, including fee-for-service providers; developed and distributed the survey forms in conjunction with the County; provided data management services of completed surveys; conducted data analyses to assess consumer satisfaction with services received, including comparisons across County regions and consumer groups; and wrote a comprehensive report on the survey findings, including recommendations for future survey design and changes that may be made in response to survey findings on the performance outcomes. 

DMH Performance Outcomes Survey was funded by the Los Angeles County Department of Mental Health, contract MH010089, from March 2012 to June 2012.

Top of Page


Los Angeles County Evaluation System (LACES):
An Outcomes Reporting Program

Desirée Crèvecoeur-MacPhail, Ph.D., Principal Investigator (desireec@ucla.edu)
Richard A. Rawson, Ph.D., Co-Principal Investigator
M. Douglas Anglin, Ph.D., Co-Investigator
Loretta L. Denering, M.S., Project Director

The Los Angeles County Evaluation System (LACES) is designed to assess the overall effectiveness of the county’s alcohol and drug treatment/recovery system and measure the effectiveness of specific service modalities and service components. To accomplish this, LACES augmented the intake and discharge questions used by the county’s alcohol and drug treatment/recovery system. The information from the revised admission and discharge questions provide information that is used to evaluate the treatment system at the county level.  In addition, LACES works with the County of Los Angeles to examine its drug court, Women’s Re-entry, Driving Under the Influence (DUI), and medication assisted treatment programs (MAT).  The information gained from these programs, as well as the continued review of annual admission and discharge data, provides client outcome and program performance information that can be used to improve the overall substance use disorder treatment system. In addition, LACES has implemented several pilot projects in order to improve treatment engagement and retention and to assess for the “optimal” level of care.  Future tasks for LACES include a follow-up of clients from the MAT study, follow-up of patients from the medical marijuana field study, and the development of report cards that will include benchmarks to improve program performance. (Additional information is available at www.laces-ucla.org.)

Los Angeles County Evaluation System (LACES): An Outcomes Reporting Program was funded by the Los Angeles County Alcohol and Drug Program Administration, Contract PH00173 (March 2004 through June 2012).

Last Updated: 12/10/2012

Top of Page

2016 ISAP Publications
2015 ISAP Publications