Program Evaluation


Evaluation of the Community Bridges Project
Christine E. Grella, Ph.D., Principal Investigator (grella@ucla.edu)
Kira Jeter, M.A., Project Director

The purpose of the Community Bridges Project is to strengthen and enhance the comprehensive treatment system in Santa Monica, California, for chronically homeless persons who have co-occurring mental illness and substance use disorders. Led by the CLARE Foundation, the project is based on a model of services collaboration among community-based providers of substance abuse treatment, mental health treatment, health services, shelter and drop-in services, outreach and engagement, and transitional and permanent housing. The integrated treatment model incorporates several evidence-based practices, including motivational interviewing and relapse prevention, with the aim of (1) increasing access to treatment, (2) increasing continuity of care and services integration, (3) improving quality of life and increasing self-determination, and (4) improving levels of functioning among participants.  In addition, the project aims to increase integration of services across the participating providers, to eliminate barriers to system entry, to improve awareness of service needs for this population within the broader community, and to improve staff competencies for delivering services to this population.  ISAP is conducting the project-level process and outcome evaluations.  A total of 300 participants will be recruited over 5 years.

Evaluation of Community Bridges Project was funded by the CLARE Foundation (Grant 20063816), through a grant from the Substance Abuse and Mental Health Services Administration (SAMHSA; September 2006 to September 2011).


Evaluation of the Homeless Interventions Treatment Options Project (HI-TOP)
Christine E. Grella, Ph.D., Principal Investigator  (grella@ucla.edu)
Kira Jeter, M.A., Project Director

ISAP conducted the evaluation of the Homeless Interventions Treatment Options Project (HI-TOP), which is an intervention project provided by Special Service for Groups, Inc. HI-TOP provided comprehensive substance abuse treatment and case management services to homeless individuals with co-occurring substance abuse and mental health disorders. The intervention model paired recovery home living with structured day treatment three days per week for 120 days, as well as individual counseling and comprehensive case management. Specific treatment approaches included cognitive behavioral and motivational enhancement therapies. The evaluation study found that project participants (N = 150) significantly improved in their alcohol and drug use, residential status, mental health status, and employment status over the period from study intake to 6-month follow-up.  In addition, the evaluation documented several challenges to implementing the integrated treatment model and strategies that were used to successfully address these barriers.

Evaluation of Homeless Interventions Treatment Options Project was funded by Special Service for Groups, Inc., Subcontract 243 (April 2005 through December  2009).


Evaluation of the Liberating our Families from Drugs and Incarceration Program
Christine E. Grella, Principal Investigator (grella@ucla.edu)
Luz Rodriguez, Project Director

ISAP is conducting the evaluation of the SAMHSA-funded project “Liberating our Families from Drugs and Incarceration (LOFFDI),” which is part of the Pregnant and Parenting Women’s Treatment Initiative at the Center for Substance Abuse Treatment.  The LOFFDI program is provided at the Walden House program in El Monte, CA, and the target population is women parolees who have a history of substance use problems and who are currently pregnant or have young children.  The program’s objectives are to improve the mental and physical health status, employment status, and parenting skills of participants so that they can successfully reintegrate into the community following their release from prison.  Intensive case management services are provided within the context of residential substance abuse treatment, which includes children’s services, family counseling, parenting skills training, vocational services, mental health services, trauma-related services, and referral for medical services for the women and their children. The evaluation study showed that approximately 60% of participants (N=110) completed the LOFFDI treatment, compared with an average of 39% for participants in other aftercare programs for female offenders in California.  Moreover, LOFFDI participants demonstrated significant improvements in their alcohol and drug use, psychological status, family functioning, employment, and quality of life at the 6-month post-discharge follow-up.  Participants had a high level of satisfaction with the services received and staff improved their competency in delivering parenting and other services over the course of project implementation. Overall, the evaluation demonstrated that LOFFDI participants improved in targeted domains that address the multiple and complex problems faced by female offenders and their families as they re-integrate into the community following parole.

Evaluation of the Liberating our Families from Drugs and Incarceration Program was funded by Walden House, Inc., Contract20064370 (October 2006 to September 2010).


Evaluation of Trauma Informed Substance Abuse Treatment (TI-SAT)
Christine E. Grella, Ph.D., Principal Investigator (cgrella@mednet.ucla.edu)
Nena Messina, Ph.D., Co-Investigator
Luz Rodriguez, B.A., Project Director

The Trauma-Informed Substance Abuse Treatment (TI-SAT) program was established at the Leo Chesney Community Correctional Facility in 2008 to provide gender-responsive and trauma-informed substance abuse treatment for women offenders.  Trauma-informed treatment is based on the recognition that childhood and adult trauma exposure is pervasive among women offenders and intricately related with their substance abuse problems.  ISAP is conducting an evaluation of the TI-SAT to determine whether the trauma-informed treatment components are implemented as planned; if obstacles are encountered in the implementation process, and if so, the strategies used to address these obstacles; and the characteristics of participants, their treatment retention and aftercare participation, and their outcomes on parole.  A total of 106 participants were recruited to participate in the outcome study; interviews are conducted at study intake and at 6-month follow-up.  Study findings will compare risk of recidivism of TI-SAT participants with comparable women offenders who were enrolled at the facility prior to implementing the TI-SAT program. Outcome analyses will examine the effects of program participation on status and functioning of participants, including their relapse to substance use, criminal behavior involvement, employment, parenting and family relationships, and overall mental health and psychosocial functioning.  In addition, surveys with program staff and observational site visits are conducted to determine whether the gender-responsive and trauma-informed program components are successfully implemented.

Evaluation of Trauma Informed Substance Abuse Treatment (TI-SAT) was funded by the State of California Department of Corrections and Rehabilitation, contract C08.106, from October 2008 to March 2011.


CLARE Follow-up Study
Christine Grella, Ph.D., Principal Investigator (cgrella@ucla.edu)

The CLARE Follow-up Study was designed to provide basic evaluation information on the socio-demographics and background characteristics of CLARE participants, their time in treatment and completion status, and outcomes across various domains of functioning over time. Four assessments were completed over a 12-month period; 95% of the study sample (N=92) was located over the follow-up period and 85.9% of the baseline sample completed at least one follow-up interview. Overall, close to two thirds of the study sample completed treatment and less than one third discharged prior to completion.  CLARE participants significantly reduced their alcohol and drug use at 3-months, although there was a slight rebound by 6-months.  Participants significantly improved in the areas of overall health status and psychological status, with greater improvements generally among those who completed treatment compared with those who did not.  Substance use at the time of follow-up was associated with poorer functioning in several areas, particularly regarding psychological problems. Overall, the odds of working increased over time by approximately 40%. Individuals who were more involved in AA/12-step participation (both during and after treatment) had lower levels of drug use severity, depressive symptoms, and severity of psychological problems over time.  There were significant gender differences in the areas of social/family functioning and psychological problems, with women reporting more anxiety, overall psychological problems, and problems in social/family functioning at all time points.

CLARE Follow-up Study was funded by the CLARE Foundation, contract 20080772, from May 2008 to April 2009.


Evaluation of Horizon's Integrated AODA Treatment/HIV
Services for Justice Involved Women
Nena Messina, Ph.D., Principal Investigator (nmessina@ucla.edu)
Project Director and degree: Kira Jeter, M.P.H.

The purpose of this SAMHSA funded project is to expand Community Advocates - Horizons Campus' program to increase access to integrated outpatient AODA treatment and HIV services for a minimum of 600 justice-involved women who are less likely to enter treatment through formal systems. Objectives are to: (1) increase the number of women who participate in integrated outpatient AODA/HIV services from 25 to 125 women per year; (2) enhance Community Advocates-Horizons Campus' program with research-based AODA/HIV curricula that will engage hard-to-reach women, their children and their partners; and (3) increase the number of women and partners who receive rapid HIV testing from 90 to 200 per year.

  • ISAP is conducting the evaluation, which (1) includes process and outcome components to monitor the implementation of the planned intervention and (2) measures observed changes in participants from before to after their participation in the project in the following domains of functioning: (a) alcohol and drug use; (b) parenting behaviors and family functioning; (c) quality of life and social functioning; (d) criminal behavior and legal status; (e) mental health and physical health status; (f) HIV/AIDS risk reduction behaviors; and (g) treatment and services received.

ISAP will determine the degree the project was able to increase access for justice-involved women to gender-specific, integrated outpatient AODA treatment and HIV prevention services.

Evaluation of Community Advocates - Horizons Campus Integrated AODA Treatment/HIV Services for Justice-Involved Women was funded by the Substance Abuse and Mental Health Services Administration, Grant TI18592 to Community Advocates-Horizons Campus (October 2007 to September 2012).


Evaluation of People in Progress’s New Elements Treatment for Homeless Program
Richard A. Rawson, Ph.D., Principal Investigator (rrawson@mednet.ucla.edu)
Anne Bellows, M.S.W., Project Director

ISAP is conducting the process and outcome evaluations of the New Elements Treatment program, which is an intervention project provided by People in Progress. The project is funded by a 5-year grant from the Substance Abuse and Mental Health Services Administration, Center for Substance Abuse Treatment. New Elements provides comprehensive substance abuse treatment and case management services to homeless individuals. The evaluation study is tracking the characteristics of project participants, and examining the delivery of project-specific services, adherence to the treatment models, barriers encountered in project implementation, and solutions devised to address those barriers. Focus groups with project participants and interviews with project staff are being conducted in order to monitor their respective perceptions and experiences of the project; feedback is provided from the evaluation in order to refine the project implementation.

Evaluation of People in Progress New Elements Treatment for Homeless Program was funded by People in Progress, Contract 20071053 (November 2006 to October 2011).


Los Angeles County Evaluation System (LACES): An Outcomes Reporting Program
Desirée Crèvecoeur-MacPhail, Ph.D., Principal Investigator (desireec@ucla.edu)
Richard A. Rawson, Ph.D., Co-Principal Investigator
M. Douglas Anglin, Ph.D., Co-Investigator

The Los Angeles County Evaluation System (LACES) is designed to assess the overall effectiveness of the county’s alcohol and drug treatment/recovery system and measure the effectiveness of specific service modalities and service components. To accomplish this, LACES augmented the intake and discharge questions used by the county’s alcohol and drug treatment/recovery system. The revised admission and discharge questions provide information that is used to evaluate the treatment system at the county level, but they also serve to satisfy the additional requirements outlined by the state under the California Outcomes Measurement System (CalOMS) and the federal government as defined by the National Outcomes Measurement System (NOMS). The information gained from the revised admission and discharge questions allow LACES to assess individual programs through the use of site reports that provide outcome and performance information based on the same areas under examination by the state and federal governments. In addition, LACES has implemented several pilot projects in order to improve treatment engagement and retention and to assess for the “optimal” level of care. Future tasks for LACES include the assessment of participants’ satisfaction with treatment and development of report cards that will include benchmarks improve program performance. (Additional information is available at www.laces-ucla.org.)

Los Angeles County Evaluation System (LACES): An Outcomes Reporting Program was funded by the Los Angeles County Alcohol and Drug Program Administration, Contract H700244 (March 2004 through June 2010).


Evaluation of the California Outcome Measurement System (CalOMS) for Treatment
Richard A. Rawson, Ph.D., Principal Investigator (rrawson@mednet.ucla.edu)
Rachel Gonzales, Ph.D., Project Director

The California Department of Alcohol and Drug Programs (ADP) implemented a statewide data collection and management system called the California Outcome Measurement System (CalOMS) to contribute to the improvement of treatment services for substance abuse. CalOMS is the first statewide data collection and management system to comprehensively measure program outcomes using performance and outcome measures based on the federally required reporting of National Outcome Measures (NOMs), developed by the Substance Abuse and Mental Heath Services Administration (SAMHSA).  In collaboration with ADP, UCLA Integrated Substance Abuse Programs (ISAP) conducted an evaluation study of state data system (CalOMS) under the guise of the following objectives:

  • Use CALOMS data to improve knowledge of alcohol and other drug (AOD) treatment services in California.
  • Enhance the capability of county administrators to use CalOMS data to improve treatment services.
  • Evaluate the quality and validity of CalOMS data.
  • Develop recommendations for improvement of the CalOMS system.

UCLA provided ADP with a final report that addressed these objectives to help the state department improve the quality and performance of AOD treatment services in California and to maximize the usability of CalOMS data to enhance treatment policies and practices in California.

Evaluation of the CalOMS Data was funded by the State of California, Alcohol and Drug Programs, Contract 06-00216 (June 2007 to December 2008).


Research Services in Performance and Outcome Management in
 Support of the Continuum of Services Redesign (COSSR)
Richard A. Rawson, Ph.D., Principal Investigator (rrawson@mednet.ucla.edu)
Rachel Gonzales, Ph.D., Project Director (rachelmg@ucla.edu)

In an effort to move towards a continuum of services platform recognizing AOD problems under a chronic illness model, the California Department of Alcohol and Drug Programs (ADP) established the Continuum of Services System Re-engineering (COSSR) Task Force in May 2006.  This effort focused on “developing a comprehensive, integrated, continuum of AOD services that includes prevention, intervention, treatment and recovery”.  This reengineering effort is consistent with the 2006 update of the Institute of Medicine’s (IOM) Quality Chasm Series which recommends that “substance use disorder treatment move toward building its standards of care, performance measurement and quality, information and cost measures, upon a chronic illness model rather than the current, acute illness-based, fragmented and deficient system of health care.” 

ADP contracted with UCLA ISAP for a 1-year period (July 2008 to June 2009) to help ADP facilitate Research Services in support of the Continuum of Services Redesign (COSSR) addressing Performance and Outcome Measurement and Management.  For the purposes of addressing theseCOSSR-related Objectives, UCLA addressed the following three objectives:

  • Objective 1: Development of a Framework for Performance and Outcome Measurement/Management System
  • Objective 2: Identification of Performance and Outcome Measurement in Support of a Continuum of Services Model
  • Objective 3: Putting it all Together: Planning for the Redesign of the System in Support of a Continuum of Services

Research Services in Performance and Outcome Management in Support of the Continuum of Services Redesign was funded by the State of California Department of Alcohol and Drug Programs, contract 07-00176, from July 2008 to June 2009.


Evaluation Services to Enhance the Data Management System in California
Richard A. Rawson, Ph.D., Principal Investigator (rrawson@mednet.ucla.edu)
Valerie Pearce, MPH, Project Director (vpearce@mednet.ucla.edu)

A top priority of the California department of Alcohol and Drug Programs (ADP) is to improve the accountability of the Substance Use Disorder (SUD) treatment system in California in terms of ensuring quality services and effective client outcomes.  Within this contract, ADP requested a two-year work plan for 2009-2011 to continue evaluation work and concepts as defined within the previous Continuum of Services System Reengineering (COSSR) state contract (07-00176) in support of establishing a more effective SUD system in California.

Within Year 1, the work plan was focused around the following objectives:

Objective 1: Examine the California Outcomes Measurement System for SUD Treatment (CalOMS-Tx) data to enhance SUD treatment services and client outcomes in California

Objective 2: Enhance CalOMS-Tx system to include performance measurement/management data

Objective 3: Enhance CalOMS-Tx system to include performance and outcome measurement in support of the COSSR model

Objective 4: Increase the capacity of SUD stakeholders (state ADP, county administrators, and providers) to understand how to apply performance and outcome data to improving the quality of treatment services

Within Year 2, the work plan was amended to address new priorities established following the passing of H.R.3590 – the federal Patient Protection and Affordable Care Act, to assist the State of California and its 58 counties for the preparation for health care reform (HCR). It is anticipated that HCR will result in (among other things) modifications in how services will be funded, the type of services delivered, the venues where they are delivered, the individuals who will receive the services, the work force that delivers the services, how services are measured, and how service benefits are evaluated.  The major focus of the revised work plan was to conduct data collection, data analysis, and related activities to determine models of service delivery that may facilitate this new system, and disseminate this knowledge to policy makers and practitioners across California.  The tasks and activities were focused around the following topic areas:

  1. Data Systems Improvements
  2. SA/MH/Health Care Integration
  3. Performance Measurement/Dashboards
  4. Development of Recovery Measurement and Resources
  5. Planning for Prevention
  6. Workforce Development

Evaluation Services to Enhance the Data Management System in California was funded by the State of California Department of Alcohol and Drug Programs,  contract 09-00115, from July 2009 to June 2012.

Last Updated: 11/16/2011

Top of Page

2016 ISAP Publications
2015 ISAP Publications