Partners In Policymaking ® Long Term Research John R. Johnson, Ph.D. & Heidi Bethke Department of Special Education San Diego State University 5500 Campanile Dr. San Diego, CA 92182-1170 (619) 594-3841 johnson7@mail.sdsu.edu Overview . Rationale & Purpose . Research Questions . Methodology . Preliminary report of findings of an exploratory data analysis . Future research activities Rationale & Purpose . Program evaluation based on inputs of key stakeholders . Evidence of effectiveness for achieving legislative mandates and outcomes . System level benefits of Partners – Individual – Family – Community – State – National Rationale & Purpose . Information as power; exercise of power as the currency of leadership . Knowing how resources are allocated and used . Right of people with disabilities to know what is being done to and with them Rationale & Purpose . Contributions persons with disabilities make to –The knowledge base –Improving their own lives –Local communities –Systemic change –Personal and community advocacy –Organizational and political leadership Research Questions . Who are the participants? . What is their level of self-reported advocacy & leadership skill before and after participation in Partners? . What outcomes have been achieved by participants related to: – Independence – Productivity – integration & inclusion – self-determination . What other outcomes both measurable and others have resulted? . What is the level of participant satisfaction with – Participation in partners – Outcomes achieved Methodology . Each participant completes an initial questionnaire that provides preliminary information related to – Demographics – Level of involvement in advocacy activities .Contact with public officials Federal, State or local level .Types of contacts – Self-reported assessment of advocacy and leadership skills .Specific types of advocacy activities .Overall rating of advocacy an leadership – Rating of ability to secure needed services –Expectations from participating in Partners 6 Month & Long-Term Follow-UP . 1st follow-up survey submitted to Partners graduates within 6 months . Additional follow-up surveys submitted to graduates 1-5 years after graduation Long-Term Follow-up Survey Items A. Year graduated B. Rating of ability to secure services after graduation C. Skills learned that use today D. Rating of extent that skills learned from Partners help secures services today E. Self-assessment of advocacy leadership skills Long-Term Follow-up Survey Items F. Evidence of advocacy involving in areas related to 1. Independence 2. Productivity 3. Self determination 4. Integration and inclusion Long-Term Follow-up Survey Items G. Evidence of advocacy & leadership 1. Engagement in civic responsibility with public officials . Frequency of contact with public officials . Types of contact with public officials 2. Types of advocacy activities – Testimony – Presentations – Committees – Articles/editorials – Online communication/newsletters Long-Term Follow-up Survey Items H. Evidence of outcomes/benefits to individual participant & community attributed to participation in Partners 1. Housing 2. Education 3. Employment 4. Family support 5. Case management 6. Technology 7. Health care 8. Friendships 9. Other Long-Term Follow-up Survey Items . Areas for program improvement . Interactions with partners graduates, advocacy organizations or others involved in leadership The Challenge & Some Limitations . Data retrieved from surveys completed and returned a number of years ago . 5 different versions of follow-up survey with number of items ranging from 9 to 45 . Number of questions related to response rate (i.e., number of graduates within a given year and number of completed surveys available) . Missing data (items left blank) . Lack of availability of pre-participation surveys . Lack of availability of data describing characteristics of graduates completing follow-up surveys Caveat . Exploratory data analysis is a means of investigating the structure, integrity and preliminary findings of a data set . Additional data collection and analysis required to test theory, assumptions and hypotheses . Does provide a means for formulating additional data collection and analysis . Does provide some insights as to what conclusions the data MAY support . Provides information about emergent issues and themes that may require further investigation Important Preliminary Findings A Selected Sub-sample of the Characteristics of Partners (2000 & 2006) Note: Rounding may cause totals to exceed 100% Sample Size . 2000 (n=37) . 2006 (n=25) . N=62 Gender . Female = 69% . Male = 31% Ethnicity . African-American = 4% . Native American = 2% . Hispanic American = 4% . Caucasian = 90% . Other = 2% AGE . < 20 = 3% . 20-29 = 16% . 30-39 = 40% . 40-49 = 34% . 50+ = 6% Residence . Urban = 27% . Suburban = 38% . Rural = 36% A Selected Sub-sample of the Characteristics of Partners (2000 & 2006) Note: Rounding may cause totals to exceed 100% Income . < $12k = 37% . $12k --$19k = 8% . $20k -$29k = 8% . $30k -$39K = 12% . $40k -$49k = 18% . $50k+ = 17% Education . < HS = 5% . HS Grad = 29% . College 2yrs = 29% . College 4yrs = 21% . Partial grad = 5% . Masters = 8% . Post Masters = 2% . Doctorate = 2% Contact with public officials prior to participation in Partners (n=61) Number of Contacts 0 Federal 67% State 53% Local 58% Number of Contacts < 6 Federal 30% State 36% Local 29% Number of Contacts 5 - 10 Federal 2% State 7% Local 3% Number of Contacts 11 - 15 Federal 2% State 2% Local 3% Number of Contacts 15+ Federal 0% State 2% Local 5% Percent of Persons Engaged in Various Types of Advocacy Activities (n=62) Type of Activity Letter No Activity 63 Not Reported 0 Phone No Activity 58 Not Reported 0 Visit No Activity 71 Not Reported 0 Testimony No Activity 17 Not Reported 76 Presentation – Parent/Comm No Activity 13 Not Reported 58 Presentation – Conf No Activity 15 Not Reported 69 Committee No Activity 6 Not Reported 65 Media No Activity 23 Not Reported 69 Articles/Editorials No Activity 21 Not Reported 73 Rating of Advocacy, Leadership and Ability to Secure Services Self-rating of advocacy & leadership MN 2.27% SD .75% 1 - 2 65% 3 – 4 35% Current ability to secure services MN 2.25% SD .70 1 - 2 63% 3 – 4 37% Expect to Receive Appropriate Services as a Result of Participating in Partners . Definitely Yes = 79% . Probably Yes = 21% No one responded probably no or definitely no! PRELIMINARY FINDINGS OF POST-GRADUATION OUTCOMES Participant & Data Characteristics . Data available for participants from 10 graduating classes and years. Sample Sizes 1994 = 8 1995 = 39 1997 = 25 1998= 7 1999= 7 2000 = 16 2001 = 17 2002 = 13 2003= 8 2004 = 1 Not Reported = 78 Sample Size & Years Data Available YEAR 1994 N = 8 1995 N = 29 1997 N = 25 1998 N = 7 1999 N = 7 2000 N = 16 2001 N = 17 2002 N = 13 2003 N = 8 2004 N = 1 YEAR NOT REPORTED N = 78 TOTAL SAMPLE SIZE: N = 219 Rating of Advocacy, Leadership and Ability to Secure Services PRE (n=62) Self-rating of advocacy & leadership MN 2.25 SD .75 1-2 65% 3-5 35% Current ability to secure services MN 2.25 SD .70 1-2 63% 3-5 37% POST (n=214) Self-rating of advocacy & leadership MN 4.06 SD .80 1-2 3.8% 3-5 96.2% Current ability to secure services MN 4.2 SD .81 1-2 2.8% 3-5 97.2% Extent that Skills Learned from Partners Help Secure Services (n=211) MN 2.8 SD .44 Most Times (3) 79.6% Sometimes (2) 19% Seldom (1) 1.4% Rating of Advocacy, Leadership and Ability to Secure Services PRE (n=62) Self-rating of advocacy & leadership MN 2.27 SD .75 1-2 65% 3-5 35% Current ability to secure services MN 2.25 SD .70 1-2 63% 3-5 37% Skills learned help secure services MN SD 1-2 3-5 POST (n=214) Self-rating of advocacy & leadership MN 4.0 SD .80 1-2 3.8% 3-5 96.2% Current ability to secure services MN 4.2 SD .81 1-2 2.8% 3-5 97.2% Skills learned help secure services MN 2.8 SD .44 1-2 3-5 Percent of Respondents Indicated Improved Performance on Mandated Outcomes OUTCOMES Significant changes attributed to Partners n 213 % 90 More Independent n 147 % 92.5 More Productive n 143 % 79.7 More Self Determined n 213 % 82.6 More Integrated or Included n 147 % 88.4 ROI Indicators: Employment, Income, & Community Involvement (1 Year) ROI INDICATOR Now Employed n 51 % 18 Career Change n 45 % 40 Earn More $ n 48 % 23 Use Food Stamps n 44 % 21 Volunteer more time n 49 % 82 ROI Indicators: Changes in Income (1 Year) PRE (n=35) < $4, 999 37% $5K -$9,999 14 10k -$14,999 9 15k -$19,999 37 $20,000+ 3 POST (n=36) < $4, 999 25% $5K -$9,999 11 10k -$14,999 6 15k -$19,999 6 $20,000+ 53 What significant changes have occurred in your life or the life of a family member with a disability that you attribute to your Partners experience? PARTICIPANT COMMENTS Significant Changes Attributed to Partners Outcome Areas Housing n=217 % Yes 16 % No 84 Education n=214 % Yes 48 % No 52 Employment n=216 % Yes 13 % No 87 Family Support n=216 % Yes 22 % No 78 Case Mgmt n=216 % Yes 20 % No 80 Technology n=216 % Yes 15 % No 85 Health Care n=215 % Yes 21 % No 79 Friendship n=215 % Yes 24 % No 76 Any n=218 % Yes 88 % No 12 “My daughter has become more assertive and communicates more to let other peopleand communicates more to let other people know her wishes…and she is doing very well given the profound degree of herwell given the profound degree of her disability. She can make choices and we have learned and taught other people tohave learned and taught other people to respect her choices and wishes... I feel it came about because of my partners training... I view her differently now and value her existence.” “I am in law school!” “I ran for our local school boardboard……and WON!” “Everything! My son is in a regular classroom inclusive. He does have classroom inclusive. He does have adaptive phys ed too... ‘He has friends’! He is having his first birthday party with other children. He is reading, swimming, andHe is reading, swimming, and playing.” “My son is ‘flourishing’ in a regular ed classroom; in a district that practices exclusion. He is successful in this class and his peers and teachers love him. I would never have known about inclusion without ‘Partners’! My senator and representative visited him in his classroom to view successful inclusion of a child with significant cognitive and physical delays—at my invitation!” “I was able to move from HUD housing to a new townhouse. I was able to secure a contract for deed. Partners gave me the confidence to return to college. I graduated summa cum laude with a BS in management and ethics. I have received raises and promotions at work.” “Possibly divorce (explanation provided)” “My husband and I work as a team for our son.” “I was hired by ARC to do parent training and advocacy.” I am president of an international disability organization that supports my child’s diagnosis. I organize a family medical conference every year and currently support all research projects.support all research projects.”” “I moved into a bigger apartment and I got more work hours. I jumped from 9 hours a week to 16 hours a week 4 days a week. And I was accepted as a volunteer at a private elementary school.” “I have stayed employed at the... and applied and accepted at the University of Minnesota graduate school of social work.” “I have three jobs now!” “Today I am a student at the JFK School of Government at Harvard University because of ‘PIP’training. Most of my friends are PIP graduates.” Some Recurring Themes . Acquired services didn’t have before . Waiver . Inclusion . Friends . Effect on graduates – Going back to school – Career changes focused on improving services for persons with disabilities – Confidence to advocate and lead . Improved housing . Engaged in the civic, political & democratic process . Expansion of issues beyond the scope of individual and family . Involvement of husbands and fathers needed . Helping others with similar issues Future Research Issues . Filling in gaps in information especially information about respondent characteristics . Investigating differential outcomes with respect to class and year graduated . Maintaining comparability of variables . Need to address response rate and representativeness of sample. . Controlling for duplicated counts. . Controlling for other factors that may account for outcomes. . Analyses that examine trends over time and pre/post training differences . Identifying whether there is a higher proportion of Partners graduates who assume clearly defined leadership roles and responsibilities THANK YOU! Special thank you to Colleen Wieck,the Partners staff and Heidi Bethke who entered the data for this presentation. For more information about Partners In Policymaking ® link to .http://www.partnersinpolicymaking.com/