Loading...
HomeMy WebLinkAboutNMK Research and Consulting-Local Tobacco Control Evaluation Services_A-18-715-1.pdfCounty of Fresno Board of Supervisors Minute Order Hall of Records, Room 301 2281 Tulare Street Fresno, California 93721-2198 Telephone: (559) 600-3529 Toll Free: 1-800-742-1011 www.co.fresno.ca.us March 24, 2020 Present:Chairman Buddy Mendes, Vice Chairman Steve Brandau, Supervisor Nathan Magsig, Supervisor Brian Pacheco, and Supervisor Sal Quintero 5 - Agenda No. 41.Public Health File ID: 19-1356 Approve and authorize the Chairman to execute a First Amendment to Agreement No. 18-715 with NMK Research and Consulting, for local tobacco control program evaluation services, effective upon execution with no change in term to June 30, 2023 and increasing the maximum by $13,384 to a total of $253,266 Re: APPROVED AS RECOMMENDED Ayes:Mendes, Brandau, Magsig, Pacheco, and Quintero5 - Agreement No. 18-715-1 Page 46County of Fresno Board Agenda Item 41 DATE:March 24, 2020 TO:Board of Supervisors SUBMITTED BY:David Pomaville, Director, Department of Public Health SUBJECT:First Amendment to Agreement No. 18-715 with NMK Research and Consulting RECOMMENDED ACTION(S): Approve and authorize the Chairman to execute a First Amendment to Agreement No. 18-715 with NMK Research and Consulting, for local tobacco control program evaluation services, effective upon execution with no change in term to June 30, 2023 and increasing the maximum by $13,384 to a total of $253,266. Approval of the recommended action will amend Agreement No. 18-715 with NMK to include additional program evaluation activities commensurate with the $13,384 increase in funding; and shifts unspent dollars from year one to year two budgets. The recommended amendment is fully funded with California Tobacco Control Program (CTCP) grant funds, with no Net County Cost. This item is countywide. ALTERNATIVE ACTION(S): There are no viable alternative actions. Should your Board not approve the recommended action, the local tobacco control program evaluation would not be completed as required by the CTCP grant. FISCAL IMPACT: There is no increase in Net County Cost associated with the recommended action. The recommended Amendment is 100% funded with CTCP grant funds. The recommended amendment increases the maximum compensation of Agreement No. 18-715 in the amount of $13,384 to a total compensation of $253,266. The revised annual budgets are as follows: ·Year One FY 2018-19 $8,230 - decrease of $28,279 ·Year Two FY 2019-20 $92,116 - increase of $41,663 or 83% ·Year Three FY 2020-21 $52,920 - no change ·Year Four FY 2021-22 $50,000 - no change ·Year Five FY 2022-23 $50,000 - no change Sufficient appropriations and estimated revenues are included in the Department’s Org 5620 FY 2019-20 Adopted Budget and will be included in future budget requests for the duration of the term. DISCUSSION: On December 11, 2018, the Board approved a five-year Agreement with NMK to satisfy the CTCP grant funding requirement for a dedicated external evaluator to provide comprehensive evaluation services of the local tobacco control program. If approved, the recommended Amendment will revise the scope of work to Page 1 County of Fresno File Number: 19-1356 File Number: 19-1356 include additional evaluator activities expanding upon the current grant objectives to include: ·Conducting additional key informant interviews with schools/parents; ·Reviewing Community Action Model (CAM) guides; ·Developing additional CAM post-training surveys and analyzing the data; ·Complete policy records/reports for two (2) reporting periods; ·Review and provide feedback on four evaluation instruments developed by CAM subcontractors; and ·Attend two (2) days at the Joining Forces Tobacco Conference The funding increase in the amount of $13,384 is proportionate to the increases in the scope of work activities. With your Board’s approval, the recommended Amendment will allow the Department to continue meeting CTCP grant requirements in the area of tobacco prevention and cessation activities through policy, system, and environmental change efforts. All other terms of the agreement remain unchanged. REFERENCE MATERIAL: BAI #58, December 11, 2018 ATTACHMENTS INCLUDED AND/OR ON FILE: On file with Clerk - First Amendment to Agreement No. 18-715 CAO ANALYST: Raul Guerra Page 2 County of Fresno File Number: 19-1356 - 1 - 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 FIRST AMENDMENT TO AGREEMENT THIS FIRST AMENDMENT TO AGREEMENT (hereinafter “Amendment”) is made and entered into this 24th day of March, 2020, by and between COUNTY OF FRESNO, a Political Subdivision of the State of California, Fresno, California (hereinafter “COUNTY”), and NMK Research and Consulting, a California Sole Proprietor, whose address is 316 Alta Vista Drive, Santa Cruz, California, 95060, (hereinafter “CONTRACTOR”). WITNESSETH: WHEREAS, COUNTY and CONTRACTOR entered into Agreement number A-18-715, dated December 11, 2018 (hereinafter “Agreement”), pursuant to which CONTRACTOR agreed to provide comprehensive public health program evaluation services for its local tobacco prevention program, through its Department of Public Health (Department); and WHEREAS, COUNTY and CONTRACTOR now desire to amend the Agreement in order to increase total compensation and modify the budget. NOW, THEREFORE, for good and valuable consideration, the receipt and adequacy of which is hereby acknowledged, COUNTY and CONTRACTOR agree as follows: 1. Section Four (4) of the Agreement, located on page Three (3), lines Four (4) through Eighteen (18), is deleted in its entirety and replaced with the following: "4. COMPENSATION/INVOICING A. COUNTY agrees to pay CONTRACTOR and CONTRACTOR agrees to receive compensation as described in Revised Exhibit B. CONTRACTOR shall invoice COUNTY monthly, by the thirtieth (30th) day of each month for the prior month’s expenditures, addressed to the County of Fresno Department of Public Health, P.O. Box 11867, Fresno, CA 93775, Attention: OFFICE OF HEALTH POLICY & WELLNESS Staff Analyst. Invoices shall detail line items as specified in Revised Exhibit B, including original budget amount(s), current month’s expenses, year-to-date expenses, and budget balances. In addition, CONTRACTOR invoices shall include all relevant supporting documentation including but not limited to copies of original statements, program expense receipts, payroll records and mileage claims. B. In no event shall services performed under this Agreement be in excess of One Agreement No. 18-715-1 - 2 - 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 Hundred Fifty-Three Thousand Two Hundred Sixty-Six and No/100 Dollars ($153,266.00) during the period of December 11, 2018 through and including June 30, 2021. In no event shall actual services performed under this Agreement be in excess of Fifty Thousand and No/100 Dollars ($50,000.00) during each of the two (2) possible one (1) year extensions. It is understood that all expenses incidental to CONTRACTOR’S performance of services under this Agreement Shall be borne by CONTRACTOR." 2.That all references in existing COUNTY Agreement No. A-18-715 to “Exhibit A” shall be changed to read “REVISED Exhibit A,” attached hereto and incorporated herein by reference. 3.That all references in existing COUNTY Agreement No. A-18-715 to “Exhibit B” shall be changed to read “REVISED Exhibit B,” attached hereto and incorporated herein by reference. COUNTY and CONTRACTOR agree that this Amendment is sufficient to amend the Agreement and, that upon execution of this Amendment, the Agreement and this Amendment together shall be considered the Agreement. The Agreement, as hereby amended, is ratified and continued. All provisions, terms, covenants, conditions and promises contained in the Agreement and not amended herein shall remain in full force and effect. /// /// /// /// /// /// /// /// /// /// /// /// 1 2 EXECUTED AND EFFECTIVE as of the date first above set forth. 3 CONTRACTOR: NMK Research and Consulting 4 5 6 7 8 9 (Authorized Signature) 10 Natasha Kowalski Print Name & Title 11 12 13 316 Alta Vista Drive, Santa Cruz, California 95060 14 Mailing Address 15 16 17 18 19 20 21 22 23 FOR ACCOUNTNG USE ONLY : 24 ORG No .: 25 Account No .: 26 27 28 Fund/Subclass : SA 56201552 7295 0001/10000 COUNTY OF FRESNO: ~~~ Ernest BuddyMens, Chairman of the Board - 3 - of Supervisors of the County of Fresno ATTEST : Bernice E. Seidel Clerk of the Board of Supervisors County of Fresno , State of California By: Deputy REVISED Exhibit A Evaluator Scope of Work Evaluation Activity Description Timeline Responsible Party Deliverables 2-E-1 Education/ Participant Survey Fresno County Tobacco-Free Coalition members will be surveyed annually to identify which priority populations are represented, ask members to rate the diversity of the Coalition, measure members' level of participation in meetings and activities, and assess their satisfaction with the function of the coalition. The Evaluator and staff will review and update the previously used instrument called the Coalition Survey, which is based on the Tobacco Control Evaluation Center's (TCEC) sample survey. All Coalition members will be asked to complete the survey online (i.e., Survey Monkey or Survey Analytics), and 14 are expected to complete the survey. Additional surveys may be collected pen-to-paper at a Coalition meeting to increase the response rate, and the pen-to-paper surveys will be entered into Survey Monkey or Survey Analytics after the meeting. The results will be used to improve recruitment efforts, gauge diversity, and assess functioning of the Coalition. A limitation of the satisfaction survey is that current, active members tend to volunteer to participate in the survey. Though we will gain valuable feedback about how to maintain a healthy coalition and suggestions for recruiting new members, we may gain limited information from inactive and new members. 01/19-06/19 01/20-06/20 01/21-06/21 Evaluator Fresno County Department of Public Health (FCDPH) Summary of Key Findings Copy of Survey Instrument 2-E-3 Final Evaluation Report In the Brief Evaluation Report (BER), all data sources will be summarized following the guidelines specified by CTCP and will identify effective training strategies, barriers and challenges, and recommendations for other programs seeking to use CAM to build capacity of new partners. A content analysis will be conducted to analyze the key informant interviews, and resulting themes will identify the strengths, weaknesses, and impact of the intervention efforts. Descriptive statistics such as frequencies and percentages will be used to analyze the post-training surveys of the CAM trainings, the post training surveys of the Coalition trainings, and the annual Coalition Survey. The post-training surveys will determine the usefulness and effectiveness of the trainings and whether additional training is needed. The Coalition Survey will assess the functioning and diversity of the Coalition. A summary of the report will be submitted to ROVER and Partners, and will be shared with the Coalition and other programs who may want to tackle a similar objective. 01/21-06/21 Evaluator FCDPH Brief Evaluation Report 2-E-4 Education/ Participant Survey After each of the six Community Action Model (CAM) trainings, 3-6 participants will complete a retrospective post-training survey to assess their level of knowledge of the training topics including the CAM process. The surveys will be administered pen-to-paper and/or online, and a census sample will be used. The survey instrument will be 01/19-06/21 Evaluator FCDPH Summary of Key Findings Survey Instrument REVISED Exhibit A Evaluator Scope of Work developed based on any existing materials from other programs and TCEC. The questions will ask participants to rate their knowledge and skill levels before and after the training, and to identify any additional training needs. After the final CAM training, survey questions will also ask participants to rate their ability to advocate for positive change in their community. The survey results will be analyzed using descriptive statistics such as percentages and frequencies. The results will be used to address gaps in knowledge and to improve future trainings. A limitation might be that participants want to please the training leader and may give a more favorable rating as a result 2-E-6 Key Informant Interview Key informant interviews will be conducted with the staff and volunteers from the funded projects to identify strengths, weaknesses, and impact of the CAM process. A total of 6-8 interviews (20-30 minutes long) will be conducted over the phone. The Evaluator will contact other programs working on similar issues (City and County of San Francisco) and TCEC to obtain any existing interview questions. Based on any existing materials, interview questions will be developed to: (1) obtain feedback on the RFP application process and the training and technical assistance received, (2) discover how the funded projects addressed tobacco issues and health inequities in their community and whether they felt they made an impact, and (3) gather suggestions for improvement. A content analysis will be used to summarize points of view and common themes. Resulting themes will be used to improve intervention strategies, inform the evaluation summary, and provide recommendations for future campaign strategies 07/19-12/20 Evaluator FCDPH Interview Questions Summary of Key Findings 2-E-7 Education/ Participant Survey After each of the trainings to build the capacity of Coalition members, 6-10 participants will complete a post-training survey to assess their level of knowledge of the training topics and their capacity to strengthen local control efforts. The surveys will be administered pen- to-paper, and a census sample will be used. The survey instrument will be developed based on any existing materials from other programs and TCEC. The survey results will be analyzed using descriptive statistics such as percentages and frequencies. The results will be used to address gaps in knowledge and to im prove future trainings. 07/17-06/21 Evaluator FCDPH Summary of Key Findings Survey Instrument 3-E-1 Public Policy Record A public policy meeting observation form will be developed to document policymaker support/opposition, key issues raised by policymakers and staff, and other insights from public policy meetings. TPP staff will observe and record this information during any public policy meetings where the policy is discussed. In addition, official policy records such as city council agendas, meeting minutes, presentations, testimony, and policies enacted will be collected and coded to track: (1) 07/17-06/21 Evaluator FCDPH Public Policy Meeting Observation Form Summary Report of Key Findings REVISED Exhibit A Evaluator Scope of Work support/opposition; (2) key issues that surface during the intervention that could help shape changes to the intervention plan during the process; (3) the policy adoption process so that it can be described in the brief evaluation report (BER); and (4) to confirm adoption of a policy that meets the objective for this project. Documentation of the signed policy will also be included. Updated Policy 3-E-2 Media Activity Record Tobacco Prevention Program (TPP) staff will track and review earned media placement through online news, area and local radio, area and local television, and/or hard-copy community newspapers, including details of media placement, data, and coverage. A content analysis will be conducted to evaluate the earned media pieces’ effectiveness at delivering/disseminating the message, their accuracy and neutrality of facts, and to gauge the level of public support for campaign strategies. Using CTCP's paid media tracking form, paid media will be similarly tracked to determine coverage and placement. 07/17-06/21 Evaluator FCDPH Earned Media Tracking Form Paid Media Tracking Form 3-E-5 Key Informant Interview In the third and fourth year, 1-2 key informant interviews will be conducted in each of the jurisdictions that adopted a policy for a total of 8-12 interviews (15-30 minutes long). The interviews will identify factors that have facilitated or impeded policy adoption and ways to improve intervention strategies. Interviews will be conducted over the phone and/or face-to-face. Interviewees may include elected officials, their staff, members of the business community (e.g., Chamber of Commerce, individual retailers), and/or and other leaders in the community. The Evaluator will contact other programs working on similar issues and the Tobacco Control Evaluation Center (TCEC) to obtain any existing interview questions. Based on any existing materials, interview questions will be developed to identify community champions and barriers, key factors for supporting or opposing policy adoption, and successes and lessons learned. A content analysis will be used to summarize points of view and common themes. Resulting themes will inform the evaluation summary and provide recommendations for future campaign strategies. Evaluator FCDPH Interview Questions Summary of Key Findings 3-E-6 Final Evaluation Report In the BER, all data sources will be summarized and will discuss which intervention strategies were successful and why, challenges faced and how they were addressed, and recommendations for future efforts. A content analysis will be used to analyze the policy records, media tracking forms, and key informant interviews. For the interviews and policy records, common themes such as the level of support for policy change and barriers to policy adoption and implementation will be identified and summarized. The media tracking forms will help determine the extent of news and media coverage and assess whether public sentiment is leaning positively or negatively towards new and 01/21-06/21 Evaluator FCDPH Brief Evaluation Report REVISED Exhibit A Evaluator Scope of Work emerging tobacco product issues. Finally, new materials will be consumer tested with members of the target audience, and results will be used to improve the materials. 3-E-7 Consumer Testing Conduct consumer testing to assess feedback on the look, feel, content, language, approach, and action steps in the newly developed educational materials for community members (3-3-1). Develop the consumer testing instrument using guidelines from TECC and/or adapt any existing samples from other project directors. To assess the appropriateness of the educational materials for the general public, TPP staff will conduct a public intercept survey with 10-15 community members and will use a convenience sample. For materials for policymakers (3-3-1), a review committee will assess feedback on the look, feel, content, language, approach, and action steps in the newly developed fact sheets and educational packets. The committee will consist of 2-3 members of the senior administration of Fresno County who understand the perspective of elected officials and their staff. Based on participant responses, materials will be revised, and changes will be tracked. 01/18-12/18 Evaluator FCDPH Consumer Testing Instrument Log of Revisions 4-E-1 Public Policy Record A public policy meeting observation form will be developed to document policymaker support/opposition, key issues raised by policymakers and staff, and other insights from public policy meetings. TPP staff will observe and record this information during the meetings. In addition, official policy records such as city council agendas, meeting minutes, presentations, testimony, and policies enacted will be collected and coded to track: (1) support/opposition; (2) key issues that surface during the intervention, that could help shape changes to the intervention plan during the process; (3) the policy adoption process so that it can be described in the final evaluation report (FER); and (4) confirmation of adoption of a policy that meets the objective for this project. Documentation of the signed policy will also be included. 07/17-06/21 Evaluator FCDPH Public Policy Meeting Observation Form Signed Policy Summary Report of Key Findings 4-E-2 Consumer Testing Conduct consumer testing to assess feedback on the look, feel, content, language, approach, and action steps in the two fact sheets for participants of the educational presentations. Develop the consumer testing instrument using guidelines from the TECC and/or adapt any existing samples from other project directors. Two public intercept surveys will be conducted, one for each fact sheet. Each survey will use a convenience sample and will include 10-15 community members. Based on participant responses, materials will be revised, and changes will be tracked. 01/19-12/19 Evaluator FCDPH Consumer Testing Instrument Log of Revisions 4-E-4 A public intercept survey will be conducted to assess whether community members in the targeted jurisdiction are concerned about 01/18-06/19 External Evaluator Summary Report of Key Findings REVISED Exhibit A Evaluator Scope of Work Public Intercept Survey/Opinion Poll the location and number of tobacco retailers, their level of support for the proposed policy, and key demographic characteristics of the respondents. Only residents of the targeted community will be eligible to complete the survey, and 139-185 surveys will be collected at community events, shopping centers, and/or other sites within the targeted jurisdiction. Using existing survey instruments collected from TCEC and from other programs addressing this issue, a survey instrument will be developed. Using a convenience sampling method, surveys will be administered online and through face-to-face interactions. The Evaluator will conduct the data analysis utilizing frequencies and percentages to document support for policy strategies, beliefs regarding proximity and density issues, and demographic information. These results will guide educational efforts and will be shared with community partners, policy makers, and the general public. FCDPH Density/Proximity Subcontractor Survey Instrument 4-E-5 Final Evaluation Report In the FER, all data sources will be summarized and will discuss which intervention strategies were successful and why, challenges faced and how they were addressed, and recommendations for future efforts. Both public intercept surveys will be analyzed using descriptive statistics such as frequencies and percentages to guide campaign strategies and to educate policy makers, community partners, and the general public. A content analysis will be used to analyze the policy records, media tracking forms, and key informant interviews. For the interviews and policy records, common themes such as the level of support for policy change and barriers to policy adoption and implementation will be identified and summarized. The media tracking forms will help determine the extent of news and media coverage and assess whether public sentiment is leaning positively or negatively towards proximity issues. New materials will be consumer tested with members of the target audience, and results will be used to improve the materials. To assess outcome, maps will be developed to compare the number of retailers near schools before and after policy adoption. Evaluator FCDPH Final Evaluation Report 4-E-6 Media Activity Record Using a tool developed by TCEC, TPP staff will track and record earned media placement through online news, social media websites and blogs, area and local radio, area and local television, hard-copy community newspapers and newsletters, including details of media placement, data, and coverage of density/proximity issues, the HSHC press event, and other related retail issues. Paid media will be similarly tracked. A media content analysis will be conducted to evaluate the earned media pieces’ effectiveness at delivering/disseminating the message, their accuracy and neutrality of facts, and to gauge the level of public support for HSHC campaign strategies including density/proximity strategies. 07/18-06/21 Evaluator FCDPH HSHC Media Record (Earned Media) Media Tracking Form (Paid Media) REVISED Exhibit A Evaluator Scope of Work 4-E-7 Observation Data Observation data will be collected in tobacco retail stores using electronic handheld devices, and an electronic survey instrument developed by CTCP. TPP staff will complete the core HSHC observation survey and may complete additional modules based on priorities and resources available. Completion of the core survey will take approximately 20 minutes. The number of stores to be surveyed (330 - 428) is based on cohort sample provided by CTCP. With technical assistance by TCEC as needed, TPP staff will review and perform initial cleaning of survey data. CTCP will coordinate remaining data cleaning and analysis and provide results to Local Lead Agencies (LLA) to support their HSHC intervention activities, Community of Excellence Needs Assessment and Media Release. Optional: More advanced analyses may be done by TPP (e.g., comparing results by store type or area demographics, looking at change over time, etc.) based on training provided via the HSHC Advanced Data Analysis Webinar. Provide stipends or gift cards for youth and adult volunteers who participate in local data collection effort including training and completion of data collection forms. 01/19-06/19 Coalition Members Density/Proximity Subcontractor Evaluator FCDPH Youth Volunteers Cleaned Survey Result Store Observation Tracing List 4-E-9 Public Intercept Survey/Public Opinion Poll Utilizing a survey instrument developed by TCEC, use electronic handheld devices to collect follow-up data of knowledge, attitudes, and perception of the community regarding the retail environment. The sample size will be 190-250 based on recommendations from the CTCP. Frequencies and percentages will be calculated to document support/opposition to policy strategies for the HSHC campaign, knowledge, and demographic information provided by survey participants. The results will be shared with project staff and Coalition members and help guide campaign interventions. 07/19-12/19 Evaluator FCDPH Density/Proximity Subcontractor Summary of Results Survey Tool 4-E-10 Mapping To assess outcome, the number of tobacco retailers near schools will be compared before and after policy adoption. Utilize local data from the HSHC campaign and/or the California Community Health Assessment tool in collaboration with County Epidemiology staff to develop detailed local maps, by jurisdiction, related to tobacco retail density and proximity to schools and parks, and by population and selected characteristics, such as income. The results will be shared with elected officials during presentations and in educational packets. Update maps at least twice over the course of the workplan. 01/18-06/21 Evaluator FCDPH Developed Maps Report Summarizing Findings 4-E-11 Key Informant Interview Prior to policy adoption, TPP staff and/or Density/Proximity Subcontractor will conduct 5-8 key informant interviews (15-30 minutes in length) about density/proximity issues with school staff and/or administration, the County Office of Education, parent groups, and other groups/agencies with close ties to the schools. Interview questions will be developed in consultation with TCEC and/or other 01/19-12/19 Density/Proximity Subcontractor Evaluator FCDPH Interview Questions Summary of Key Findings REVISED Exhibit A Evaluator Scope of Work programs working on similar issues. Questions will assess whether key decision makers connected to the school system are concerned about the tobacco retailers near their schools and whether they are interested in addressing the issue. Interviews will be conducted in a conversational tone and either face-to-face or over the phone. The Evaluator will create an interview protocol and will review the protocol with TPP staff/Subcontractor to ensure high quality data is collected. Qualitative analysis of interview results will be used to summarize and report interview findings. The results will help support and guide campaign interventions. 5-E-1 Observation Survey A pre-observation survey of smoking incidents and tobacco litter will be conducted at the eight targeted facilities (purposive sample) during the months prior to policy adoption. At each of these facilities that adopt a policy, a post-observation survey will be conducted during the months immediately following policy implementation. Results from the pre- and post-observations will be compared to assess any changes after policy adoption. Based on any existing instruments from other programs and TCEC, the Evaluator and TPP staff will develop an observation form and survey protocol. The form will include the name of the facility, the time and duration of the observation, the number of smokers, and evidence that smoking is occurring (i.e., butt litter, ashtrays, and smell). Descriptive statistics such as percentages and frequencies will be used to analyze the results, and pre- and post-surveys will be compared. Results will be shared with the facilities to improve implementation efforts and will inform the Final Evaluation Report (FER). A summary of key findings will be submitted in the fourth year. 07/18-06/21 Evaluator FCDPH Summary of Key Findings Survey Instrument 5-E-3 Education/ Participant Survey Following the completion of the cessation facilitator training program, participants will be surveyed to assess their knowledge about tobacco cessation and their readiness to implement the curriculum. The instrument will be based on the material from the curriculum as well as samples collected from other projects. A census sample will be used, and 10-20 participants will complete the survey either pen-to-paper and/or online. The survey results will be analyzed using descriptive statistics such as percentages and frequencies. 01/20 – 06/21 Evaluator FCDPH Summary of Key Findings Survey Instrument 5-E-4 Key Informant Interview To determine which intervention strategies were successful, TPP staff will conduct 6-8 post-intervention interviews with site administrators, supervisors, and/or key staff members of the targeted facilities. Questions will identify factors that facilitated or impeded policy adoption and/or implementation, observed changes in the level of tobacco use, and the respondents’ knowledge and beliefs. Interviews (15-30 minutes long) will be conducted face-to-face and/or over the phone, and a purposive sample will be used. Interview questions and 01/20-06/21 Evaluator FCDPH Summary of Key Findings Survey Instrument REVISED Exhibit A Evaluator Scope of Work protocol will be adapted from any existing materials from other programs and TCEC, and the Evaluator will review the protocol with TPP staff. A content analysis will be used to analyze the results, and the results will provide qualitative information for the FER. 5-E-6 Media Activity Record TPP staff will monitor and track earned media placement through online news, area and local radio, area and local television, and/or hard-copy community newspapers, including details of media placement and coverage. Using the paid media tracking form, paid media will be similarly tracked. A content analysis will be conducted to evaluate the earned media pieces’ effectiveness at delivering/disseminating the message, their accuracy and neutrality of facts, and to gauge the level of public support for campaign strategies. Review and update if necessary the media tracking tool that was used during the previous plan and was based on the Tobacco Control Evaluation Center's (TCEC) sample. 07/18-06/21 External Evaluator FCDPH Earned Media Tracking Form Paid Media Tracking Form 5-E-8 Public Policy Record Baseline data will be collected as part of activity 5-1-1 to establish any existing smoke-free policies. After policy adoption, TPP staff will collect written copies of the new smoke-free policies from each facility. The provisions of the adopted policy will be compared to the model policy and to the baseline data. A content analysis will be conducted to identify strengths and weakness of the adopted language. Results will help determine if the objective was met. 01/18-06/21 Evaluator FCDPH Content Analysis Summary Directory From 5- 1-1 Signed Policies 5-E-9 Data Analysis Plan In the FER, all data sources will be summarized and will discuss which intervention strategies were successful and why, challenges faced and how they were addressed, and recommendations for future efforts. The post-training survey and observation surveys will be analyzed using descriptive statistics such as frequencies and percentages, and a content analysis will be used to analyze the policy records, media tracking forms, and key informant interviews. A comparison of the observation surveys will determine if the amount of smoking occurring at the targeted facilities changes after policy adoption. The post- training survey will determine the usefulness and effectiveness of the cessation facilitator trainings and whether additional training is needed. Pre-key informant interviews will help guide intervention strategies, and post-key informant interviews will help assess perceived barriers and benefits of policies after the intervention. The policy record review will identify strengths and weakness of the adopted language and whether the objective was met. Finally, the media tracking form will help determine the extent of news and media coverage and assess whether public sentiment is leaning positively or negatively towards SHS issues. An interim evaluation report will be written during the 7/19 to 12/19 progress report period. 07/19-06/21 Evaluator FCDPH Final Evaluation Report Interim Report REVISED Exhibit A Evaluator Scope of Work 6-E-1 Focus Group With assistance from TPP staff, the Smoke-free MUH Subcontractor will conduct 2 focus groups (1-1.5 hours in length) with residents of MUH complexes to discover how exposure to secondhand smoke (SHS) and thirdhand smoke has impacted their families. The results will help guide campaign strategies. Each focus group will include a purposive sample of 6-8 people, and at least one of the focus groups will recruit participants from housing complexes serving residents with low social economic status. The Evaluator and Project Director will develop a protocol and questions for the focus group based on any existing instruments and/or in consultation with Tobacco Control Evaluation Center (TCEC) and TECC. Each focus group will be audio- taped and transcribed, and an observer of the group will take notes of participant responses. A content analysis will be conducted to identify key themes. Results will be shared with elected officials and other community leaders and will help advance the objective by illustrating how SHS impacts families living in MUH. A possible challenge is recruiting residents to participate in the survey. To address this possible challenge, staff will work with MUH complexes where they have successfully conducted outreach and surveys in the past. 07/19-06/20 Evaluator FCDPH Smoke-free Multi-unit Housing Subcontractor Focus Group Question Summary of Key Findings 6-E-2 Public Policy Record A public policy meeting observation form will be developed to document policymaker support/opposition, key issues raised by policymakers and staff, and other insights from public policy meetings. TPP staff will observe and record this information during any public policy the meetings where the policy is discussed. In addition, official policy records such as city council agendas, meeting minutes, presentations, testimony, and policies enacted will be collected and coded to track: (1) support/opposition; (2) key issues that surface during the intervention that could help shape changes to the intervention plan during the process; (3) the policy adoption process so that it can be described in the brief evaluation report (BER); and (4) to confirm adoption of a policy that meets the objective for this project. Documentation of the signed policy will also be included. 07/19-06/21 Evaluator FCDPH Public Policy Meeting Observation Form Signed Policy 6-E-3 Final Evaluation Report In the BER, all data sources will be summarized following the guidelines specified by CTCP and will discuss which intervention strategies were successful and why, challenges faced and how they were addressed, and recommendations for future efforts. Descriptive statistics such as frequencies and percentages will be used to analyze the public intercept survey, and results will be shared with policy makers and other community leaders. A content analysis will be used to analyze the focus groups, earned and paid media tracking forms, and the public policy meeting observation form. The results from the focus groups will help advance the objective by illustrating to 01/21-06/21 Evaluator FCDPH Smoke-free Multi-unit Housing Subcontractor Brief Evaluation Report REVISED Exhibit A Evaluator Scope of Work community leaders how SHS impacts families living in MUH. The results from the meeting observation form will help identify factors that facilitated or impeded policy adoption. The media tracking forms will help determine the extent of news and media coverage and assess whether public sentiment is leaning positively or negatively towards SHS and thirdhand smoke issues. A summary of the report will be submitted to Partners and shared with Fresno County Tobacco-Free Coalition members, community members, and other programs who may want to tackle a similar objective 6-E-4 Media Activity Record TPP staff will track and review earned media placement through online news, area and local radio, area and local television, and/or hard-copy community newspapers, including details of media placement, data, and coverage. A content analysis will be conducted to evaluate the earned media pieces’ effectiveness at delivering/disseminating the message, their accuracy and neutrality of facts, and to gauge the level of public support for campaign strategies. Paid media will be similarly tracked using the paid media tracking form to determine coverage and placement. 01/18-06/21 Evaluator FCDPH Earned Media Tracking Form Paid Media Tracking Form 6-E-5 Public Intercept Survey/Opinion Poll To help support intervention efforts, a public intercept survey will be conducted with 140-190 MUH residents living in one of the targeted jurisdictions. A public intercept survey has already been completed in the City of Fowler, and an additional 15-20 surveys will be collected. Key questions will assess respondents’ exposure to SHS in their homes, their level of support for policy change, and demographic characteristics. A screening question will ensure only MUH residents living in the targeted jurisdiction are surveyed. The instrument that was developed during the previous contract period will be updated and translated into Spanish. Using a convenience sample, surveys will be collected in English and Spanish at housing complexes, community events, shopping centers, and/or other sites within the targeted jurisdiction. The Evaluator will conduct the data analysis utilizing frequencies and percentages to document support for policy strategies. These results will guide educational efforts and will be shared with community partners, policy makers, and the general public. 07/19-06/21 Evaluator FCDPH Smoke-free Multi-unit Housing Subcontractor Summary of Results Survey Tool Administrative Activity Description Timeline Responsible Party Deliverables REVISED Exhibit A Evaluator Scope of Work 7.1.1 Meet or talk with TPP staff monthly and/or as needed to discuss contract activities. Ongoing Evaluator FCDPH Meeting notes 7.1.2 Attend training sessions, conferences, conference calls, and webinars on specific content areas. Ongoing Evaluator FCDPH Copies of training agendas 7.1.3 Work collaboratively with FCDPH and California Tobacco Control Program to develop the Evaluation Plan. Update as needed. Ongoing Evaluator FCDPH Evaluation Plan and Plan Updates 7.1.4 Work collaboratively with Community Action Model Subcontractors in the implementation of the Evaluation Plan including phone calls, meetings with subcontractors for planning, training and direction of data collection activities by staff and volunteers. Ongoing Evaluator FCDPH Subcontractors Meeting agendas, Developed materials 7.1.5 Collaborate in the development and refinement of at least three success stories that can be used for reports and public communication/media purposes. Due to FCDPH 04/30/21 Evaluator FCDPH Developed stories 7.1.6 Work with FCDPH to develop submit progress bi-annual and evaluation reports (including the Interim Evaluation Report for Objective 3) following guidelines from FCDPH. Collaborate with FCDPH in the development of report deliverables. Submit electronic documentation of deliverables. Due to FCDPH 60 days before progress report is due Year 1 and 2 reports are due to CTCP 01/31 and 07/31 Year 3 reports are due to CTCP 01/29 and 06/30 Evaluator FCDPH Semi-annual progress report with supporting documents 7.1.7 Work with FCDPH to develop and submit Brief Final Evaluation Reports upon completion of Objectives 2, 3 and 6 and Long Final Evaluation Reports for Objective 4 and 5. Due to FCDPH 04/30/21 Reports are due to the CTCP 06/30/21 Evaluator FCDPH Developed reports 7.1.8 Submit monthly invoices to DPH within 30 days of the end of the previous month. Ongoing Evaluator Monthly invoices REVISED Exhibit A Evaluator Scope of Work Additional Scope of Work Activities Timeline: July 2019-June 2020 1. Conducting an additional 5-8 key informant interviews with schools/parents 2. Reviewing 11 Community Action Model (CAM) guides 3. Four additional CAM post-trainings 4. Complete policy records/reports for 2 reporting periods 5. Review 4 instruments developed by CAM subcontractors 6. Attend 2 days at the tobacco conference REVISED EXHIBIT B Tobacco Prevention Program Evaluator Budget Year 1 (12/11/18 - 6/30/19) Personnel Hours Rate Total Cost Evaluator 82.5 $95.00 $7,837.50 Benefits (i.e. health, dental, and vision costs; FICA –Social Security and Medicare ) Provided in-kind Subtotal $7,838 Materials/Supplies Unit Unit Cost Total Cost Office supplies (i.e. general office supplies, computer-related supplies such as laser printer/copier cartridges, etc.) Provided in-kind Subtotal $0 Travel Unit Unit Cost Total Cost Mileage, conference registration, hotels, etc. To be determined at budget revision Subtotal TBD Other Costs Unit Unit Cost Total Cost Communications (i.e. internet & phone provider) Provided in-kind Equipment (i.e. computer, printer, etc.) Provided in-kind Commercial general liability insurance Provided in-kind Subtotal $0 Indirect Total Cost 5% of Personnel costs (includes: administrative expenses such as overhead, accounting, utilities, building and equipment maintenance, etc.) $391.88 Subtotal $392 Year 1 Total $8,230 REVISED EXHIBIT B Tobacco Prevention Program Evaluator Budget Year 2 (7/1/19 - 6/30/20) Personnel Hours Rate Total Cost Evaluator 836 $100.00 $86,300 Benefits (i.e. health, dental, and vision costs; FICA –Social Security and Medicare ) Provided in-kind Subtotal $86,300.00 Materials/Supplies Unit Unit Cost Total Cost Office supplies (i.e. general office supplies, computer-related supplies such as laser printer/copier cartridges, etc.) Provided in-kind Subtotal $0 Travel Unit Unit Cost Total Cost Mileage, conference registration, hotels, etc. $1,500 Subtotal $1,500.00 Other Costs Unit Unit Cost Total Cost Communications (i.e. internet & phone provider) Provided in-kind Equipment (i.e. computer, printer, etc.) Provided in-kind Commercial general liability insurance Provided in-kind Subtotal $0 Indirect Total Cost 5% of Personnel costs (includes: administrative expenses such as overhead, accounting, utilities, building and equipment maintenance, etc.) $4,316 Subtotal $4,316.00 Year 2 Total $92,116 REVISED EXHIBIT B Tobacco Prevention Program Evaluator Budget Year 3 (7/1/20 - 6/30/21) Personnel Hours Rate Total Cost Evaluator 504.00 $100.00 $50,400 Benefits (i.e. health, dental, and vision costs; FICA –Social Security and Medicare ) Provided in-kind Subtotal $50,400 Materials/Supplies Unit Unit Cost Total Cost Office supplies (i.e. general office supplies, computer-related supplies such as laser printer/copier cartridges, etc.) Provided in-kind Subtotal $0 Travel Unit Unit Cost Total Cost Mileage, conference registration, hotels, etc. To be determined at budget revision Subtotal TBD Other Costs Unit Unit Cost Total Cost Communications (i.e. Internet & phone provider) Provided in-kind Equipment (i.e. computer, printer, etc.) Provided in-kind Commercial general liability insurance Provided in-kind Subtotal $0 Indirect Total Cost 5% of Personnel costs (includes: administrative expenses such as overhead, accounting, utilities, building and equipment maintenance, etc.) $2,520 Subtotal $2,520 Year 3 Total $52,920