Community Development Evaluation Framework

 

The Community Development Framework is a new approach to community development across the City of Ottawa.

Goals of the CDF are:

Goal #1: Increase neighbourhood capacity to enact positive change (includes collective efficacy/social cohesion and resources)
Goal #2:  Improved planning and service delivery to achieve neighbourhood defined goals by increasing collaboration and coordination between and within city services, community agencies and their partners and residents
Goal #3:  Improve health of neighbourhoods (includes the social determinants of health)
Goal #4:  Improve neighbourhood safety and perceptions of safety in an inclusive manner.
Goal #5:  Promote sustainability of positive change at the neighbourhood and systems levels

The Evaluation Framework provides a standardized, integrated system with clear goals that will provide quantitative and qualitative evidence on whether the CDF is achieving its goals, at all levels. Reporting on the evaluation measures will require additional funding, but this is essential in order to have the evaluations stand up to careful scrutiny. Consistent reporting will enable all CDF stakeholders to assess the progress of this initiative, provide ongoing feedback and implement necessary changes. 

Methods used in developing the framework.

This evaluation framework is based on a literature review.  In particular, we focused on: the New Deal for Communities Evaluation (Lawless, 2005), Wolff’s work on evaluating community collaborations (Wolff,  2003), Action for Neighbourhood Change (United Way, Toronto), and the Portland Performance Measurement System (PPMS).

We also had discussions and e-mail conversations with KTT table members and city staff, conducted an expert survey (through e-mail and interviews) and drew data from the Ottawa Neighbourhood Study.

Evaluation Framework components

The evaluation framework comprises six elements:

  • An evaluation matrix
  •  Neighbourhood level tools:
  • annual neighbourhood survey
  • community developer template
  • community developer tools

Systems level tools:

  • annual systems survey
  • internal City of Ottawa staff survey
  • community developer template
  • interviews with key stakeholders

Additional outcome measurements:

  • Ottawa Neighbourhood Study
  • Other sources
    Annual reflection sessions at each CDF table
    Celebration and reporting forums every two years

Description of Evaluation Framework Components

1. Evaluation Matrix (Appendix A)

This tool outlines the CDF goals, and for each, which outcome measures will be used, who will administer them, and how often.

2. Neighbourhood level tools

a) Neighbourhood surveys (Appendix C)
This survey questionnaire is designed to elicit the reflections and views of people who live in the neighbourhood, and will be used as one important driver in project development.  The survey will help measure whether the CDF is actually working as well as function as a planning and discussion tool. The survey comprises both qualitative and quantitative questions. It covers basic demographics, and measures the social determinants of health.
Surveys will be supplemented by focus groups to ensure that the voices of the most marginalised groups are heard.
b) Community Developer template
Community developers are in an excellent position for data collection through their existing contacts in neighbourhoods and their daily interactions with residents. A template has been developed, in partnership with community developers and the Knowledge Transfer Table, which will simplify the data collection process.

c) Community Developer tools
Community developers use a variety of tools to engage with and mobilise residents. These will be supplemented by tools developed by No Communities Left Behind (NCLB) which guide community developers in the development, implementation and evaluation of a neighbourhood planning process.  These include:

  • Asset Resource Inventory
  • Neighbourhood Action Plans
  • Annual assessments of the Neighbourhood Action Plans
  • Photovoice


3.  Systems level tools

a) Systems Surveys (Appendix B)
All CDF tables and each neighbourhood steering committee will be surveyed on an annual basis in order to get participants' views on how CDF implementation is going, and what needs to be improved. 

b) Internal City of Ottawa staff survey
This survey will measure coordination and collaboration between City of Ottawa staff at all levels. (Still needs to be developed)

c) Community Developer template
Community developers are in a unique position to track whether systems level supports are supporting neighbourhood efforts. The community developer template will include space to comment on whether and how neighbourhood initiatives have been supported by systems level supports.

d) Interviews with key stakeholders
Key stakeholder interviews will enable the CDF team to gather important qualitative information. 

4. Outcome measurements

a) Ottawa Neighbourhood Study (Indicators in Appendix D)
This study has been used to identify initial CDF neighbourhoods, and will be used to assess the effectiveness of the Community Development Framework. New data will be added to this study and all neighbourhood profiles updated every 5 years to coincide with Statscan data

b) Other sources
Other sources for outcome measurements include Ottawa Police Data, United Way (Success by Six), City of Ottawa records, School Boards data, the Annual Citizen Satisfaction survey, Ottawa Public Health, Ottawa Community Housing, the City of Ottawa's Neighbourhood Planning Initiative, private sector partner sources and other sources.

5.  Annual Reflection Session (Appendix E)

An annual reflection session at each systems level and neighbourhood table allows all CDF stakeholders to reflect upon the progress of their work as a group. Allowing time and space for this oral evaluation can help ground each table in their strengths and weaknesses, as well as revisit the overall goals and vision

This reflection session can be self-facilitated and will focus on process measures. Suggested tools are taken from:

  • Diagnosing your Coalition:  Risk Factors for Participation (Kaye 1993).  Worksheet 2 in Wolff (2003).  In Appendix D
  • Climate Diagnostic Tool.  The six Rs of participation (based on Kaye and Resnick, 1994). Worksheet Four from Wolff (2003). 

6.  Celebration and Reporting Forum:

Held every two years, this City-led forum will provide all CDF stakeholders an opportunity to come together, share their success stories and get to know one another. It will showcase efforts by CDF neighbourhoods as well as their respective Community Resource Centres.

Celebrating success is a key pillar for maintaining momentum in collaborative efforts. Data collected up to that point can be presented orally by residents, community developers and City of Ottawa staff.

 

© 2008-09 City of Ottawa and Partner Agencies

Collaboration, Coordination, Community Participation and Leveraging of Resources