Terry Amsler is program director of the Institute for Local Government’s Collaborative Governance Initiative and can be reached at tamsler@ca-ilg.org. For more information about the Collaborative Governance Initiative, visit www.ca-ilg.org/cgi.

You’ve devoted time, money and staff energy to a public involvement effort about an important plan, issue or policy. But did the process make any difference? Did the local agency benefit? Were the participants satisfied with their experience? What worked and what didn’t? And what lessons will your city or county take from this experience into future civic participation activities?

Each local effort to develop and implement a public involvement process is unique. Each issue or policy you address is different, including the number or demographics of those participating, the specific means through which participants generate ideas and recommendations, and the way local officials ultimately use the results in decision-making. The history, conditions and dynamics of each community setting also differ.

Regardless of the approach, local agencies that take the time to assess their civic engagement activities are more likely to learn from the experience and improve their future efforts.

Looking at Outcomes

Local officials can use a minimum of four outcomes to assess the success of their agency’s efforts to involve the 
public, including:

1. The appropriateness and effectiveness of the public involvement process design and implementation, including the participants’ satisfaction with the process. Did the chosen process or approach fit the problem, and was it done well?
2. The real impacts on public decisions, policies and actions. Were the ultimate decisions different — and better — than would otherwise have been the case?
3. The effect on the community’s capacity for democratic participation. Has the public involvement process made it more or less likely that the necessary information, skills and willingness to get involved are present in the community?
4. How, if at all, has a particular public participation effort enhanced a local agency’s ability to effectively sustain and support civic engagement? Was the public engagement process considered a one-time affair, or have sponsors used it to build a more sustained capacity for soliciting the public’s ideas and recommendations?

Most formal studies by academics and others seem to focus on the first category; fewer focus on the second; still fewer on the third; and the fourth often receives very little attention. Relatively few local agencies appear to formally review their public engagement in a systemic way. Rigorous evaluations of more complex and longer-term outcomes can take significant time and resources.

However, civic participation sponsors and organizers can ask basic but crucial questions that provide important feedback to assess a specific activity and improve future efforts. While the result may not be a rigorous scientific study, the information is useful, especially if the results are shared broadly within the local agency.

Key Questions to Ask

While these questions are broadly applicable, local agencies should choose those that are most appropriate and helpful. These questions are most relevant to assessing stand-alone civic engagement events that seek the public’s ideas and recommendations.

Was a comprehensive plan in place? Did appropriate local officials develop and support a clear public involvement plan that included a stated purpose, ties to city vision or goals, participation targets, a process design, a timeline, clear staff roles, a budget and how local officials would integrate any developed recommendations into their ultimate decision-making?

Did participation meet your goals? Was the actual participation appropriate to the issue? Was the participant selection process effective? Did local officials make successful efforts to involve the community’s diverse population? What worked and what didn’t to help secure the participation you intended?

Was the process appropriate for participants? Was background information provided to participants so they were prepared to take part? Were materials used in the process helpful? Were there sufficient opportunities for deliberation among participants that allowed for the exchange of informed views, consideration of alternatives and the formulation of recommendations? Was the process appropriate for the nature of the input and the degree of specificity you were seeking? Were participants’ language capacities taken into consideration? If a consultant or facilitator was used, did he or she provide a safe and well-managed environment for people to participate effectively?

Did discussions lead to action? Did public officials demonstrably consider the ideas or recommendations resulting from the public involvement process in their final decision-making? Did the public involvement process result in local officials making a more informed and/or better decision? Was there greater support for the resulting new policy or action? Did participants receive feedback on how their recommendations were or were not used?

Were participants satisfied? Did participants view the public involvement process as transparent, well-managed and appropriate to the issue(s) under consideration? Did they believe their input was welcomed, heard and considered? Would they be more or less likely to participate in other such processes in the future?

Was communication effective? Was information about the public involvement process and its outcomes communicated effectively to the broader public?

Did the process enhance capacity? Did the public engagement process provide residents with additional skills, knowledge and experiences likely to encourage their role as committed and effective community members? How can the local agency continue to draw on and develop these capacities?

What are the key lessons? Has the review of the public engagement effort involved the community and other sectors? What are the most important lessons identified, and how can the local agency apply them to its future public involvement efforts? And will these lessons be shared across those city or county offices that often engage the public?

While not exhaustive, grappling with these questions helps to ensure that individual civic engagement efforts are indeed reviewed and that the local agency develops a shared and growing body of knowledge about public involvement. These efforts result in more strategic and effective civic participation over time.

Broader Measurements and Results

A handful of cities are taking steps to build assessment and organizational learning into broader civic engagement efforts.

Colin Gallagher, neighborhood services coordinator for the City of Salinas, reports that his office is in the process of developing performance indicators and a budget for assessing its outreach and civic engagement efforts. The performance planning work is being carried out in collaboration with city residents and within the work plan set for neighborhood services by the city council. The City of Salinas will publish measurement results on its website. Gallagher explains that, to date, the city has assessed individual public meetings 
using such questions as:

• Was there a clear purpose for the meeting?
• Did the purpose tie into the city council goals?
• Was contact information generated as a result of the meeting?
• How many were contacted and consequently attended?
• Was an actionable item developed as a result of the meeting?

Anne Hallock, Ventura’s civic engagement manager, says that the city now measures several civic participation factors annually in a statistically significant survey. This includes a measurement of residents’ satisfaction with the city’s communication with the public, volunteerism rates and residents’ awareness of local government activities. Qualitatively, she suggests that the results of Ventura’s civic engagement are reflected in the rejection or acceptance of public decisions.

Cherise Brandell, Menlo Park’s civic engagement manager, notes that informal feedback systems have been used in the past, but the city is currently developing a more consistent method for evaluating individual activities and overall efforts. These measures assess the outcomes and indicators identified in a community engagement program “logic model” that lays out specific strategies, short- and long-term desired outcomes and indicators of success.

These local developments reflect a growing movement among cities and counties to better understand the effectiveness and outcomes of their civic engagement efforts. In 2009, the Institute for Local Government will prepare a short guide to help local agencies assess civic engagement. Contact Terry Amsler, program director, at tamsler@ca-ilg.org with information about your assessment plans and experiences that can be shared with other local officials throughout California.

This article appears in the December 2008 issue of Western City
Did you like what you read here? Subscribe to Western City