In 2022, one of Zencity’s partner communities in Virginia considered constructing a roundabout as a traffic calming measure. Before making the decision, the city organized a town hall meeting to present the plan to residents and get their feedback. The room was packed with residents lining the walls, clamoring for the microphone, and raising their voices in an attempt to be heard. Clearly, there was no shortage of feedback on the roundabout proposal. The city manager and councilmembers were delighted at the turnout and the level of engagement; who knew residents are so passionate about roundabouts? (Just kidding, we knew.)

When it came time to go over the meeting notes and analyze the feedback provided, the city realized that they had little insight into where the community actually stood on this issue; were they more in favor or against the roundabout? This was puzzling, given the participation numbers. How was it possible that with so many people attending and speaking up, there was so little actionable data?

A packed room, hundreds of comments on a city post, thousands of survey responses, a city inbox crammed with resident emails; these appear like hallmarks of successful community engagement efforts, glistening with the promise of rich resident feedback.

However, participation is only one limited parameter of effective and impactful engagement. Sure, the community showed up, but participation levels alone fail to capture the quality of the feedback provided.

What then, aside from sheer participation numbers, can help us evaluate the effectiveness and quality of your community engagement efforts? Here are four parameters you should consider when evaluating a community engagement action: depth, audience, representativeness, and effort—or DARE.

Depth

What can the feedback teach you and what can it not?

The first parameter refers to the depth of input or level of detail the local government can get out of the data gathered in the engagement process. Feedback comes in many forms: a “like,” a “retweet,” a meme or a GIF, a long well-reasoned comment detailing reasons for support or objection, a 1-5 scale score on a survey question. Clearly, each one of these provides a different level of insight.

A successful and effective community engagement effort will generate the depth of feedback that is most useful and relevant to you in this particular case: will an average score calculated based on basic-binary data be sufficient, or do you require rich discourse that is generated in social media discussions or in-person meetings to reach a decision?

Remember the town hall meeting concerning the roundabout? The city wanted to get a sense of support versus opposition, but ended up with long tirades about roundabouts and general arguments about road safety, traffic, and of course, potholes. This was not invaluable, but it was not the depth of data the city was after. In fact, in this specific case, a survey or a poll sent out to residents would have provided the desired level of data the city needed to make its decision.

Therefore, ask yourself, “is the data I am likely to get through this engagement effort helpful and appropriately nuanced to support decision-making?” If you need a bottom line, an overall score outlining support or satisfaction, yet you have pages upon pages of resident commentary about their perceptions, preferences, and concerns, then this was not an effective community engagement.

Audience

Who are the crucial stakeholders? Whose feedback is essential?

In one Texas partner community, the city wanted to understand the resident sentiment around the possible recall of one councilmember. The city posted on its social media and encouraged residents to voice their opinions on the matter, but the leading comment was that this was not an issue for all residents to decide on, but for the residents of the councilmember’s district. “Are you making sure you’re listening to the feedback coming from this specific community?” commenters asked the city. “That is your audience.”

The second parameter examines the audience that participates in the engagement process. It can be a large or a small group, with a mix or a homogeneous representation of stakeholders. However, the crucial factor here is that the audience targeted and included in the engagement effort is relevant to the issue at hand. If this is an issue pertaining to a specific area in the city, the audience has to include residents from that area. For example, if it is an issue affecting communities of color, they are a critical stakeholder and the engagement must be primarily targeted toward them, even at the expense of participation levels.

When you set out to evaluate this parameter, ask yourself whether the audience accurately and fairly reflected the most relevant stakeholders and if the feedback came from the right people.

Representativeness

Whose voices are heard and whose voices are marginalized?

Representativeness refers to the question of whether the data shared in the engagement process is representative of the community, either statistically and/or in terms of equity and inclusion. This is a critical question irrespective of the issue at hand, as within every target audience you are likely to have a diverse range of community members, with different perceptions, opinions, and concerns.

An effective and successful community engagement effort will be designed to ensure that the feedback collected reflects a diverse range of voices, either within the general population or a specific group of residents.

Ask yourself: “Is the feedback I collected accurately representative of my community and am I giving an equal voice to all community members?” There are several ways to achieve representativeness: a statistically representative survey, engagement methods in different languages, and distribution and promotion across multiple channels. It is critical to create the necessary infrastructure to ensure that people of all age groups, backgrounds, socio-economic circumstances, racial and ethnic groups, gender groups, and faiths are able to participate and provide input.

Effort

How difficult (or easy) is it for residents to provide feedback?

This is the easiest parameter to evaluate; it measures the level of effort participants had to invest to voice their opinion and provide their feedback. Did they have to find a babysitter, get in their car, drive to the town hall, and wait in line to say their piece? Or did they share their feedback by phone while walking their dog? Obviously, lower effort usually means fewer hurdles and therefore results in more feedback from a more diverse set of voices. This includes people who have less-flexible schedules or can’t physically come to the meeting, and whose voices, as a result, are heard less often or become discounted.

By reducing the amount of effort required by the participant, the engagement process is more accessible and makes participation easier for harder-to-reach, marginalized, or vulnerable communities.

Overall, the aim is to lower barriers to participation and make the process as effortless as possible. How? Make sure you offer multiple ways to provide feedback that suits diverse levels of lifestyles, preferences, resources, and capabilities.

Conclusion

By now, you may have realized that community engagement efforts that meet all four of the parameters discussed are difficult to create. Representativeness can be best achieved in a statistically representative survey, but it might mean compromising on the depth of the feedback. Ensuring that you have the appropriate audience might require more effort from participants.

Do not despair. The most effective community engagement effort is one that is designed with the four parameters in mind in a way that answers your specific objectives, but is not necessarily perfect on all four. A variety of community engagement solutions is the best guarantee for a flexible and adaptable engagement strategy that can emphasize each parameter when it is most needed.

And herein lies the twist. These four parameters serve a dual purpose. Yes, they are a helpful prism through which to evaluate the engagement after the fact, but, more importantly, they are a crucial framework with which to design community engagement activities before they are implemented.

This four-parameter approach to designing and evaluating enables local government organizations to identify areas for improvement and design more effective and inclusive engagement processes that reach a wider audience and gather diverse, and actionable, insights from their constituents.

The names that appear in images and social accounts are fabricated for the purpose of an example.

NOAM RABINOVICH is the head of content strategy at Zencity, where she conducts high-level data analysis and research into overarching trends across cities and counties, such as public safety, homelessness, economic issues, and public health.

LEE FELDMAN, ICMA-CM (retired), is senior advisor for local government strategy at Zencity.

Zencity is a New York- and Tel Aviv-based govtech company that serves more than 300 local government agencies in the United States, Canada, Australia, the United Kingdom, and Israel. Founded in 2016, Zencity works with mayors, police chiefs, city managers, and other public sector leaders to build trust between governments and the people that they serve by improving responsiveness.