Aligning Public Participation to Stakeholders’ Sustainability Literacy—A Case Study on Sustainable Urban Development in Phoenix, Arizona

Abstract

In public planning processes for sustainable urban development, planners and experts often face the challenge of engaging a public that is not familiar with sustainability principles or does not subscribe to sustainability values. Although there are calls to build the public’s sustainability literacy through social learning, such efforts require sufficient time and other resources that are not always available. Alternatively, public participation processes may be realigned with the sustainability literacy the participants possess, and their capacity can modestly be built during the engagement. Asking what tools might successfully align public participation with participants’ sustainability literacy, this article describes and evaluates a public participation process in Phoenix, Arizona, in which researchers, in collaboration with city planners, facilitated sustainability conversations as part of an urban development process. The tool employed for Visually Enhanced Sustainability Conversation (VESC) was specifically designed to better align public participation with stakeholders’ sustainability literacy. We tested and evaluated VESC through interviews with participants, city planners, and members of the research team, as well as an analysis of project reports. We found that the use of VESC successfully facilitated discussions on pertinent sustainability issues and embedded sustainability objectives into the project reports. We close with recommendations for strengthening tools like VESC for future public engagements.

1. Introduction

Both sustainability science and urban planning literature identify public participation as an important decision making procedure [1,2,3,4,5]. Yet, high-quality public participation is not always achieved. Some challenges include overly dominant government agencies, participants who lack capacities to engage, and inequitable distributions of resources [6]. Particularly, in planning for sustainable urban development, public participation requires not only engaged participants, but also participants that are literate in sustainability principles, norms, and behaviors [7,8].

Engaging with a public that lacks sustainability literacy is a significant challenge for planners and experts working towards sustainable urban development. Cohen and Wiek [9] identify ten ways in which the public participation process (as designed by planners and experts) is often misaligned with the local context. Participants’ sustainability literacy is one key misalignment that can impair the quality of public conversations on urban sustainability and ultimately compromise the robustness of sustainability outcomes in project planning and implementation.

Obstacles to high-quality public participation, including low sustainability literacy, can often be avoided with careful process design [10,11]. However, the body of planning literature on public participation is weak in providing empirically informed directives for designing good processes. The majority of published papers on public participation are theory-based, and there are far fewer examples of empirical studies [12]. The magnitude of empirical research on the topic lags behind both the theoretical and practitioner communities [13,11], due in part to a lack of evaluative studies on public participation and stakeholder engagement processes [14,15,16].

Therefore, this article describes and evaluates a case in which researchers, in collaboration with city planners, attempted to align the public participation process to stakeholders’ sustainability literacy in a sustainable urban development process in Phoenix, Arizona. The project team encountered low sustainability literacy amongst participants and redesigned engagement activities to strengthen sustainability conversations at public visioning workshops. This study asks how public participation processes might be structured to cope with low sustainability literacy, with the objective of generating evidence-supported directives for designing public participation for sustainable urban development.

2. Public Participation Processes in Urban Development Projects

2.1. Defining Public Participation

“Public participation” may describe a variety of processes applied across diverse contexts [10]. For the purposes of this study, we define public participation as a process that is part of an official urban development project. It is situated within the urban development project, which itself is positioned within a specific context. Strategic Agents (i.e., elected officials and investors) supervise the process, which is carried out by Operating Agents (i.e., city staff and project partners), and Participating Stakeholders (i.e., residents, non-profits, businesses) provide input through a structured process. Stakeholders may be involved throughout multiple project phases, including preparing, planning, implementing, and evaluating project outcomes. Stakeholders may be asked to engage through different methods, including public meetings, focus groups, workshops, citizen juries, and other protocols [9,17].

2.2. Misalignments between Public Participation Process and Local Context

The literature on public participation features theoretical debates about whether participation yields positive outcomes or instead produces adverse effects. In planning literature, little empirical evidence supports either argument [12], yet some scholars contend that most shortcomings of participation processes can be mitigated through careful process design [10,11].

To avoid challenges to high-quality public participation, one must first identify these pitfalls. Through a broad review of the literature on public participation in urban development, Cohen and Wiek [9] found common challenges to public participation and organized these issues into ten categories, conceptualized as misalignments between the public participation process and the local context, including policy maker support, community civic engagement, and stakeholders’ collaborative capacity, among others. These common challenges to public participation in urban development occur when the public participation process (as designed by experts and planners) does not align with the local context where participation is taking place.

This article is concerned with one specific misalignment between the public participation process and the local context: Stakeholders’ sustainability literacy. Generating robust sustainability outcomes through public participation processes requires stakeholders to engage in sustainability-oriented conversations. Many members of the public, however, lack a strong grounding in sustainability principles, and their values and behaviors may be in conflict with sustainability. Meaningful engagement around sustainability is challenging when participants are not, in this sense, “literate” in sustainability.

When a public participation process is not aligned with stakeholders’ sustainability literacy, there is a knowledge and/or values gap on sustainability between experts and stakeholders. When this problem persists, participants may feel confused, they may harbor frustrations or distrust, and their input may be incompatible with sustainability goals and objectives [9]. This is not to imply that sustainability-literate participants would guarantee a consensus around sustainability outcomes. Rather, a public participation process aligned to participants’ sustainability literacy might yield constructive dialogue, seek compromise, find common understanding, and enable robust sustainability-oriented outcomes to influence subsequent policy decisions.

2.3. Approaches for Aligning Public Participation Processes to Participants’ Sustainability Literacy

Building stakeholders’ capacity for sustainability can strengthen the quality of public participation in urban sustainability governance [7]. Sustainability competence has been defined as a bundle of knowledge, attitudes, and skills about sustainability [18], and we adopt this comprehensive perspective on sustainability competence as the ideal for high sustainability capacity. Stakeholders’ capacity for sustainability can be developed through social learning, i.e., experiences in which participants build understanding and shape their values through collaboration with others. Social learning is a common potential benefit of public participation, and there are many cases of facilitating social learning of sustainability [19,20,21]. Although social learning is popular in sustainability governance literature, it is important to note, that this form of capacity building requires a significant investment of time and other resources that may not be available in all participatory processes [20]. In an empirical study of social learning in public participation for sustainability outcomes, Garmendia and Stagl [22], for example, found that participants need ample time for interaction and deliberation.

As an alternative to enhancing sustainability literacy through separate capacity building events, the planning process itself can be designed as a learning experience, using guidelines from the literature. In the following, we review a set of guidelines for aligning public participation processes to participants’ sustainability literacy.

Innes and Booher [6] identify keys to successful public participation, one of which is dialogue. Dialogue can be transformative because participants who listen to and inform each other can develop new ideas and shared meanings. They recommend that public agencies promote procedures that foster deliberation between stakeholders and that planners be trained to design and manage collaborative processes.

Other scholars repeat this recommendation. Through case studies of participatory processes, Fung and Wright [23] synthesize key principles of empowered participatory governance. They state that procedures should lead dialogue beyond an abstract discourse on values and instead focus on conversations about practical issues and concrete problems. People affected by the problems being discussed should be given an opportunity to deliberate solutions to the problems. Healey [24] discusses creative urban governance as an alternative to the “established routines (and) rule-bound bureaucratic procedures (p. 88)” that typify planning processes. Under creative urban governance, processes would support informative conversations that are facilitated through experimental practices (cf. [5]).

Rowe and Frewer [16] propose four criteria for evaluating a public participation procedure: Criterion of resource accessibility, criterion of task definition, criterion of structured decision making, and criterion of cost-effectiveness. Two of these criterion—resource accessibility and structured decision making are particularly helpful for guiding the construction of participatory procedures to foster social learning and structure participant interaction and deliberation. The first vital criterion, resource accessibility, stipulates the resources that participants need to make a decision. This includes (a) information resources, or the facts needed to make an informed decision; (b) human resources, or access to experts and other individuals that can provide needed information; and (c) material resources, or objects like projectors or whiteboards to facilitate understanding. These resources would all be important for communicating sustainability concepts, problems, and solutions to diverse participants.

The second vital criterion, structured decision making, states that activities should follow clear mechanisms for facilitating decision making. Coping with low participant sustainability literacy in participatory procedures would lend to a need for competent facilitation. Subsequent works [15,25] question whether this criterion should be further broken into more specific criteria, and Rowe et al. [25] consider including assessments of adequate and fair elicitation as well as information presentation.

One way to present information is through the use of images. In a study on the use of imagery for public engagement on climate change, O’Neill et al. [26] found that images of climate change impacts made participants feel that climate change was important (salience), but reduced their feeling that they could do something to address the problem (self-efficacy). On the other hand, images of energy futures increased participant efficacy. While images of the problem reduced participant self-efficacy, images of potential solutions awoke in participants a sense that they could tackle the challenge. This finding would support Fung and Wright’s assertion that participants should be engaged on the topic of solutions, which is doubly relevant to sustainability planning, as sustainability science is framed as a solution-oriented endeavor [27]. The framing of issues significantly impacts participant input [22,28,29], and in the case of sustainability, these examples from the literature indicate that such topics should be framed as solutions to real problems.

In summary, a public participation procedure to support sustainable urban development may be effective if it fosters deliberation about real problems and their potential solutions, and if it supports social learning. Due to common time constraints of participatory planning processes, we focus particularly on information resources and presentation as well as facilitation to enable conversations about sustainability outcomes amongst stakeholders who may not be comfortable or familiar with sustainability. Therefore, facilitators need to be specially trained to lead deliberations on sustainability supported by materials like visuals to aid participant understanding.

3. Research Methods: Evaluation of a Public Participation Procedure

This study was conducted on a participatory visioning process for an urban development project in Phoenix, Arizona. The authors were members of a research team that designed and implemented a participatory process to elicit stakeholder input for sustainability visions for districts along the City of Phoenix’s light rail corridor. Through one-on-one interviews, surveys, community organization meetings, and public workshops engaging nearly 300 participants in the first district (Gateway District), researchers observed a challenge of facilitating discussions around sustainability with stakeholders who had little background on the subject. The researchers identified stakeholders’ low sustainability literacy as a barrier to quality public participation and redesigned workshop activities and materials for use in other transit districts (including the Midtown District, which serves as the reference district in this study).

This article evaluates the tool of Visually Enhanced Sustainability Conversation (VESC) that was designed to better facilitate deliberation on sustainability options during the public visioning process. We evaluate VESC using select criteria from Rowe and Frewer [16] that would support participant deliberation over sustainability issues (resource accessibility, including information resources, human resources, and material resources; and structured decision making, including elicitation and information presentation), and we apply a fifth criterion that the activity must facilitate public discussion on sustainability. The ultimate research question is whether or not the application of VESC effectively facilitated conversations and decision-making about sustainability options.

To assess VESC in terms of the above criteria, the evaluation includes researchers’ direct observations; document analysis of vision reports from two of the transit districts (Gateway and Midtown); and 11 interviews with participant stakeholders, project partners, and members of the research team. Table 1 outlines the data inputs used in the evaluation.

After public participation concluded in each district, a steering committee of stakeholders was formed to support the visions towards implementation. Steering Committee members were co-selected by researchers and staff from the City of Phoenix Neighborhood Services Department and the Planning and Development Department. Selection criteria emphasized a diversity of perspectives, including anchor institutions (i.e., hospitals, universities, and religious institutions), businesses and business coalitions, K-12 schools, developers, landowners, neighborhood associations, and residents. Special attention was given to underrepresented populations, including refugee communities, recent immigrants, public housing residents, and American Indians. Those recommended were contacted and interviewed by researchers to determine their availability and interest in serving on the committees. Over a span of 18–24 months, Steering Committees reviewed and edited planning documents, recommended potential investments (i.e., support for road improvements or development of business incubators), and supported grants for infrastructure projects.

Table 1. Data inputs for evaluating visually enhanced sustainability conversations.

Data Type:

Direct observations

Document Analysis:

District Vision Report Midtown District Vision Report

Interviews:

Gateway District Steering Committee Member #1, interviewed 6 January 2015 Gateway District Steering Committee Member #2, interviewed 9 January 2015 Gateway District Steering Committee Member #3, interviewed 14 January 2015 Midtown District Steering Committee Member #1, interviewed 5 December 2014 Midtown District Steering Committee Member #2, interviewed 11 December 2014 Midtown District Steering Committee Member #3, interviewed 12 December 2014 Midtown District Steering Committee Member #4, interviewed 14 December 2014 Midtown District Steering Committee Member #5, interviewed 19 December 2014 City of Phoenix Planner, interviewed 17 December 2014 Research Team Member #1, interviewed 30 January 2015 Research Team Member #2, interviewed 3 February 2015

Table 1. Data inputs for evaluating visually enhanced sustainability conversations.

Data Type:

Direct observations

Document Analysis:

District Vision Report Midtown District Vision Report

Interviews:

Gateway District Steering Committee Member #1, interviewed 6 January 2015 Gateway District Steering Committee Member #2, interviewed 9 January 2015 Gateway District Steering Committee Member #3, interviewed 14 January 2015 Midtown District Steering Committee Member #1, interviewed 5 December 2014 Midtown District Steering Committee Member #2, interviewed 11 December 2014 Midtown District Steering Committee Member #3, interviewed 12 December 2014 Midtown District Steering Committee Member #4, interviewed 14 December 2014 Midtown District Steering Committee Member #5, interviewed 19 December 2014 City of Phoenix Planner, interviewed 17 December 2014 Research Team Member #1, interviewed 30 January 2015 Research Team Member #2, interviewed 3 February 2015

For this study, researchers identified steering committee members as ideal respondents because they were involved at public engagements during the visioning process, they hold a big-picture perspective of development goals in their districts, and they were selected to be representative of significant stakeholder groups in their districts. Researchers conducted interviews with three Gateway Steering Committee members and five Midtown Steering Committee members to compare the workshop experiences in the two districts. A City of Phoenix planner was interviewed to gain insight from a project partner. The planner also recommended the steering committee members to be interviewed for this evaluation. Finally, to reduce bias in the evaluation, interviews with two members of the research team provide feedback from individuals that helped design the VESC tool and facilitate public participation, but are not authors of this article. The interviews with the research team members are particularly valuable because both respondents are now practicing urban planners in major metropolitan areas in the U.S.

Respondents participated in semi-structured interviews. A researcher met with the respondent, reviewed copies of workshop activity posters from both districts and VESC posters from the Midtown district. In each interview, respondents compared the experiences from the two districts and provided feedback on the tools that were used. After each interview, responses were coded by evaluative criteria, and the researcher assessed whether feedback was negative, ambivalent, or positive. This approach relies to some extent on the researchers’ own judgments (coding and assessment), but respondent quotes provide rich details that support the assessment decisions.

4. Reinvent Phoenix: Aligning Public Participation Process to Stakeholders’ Sustainability Literacy

4.1. The Reinvent Phoenix Participatory Visioning Process

Reinvent Phoenix was an urban development project in Phoenix, Arizona. Funded by the U.S. Department of Housing and Urban Development, Reinvent Phoenix was a partnership between the City of Phoenix, Arizona State University, St. Luke’s Health Initiative, and other community organizations. The project sought to promote transit-oriented and sustainable urban development along Phoenix’s light rail corridor. This goal was to be achieved over multiple phases that included a public participation process to develop sustainability visions for five specific transit districts: Gateway, Eastlake-Garfield, Midtown, Uptown, and Solano (Figure 1). The visions would then inform a zoning process to create form-based codes that support transit-oriented and sustainable urban development.

Figure 1. Reinvent Phoenix project map.

Figure 1. Reinvent Phoenix project map.

The authors and their research team managed the public participation process to create the sustainability visions for each district. This article focuses on a particular aspect of the visioning processes in the Gateway and Midtown Districts. The Gateway District is the farthest east district of Phoenix’ light rail corridor. It is one of the most ethnically diverse transit districts, and it features the corridor’s highest poverty rates and lowest educational levels. The Midtown District, by contrast, is the most affluent of Phoenix’s transit districts. Comparing participation between these two districts is challenging as socioeconomic, educational, and other factors significantly impact participation [30]. However, Gateway’s visioning process provides a convenient baseline against which to measure the effectiveness of VESC as this tool was not designed until after public participation in Gateway had concluded.

A pilot process was conducted in the Gateway District from September–December 2012. During this period, researchers engaged with stakeholders, who live, work, do business in, or visit the district, through one-on-one interviews, community organization meetings, two public mapping forums, and two public visioning workshops. Through the engagements leading to the visioning workshops, participants identified areas they would like to see preserved or changed, and they discussed the types of changes they would like to see occur. Researchers identified consensus areas for change (transition areas) and prepared a visioning workshop to enable participants to discuss in detail how each of the transition areas might look in the future.

Accordingly, researchers engaged workshop participants in discussions about specific changes for the identified transition areas in the Gateway District. These conversations revolved around a Visual Preference Survey (VPS) in which participants discussed and voted on preferences for issues including building height, street design, and landscaping. The VPS facilitated form-based discussions that focused primarily on the district’s physical form. Sustainability outcomes were tacitly included in certain options. For instance, taller building heights promoted density, complete street designs fostered multi-modal transportation options, and low-water landscaping would use fewer natural resources. Sustainability was not explicitly addressed in these conversations, and function-oriented conversations (i.e., how participants would live and work in the buildings; how participants would travel on streets) occurred organically, if at all.

At the end of the public participation process in the Gateway District, researchers drafted a vision based strictly on stakeholder input. After the report was complete, one member of the research team conducted a criteria-based sustainability appraisal of the Gateway vision. The appraisal showed that the initial vision lacked sustainability substance [31]. Throughout the process, the researchers felt that public discussions about sustainability outcomes could have been stronger and that more targeted discussions would have further infused sustainability into the vision.

As the Gateway District was the first transit district in which visioning activities occurred, researchers had an opportunity to take lessons from that experience and revise the approach for subsequent districts. Identifying low sustainability literacy of participants and seeking to strengthen sustainability conversations at visioning workshops, the researchers devised new workshop activities and materials for facilitating public discussions. The Midtown District is one of the districts in which the revised participation procedures were implemented.

The Midtown visioning process occurred January–May 2013. Researchers employed a slightly altered process for identifying transition areas, including one-on-one interviews, forums at existing neighborhood meetings, and tabling at community events. At the visioning workshops, researchers again led participants through a VPS activity. After concluding the VPS, researchers facilitated a new activity, titled Visually Enhanced Sustainability Conversation (VESC).

4.2. Designing the Visually Enhanced Sustainability Conversation

The intention of VESC was to facilitate a public discussion to prioritize sustainability objectives and identify means (vision elements) for achieving these objectives that would be acceptable to stakeholders. To foster deliberation towards sustainability outcomes, researchers pre-selected the objectives and vision elements prior to the visioning event. Figure 2 shows the hierarchy of a sustainability vision, from the most general component (guiding principle) to the most specific (vision element).

Figure 2. Hierarchy of a sustainability vision.

Figure 2. Hierarchy of a sustainability vision.

In the case of Reinvent Phoenix, researchers populated this sustainability vision hierarchy through a transparent process. Guiding sustainability principles were adapted from Gibson’s widely accepted synthesized set of six criteria [32]. For each sustainability principle, neighborhood-specific objectives were identified through intensive public engagements via one-on-one stakeholder interviews, surveys, community organization meetings, and public forums [33]. Researchers then identified criteria-based elements that could be operationalized to achieve the neighborhood-specific objectives and the guiding sustainability principles. Researchers then vetted the objectives and vision elements with project partners in the City of Phoenix Planning Department. Table 2 outlines the objectives and their related vision elements for each of Midtown’s three transition areas.

Researchers sought local and regional examples of each vision element in order to capture visual evidence of the element’s use in the Phoenix area. In some cases, local examples did not exist, and researchers attempted to locate images as relevant to the Phoenix context as possible.

Although the content of the VESC tool is critical, it is not the only success factor for a participatory vision. The facilitators themselves are equally as important as the facilitation tools. In preparation for public visioning workshops, members of the research team were paired as facilitators and note-takers. Facilitators and note-takers underwent multiple trainings. Through a series of dry runs, the facilitators and note-takers practiced the visioning activity that revolved around VESC. The facilitator played a focal role leading the exercise, and note-takers played a support role by recording all conversation onto a laptop, joining the group discussion when the facilitator couldn’t manage multiple conversations, and organizing support materials like pens, markers, and post-it notes. In some cases, when group dynamic dictated, the note-taker became a secondary facilitator.

Table 2. Sustainability objectives and vision elements by transition area.

Transition Area	Objective	Vision Element 1	Vision Element 2	Vision Element 3
Park Central Mall	Economic vitality through strong local businesses	Buy-local initiative	Small business support organization	Business in mixed-use building
	Diverse employment and training opportunities	Co-working spaces	University-community partnership	Participant suggestions
	Cool neighborhoods	Cool pavement	Vegetation	Living roof
	Walkable and bikeable neighborhoods	Neighborhood circulator	Pedestrian malls and promenades	Park-and-ride
Central Avenue Corridor	Diverse employment and training opportunities	Co-working spaces	University-community partnership	Participant suggestions
	Cool neighborhoods	Cool pavement	Living roof	Vegetation
	Walkable and bikeable neighborhoods	Neighborhood circulator	Pedestrian malls and promenades	Participant suggestions
	Saving money through conserving natural resources	Adaptive reuse	Energy efficient homes	Solar houses
Third Street Corridor	Economic vitality through strong local businesses	Business incubator	Small business support organization	Buy-local initiative
	Cool neighborhoods	Cool pavement	Living roof	Vegetation
	Walkable and bikeable neighborhoods	Neighborhood circulator	Pedestrian malls and promenades	Participant suggestions
	Saving money through conserving natural resources	Adaptive reuse	Solar houses	Energy efficient homes

Table 2. Sustainability objectives and vision elements by transition area.

Transition Area	Objective	Vision Element 1	Vision Element 2	Vision Element 3
Park Central Mall	Economic vitality through strong local businesses	Buy-local initiative	Small business support organization	Business in mixed-use building
	Diverse employment and training opportunities	Co-working spaces	University-community partnership	Participant suggestions
	Cool neighborhoods	Cool pavement	Vegetation	Living roof
	Walkable and bikeable neighborhoods	Neighborhood circulator	Pedestrian malls and promenades	Park-and-ride
Central Avenue Corridor	Diverse employment and training opportunities	Co-working spaces	University-community partnership	Participant suggestions
	Cool neighborhoods	Cool pavement	Living roof	Vegetation
	Walkable and bikeable neighborhoods	Neighborhood circulator	Pedestrian malls and promenades	Participant suggestions
	Saving money through conserving natural resources	Adaptive reuse	Energy efficient homes	Solar houses
Third Street Corridor	Economic vitality through strong local businesses	Business incubator	Small business support organization	Buy-local initiative
	Cool neighborhoods	Cool pavement	Living roof	Vegetation
	Walkable and bikeable neighborhoods	Neighborhood circulator	Pedestrian malls and promenades	Participant suggestions
	Saving money through conserving natural resources	Adaptive reuse	Solar houses	Energy efficient homes

4.3. Facilitating the Visually Enhanced Sustainability Conversation

During the visioning activities, the facilitator employed the VESC tool by first showing participants a poster stating a small number (3–6) of sustainability objectives to discuss. Participants voted on the objectives that most interested or resonated with them (top 1–3). This voting served as an input for planners on how to prioritize objectives in the planning and implementation process of the Reinvent Phoenix project. The facilitator then led structured conversations about the most popular objectives, sharing a poster for each objective that illustrated potential vision elements for achieving the objective (Figure 3).

The poster for each objective listed two to three potential vision elements, and some posters prompted participants to suggest additional elements. For each vision element, there was a short description and a photograph providing visual representation. For issues involving building height and street design, we used photo-realistic visuals that depicted the vision elements (i.e., 5–8 story buildings) as they would actually look in a given location. The facilitator described each vision element and fielded questions. The facilitator then guided a pros/cons conversation in which participants provided strengths for each vision element as well as potential obstacles to successful implementation. The facilitator noted participant responses on sticky notes and placed these inputs on the poster.

Participant input was inserted directly into the Midtown District vision. The vision for each transition area was organized around the sustainability objectives that gained the greatest stakeholder interest. The specific vision elements that participants supported were included in the report, and any nuances in participant preferences were addressed. For instance, in discussing rooftop photovoltaics (strategy: Solar houses) as a vision element for saving money through conserving natural resources along the Central Avenue Corridor, one participant noted that he was not comfortable with photovoltaic panels being visible from the street in the historic neighborhoods near Central Avenue. Through deliberation, he acknowledged that photovoltaic panels on historic homes were acceptable provided they were visible only from backyards. The note-taker recorded this request, and the vision stipulated that photovoltaic installations should not compromise historic character in such neighborhoods [34].

Figure 3. Example poster from a Visually Enhanced Sustainability Conversation.

Figure 3. Example poster from a Visually Enhanced Sustainability Conversation.

Because the vision was oriented around pre-selected sustainability objectives, researchers conducted the criteria-based sustainability appraisal during the design of VESC. As described in Section 4.2, sustainability scientists vetted potential vision elements prior to the visioning workshops (based on objectives that aligned criteria identified by stakeholders and expert-based sustainability criteria), ensuring that the public discussions revolved around sustainability-oriented outcomes. A cursory sustainability appraisal of the vision reinforced that the vision does indeed describe a sustainable Midtown [34].

5. Evaluating the Alignment between Participatory Visioning and Participants’ Sustainability Literacy in Reinvent Phoenix

The central focus of this article is to determine if Visually Enhanced Sustainability Conversation (VESC) aligned Reinvent Phoenix’s participatory process to stakeholders’ sustainability literacy. We therefore evaluate the tool in terms of how it facilitated participant conversations about sustainability. Respondents provided predominantly positive feedback on VESC for each of the evaluation criteria, and Table 3 shows which inputs (i.e., Steering Committee members, research team members, direct observations) yielded positive or negative assessments for each criterion.

Table 3. Results of evaluation of visually enhanced sustainability conversations.

Evaluation Criteria	Assessment
	Negative	Positive
Information Resources	2 Midtown Steering Committee members	2 Gateway Steering Committee members 2 Midtown Steering Committee members 1 research team member
Human Resources	1 Direct observation	1 Midtown Steering Committee member 1 research team member 1 Direct observation
Material Resources		1 Gateway Steering Committee member 1 Research team member
Elicitation	1 Research team member	City of Phoenix planner 2 Gateway Steering Committee members 5 Midtown Steering Committee members 1 Research team member 1 Direct observation
Information presentation	City of Phoenix planner 1 Gateway Steering Committee member 2 Midtown Steering Committee members	3 Gateway Steering Committee members 3 Midtown Steering Committee members 1 Research team member
Sustainability Discussions		City of Phoenix planner 2 Gateway Steering Committee members 5 Midtown Steering Committee members 2 Research team members 1 Direct observation Midtown Vision Report [34]

Table 3. Results of evaluation of visually enhanced sustainability conversations.

Evaluation Criteria	Assessment
	Negative	Positive
Information Resources	2 Midtown Steering Committee members	2 Gateway Steering Committee members 2 Midtown Steering Committee members 1 research team member
Human Resources	1 Direct observation	1 Midtown Steering Committee member 1 research team member 1 Direct observation
Material Resources		1 Gateway Steering Committee member 1 Research team member
Elicitation	1 Research team member	City of Phoenix planner 2 Gateway Steering Committee members 5 Midtown Steering Committee members 1 Research team member 1 Direct observation
Information presentation	City of Phoenix planner 1 Gateway Steering Committee member 2 Midtown Steering Committee members	3 Gateway Steering Committee members 3 Midtown Steering Committee members 1 Research team member
Sustainability Discussions		City of Phoenix planner 2 Gateway Steering Committee members 5 Midtown Steering Committee members 2 Research team members 1 Direct observation Midtown Vision Report [34]

In general, respondents regarded VESC as a useful tool for facilitating public discussions on sustainability objectives and vision elements. Respondents from Midtown reported that VESC did support the goal of infusing sustainability into both discussions at workshops and the resulting vision. Respondents from Gateway reviewed the VESC materials and felt that the tool would have been helpful in their district as well. The City of Phoenix planner and members of the research team also acknowledged the benefits of VESC. Although respondents supported the use of VESC, some also provided critiques and made recommendations for improving the tool. The most salient remarks and observations are discussed below.

6. Discussion

This section discusses the results for each evaluation criterion and highlights key quotes from respondents that illustrate both the strengths and weaknesses of VESC as it was implemented in Reinvent Phoenix. The following codes are used to cite each evaluation source:

• Direct observations: DO
• Gateway District Vision Report: [31]
• Midtown District Vision Report: [34]
• Three Gateway Steering Committee members: GW01; GW02; GW03
• Five Midtown District Steering Committee members: MT01; MT02; MT03; MT04; MT05
• City of Phoenix Planner: CP01
• Two research team members: RT01; RT02

6.1. Information Resources

The VESC posters, as information resources, were intended to define sustainability-oriented development objectives and present details about potential vision elements for achieving those objectives. The goal here was to foster informative conversations [24]. One Gateway respondent noted, “Everybody needs a starting point to enter the discussion” (GW03). A member of the research team pointed out that by placing objectives and vision elements on the posters, the researchers were reassuring participants “that these options are sustainable” (RT01). By presenting to participants sustainable options to discuss with examples and details, VESC gave participants the starting point they needed to enter the discussion about sustainability options.

To improve VESC posters as information resources, a Midtown respondent recommended providing more examples for each objective and offering more local examples so that participants could understand the examples through a context with which they are familiar (MT02). Researchers attempted to provide local images of vision elements, but not all vision elements had been implemented locally. The research team included external examples so as not to limit the vision to only what had already been accomplished in Phoenix. O’Neill et al. [26] reported on the importance of images in facilitating public discussions on sustainability issues, and respondent feedback in this evaluation seems to support this assertion. One Midtown respondent was concerned that participants that were learning of a potential objective or vision element for the first time might walk away from the activity with an overly optimistic view if the group is unable to identify shortcoming and pitfalls (MT04). For a balanced perspective [35], the posters may need to include disadvantages and facilitators might need to be more transparent about the pros and cons of each option. To alleviate this concern, the VESC poster and facilitation included an opportunity to discuss shortcomings of each vision element. Ensuring then that participants have a firm grasp of each option depends on competent and transparent facilitation.

6.2. Human Resources

Rowe and Frewer [16] highlight the importance of human resources to a participatory exercise, and VESC involved two groups of people: (1) the facilitator and note-taker; and (2) the participating stakeholders. A member of the research team was very positive about the quality of facilitation and attributed facilitator competence to the training and preparation of facilitators prior to the event (RT02). Facilitators helped research the sustainability objectives and vision elements and therefore were familiar with each option. A respondent from the Midtown District also noted the human resource value of fellow participants, saying, “It was good to have people with different ideas that can consider something different than their own point of view. There were times that I changed my opinion when I heard other people’s ideas” (MT01). This form of social learning was fostered by the facilitated discussions about the benefits and obstacles for each vision element. Furthermore, this outcome from VESC supports the importance of dialogue and deliberation emphasized by Innes and Booher [6] and Fung and Wright [23].

6.3. Material Resources

During the interviews, respondents discussed the VESC posters as a mode for sharing information with participants. One member of the research team felt strongly that posters were resources that aided “people that didn’t necessarily have a literacy in sustainability or urban planning” (RT02). A Gateway Respondent said, “I think the materials are fine. I look at this and I see ‘here is a priority and here are three strategies to do that.’ I think that is great” (GW03). The objectives and vision elements offer solutions to sustainability problems in the districts, and the VESC materials focus deliberation on solutions, as recommended by Fung and Wright [23].

6.4. Elicitation

Interviews showed a favorable assessment of VESC as an elicitation tool. Respondents tended to like the structure of the activity, and they felt that VESC prompted discussion by first providing examples that participants could see and understand. One respondent noted, “If you give them tangible examples, then they can see themselves in it” (MT02). Another respondent liked the structure, saying “It might help with people that aren’t as knowledgeable, getting them on the right path” (MT04). The City of Phoenix Planner said VESC “helps because (…) you need to have directions so you can prompt them to think and then go with it. You initiate some conversations and then you elicit additional ideas because you prompted” (CP01). Contrasting VESC against the experience in Gateway, a member of the research team said, “In Gateway, it was hard to facilitate without the material to guide the conversations. We were less able to elicit responses because we didn’t have the tools to do that” (RT02). She felt the VESC tool solved this issue. These comments highlight VESC’s strength at structuring decision-making, emphasized by Rowe and Frewer [16], and facilitating informative conversations, promoted by Healey [24] and Zint and Wolske [5].

In critique of elicitation under VESC, one Gateway Respondent said, “I think you should always leave an opportunity for people to come up with ideas that you might not have already thought of” (GW03). Some objective posters presented two options (vision elements) and asked for additional ideas, while other posters presented three options and did not elicit additional input. It would be possible to design posters that always ask for additional ideas. A member of the research team also felt that while the structured conversation around pre-selected options was helpful, “there would be more value if (…) there could have been a more organic discussion about, for example, what sustainable land use looks like” (RT01). One solution to this concern might be to initiate the conversation with the objective/element discussion and then facilitate a bigger-picture discussion of general sustainability once the participants have been prepared by first discussing tangible examples. However, such a structure may not be reasonable if an event is facilitated under time constraints.

6.5. Information Presentation

O’Neill et al. [26] show that images of solutions inspire participants and increase participants’ self-efficacy. The VESC sought to accomplish similar goals by including images of potential vision elements for achieving the sustainability objectives. The images allowed participants to see themselves in the sustainable future (MT02). VESC posters would also help participants who were visual thinkers (GW02), and visuals help move “the discussion along quicker in terms of people comprehending what we’re comparing in terms of several options” (GW03). All five respondents that participated in VESC in Midtown District felt that the visuals improved the activity and fostered good conversation.

There were several issues that should be improved so that VESC can better present information. The City of Phoenix Planner felt that “the language was very planner wonky. Why can’t it just say “good job choices?” (instead of “diverse employment opportunities”) (CP01). Although there is credence to the respondent’s opinion, the language on each poster was negotiated between project partners, evaluated in pre-tests, and revised multiple times. Another critique focused on the quality of translation, because some of the Spanish language translation was considered inaccurate, making the activity confusing for Spanish speakers (GW01). This critique is especially troublesome because the Reinvent Phoenix research team included native Spanish speakers who translated the materials. Pre-testing VESC with Spanish speaking participants is one potential solution for checking translation quality. The presentation of vision options would also have been better if images depicted implementations local to the Phoenix area (RT01). The goal was to show local examples, but there was not always local evidence available for the selected vision elements.

6.6. Sustainability Discussions

Respondents overwhelmingly rated VESC positively as a tool for leading discussions about sustainability outcomes, and researcher observations support these conclusions. A member of the research team succinctly justified VESC and highlighted the activity’s outcomes: “Our mandate was from a grant which stipulated that the vision had to be something sustainable. We weren’t just talking about sustainability in general terms. Because the future has to be sustainable—what options would you support out of this pool of ideas? The VESC guided the conversation in a particular direction” (RT02). One respondent speculating on how VESC would have supported visioning in Gateway said, “I think you have to present some sustainable strategies and put those forth rather than work through 15 ideas people throw out that aren’t sustainable. And if it is an opportunity for folks to learn about sustainability by discussing strategies that are based on sustainability, it allows people to meaningfully engage” (GW03). A Midtown respondent said, “If you just give someone a question, their mind goes blanker than anything. These posters were a good way to start” (MT01). Another Midtown respondent praised VESC for supporting his own thought process, saying, “I’m thinking along these lines anyways. I might organize my ideas. But without these objectives, my ideas might not be so formalized” (MT04).

7. Limitations

The evaluation of VESC provides insights about designing tools for supporting public discussion about sustainability, but the evaluation did have some limitations that are discussed below.

The evaluator was a member of the research team: While there is opportunity for bias to cloud the evaluation, the authors’ role in the process afforded rich opportunities to collect direct observations of the design of visioning workshop, the VESC tool, workshop implementation, and participant experiences. Because the evaluation occurred up to two years after the public participation events, the authors were also able to distance themselves from the research project and approach the evaluation more objectively.

The evaluation occurred as much as two years after the public participation events: While this gap in time afforded greater objectivity, it might have slightly undermined the quality of participant reflection (cf. [36]). Some respondents clearly remembered the participation events and activities. Others were able to recall what occurred after brief conversations about the process. Others had trouble remembering specific conversations and themes. Ideally the research questions and evaluation would have been established prior to the visioning process in each district and conducted during and immediately after. Still, given the circumstances, the authors felt that there was value in collecting feedback to learn from the experience.

No formal assessment of stakeholders’ sustainability literacy: The researchers did not formally evaluate stakeholders’ sustainability literacy, and the decision to create tools to better align the engagement process with participants’ sustainability literacy was based on direct observations and a heuristic process. Were more time available, critically assessing participant capacities could have further informed the design of the engagement tools and procedures. Still, although there was no formal pre-testing of literacy, through an informal appraisal we found sustainability literacy was very low, and in Gateway, this experience was repeated at four separate tables at each of four public events. Facilitators at each table faced the same challenges in engaging participants on issues of sustainability. From this low level of observed literacy, researchers set a baseline from which to communicate through VESC.

Sustainability not publicly defined with participants: While neighborhood-specific sustainability objectives were derived through robust stakeholder engagement, guiding sustainability principles were drawn from the literature, and a shared conceptualization of sustainability was not co-created with participating stakeholders. In-depth discussions about what sustainability means to a community is a way to engage on a vision. Pragmatically, in this project researchers wanted visioning workshops to produce tangible outcomes so that the vision could produce implementations and experiments. In most cases, VESC fostered discussions that went beyond the preselected options, and the vision elements were often just a starting point, as noted above by respondents MT01 and MT04.

No interviews with participants outside of steering committee members: Because the evaluation took place long after public visioning concluded, the authors chose to interview steering committee members out of convenience. Steering committee members were easy to contact because they were still involved in Reinvent Phoenix. Steering committee members were also representative of certain constituents in each district, and the authors felt that their perspectives would be valuable, and that they could speak on behalf of other participants.

The evaluation lacks quantitative data: Prior to the evaluation, the authors did not establish clear metrics for what would be negative or positive assessments of each criterion. Respondent interviews do, however, provide rich details about VESC and the experience of participants at Reinvent Phoenix visioning workshops.

8. Conclusions

In Reinvent Phoenix, researchers experienced low sustainability literacy amongst participants. Researchers went from a model in which participants couldn’t engage with sustainability principles to a model under Visually Enhanced Sustainability Conversations (VESC) in which they could. VESC helped sustainability experts translate abstract, hard to understand sustainability principles into something that is tangible, down to earth, and reasonably easy to understand.

One of the goals of VESC was to align a public participation process with stakeholders’ sustainability literacy in order to improve sustainability-oriented discussions at public visioning workshops. To achieve this goal, the tool would have to provide participants with information resources and structure decision making [16]. Through an evaluation of VESC, the study investigated in how far the tool effectively facilitated public deliberation about sustainability outcomes and in how far VESC improved public discussion about sustainability compared to previous engagements.

In general, all respondents had favorable opinions of VESC. Midtown District respondents thought the activity was successful as they experienced it, and Gateway District respondents thought the exercise would have been beneficial to visioning in their own district. Respondents thought VESC was strong because it stimulated conversation. By seeding participants with example ideas, participants were then able to think more creatively. In terms of meeting sustainability goals, presenting vision elements that were already vetted as sustainable steered the conversation towards additional ideas that were more likely to lead to sustainability outcomes.

VESC did have some flaws. For instance, despite the presence of native Spanish speakers on the research team, the Spanish translations were not perfect, and some of the technical language should have been better translated. Terminology in English could have been simplified, yet all language was negotiated between project partners. More local examples of successful strategies could have inspired further support from participants, and additional images of each vision element could have made the options even more tangible. However, despite detailed research, few local examples of vision elements existed. These illustrate clear concerns regarding VESC, but the solutions for improving the tool are not simple.

There are some concerns regarding the evaluation of VESC, but the evaluation is transparent and provides a discussion of the tool’s strengths and weaknesses. The evaluation describes the tool and highlights the aspects to be replicated as well as aspects to improve upon. VESC is a tool that was created through academic work. This study describes the intention, design process, implementation, and outcome of the tool to support participatory researchers and planning professionals in utilizing similar engagement tools to align public participation processes to the local context. We fully acknowledge that there was no formal testing. As we are moving forward with VESC, it would be important to include a formal pre/post test.

Misalignments between the public participation process and local context, like low sustainability literacy of participating stakeholders, can undermine sustainability outcomes in public participation processes in urban development projects. Facilitation and deliberation tools can improve discussions amongst members of the public. This study offers insights from which planners and experts can learn when designing their own public participation activities and materials.