Measuring and Assessing Perceptions of Success in a Transit Agency's Stakeholder Involvement Program

Stakeholder involvement is often a legally required activity for transit agencies. Optimally this process begins at the earliest stages of any given project and involves two-way communication and interaction that can be reflected in the resulting plans or project designs. Despite federal statutes and regulations mandating meaningful public input for federal capital funding consideration, approaches to fulfill these mandates successfully remain difficult, and stakeholder involvement practices often fail to have a deep impact on planning outcomes. A gap often exists between implementation plans and the satisfaction of stakeholders. Guided by prior research and applying cognitive mapping methods, this study uses a rail line reconstruction project of the Chicago Transit Authority in Illinois as a case study to systematically understand the gap in expectations and interpretations of success among stakeholders. Findings identify four important categories of goal nonalignment that act as barriers to optimal interaction that can lead to mistrust. Conclusions identify potential solutions for bridging the gaps.

general policies into effective practice. At the same time, citizens and external stakeholder groups increasingly demand greater involvement and access to agency staff and are quick to voice their dissatisfaction with participation processes. As a result, stakeholder involvement practices often fail to have a deep impact on planning outcomes.
Overall, a substantial gap in understanding exists between (a) public agency implementation of legal mandates to integrate stakeholder needs in decision making while simultaneously meeting project and agency goals and (b) satisfaction of external groups that the agency has sufficiently understood, considered, and included their perspectives. This paper is an attempt to better understand this conceptual gap by systematically assessing internal and external stakeholder perceptions of the success of a recent stakeholder involvement program for a rail line rehabilitation project at the Chicago Transit Authority (CTA) in Illinois. This case study research responds to the need to better understand the theoretical bases for this conceptual gap and to provide stakeholder involvement practitioners with the knowledge and understanding to better anticipate and frame expectations of stakeholders during involvement processes, and more appropriately balance the costs of engagement with the benefits of improved outcomes.

LITERATURE REVIEW FINDINGS
The stakeholder literature makes clear that there are both legal requirements and clear benefits to the involvement of stakeholders in public planning and decision-making processes. In addition, the literature highlights key areas in which expectations of both agency and external stakeholder groups may not align.

Legal Requirements for Stakeholder Involvement in Transportation Planning
Innes (3) points out that stakeholder participation is something planners and public officials must do because it is legally required. The last two decades have witnessed the passage of several federal mandates, which call for active involvement of the public in the transportation planning process. These include the Intermodal Surface Transportation Efficiency Act of 1991; the Transportation Equity Act for the 21st Century (1998); and the Safe, Accountable, Flexible, Efficient Transportation Equity Act: A Legacy for Users (2005).
Enforcement of these mandates depends upon federal action; however, agencies are required to inform and gather comments from stakeholders (4). Stakeholder participation is widely interpreted as involvement in decision making with the purpose of gathering input

Measuring and Assessing Perceptions of Success in a Transit Agency's Stakeholder Involvement Program
Meghan Mattingly, P. S. Sriraj, Eric W. Welch, and Barbara (Basia) Bhojraj 89 Stakeholder involvement is often a legally required activity for transit agencies. Optimally this process begins at the earliest stages of any given project and involves two-way communication and interaction that can be reflected in the resulting plans or project designs. Despite federal statutes and regulations mandating meaningful public input for federal capital funding consideration, approaches to fulfill these mandates successfully remain difficult, and stakeholder involvement practices often fail to have a deep impact on planning outcomes. A gap often exists between implementation plans and the satisfaction of stakeholders.

Guided by prior research and applying cognitive mapping methods, this study uses a rail line reconstruction project of the Chicago Transit Authority in Illinois as a case study to systematically understand the gap in expectations and interpretations of success among stakeholders. Findings identify four important categories of goal nonalignment that act as barriers to optimal interaction that can lead to mistrust. Conclusions identify potential solutions for bridging the gaps.
Public agencies are increasingly required to involve stakeholders in their planning and decision-making processes. Many transit agencies invest significant resources in terms of staff, time, and money to realize their stakeholder involvement efforts. This causes them to hold practical reservations about stakeholder involvement programs, believing they increase costs, create delay, open the door to emotional considerations and self-interest, and ignite controversy rather than consensus (1). Despite these reservations, stakeholder involvement is often a legally required activity that affects multiple departments within a transit organization.
Current federal statutes and regulations mandate meaningful public input as a quid pro quo for federal capital funding consideration. However, approaches to successfully fulfill these mandates remain immensely complex. Szyliowicz (2) argues that despite rapidly proliferating literature, practitioners engaged in public involvement can find only limited guidance in dealing with difficulties often encountered in the implementation of public involvement programs. Current federal statutes and regulations fall short of providing specific parameters on conducting meaningful and effective stakeholder involvement programs. Federal guidelines also assume practitioners possess the time, resources, and skills needed to appropriately translate these to influence the choices being made (5), whereby input is the key phrase that differentiates participation methods from other communications strategies (6). Input includes procedures designed to consult, involve, and inform the public, and to provide those affected by a decision with the opportunity to have a voice in the process.

Benefits of Successful Stakeholder Involvement
The complexity of the involvement process means that meaningful public input in transportation projects is often limited or absent. Szyliowicz (2) argues despite general consensus that extensive public involvement is desirable and despite significant legislation few public involvement efforts have had an impact. Prevost (4) cites several studies showing that citizens perceive the process of informing and gathering comments allows agencies to adopt a one-way communication policy and ignore more complex interaction.
The resource constraints that public agencies face often compel them to fulfill only the basic requirements for compliance with legal mandates, even though the benefits to meaningful stakeholder involvement include • Enhancing the effectiveness and quality of agency plans and decisions, • Increasing public acceptance of plans through use of local knowledge, • Building the public's trust in the agency, and • Assuring cost-effective decision making.

Enhancing Effectiveness and Quality of Agency Plans and Decisions
A generally holistic approach toward stakeholder involvement within an agency is more likely to enhance the success of project outcomes than an ad hoc, reactionary approach. A holistic approach integrates public involvement in the planning process rather than treating it as a necessary add-on. As O'Connor et al. (7 ) note, a twoway approach to communication allows transit agency stakeholders to develop a better understanding of agency operations, and agency officials have a better understanding of public thinking. Informing the public continuously and involving them in the early planning stages can legitimize the project and minimize negative interactions. For instance, when project setbacks occur, an informed public that understands the project more thoroughly can better appreciate the cause for delay. A well-informed and engaged citizenry can help overcome barriers to implementation created by the sound-bite media culture (8) because stakeholders have a sense of ownership of planning proposals (1,7).

Increasing Public Acceptance of Plans Through Use of Local Knowledge
An often-cited purpose to include the public in plan making is that citizens possess ordinary knowledge that can help ensure that policies and plans reflect local conditions and values (1). Only a local constituency knows what is best for its community. Reliance on local knowledge to formulate plans and policies not only provides citizens with a voice in planning and decision making but also improves plans, decisions, and service delivery by incorporating citizens' local knowledge into the final outcome (3,9,10). Through citizen participation programs, policies and plans become more realistically grounded in citizen preferences. This serves to enhance the overall effectiveness of government and implementation of major projects by improving the fit of programs to citizen needs and by encouraging citizen assistance in operating programs (1,11). When issues do not attract the interest of potential stakeholders, planners do not benefit from this local knowledge, and moreover, the policies they propose may seem irrelevant to those they are supposed to benefit (1).
A more practical and perhaps more compelling reason to include the public in plan making is that through stakeholder participation programs, public officials are more likely to learn about potential opposition to their proposals early on and can find ways to avoid potential policy failures (1,8). Public agencies can benefit greatly from robust public involvement processes by learning about the problems facing communities and various stakeholders, gaining the opportunity to adapt planning goals to local circumstances, and enhancing political feasibility of plans and policies by adopting designs that reflect local values and knowledge (1).

Building Public's Trust in the Agency
Early, two-way communication allows trust to develop, rather than the perpetual skepticism and antagonism that typically results from one-way communication, whereby decisions are announced to the public and consequently defended. From the perspective of public agencies, one purpose of involving citizens in planning and other governmental processes is to increase the trust and confidence of the citizenry in government (9). Moreover, some assert that the need for stakeholder involvement programs stems from the practical recognition that implementing unpopular policies may result in widespread protest and reduced trust in governing bodies (6).

Assuring Cost-Effective Decision Making
Public choice theory argues that people choose not to participate in the policy process if the personal costs of participating outweigh the benefits. For example, when people see that their concerns or issues are taken seriously, they will continue to participate, and perhaps more importantly, if participants see their input leading to results, they will be more willing to participate a second time (12). Overlooking the concerns of the local community, especially during the early stages of the planning process, can lead to critical and incredibly costly mistakes to the agency (7,13). Citizen dissatisfaction with the public involvement process can negatively affect their support for the project, the agency, and even the service provided by the agency. Finally, strong stakeholder participation is assumed to be a cost-effective planning paradigm because it reduces the probability of litigation (8).

Differences in Expectations of Public Agencies and External Stakeholders
According to Rowe and Frewer (6), public agencies often assume that public involvement is an end, rather than a means to an end. Moreover, agency goals for stakeholder involvement and the definition of "successful involvement" are often not clearly articulated, even though these elements guide the selection of participation activities and techniques.
Most stakeholder participation practitioners reject the notion of a linear relationship between increased participation and increased effectiveness (11). Instead, careful attention needs to be paid to which specific participatory techniques are appropriate for the attainment of such varied objectives as informing citizens, educating citizens, generating support among citizens, or utilizing input in decision making. In practice, participatory techniques are often selected prior to the identification of the desired objectives. When the relationship between objectives and techniques is ignored, the probability of a successful outcome decreases (9).
Literature reveals that stakeholder satisfaction with involvement depends on two types of criteria: (a) process criteria and (b) acceptance criteria (6). Acceptance criteria relate to the effective construction and implementation of a procedure, while process criteria relate to the potential public acceptance of a procedure.
Acceptance criteria include representativeness, early involvement, influence, and transparency. Participants should be representatives of the broader public rather than simply some self-selected subset. Consensus exists in the literature that public participation should happen as soon as reasonably practical. While it might not be practical to have the public participate in highly technical issues, consultation is important for value judgments. The outcomes of stakeholder participation processes should have an impact on policy and be perceived to do so. A main complaint is that participation methods are often used as a mechanism to legitimize predetermined decisions or give the appearance of consultation (6). Such symbolism may result in civic distrust of public agencies. Transparency may reduce tendencies toward mistrust. Clarity of decision parameters and processes may reduce public suspicions about the agency.
Process criteria include resource accessibility, task definition, structured decision making, and cost effectiveness. Information resources, human resources, and time resources all affect the quality of participation processes. The nature and scope of participation should be clearly defined and the appropriate resources should be available to meet the task. The degree of involvement desirable in making a decision depends on the attributes of the core problem (11). Therefore, stakeholder involvement techniques should be chosen in the context of the overall goals and objectives of a public outreach program (10) and the choice of the public involvement method(s) will depend on the needs of a particular circumstance.
If disconnect exists in either the process or in acceptance, it tends to result in a gap in the expectation and in the strength of the message. Therefore, two propositions are posited: P1. Involvement exercises for transit planning result in gaps in expectations and understanding between the agency and stakeholders. These gaps can lead to mistrust among the different parties.
P2. Gaps in expectations are attributable to non-attainment of acceptance criteria, process criteria, or both. As such, it is possible to suggest solutions aimed at criteria attainment that reduce gaps and increase trust.
To explore whether the propositions are supported empirically, a case study approach was adapted, and the CTA's Brown Line Rehabilitation Project was used as the focus. To organize the study, each of the four benefits of stakeholder involvement identified above was explored as a basis for comparison of internal and external stakeholder perceptions.

Case Study: Brown Line Rehabilitation Project
CTA is the transit provider in the city of Chicago and parts of suburban Cook County in Northeastern Illinois. The rail network of CTA consists of eight lines and this analysis is about the Brown Line.
Since 1998, ridership has increased on the Brown Line by 21%, marking the highest rate of growth in CTA's rail system. The CTA began introducing operational changes in the mid-1990s to accommodate increasing Brown Line demand. Despite service changes, persistent crowding on Brown Line platforms and rail cars continued to negatively affect the rail transit experience for passengers.
In 1997, the CTA began to plan a significant rehabilitation project for the Brown Line. This was a large-scale construction project with substantial impacts throughout some of the most densely built neighborhoods of Chicago's north side. Prior to the rehabilitation project, all Brown Line stations outside of the loop-with the exception of Merchandise Mart, Fullerton, and Belmont stations-could only accommodate six-car trains instead of the normal eight. The major objectives of the project were to lengthen all station platforms between Chicago and Kimball stations to accommodate eight-car trains and rehabilitate rail infrastructure and stations to meet Americans with Disabilities Act accessibility requirements.
During this process, a Brown Line Task Force was set up to serve as a major conduit for communication among key internal and external stakeholders throughout this project. The task force met on a monthly basis to supplement rather than supplant the CTA communications platform to the community. According to the literature, public involvement should not be a one-size-fits-all approach to participation because good participation depends on the project-specific circumstances (14,15). On the Brown Line rehabilitation project, the CTA's outreach and involvement approach catered to the needs and characteristics of the demographic profile of the project impact area. In the project impact area, elected officials and communities groups are highly organized and residents are highly educated. As such, the agency was expected to undertake a high level of engagement. The CTA was aggressive in sending messages out to the community about anticipated delays, provision of alternative bus services, and scheduled construction activities. The agency, according to interviews, established an intense outreach effort involving a wide range of activities including aldermanic briefings, distribution of flyers, media announcements, and community meetings. CTA perceived that their efforts were effective. However, not all external groups agreed.

METHODOLOGICAL APPROACH
The purpose of this research is to measure and assess internal and external stakeholder perceptions and expectations of a meaningful and effective involvement effort in public transit planning. This research methodology is centered upon three sets of structured interviews with distinct internal and external stakeholder groupings for CTA outreach and involvement. The interviews serve to identify the gaps in the process and in the acceptance, if any, of stakeholder involvement. This process, with theoretical underpinnings in cognitive mapping, makes use of Strategic Options Development Analysis, a qualitative approach steeped in public participation theory (16) through the software Decision Explorer (17). The method is systemic in nature and looks at the system as a whole rather than taking the reductionist approach of breaking it into parts. The interviews with each stakeholder grouping carried a parallel set of questions based on the key findings of the literature review including assessing the extent to which the process enhanced the effectiveness and quality of the plan, increased public acceptance of the plan through use of local knowledge, built public trust in the CTA, and assured cost-effective decision making. The hypothesis enveloping each of these concepts is that there exists a gap in either the process or in the acceptance of each concept. CTA outreach and involvement activities are theoretically channeled through its government and community relations (GCR) department. The first phase of this research entailed conducting structured, one-on-one interviews with all eight GCR department members to gather a baseline understanding of the current state of CTA stakeholder outreach and involvement practices. Through the baseline study, the Brown Line rehabilitation project was identified as a case study for this research. Two sets of structured interviews were conducted with internal and external CTA stakeholders on the Brown Line rehabilitation project. Internally within the CTA, the outreach and involvement process for this project has been heralded as a success. The major objective of this case study is to assess the extent to which a gap exists between the CTA's conception of "success" for public outreach and involvement in this case and the public's conception of "success" for the CTA's outreach and involvement efforts.
A Brown Line community outreach database, which dates back to project inception in 1997 and recorded all meetings held in reference to the rehabilitation project, was utilized to identify the most involved internal and external stakeholders to interview for this case study. Ten internal CTA employees who were involved in the Brown Line rehabilitation project from a range of departments (e.g., planning, the president's office, rail and bus operations, construction, and customer communications) were interviewed. Nine external stakeholders were interviewed including elected officials, university representatives, community and neighborhood group representatives, chambers of commerce, and business owners.
On completion, each interview was digitally transcribed. Based on direct quotes pulled from each interview transcript in response to the same question sets, a gap analysis of internal and external understandings of the outcome of the stakeholder involvement process was performed. Decision Explorer cognitive mapping software was used to complete this gap analysis. Decision Explorer allows for structured analysis of qualitative data by creating a model of interlinked ideas using a cognitive map (Figure 1). The map consists of short phrases, or rather concepts gathered directly from the interviews, whose relationships are indicated by the causal links drawn between them. Together, concepts and links form a model. The term "cognitive map" refers to the data set as a whole. The benefit of this cognitive mapping software is that it maintains the richness of the data by managing the complexity of it, rather than having to rely on weaker, generalized statements to summarize the interview data.
A set of individual, comparative models was created to assess the internal versus external perspectives on the successfulness of the Brown Line rehabilitation outreach and involvement process with respect to the following four conceptual areas identified in the literature: 1. Enhancing the effectiveness and quality of agency plans and decisions, 2. Increasing public acceptance of plans through use of local knowledge, 3. Building the public's trust in the agency, and 4. Assuring cost-effective decision making.
This was an iterative process to verify the intercoder reliability of the links between concepts. Beyond mapping, Decision Explorer contains advanced techniques used to analyze relationships in the interview findings. The software produces two types of statistics: central scores and potency scores. The central score represents the level of influence that a concept has in relation to other concepts in the model, while the potency score reflects the extent to which a peripheral concept contributes to the higher-order central concepts. Central and periphery scores were calculated from cognitive maps developed for external and internal stakeholders for each of the four concepts. In this way the conceptual gaps between internal and external stakeholders about the success of different dimensions of the Brown Line rehabilitation stakeholder involvement process can be assessed. For analysis purposes, the top five central and potency scores from each category were selected and analyzed to assess the gaps in internal and external perceptions of the success of this rehabilitation project. This method provides a robust, systematic technique to measure and analyze stakeholder perceptions using qualitative data.

FINDINGS
The findings demonstrate systematic gaps in internal and external CTA stakeholders' perceptions of the four conceptual areas as well as of the overall goals of the stakeholder involvement process. These gaps occur when the outreach and involvement process fails to adhere to process criteria or acceptance criteria or both. This is explored in detail below.

Enhancing the Effectiveness and Quality of Agency Plans and Decisions
Gap Table 1 demonstrates a gap in perception of how this process enhanced the effectiveness and quality of agency plans and decisions. The gap is measured by comparing the robust central scores for the internal stakeholders with those of the external stakeholders. As Rowe and Frewer's theory of process criteria explains, information resources, human resources, and time resources all affect the quality of participation processes (6). In this case, the CTA deemed its outreach and involvement efforts "intense" in citing a range of participation techniques utilized throughout the process; however, a gap in understanding exists as the external stakeholders did not believe they had adequate access to information or human resources throughout this process.

Analysis
The top five potency scores behind this map provide further insight into this disconnect. As the literature demonstrates, more involvement does not always necessarily yield improved outcomes; rather, transit agencies could stand to benefit from more discriminating use of outreach and involvement techniques rather than additional methods over their baseline approach at the behest of community demands for more involvement. External stakeholders value two-way forms of communication and hold a greater sense of satisfaction and success with an outreach and involvement process when they are greeted with agency staff empowered and prepared to make decisions and when they have more direct access to higher levels of agency leadership.

Solution
The agency can improve the perception and the gap by assigning a decision maker to attend a select number of meetings with external stakeholders to address their concerns.

Increasing Public Acceptance of Plans Through Use of Local Knowledge
Gap Table 2 highlights the internal and external stakeholder gap in perceptions on how the CTA process increased public acceptance of plans through use of local knowledge. The findings indicate that the internal CTA stakeholders take recommendations from the public seriously and they find their process useful in terms of the community holding an impact on more peripheral issues including station components and the timing of construction activities. By contrast, the findings indicate that the external stakeholders felt disenfranchised from the involvement process because they did not have enough information to actually influence the process. In addition, they lacked a clear set of expectations about where their contributions would influence the process. These gaps speak to Rowe and Frewer's process and acceptance criteria (6).

Analysis
In this case, the central scores and potency scores when taken together provide a more enlightening explanation about the point of divergence between internal and external perceptions of the usefulness of the outreach and involvement process. Announcing decisions rather than soliciting input thwarts opportunities for input as the project evolves. Public transit agencies have a mandate as governmental bodies and stewards of public fiscal resources to serve their constituent base and be responsive to their needs in the provision of transit services. Use of local knowledge should inform this process. To uphold the acceptance criteria the outcomes of stakeholder participation processes should have a genuine impact on policy. In this case, the CTA needed to withhold proprietary budget information; however, the acceptance criteria mandate a logical explanation as to what information is being withheld and why. Solution Transit agencies may benefit from a more proactive approach to stakeholder involvement that provides transparency around their expectations of the process to the public. Earlier and continuous external stakeholder involvement could secure stakeholder buy-in and mitigate some of the more central frustrations with the process noted above. The nature and scope of participation should be clearly defined and the appropriate resources, including financial, human, and time, should be corralled to meet the task.

Building Public's Trust in the Agency
Gap Table 3 highlights the gap in internal and external stakeholder perceptions on how the CTA outreach and involvement process built public trust in the agency. While the CTA claims its process in this case was thorough in maintaining open lines of communication and in providing advance warning, this does not temper baseline community mistrust of the agency.

Analysis
Based on the internal central scores, the CTA understands the elements of a successfully conducted outreach and involvement process that serve to build trust in the agency. However, the findings indicate that key actions and turning points within this process are valued and weighted very differently on each side of this paradigm. The CTA originally promised that no stations would be closed throughout this rehabilitation process; however, this promise proved cost prohibitive and was therefore not possible. From the external stakeholder perspective, this proves a stumbling block to securing community trust. In this case, it seems the CTA's understanding of Intense outreach effort because of the magnitude of project impacts. Gain input. Aldermanic briefings, door-to-door flyers, radio ads, direct mailings, specific website, open houses and public meetings, monthly task force meetings. Task force instrumental in keeping the whole thing going. Level of interaction depends on project needs and timing.
CTA representatives could not provide answers to community questions. Frustration with the process. Difficult to get accurate information from the CTA. One-way communication is achieved rather than two-way communication. Very little communication from CTA between task force meetings.
Formal public notice processes let people know where to get more information about changes. Gain input. Answer public questions in community meeting forums. Introduce contractor and share construction plans and timeline. Level of interaction depends on project needs and timing.
CTA representatives were not actual decision makers. CTA representatives could not provide answers to community questions. No formal transition process between personnel changes. CTA did not adequately prepare representatives for meetings. Better outcomes in meetings where top CTA leadership was present rather than frustration with the process. Every recommendation received was taken to heart. Construction schedules were adjusted because of community events. It is best to find out community concerns to prevent being sued. Community brought in good suggestions about station components rather than actual design and engineering. Few comments were received that would actually be implemented or that had not been thought of.
Frustration with the process. Community does not have enough information to provide appropriate input. CTA allows community demands to get unwieldy rather than clearly state expectations of process from onset. Budget used as a blanket excuse when things changed and promises were broken. People were suspicious about justifications provided by CTA throughout process.
Few comments received that would actually be implemented or that had not already been thought of. Input received either wasn't feasible or wasn't 100% thought through. If notices had been put out earlier, maybe more people would have heard about things in advance. Every recommendation received was taken to heart. Community brought in good suggestions about station components rather than actual design and engineering.
Frustration with the process. Things had already been decided, which leaves no room for community input.
Involving the public needs to happen earlier.
CTA believed so much in their plan that they couldn't understand why anyone would question elements of the plan. Community meetings to discuss station closures were only set up after everything had been decided. If we promised something that couldn't happen we gave enough advance notice. CTA communicated from the very beginning exactly what they needed to do and why. Transparency and timing matter.

Score
More accountability would have yielded less community resistance. Community feels like CTA doesn't care about their concerns. CTA just announces plan and station design changes rather than involve and consult external stakeholders as things change. CTA breaks promise to not close any stations because it is cost prohibitive to keep stations open. Lack of community trust in CTA.
Early investment in doing outreach activities builds community trust. The more notice you give people the better things go, rather than when you surprise them it degrades trust. Development of trust on project between community residents and CTA rather than bringing about an increase in opposition. Adhering to our schedule. President attended all initial meetings, which made the community feel important and listened to.
Lack of community trust in CTA. Internal CTA employees on task force were good rather than executive leadership of organization. Affected stakeholders rely on aldermen to express their concerns rather than approach the CTA directly. More interaction leads to more favorable community impression of agency. Skepticism about competence and decisions of CTA. providing notice, or rather disseminating information, does not match the community's expectation of being involved and consulted as things change. This misunderstanding exacerbates external distrust of the agency.

Solution
Based on the potency scores of the external stakeholders, the more stakeholder involvement an agency engages in, the more the image of the agency improves. Distrust is particularly mitigated through positive, personal interactions with agency employees throughout an outreach and involvement process. In doing so, the agency is likely to incur more cost than it budgeted.

Assuring Cost-Effective Decision Making
Gap Table 4 highlights the gaps in internal and external stakeholder perceptions on how the CTA outreach and involvement process served to assure cost-effective decision making. The CTA deems its processes an excellent investment of resources and time in terms of preventing backlash and outcry later in the project, while the external stakeholders interviewed do not believe that the methods utilized are the best use of their time and that they ultimately fail to accomplish what the process sets out to do.

Analysis
The process criteria dictate that an effectively conducted process is one where the agency chooses mechanisms of engagement that are cost effective in terms of time, money, risk, and opportunity cost of achievements relative to other mechanisms that could be expected to 96 Transportation Research Record 2174 achieve the same result. But perhaps more importantly, an effective and meaningfully conducted stakeholder involvement program can help the agency save resources or avoid costs down the line by becoming aware of potential issues that can be costly later. While the CTA points out that from a labor dollar standpoint some costs may be inconsequential-especially those of professional staff who do not receive overtime payment-this understanding may obscure consideration of opportunity costs of time and resources as well as the costs of more physical outputs related to outreach and involvement including advertising for businesses and production of public notifications. If an agency does not conduct an outreach and involvement process that reflects stakeholder needs they may continue to expend time and resources without the expected return on their efforts.

Solution
A more proactive and continuous involvement process may allow an agency to be more in touch with community concerns and direct resources accordingly. With a goal of assuring cost-effective decision making, an agency could benefit from an evaluative component to its outreach and involvement processes. This would provide insights into methods of outreach and engagement that external stakeholders deem a waste of time and would allow more targeted resource allocation decisions.

Summary: Goals and Expectations
Gap Table 5 highlights the gaps in priorities between internal and external stakeholders on how this process could be improved. The agency strongly identifies with goals and expectations that advance dissemination of information to external stakeholders, whereas the external stakeholders cling to a set of expectations centered upon the idea Outreach and involvement was so minimal that it was cost-effective. Outreach and involvement methods selected probably not the best use of resources. CTA wants input to be representative, not all-inclusive. CTA doesn't want to hear from every single person affected on issue. Earlier participation would have helped CTA gauge concerns. of involving external stakeholders earlier in such a way that their input will affect the final plans and decisions. As noted above, providing notice, or rather disseminating information, does not match the community's expectation of being involved and consulted.

Analysis
The cleavages in these analysis scores are telling; internal and external stakeholders carry a distinct set of expectations into the interactive planning process. External stakeholders demand more involvement, earlier involvement, and proactive agency leadership. These demands build toward the expectation that their input is used to shape the final plan. The agency asserts concern about cost-effective outreach and involvement activities. The agency will benefit from more careful selection of the specific participatory techniques they select to meet project specific goals for outreach and involvement.

Solution
The external understanding that increased outreach and involvement activities will lead to increased agency effectiveness may continue to exacerbate external dissatisfaction with the agency engagement processes. Instead, the agency may benefit from a more transparent articulation of goals and the resources available to meet project specific goals to arrest this gap in expectations of their outreach and involvement processes.

CONCLUSIONS
The purpose of this paper is to qualitatively measure and assess gaps in internal and external perceptions of success of an outreach and involvement program. Two propositions were employed as guides. Across all four categories is evidence for the first proposition: there are clear gaps in perceptions of the successfulness of interaction and engagement processes across the four main conceptual areas of this paper. Secondly, the gaps are largely explained non-attainment of process and acceptance criteria. A brief review of the findings and implications for transit agencies follows.

Importance of Interaction from All Levels of the Agency
Findings reinforce the notion that successful stakeholder involvement depends not only on what techniques are utilized, but also who is involved and the level of commitment by an agency to such a program. While the acceptance criteria dictate a representative process whereby the participants in the process represent and reflect the broader public, public agencies similarly must involve a broad spectrum of their internal staff, including those at the highest levels of leadership and power within the organization. This research shows that external stakeholders hold a greater sense of satisfaction and success with an outreach and involvement process when they are greeted with agency staff empowered and prepared to make decisions and when they have more direct access to higher levels of agency leadership.

Importance of Transparency
Findings show that external stakeholders can feel disenfranchised from an involvement process when (a) they do not have enough information to inform the process and (b) they lack a clear set of expectations about how their contributions will influence decisions. A decision-making process that follows a recognizable involvement structure enhances transparency. As Falcocchio (18) explains, structure allows participants in the process to be more aware of the impact of their decision on transportation goals and objectives, and informs them of the tradeoffs of decisions made by public officials that may depart from the ini- CTA is more proactive rather than reactive. CTA takes leadership role in forming task force rather than let outside group form it, run it, host it, control it. Community has better understanding of CTA goals and constraints. Early involvement rather than involving public once decisions have been made. Use community input to shape final plans.
Give advance notice on construction activities rather than last-minute notice or failure to notify. Community can plan appropriately during period of impact. Better community trust in CTA. Vary location of public hearings and open houses. Provide geographical spread and coverage for public hearings and open houses.
Early involvement rather than involving the public once decisions have been made. More involvement. CTA has better understanding of community needs and concerns. More appropriate use of resources and technology. Reduce impacts and inconveniences on neighboring residents and businesses. tial plan. Moreover, fulfillment of process criteria would demand that the nature and scope of participation be clearly defined. While each project is unique, the agency should clarify how participation and project objectives align so as to appropriately frame expectations.

Timing and Continuity Matter
The findings indicate that a large disconnect will begin to emerge when involvement does not begin in the earliest stages of project planning. If transit agencies do not utilize early engagement as a way to identify community concerns and needs and use these findings to shape their outreach and involvement accordingly, ordinary citizens will continue to remain marginalized from involvement. The findings show that stakeholder involvement should be an iterative process characterized by involvement of stakeholders not only early in the process, but often and continuously throughout the process. Earlier identification of community needs and concerns may temper community suspicions that they are disenfranchised from the process and would introduce an enhanced level of accountability to this process.

Importance of Evaluating Stakeholder Involvement Performance
Transit agencies should clearly define what constitutes success in their use of stakeholder involvement resources to provide a baseline and framework to assess their efforts. In this way, agencies can be more judicious in their selection and use of involvement techniques to reach their predefined and accepted benchmark for success. Again, the literature demonstrates that there is no one-size-fits-all approach to stakeholder involvement. Rather, agencies need to identify and select the techniques and methods appropriate on a case-bycase circumstance to not only fulfill legal mandates but also meet their agency and project goals. Individual public agencies should take the added steps to internally assess their efforts and seek improvement. Because public involvement programs are expensive and require investment of human and financial resources, they should have a built-in, carefully designed, evaluative component to measure their effectiveness (2). The mere act of complying with regulatory stakeholder participation features or implementing a public involvement agency standard can become an empty ritual over time-as seen when public comments, opinions, and controversy are systematically captured, documented, and filed away, with no real intention incorporating the content of public comments into planning outcomes. Without techniques to evaluate their practice, current methods employed by agencies may continue to waste resources and may fail to promote agency goals (2).
The findings demonstrate that practices deemed successful by the agency are not necessarily valued and weighed in the external realm as successful in the context of this case study. Without evaluative techniques over time and project types, public agencies may institutionalize less-than-successful processes and waste resources.
While it is clear that stakeholder involvement is costly and time consuming and, as this research shows, requires a complex balance of external stakeholder expectations of the process with internal agency goals and objectives, there are benefits to meaningful stakeholder involvement and compliance with legal mandates. Transit agencies can improve the success of their efforts and close gaps in expectations through greater attention to process and attainment criteria, as well as the establishment of a strong evaluative component for involvement.