Intention and Action: Evaluating the Policy Antecedents of Development

Abstract

Development management ordinances are central components of the development process, and yet there is not an established set of principles to guide their evaluation. We build upon the established plan quality literature to develop a protocol to assess ordinances based on their content and their administration. Using substantive and procedural principles enables the examination of how ordinances incorporate both scientific information and administrative practices to support policy implementation. Our cross-sectional study of 22 jurisdictions in two different states compared riparian buffer policies, single-purpose mandates, and sociodemographic variables. We found (1) overall low ordinance quality scores, (2) statistically significant differences between the watersheds at the sub-principle level, and (3) multiple, moderate correlations among ordinance quality scores, population density, and planning capacity. The findings suggest opportunities to increase the usage of best available science and promising administrative practices within ordinances aimed at protecting water quality.

1. Introduction

Delineating current, purposeful actions to shape desired future conditions is a central tenet of planning, and a plan’s utility depends, in part, on its quality and implementation [1,2,3]. Decades of plan quality research examine the content of plans and their association with positive outcomes. Studies of hazard mitigation dominate the research literature [4,5,6,7], but there are plan quality studies on affordable housing [8], transportation [9], watershed protection [10], ecosystem services [11], and climate action [12]. These studies contribute to an ongoing debate on the gap between plan creation and implementation [13]. This study seeks to bridge the implementation gap by focusing on a common tool of policy implementation: development management ordinances [14]. Although the term development management ordinance is primarily associated with policies and procedures within the United States, our work has applicability to a wide range of development documents that outline an administrative review process and use regulatory, incentive, and/or investment provisions to relate the goals, information, and policies from a plan to a development’s use, form, and orientation. Studies examining the policies governing development include adaptive reuse [15], community benefits [16], parking lots [17], physical activity [18], sustainability [19,20], and urban tree canopy [21,22]. However, a set of principles enabling a systematic assessment of the quality of development management ordinances does not yet exist.

We build upon the conceptual framework offered by the plan quality literature to identify a set of ordinance quality principles and examine riparian buffer provisions within development management ordinances adopted by 22 local governments in two watersheds in Maryland and North Carolina. Two research questions guide our research: (1) To what extent do the buffer protection provisions within development management ordinances incorporate scientific information and promising administrative practices and (2) are socio-demographic characteristics associated with higher-quality ordinances? The study includes population size, population density, growth rate, median income, median housing value, and planning capacity, which other studies identify as variables with an impact on plan quality [4,23,24,25,26]. The study jurisdictions are in two states planning under different single purpose state mandates, allowing the investigation of differences in ordinance quality at the state level.

Calls for evidence-based policy-making or the use of best available science acknowledge that policy creation and implementation occur in a sociopolitical context where scientific information vies for influence alongside political motivations and social pressures, and thus, is unlikely to be the only (or even major) influence on the final adopted policy [27]. Gaps in information availability and credibility, the disregard for power, secrecy, time constraints, and political expediency are among the reasons evidence only weakly informs policy [28,29]. Incorporating scientific information into the policy-making process requires consideration of the process used to generate evidence, its quality and fit, its dissemination and communication structures, and its incorporation into policy and practice [30].

Several environmental studies highlight the imperfect transfer of research, technical findings, or best practices into plans and policies. In their examination of 49 management plans across eight sites, Arkema, Abramson, and Dewbury found that most plans did not include specific marine ecosystem-based management criteria drawn from the research literature despite six sites that espoused support for scientifically based decision-making [31]. Berke and colleagues found less detailed information about local water resources and fewer policies aimed at protecting water resources [32]. Kim and Li’s study of 76 comprehensive plans in the Chesapeake Bay watershed found weak incorporation of sustainable stormwater management concepts [33]. Mills and colleagues interviewed 41 planners and consultants about the “best available science” requirement of Washington’s Growth Management Act, and their findings suggest that planners face difficulty in translating ecological data into their local context and defending it when under political scrutiny [34]. Interviews with planners and ecologists in Finland suggest that data are often outdated, unavailable, or incomplete [35]. Data usage can depend on the interest and attitude of an individual planner, especially if they believed it could increase their workload or hinder creative problem solving. Direct interaction with the public may temper the inclusion of scientific information as practitioners’ comprehension and ability to communicate a policy’s utility alters both their willingness to implement and their perceptions about the value of the policy to the public [36].

Development management ordinances connect how a jurisdiction plans to develop (intention) to the detailed provisions governing development (action). Studies focused on development management ordinances suggest that there is wide variation in the incorporation of scientific and technical information even in ordinances drafted under a mandate. Garde and colleagues found differential incorporation of sustainable design criteria into conventional zoning codes versus form-based codes for Denver, Colorado and 26 cities in southern California using the Leadership in Energy and Environmental Design for Neighborhood Development (LEED-ND) rating system [37,38]. Jepson and Haines evaluated nine sustainability principles and 53 regulatory items in 32 zoning ordinances from across the United States and concluded some principles (e.g., promote mixed use; protect ecosystems) occurring in most ordinances, while they found others in only a few (e.g., promote local food systems; encourage higher-density development) [19]. Stevens and Hanschka found that 66% of the jurisdictions did not adopt a flood bylaw or include flood risk management provisions within their zoning bylaws despite a government mandate [39]. Given the heterogeneity in research approaches, settings, and sociopolitical circumstances alongside inconsistent availability, the conversion of scientific information into policy is not a straightforward process. However, these challenges do not preclude an evaluation of the presence of existing scientific information within local policy to identify gaps and opportunities.

A standardized evaluation approach for ordinances is an under-studied portion of the development process. We provide a proof of concept for our approach by assessing both the presence and quality of riparian buffer provisions grounded in best practices and recommendations from the research literature [40]. We propose a set of eight ordinance quality principles divided into two conceptual groupings (Policy Content and Policy Administration) reflecting the need for substantive and administrative components and based on the planning research literature, the concept of street-level bureaucracy, and planning practitioner resources. Four principles make up the Policy Content conceptual framework: Goals, Fact Base, Policy Description, and Policy Restrictions. Psychological and organizational theories on goal-setting highlight the importance of specificity for performance improvement [41,42]. Goals can focus organizational and individual energy on goal-relevant activities, improve and prolong effort, and enhance achievement. Within development management ordinances, goals serve the dual purpose of outlining the intended consequence of the ordinance and creating a legal foundation by articulating their purpose and necessity [43,44]. The Fact Base, Policy Description, and Policy Restriction principles center on data identification and utilization to examine this disconnect. The Fact Base principle identifies the informational inputs required by the policy, including acceptable sources and processes to resolve disputes over interpretation. Policy Description explains the policy’s provisions, including when parameters apply. Policy Restrictions describes specific limitations and highlights the necessity to communicate the constraints placed on policy [45,46].

The administration framework includes four principles that either facilitate or complicate policy implementation (Policy Flexibility, Complexity, Monitoring and Enforcement, and Discretion). Alterman and Hill found that flexibility and the complexity of the review process affected plan deviations [26]. Policy Flexibility covers how ordinances account for circumstances leading to departures from ordinance provisions and/or allow for unique solutions as uncertainty is a fundamental reason to plan [47]. Policy provisions such as variances allow for adaptation to different circumstances and reflect the flexibility advocated for by planning scholars [48,49,50]. Complexity is a measure of the difficulty of administering a particular policy and gauges the effort necessary to navigate overlapping provisions and intensive data demands. Hill, Dorfman, and Kramer found that each additional ordinance clause (increase in complexity) resulted in a 1.03% increase in county land area covered with tree canopy at the end of the ten-year period [51]. However, lack of specificity and clear language can be barriers to implementation [52]. Brotherton’s study of permit quality posits a linkage between conformance of development applications with ordinance provisions and an applicant’s comprehension of the policy, which decreases with complexity [53].

Implementation of ordinance provisions is a growing research area and the Monitoring and Enforcement principle outlines the ongoing process to oversee and manage the actions and practices stipulated by the ordinance [48,54]. A study of 15 Sacramento parking lots for compliance with a shading ordinance found average levels of 22%, which is below the 50% level stipulated in the ordinance [17]. Spatial analysis demonstrated how a Michigan zoning ordinance on open space fell short of its objective [55]. Ozawa and Yeakley found that more stringent management strategies in the 7.5 m riparian zone resulted in less substantial riparian vegetation losses [56]. Similarly, Weilert, Ji, and Zubair studied riparian vegetation over time in the presence and absence of streamside ordinances and found a loss of vegetation in areas without ordinances [57]. They also identified varying levels of protection because of inconsistent ordinance content and implementation. Their recommendations include implementation with adequate spatial coverage and provisions encompassing both content and administration.

Discretion refers to instances where staff charged with policy implementation can make an interpretation or judgment and draws on the theory of street-level bureaucracy, a framework for examining the actions of the public agencies and employees charged with implementing policy [58]. Street-level bureaucrats often possess specialized knowledge, interpret imprecise provisions, and perform the actions that implement laws. As a result, street-level bureaucrats have differential discretion in policy interpretation and administration. Unlike the Flexibility principle, which provides approved rules, standards, and tools to adapt to changing conditions, this principle refers to policy alterations based on the judgment of staff involved in the review process.

Several studies associate mandates with higher-quality plans [5,6,25,59] although mandate design may affect their impact [10,60]. Using an adaptation of Burby and colleagues’ approach, we examined mandate complexity, implementation style, and capacity and commitment building provisions [60] (p. 197). Less mandate complexity supports implementation (although local commitment and capacity can offset negative effects) [4, 61]. Implementation style characterizes the relationship between agencies and can range from informal, cooperative styles (incentive-based tools such as financial and technical assistance) to formal, legalistic (e.g., coercive approaches such as monitoring and/or sanctions) [62,63]. Dedicated financial resources or evaluation and monitoring activities act on local commitment (i.e., willingness to act) and local capacity (i.e., the ability to act), including education and training opportunities, funding for personnel and equipment, and authorization for local fees and taxes.

2. Materials and Methods

2.1. Study Sites and Sample

We selected the Gunpowder-Patapsco watershed in Maryland and B. Everett Jordan Reservoir (hence Jordan Lake) in North Carolina based on their impairment history, different state mandates aimed at riparian buffers, and similar demographic profiles. The Gunpowder-Patapsco watershed is a part of the Chesapeake Bay drainage basin, which was targeted for restoration in 1983 [64]. Jordan Lake received a nutrient-sensitive water designation upon the reservoir’s completion in 1983 and has consistently tested eutrophic or hyper-eutrophic [65,66].

The Gunpowder-Patapsco watershed contains portions of six Maryland counties and nine jurisdictions. We eliminated two jurisdictions (Frederick County—small area within the watershed (~0.024 sq. miles); Aberdeen—data access issues). In 1984, the Maryland General Assembly enacted the Chesapeake Bay Critical Area Protection Act (hence Critical Area Act) for areas located 1000 feet from all tidal waters and wetlands. In 1986, the Critical Area Act added a minimum 100 ft buffer of natural vegetation from the mean high water line of tributary streams [67]. This mandate does not cover all riparian areas, although the legislation encourages the extension of these protections.

The Jordan Lake watershed encompasses portions of 10 counties and 13 jurisdictions. We eliminated one jurisdiction without a comprehensive plan and twelve jurisdictions based on distance, which would diminish nutrient loading and issue salience [68]. The City of Durham and Durham County produce a single plan and ordinance. In 1989, the North Carolina General Assembly passed the Water Supply Protection Act requiring riparian buffers and restricting development density. Revisions to this Act required a 30-foot vegetated stream buffer for low-density development or a 100-foot vegetated stream buffer for high-density development (North Carolina § 143.214.5, 15A NCAC 02B.0214-.0216). Similar to the mandate in Maryland, the Act is geographically limited, although local jurisdictions can adopt requirements that are more stringent.

There were no statistically significant differences between the two watersheds for mean population size (GP—209,939.1; JL—179,794.2), population density per square mile (GP—2316.2; JL—1470.5), or planning capacity per 1000 residents (GP—0.019; Jordan Lake—0.013). Median home value for Gunpowder-Patapsco jurisdictions was $306,054 while the median for the Jordan Lake watershed was $246,278 (p = 0.066). Jurisdictions in the Jordan Lake watershed were growing faster (76.1%), on average, than jurisdictions in the Gunpowder-Patapsco (15.1%) and the observed difference is statistically significant (p = 0.006).

Figure 1 summarizes our evaluation approach. The study uses ordinances in effect in 2008 to predate major legislative changes in North Carolina. We downloaded ordinances from county and municipal websites or received electronic versions from planning staff.

2.2. Protocol Creation

Hirt advocates for direct evaluation of ordinances rather than surveys to address issues of reliability and we heed Norton’s recommendation that content analysis reflects policy focus and quality [40,69]. We operationalized the eight principles using studies investigating the optimal design and functioning of riparian buffers and model riparian buffer ordinances [70,71,72,73]. The complete protocol is available in Supplementary Materials. The indicators for Goals principle were binary (0—not mentioned, 1—mentioned). Indicators for Fact Base, Policy Description, Policy Restrictions, and Monitoring and Enforcement used an ordinal scale to reflect increasing levels of protection (i.e., 0 = not mentioned, 1 = standard information requirement, 2 = enhanced information requirement or 0 = no policy, 1 = basic policy, 2 = standard policy, 3 = enhanced policy requirement). Policy Flexibility indicators used a binary scale to denote protective and incentive policies and the aforementioned ordinal scale. The Complexity principle is a summation of indicators coded at the Enhanced level from the Policy Description, Policy Restrictions, Policy Flexibility, and Monitoring and Enforcement principles. The Discretion principle examines instances where staff responsible for policy implementation can interpret the applicability or administration of policy provisions. Coders assigned the three levels of this principle (High, Intermediate, and Bounded), along with indicators from other principles.

2.3. Content Analysis

Two independent, trained coders content analyzed using Atlas. ti software. We double coded ten ordinances and calculated percentage agreement and Krippendorff’s alpha as measures of inter-coder reliability using ReCal [74,75]. The inter-coder reliability scores for ordinance quality were Goals (70%, 0.33); Fact Base (80%, 0.54); Policy Description (84%, 0.59); Policy Restrictions (89%, 0.75), Policy Flexibility (77%, 0.41); and Monitoring and Enforcement (83%, 0.67).

After reconciliation, we normalized total scores for principles and sub-principle topic areas on a scale of 0 to 10 and tested each variable for normality and unequal variance using graphic and numeric methods. Three comparison of means tests (t-tests, Welch’s t-test, Mann-Whitney U test) determined if differences between scores achieved statistical significance. For analyses using demographic characteristics, we used Pearson r and Spearmen rank correlation.

3. Results

3.1. Mandate Influence of Quality

Jurisdictions in Maryland and North Carolina have the legal latitude to exceed the minimum requirements. The Maryland mandate encourages (but does not require) local jurisdictions to “apply protection measures similar to those contained in their Critical Area program to land disturbances beyond the Critical Area boundary” (MD. COMAR, Title 27, Chapter 10 (K)) while the North Carolina mandate emphasizes procedural compliance if a jurisdiction “imposes” more stringent regulations (160A-384. §143-214.5(d)).

The Maryland mandate includes a limited number of specific goals and objectives that are direct, specific, and narrow in focus, while North Carolina’s mandate contains fewer, simpler objectives but more goals that use vague language. For implementation complexity, Maryland jurisdictions must adhere to deadlines for approval, submit annual reports on developments in the critical area, and local jurisdictions are encouraged (but not required) to establish cooperative arrangements. In North Carolina, jurisdictions must submit their program for approval, but beyond the initial submission deadline, there are no ongoing mandated implementation actions. The Maryland mandate contains more capacity-building elements, such as technical assistance and state assistance in mapping. North Carolina’s mandate has provisions for technical assistance, a model ordinance, and workshops, but does not include the same level of funding opportunities as Maryland. Commitment-building elements were more comparable for the two states, as both include deadlines, sanctions for failure to meet deadlines, financial penalties, and the possibilities of state preemption of local authority. We expected the Gunpowder-Patapsco watershed in Maryland to have higher scores when compared to the Jordan Lake watershed in North Carolina based on the presence and design of the state mandate. The clarity of the goals and policy objectives of the Maryland mandate coupled with commitment and capacity-building elements should support the implementation of the mandate.

Table 1. Ordinance quality principle scores by watershed.

	Gunpowder-Patapsco			Jordan Lake
	Mean	SD	Range	Mean	SD	Range	p Value 1
Policy Content	3.04	1.81	0.6–6.7	3.37	1.13	2.1–5.5	0.634
Goal	4.62	2.50	0.0–8.0	5.11	1.45	4.0–8.0	0.600
Fact Base	3.08	2.61	0.0–7.6	2.31	1.43	1.0–5.3	0.477
Policy Description	2.84	1.82	0.2–6.3	3.72	1.38	1.6–5.8	0.237
Width	3.08	1.66	0.6–6.1	3.23	1.75	1.2–5.5	0.835
Vegetation	3.93	2.55	0.0–6.7	4.94	3.00	0.0–10.0	0.407
Habitat	1.41	2.24	0.0–5.0	0.74	1.47	0.0–3.3	0.408
Site Design	4.62	3.39	0.0–8.9	5.19	2.72	2.2–10.0	0.668
Allowable Uses	2.18	1.94	0.0–4.2	5.00	1.38	3.3–7.5	0.001 ***
Exemptions and Exceptions	2.31	2.96	0.0–8.9	5.06	2.43	1.1–8.9	0.027 **
Owner Activities	2.18	3.00	0.0–10.0	2.59	2.06	0.0–5.0	0.428
Policy Restrictions	1.62	1.43	0.0–5.0	2.26	1.43	1.0–5.0	0.373
Hazardous Land Uses	2.31	1.99	0.0–5.0	1.30	2.17	0.0–5.0	0.282
Waste Disposal	1.67	1.23	0.0–4.2	2.78	3.00	0.0–8.3	0.679
Agriculture	0.26	0.71	0.0–2.5	0.19	0.56	0.0–1.7	0.780
Impervious Surface	1.79	3.22	0.0–10.0	4.44	3.73	0.0–10.0	0.084 *
Mining	3.73	3.98	0.0–10.0	3.70	3.51	0.0–6.7	0.272
Policy Administration	2.92	2.07	0.4–6.6	3.59	1.96	1.0–6.6	0.441
Policy Flexibility	3.31	2.27	0.0–7.8	4.88	2.75	0.0–9.1	0.159
Buffer Averaging	0.00	0.00	0.0–10.0	1.11	3.33	0.0–10.0	0.229
Overlay Zoning	1.28	3.20	0.0–10.0	5.19	5.03	0.0–10.0	0.042 **
Protective Policies	5.00	3.06	0.0–10.0	5.00	2.80	0.0–10.0	1.00
Incentives	2.88	2.47	0.0–7.5	3.61	3.77	0.0–10.0	0.621
Variances	3.40	2.71	0.0–10.0	4.91	2.52	0.0–7.5	0.196
Complexity	2.01	1.85	0.0–6.7	2.16	1.09	0.6–4.2	0.837
Monitoring and Enforcement	3.43	2.74	0.0–6.9	3.73	1.63	1.3–6.9	0.772
BMP	3.72	4.62	0.0–10.0	4.44	4.41	0.0–10.0	0.714
Inspection	1.44	1.92	0.0–4.0	1.56	1.45	0.0–4.0	0.870
Notification	4.23	4.65	0.0–10.0	3.70	3.89	0.0–10.0	0.776
Administration	2.95	2.47	0.0–5.0	4.44	1.67	0.0–5.0	0.101
Violation	7.82	3.56	0.0–10.0	7.78	3.33	0.0–10.0	0.885

¹ * p-values ≤ 0.1, ** p-values ≤ 0.05, *** p-values ≤ 0.001.

includes the mean score and standard deviation for each of the ordinance quality principles by watershed and the p-values from the comparison of means tests.

3.2. Policy Content Framework

Goals was the highest scoring principle for both watersheds and the only principle where a watershed (Jordan Lake) exceeded 50% of the points. The first of the five goals (Plan Connection) refers to linkage between the comprehensive plan and the ordinance, and the next four indicators increase in specificity from General Welfare to Natural Resource Protection and Water Resource Protection to Continuous Buffer System. Only one jurisdiction in the Gunpowder-Patapsco watershed (8%) and three jurisdictions in the Jordan Lake watershed (33%) included a linkage statement. At least seventy percent of the Gunpowder-Patapsco jurisdictions included each of the next three goals—General Welfare (9 ordinances or 70%), Natural Resource Protection (11 ordinances or 85%), and Water Resource Protection (9 ordinances or 70%). Within the Jordan Lake watershed, a majority of the jurisdictions included these three goals—General Welfare (7 ordinances or 78%), Natural Resource Protection (5 ordinances or 56%), and Water Resource Protection (8 ordinances or 89%). No jurisdiction in either watershed included the most specific goal (Continuous).

Fact Base is the only principle where the Gunpowder-Patapsco watershed’s score is higher than Jordan Lake’s score, but the difference was not statistically significant. Stream ID (sources to identify/classify streams) appears in most jurisdictions in both watersheds (54% of the Gunpowder-Patapsco watershed and 100% of the Jordan Lake watershed), and is the only individual fact base indicator with a significant difference (p = 0.020). Few ordinances within either watershed identified the 100-year floodplain, erodible soils, topographic information, wetlands, pre-development vegetation, and the authority to require a sub-drainage assessment and there were no significant differences between the watersheds on these indicators.

The Policy Description principle outlines the core policy provisions and the twenty-nine indicators fall within seven topic areas (Width, Vegetation, Habitat, Site Design, Allowable Uses, Exemptions and Exceptions, and Owner Activities). Width builds most directly on the information collected as part of the Fact Base principle. Neither watershed scored over 50% of the points and there was not a statistically significant difference between the two watersheds for this topic area (p = 0.835). The mean score for Vegetation (3.93 in Gunpowder-Patapsco and 4.94 in Jordan Lake) suggests that more jurisdictions account for vegetative target, management strategy, and buffer restoration efforts. However, the scores for Habitat in both watersheds (1.41—GP; 0.74—JL) suggest that few jurisdictions require habitat protection plans, reference habitat fragmentation, or include policies to protect aquatic and wildlife species. Statistical tests found no differences between the watersheds for the Width, Vegetation, or Habitat.

Site Design (i.e., grading, clearing, and setbacks from the buffer’s outer boundary) was the highest scoring topic area for both watersheds (4.62—GP; 5.19—JL) although there was not a statistically significant difference between the scores (p = 0.680). Owner Activities scored lower (2.18—GP; 2.59—JL) and there were not statistically significant differences between the watersheds. Allowable Uses and Exemptions/Exceptions focus on the permissive uses within or adjacent to the buffer (i.e., buffer crossing, agriculture, and recreation). There was a statistically significant difference between the watersheds for both topic areas (Allowable Uses, p = 0.001; Exemption/Exceptions, p = 0.027). Jurisdictions within the Jordan Lake watershed more frequently included provisions regulating (1) timber extraction within the buffer, (2) stream-dependent uses, (3) buffer crossings, and (4) use and location of stormwater best management practices. Jurisdictions within the Gunpowder-Patapsco watershed included policies governing agriculture exceptions slightly more than Jordan Lake jurisdictions (38% vs. 33%) while the ordinances within the Jordan Lake watershed more often included provisions governing recreation exceptions (67% vs. 31%) and general exemption policies (78% vs. 23%).

The Policy Restriction principle describes constraints or specific limitations and grouped ten indicators into five topic areas (Hazardous Land Uses, Waste Disposal, Agriculture, Impervious Surface, and Mining). None of the five topic areas scored over 50% of the points and the only significant difference between the watersheds was at the 0.1 level (Impervious Surface).

3.3. Policy Administration Framework

The Policy Flexibility principle includes the provisions to adapt to different circumstances and covers six topic areas (Buffer Averaging, Overlay Zones, Protective Policies, Incentives, and Variances). There was no statistical difference on buffer averaging provisions (only one Jordan Lake jurisdiction included this provision) while there was a statistical difference (p = 0.0417) for Overlay Zones (5 ordinances or 56% in Jordan Lake and 2 ordinances or 15% in Gunpowder-Patapsco). Protective policies such as conservation easements or fee simple acquisition were the highest scoring topic area in both watersheds (5.0 in both watersheds). However, neither Protective Policies nor Incentives (e.g., counting buffers against open space requirements) registered a significant difference between the watersheds. Variance included provisions around the administration of and limitation on variances, and the study watersheds (3.4—GP; 4.9—JL) were statistically significant at the 0.1 level, with Jordan Lake averaging a higher score (p = 0.0983).

The Monitoring and Enforcement principle refers to ongoing oversight and management practices with thirteen indicators within five topics areas (BMP, Inspection, Notification, Administration, and Violation). Few ordinances in either watershed included (1) detailed provisions for buffer inspections or (2) specific policies around the monitoring and maintenance of stormwater best management practices, but the observed differences were not significant. Notice about buffer boundaries (Notification), the administration structure for buffer monitoring (Administration) did not exceed 50% of the points and the watersheds were not statistically different. Management of violations (Violation) is the highest scoring topic area across all the principles due, in part, to the high percentage of jurisdictions in both watersheds that included general sanctions at an enhanced level (8 ordinances or 89% of Jordan Lake and 11 ordinances or 85% of Gunpowder-Patapsco). Fewer ordinances included a violation description specific to riparian buffers (4 ordinances or 44% of Jordan Lake watershed and 6 ordinances or 46% of Gunpowder-Patapsco watershed) and the difference was not significant.

The Complexity principle gauges the difficulty of navigating detailed provisions and intensive data demands. Low scores for the Complexity principle reflect the low scores on the component principles (Fact Base, Policy Description, and Policy Restriction, Monitoring and Enforcement), and there was not a significant difference between the watersheds (p = 0.837).

3.4. Discretion

Nine jurisdictions (five in the Gunpowder-Patapsco and four in the Jordan Lake) included at least one instance of discretion, and there were 34 co-occurrences of Discretion with other indicators.

Table 2. Frequency of discretion by principle by watershed.

	Goals	Fact Base	Policy Description	Policy Restrictions	Policy Flexibility	Total
Gunpowder-Patapsco	2	1	11	1	6	21
Jordan Lake	0	0	6	3	4	13

contains the frequency by five ordinance quality principles (no co-occurrences with Complexity or Monitoring and Enforcement). Three instances of Discretion were coded as High (i.e., application reviewer has sole authority to interpret ordinance provision). The remaining 31 instances were Intermediate, meaning an additional agency, department, or organization with higher positions within a bureaucracy (e.g., department or division directors) or agencies with perceived expertise (e.g., Department of Environmental Protection, Department of Natural Resources, Department of Health, Department of Public Works) became involved in the review process.

The co-occurrences of Discretion with Goals indicators limited the ordinance’s application to particular geographic areas. For example, areas may be exempted from buffer requirements with approval from the State Critical Area Commission if the “existing pattern of development prevents the buffer from fulfilling its intended function” [76] (p. 114). Co-occurrences with Fact Base indicators provided staff with discretion in determining the fulfilment of informational requirements. The Policy Restriction co-occurrences with Discretion were for septic systems, sewer pipes, impervious surfaces, and each of these instances allowed for the relaxation of the restrictions with justification. Similar to the findings for Policy Restrictions, the co-occurrences of Policy Description indicators relaxed policy provisions. Most frequently, these instances were for site design (i.e., clearing, grading) or allowable uses within the buffer and required staff or agency approval. Discretion with Policy Flexibility occurred with the Variance indicators and imposed additional requirements or conditions.

3.5. Relationship among Ordinance Quality and Sociodemographic Variables

Table 3. Ordinance quality principle scores and demographic variables correlations.

	Population Size	Population Density	Growth Rate	Median Income	Median Housing Value	Planning Capacity
Policy Content	0.16	−0.43	−0.01	−0.04	−0.22	−0.50
Goal	0.24	−0.12	−0.19	−0.21	−0.34	−0.47
Fact Base	−0.05	−0.35	0.04	0.17	−0.04	−0.49
Policy Description	0.22	−0.43	0.08	0.11	−0.10	−0.41
Width	0.20	−0.39	−0.10	0.12	−0.04	−0.54
Vegetation	0.29	−0.15	0.02	−0.04	−0.13	−0.42
Habitat	0.30	−0.01	−0.20	0.07	0.28	0.09
Site Design	0.09	−0.47	0.12	0.39	0.16	−0.38
Allowable Uses	0.17	−0.37	0.37	0.12	−0.14	−0.18
Exemptions and Exceptions	0.37	−0.46	0.19	−0.16	−0.33	−0.04
Owner Activities	0.06	−0.44	0.20	0.01	−0.15	−0.29
Policy Restrictions	0.10	−0.49	−0.03	−0.20	−0.19	−0.31
Hazardous Land Uses	−0.16	−0.29	−0.10	−0.19	−0.25	−0.32
Waste Disposal	0.14	−0.55	−0.02	0.06	−0.03	−0.27
Agriculture	0.13	0.07	−0.03	0.15	−0.02	−0.25
Impervious Surface	0.25	−0.31	0.10	−0.36	−0.03	0.04
Mining	0.21	−0.09	0.07	−0.10	−0.09	0.20
Policy Administration	−0.01	−0.34	0.08	0.15	−0.07	−0.29
Policy Flexibility	0.18	−0.36	0.08	0.04	−0.12	−0.21
Buffer Averaging	−0.05	0.15	0.33	0.22	−0.12	0.29
Overlay Zoning	0.05	−0.37	0.25	−0.04	−0.22	0.11
Protective Policies	−0.11	−0.20	0.07	0.15	−0.06	−0.39
Incentives	−0.04	−0.20	0.13	0.16	0.00	−0.20
Variances	0.24	−0.35	0.03	−0.03	−0.07	−0.27
Complexity	0.17	−0.53	0.06	0.10	−0.07	−0.32
Monitoring and Enforcement	−0.21	−0.25	0.15	0.26	0.01	−0.29
BMP	−0.29	−0.10	0.08	0.17	−0.10	−0.41
Inspection	−0.28	0.02	0.10	0.19	0.11	−0.24
Notification	−0.14	−0.32	0.10	0.25	−0.02	−0.38
Administration	−0.08	−0.43	0.28	0.19	0.14	−0.04
Violation	−0.23	−0.03	0.19	0.34	0.03	−0.10

reports correlation (Pearson r or Spearman’s rho) for the relationships among ordinance quality principles and demographic variables. We found moderate inverse relationships with planning capacity (−0.47) and the Goals principle and the Fact Base principle and population density (−0.35) and planning capacity (−0.49). Within the Policy Description principles, population density had a moderate negative correlation with Width (−0.39), Site Design (−0.47), Allowable Uses (−0.37), Exemptions/Exceptions (−0.46), and Owner Activities (−0.44). Planning capacity has a moderate, negative correlation with Width (−0.54), Vegetation (−0.42), and Site Design (−0.38). Site Design was positively correlated with median income (0.39) and Growth rate was positively correlated with Allowable Uses (0.37). Population density was negatively correlated with Policy Restrictions (−0.49) and Waste Disposal (−0.55). Impervious Surface had a moderate, inverse relationship with median income (−0.36). Population density was negatively correlated with Policy Flexibility (−0.36), Overlay Zoning (−0.37) and Variances (−0.35) while the Protective Policies topic area negatively correlated with planner capacity (−0.39). The Complexity principle has a moderate to strong negative correlation with population density (−0.53). Within Monitoring and Enforcement, population density is negatively correlated with Administration (−0.43) while planning capacity was negatively correlated with BMP (−0.41) and Notification (−0.38).

4. Discussion

The scores for the Policy Content framework (Goals, Fact Base, Policy Description, and Policy Restrictions) were low, with few ordinances scoring over half of the points for any single principle. Only one watershed averaged more than half of the points on a single principle (Jordan Lake watershed, Goals principle). Within the Goal principle, most jurisdictions include goals about general welfare, natural resource protection, and water resources protection. While these goals drawn directly on police power and cover topics with strong ties to planning practice, their broad construction coupled with the complexity necessary to accomplish the goal negatively affects implementation [60]. Beyond the legal ramifications of separating plan intention from action, the absence of these goals reinforces the separation between plans and ordinances, which can undermine the effectiveness of plans. Deviation from plans should be anticipated, but failing to connect intention and action during initial goal-setting diminishes the utility of planning [3].

The absence of a goal may reflect the overall lack of specificity in goals within the sample. In particular, the goal for a continuous buffer is directly linked to scientific findings that support the optimal functioning of riparian buffers [70,71,72,73]. This missing linkage mirrors informational deficits under the Fact Base and Policy Description Principles. The majority of jurisdictions do not require the site-specific information recommended to help determine the width or vegetative target for riparian buffers. Failure to require baseline information on factors such as floodplain extent, topography, the presence of erodible soils, the location of wetlands, and pre-development vegetation represents a missed opportunity to utilize information that might support a wider buffer or restorative actions that can improve the design and functioning of a particular stretch of buffer and its associated water quality. Similarly, for Policy Description and Policy Restrictions, key topic areas highlighted in the research literature are not uniformly represented, including allowable uses, exceptions and exemptions, and owner activities. The low mean scores on the Fact Base and Policy Description principles, combined with the ranges for each of these principles, suggest that there is an opportunity for the policies in both watersheds to incorporate more of the best practices and design features. Few ordinances in either watershed included the indicators of the Policy Restrictions principle. The low scores suggest that ordinances in both watersheds do not include explicit restrictions against uses that may reduce the effectiveness of the riparian buffer.

The inclusion of research-based evidence in a policy may be a necessary condition, but it is not sufficient to ensure better outcomes. Administrative elements of policies must also include the best available evidence supportive of implementation [45,46]. Neither watershed averaged over 50% of the points for the Policy Administration framework, but the interpretation is different. The low scores on Monitoring and Enforcement suggest that ordinances could better support ongoing oversight and management. While some ordinances included specific policies around the monitoring and maintenance of stormwater best management practices, few ordinances in the study watersheds included detailed provisions for buffer inspections (i.e., the timeline of inspection, the initiating factors for an inspection, ongoing water quality monitoring). Thus, there are no provisions in place to detect changes in functionality. This shortcoming stands in contrast with Violation as the highest scoring topic area of all the ordinance quality principles. Although ordinances included sanctions, the mechanism to detect violations through inspection is lacking, highlighting instances to create linkages among ordinance elements.

The lower scores on Flexibility and Complexity do not necessarily signal barriers to administration [26,40]. While a higher score on complexity may represent a higher level of protection based on the research literature, more complexity may be associated with more deviation from policy provisions during implementation [26]. In this sample, the lower Complexity score may indicate straightforward administration. However, the very low mean scores also reflect the low scores for the Policy Content principles, and by extension the limited incorporation of the information and policy requirements suggested by the research literature. Additional research is required to understand how to balance the complexity associated with increasing the incorporation of evidence with administration and implementation.

The presence of Flexibility is necessary to adapt to unique circumstances [40]. Regulatory policies offering flexibility such as buffer averaging and overlay zones were present but not widely utilized while the higher score for Variance was partially attributed to the inclusion of more general best administrative practices for governing authorized deviations from regulations. Protective policies such as conservation easements or fee simple acquisition and incentives such as off-site mitigation, restoration, and open space, which incorporate more voluntary action from the landholder, were also higher scoring. The moderate scores in this topic area suggest that these watersheds are incorporating some flexibility within their riparian buffer policies. More investigation, however, is necessary to understand how the provision of policies ranging from general to specific and regulatory to incentive affects implementation.

Unlike the Flexibility principle, Discretion alters policy provisions based on the judgment of staff involved in the review process [28,58]. The limited presence of this principle and the high percentage of Intermediate cases suggests that application reviewers are rarely granted sole authority to alter policy provisions. However, the addition of a secondary party could act both as a check on a single individual’s authority or introduce a party less familiar with the application or subject to political influences. This uncertainty is important as most cases co-occurred with Policy Description and relaxed policy provisions.

Our characterization of both mandates suggests that the Maryland Critical Areas mandate included more supportive design features, and we expected the Gunpowder-Patapsco watershed to have higher scores on individual ordinance quality principles [60]. Contrary to this hypothesis, the two study watersheds were not significantly different on any single principle. Further, Jordan Lake scored higher than the Gunpowder-Patapsco watershed on all six of the scored ordinance quality principles. It is possible that the mandate design features linked to plan quality by other studies did not promote higher ordinance quality given the definition of quality used in this study, which emphasized the inclusion of policy elements drawn from the research literature. The North Carolina mandate provided only limited guidance on the range of actions and policies to include, which may contribute to the lower Policy Content score at the watershed level. The Maryland mandate included more guidance on the substantial content of policies affecting riparian buffers, but the suggestion rather than requirement to extend those provisions outside of the critical area may be insufficient to translate to better ordinance quality at the watershed level.

Findings from the bivariate analysis of sociodemographic variables and ordinance quality are consistent with relationships observed with plan quality variables. Population density’s negative relationships with ordinance quality variables substantiate findings linking development pressure with lower quality plans. Specifically, higher population density was negatively correlated with Fact Base, Policy Description, and Policy Restrictions, which are the principles associated with riparian buffer policies with more of the facts, provisions, and restrictions that are protective of water resources. Similarly, planning capacity demonstrated moderate to strong, negative correlations with Goals, Fact Base, and Policy Description. While other studies associated planning capacity with higher-quality plans, the qualitative work of Mills and colleagues and Yli-Pelkonen and Niemelä helps explain the inverse relationship with planning capacity [34,35]. As planning capacity increases, ordinance quality scores on principles linked to incorporating scientific information decrease. The in-depth interviews suggest that lack of specialized expertise, difficulty communicating the justification to a wider audience and/or anticipation of conflict impede the translation of data into policy. These barriers may influence the content of ordinance as areas with more planning capacity expect conflict, especially in areas experiencing development pressures. The notable correlations within the Policy Administration (Protective Policies, BMP, Notification) lend support to this possibility as they reference policies that build on technical information and may require public justification.

5. Conclusions

As a caveat to our approach, the extensive literature on optimal riparian buffer design and functioning highlights a study limitation. For planning topics without an established body of research to draw upon for protocol creation, the approach risks equating policy focus with ordinance quality [40]. However, the low overall ordinance quality scores highlight topic areas where there is a gap between the substantial scientific information accumulated about riparian buffer design and functioning and the ordinances created and used by the planning profession. Our findings also reinforce conclusions from previous studies that a mandate is not sufficient to achieve better outcomes. Mandates that do not provide substantive guidance or geographically limited mandates that only encourage extension to other areas represent missed opportunities to safeguard water resources. Methodologically, this project demonstrates how this approach could improve both the substantive content of development management ordinances and the procedures in place to implement them. Future studies apply ordinance quality to other planning topics to contribute to the evaluation literature and deepen our understanding of how local policy quality affects implementation.