SAN DIEGO COUNTY'S PERFORMANCE METRICS INITIATIVE

Measuring What Matters: County Launches Key Performance Indicators | News | San Diego County News Center

PROMISE AND GAPS IN SYSTEMS ENGINEERING FUNDAMENTALS

BLUF (Bottom Line Up Front): San Diego County's new framework of 26 Key Performance Indicators represents progress toward data-driven governance but exhibits critical deficiencies in systems engineering fundamentals—including missing baselines, incomplete metric definitions, absent verification protocols, and inadequate stakeholder validation. While aligned with strategic priorities, the framework requires substantial technical maturation to achieve its stated objectives.

INTRODUCTION

San Diego County announced January 13, 2025, implementation of 26 Key Performance Indicators designed to measure organizational effectiveness across five strategic initiatives: community, empower, equity, justice, and sustainability[1]. The initiative consolidates previously fragmented performance data into a unified public measurement framework.

From a systems engineering perspective, effective KPI frameworks must satisfy specific requirements: clear definition of success criteria, measurable metrics with defined collection methods, achievable targets, mission relevance, and time-bound milestones[2][3]. This analysis evaluates the County's framework against these established principles and identifies gaps requiring remediation.

CRITICAL DEFICIENCIES IN KPI FRAMEWORK

Incomplete Baseline Definitions

The County's published KPIs demonstrate inconsistent baseline establishment. The Small-Local Contract Spend KPI (18% current state) and Employee Engagement KPI (86% engagement rate from biennial surveys) provide clear starting points[1]. However, critical baselines are missing:

Greenhouse Gas Emission KPI: Specifies "at least 1% each year" reduction target but provides no current emission levels, baseline year, measurement methodology, or scope definition (Scope 1, 2, or 3 emissions)[1]. This violates fundamental requirements for measurable objectives in systems engineering management standards[4].

Justice System Data Availability KPI: Reports 80% of programs sharing data publicly but lacks definition of total program count, criteria for "publicly available," update frequency requirements, or data quality standards[1].

These omissions prevent performance tracking and render the metrics unverifiable—fundamental failures in measurement system design.

Missing Measurability and Verification Protocols

Effective performance measurement requires defined data collection methods, verification procedures, and quality assurance mechanisms[5]. The County's announcement provides no information about:

• Data validation and verification procedures
• Measurement uncertainty and error analysis
• Inter-departmental data integration methods
• Independent verification of reported metrics
• Statistical process control methods

Research on government performance measurement demonstrates that third-party verification and transparent methodology disclosure are essential for credibility[6]. The County's commitment to annual reporting with quarterly internal updates represents baseline practice, but public documentation should include detailed measurement protocols to meet systems engineering standards.

Absent Stakeholder Validation

Systems engineering principles require stakeholder input throughout requirements definition[7]. The announcement indicates alignment with strategic initiatives but does not describe:

• Stakeholder engagement process for selecting the 26 KPIs
• Public input mechanisms during development
• Validation that selected metrics represent community priorities
• Trade-off analysis among competing performance objectives

Municipal performance management research indicates that citizen engagement in KPI selection significantly improves relevance and public trust[8][9]. The County provides no evidence of this foundational step.

Insufficient Definition Rigor

Analysis of the four published example KPIs reveals systematic definition gaps:

Small-Local Contract Spend (18%): No stated improvement target, no baseline year, no definition of "small" (revenue threshold? employee count?) or "local" (geographic boundary?), no discussion of quality versus cost trade-offs in procurement decisions.

Employee Engagement (86%): No trend data, no comparison to public sector benchmarks, no discussion of survey methodology or response rates, no disaggregation by department or demographic group to identify equity issues.

Justice System Data Availability (80%): Numerator and denominator undefined, no format or accessibility standards specified, no baseline or improvement trajectory established.

Greenhouse Gas Emissions: As noted above, lacks all essential baseline and methodology information required for meaningful climate accountability.

Each deficiency represents a violation of basic measurement principles documented in standards including ISO/IEC/IEEE 15288[4] and INCOSE Systems Engineering Handbook[3].

COMPARATIVE CONTEXT

National Best Practices

Leading jurisdictions including King County, Washington; Austin, Texas; and Montgomery County, Maryland implement performance measurement systems with[12][13]:

• Public dashboards with real-time or near-real-time data
• Detailed methodology documentation for each metric
• Longitudinal trend analysis with multi-year baselines
• Integration with budget and resource allocation processes
• Regular independent audits of data quality

San Diego County's initial framework lacks these essential elements. California's Open Data Initiative mandates that state and local agencies publish performance data in machine-readable formats with clear metadata[14]—requirements not yet met by the County's announcement.

Performance Measurement Research Findings

Academic research identifies critical success factors for public sector KPIs:

Data Quality Primacy: Behn (2003) and Moynihan (2008) demonstrate that performance measurement systems fail when data quality is poor or when metrics incentivize gaming behaviors rather than genuine improvement[15][16].

Balanced Measurement: The Kaplan and Norton balanced scorecard framework recommends measuring performance across multiple dimensions (financial, customer, internal processes, learning/growth) to prevent single-metric optimization at the expense of overall mission[17]. The County's framework provides insufficient information to assess balance.

Leading and Lagging Indicators: Effective frameworks include both predictive metrics (leading indicators) and outcome measures (lagging indicators). The County's examples appear weighted toward lagging indicators with limited early warning capability[18].

Climate Accountability Standards

California's Senate Bill 253 (Climate Corporate Data Accountability Act) establishes rigorous requirements for emission measurement and disclosure[23]. While primarily targeting corporations, these standards represent best practices applicable to government operations. Research demonstrates that jurisdictions with transparent, frequently updated, and third-party verified emission inventories achieve significantly better climate outcomes than those with opaque or infrequent reporting[24][25].

The County's greenhouse gas KPI falls substantially short of these standards.

SYSTEMS ENGINEERING RECOMMENDATIONS

Immediate Technical Corrections

1. Publish Comprehensive Metric Definitions: For each KPI, document operational definition, measurement methodology, data sources, collection frequency, responsible parties, baseline values with dates, target values with timeframes, and verification procedures.

2. Establish Greenhouse Gas Baseline: Immediately publish complete emission inventory including baseline year, Scope 1/2/3 breakdown, methodology aligned with Greenhouse Gas Protocol standards, and third-party verification status.

3. Document Data Quality Framework: Implement and publish data validation procedures, uncertainty analysis, statistical process control methods, and independent verification mechanisms.

4. Define All Terms: Provide precise definitions for terms like "small-local business," "publicly available," "engaged employee," eliminating ambiguity that prevents verification.

Process Enhancements

5. Implement Stakeholder Validation: Document community engagement process for KPI selection and establish ongoing feedback mechanisms.

6. Develop Leading Indicators: Supplement outcome-focused KPIs with predictive metrics enabling proactive management.

7. Enable Continuous Monitoring: Transition from annual reporting to real-time dashboards where technically feasible.

8. Integrate Resource Allocation: Establish explicit linkages between KPI performance and budget decisions.

9. Commission Independent Audits: Engage third parties to review data quality, methodology appropriateness, and alignment with best practices.

10. Publish Improvement Plans: For underperforming KPIs, document root cause analysis and corrective actions.

BROADER IMPLICATIONS

San Diego County's initiative occurs within increasing demand for government transparency and performance accountability. California's SB 272 (2023) strengthened open government requirements, mandating accessible public records and performance data[26]. Technology advances have lowered barriers to sophisticated measurement, but research indicates that organizational culture, leadership commitment, and staff capacity are equally critical to success[27].

Pew Research Center data shows declining public trust in government institutions, with performance transparency cited as key to rebuilding confidence[28]. Well-designed KPI frameworks contribute to trust-building when they demonstrate genuine accountability rather than selective reporting.

The County's framework represents a foundation requiring substantial development to meet this standard. Success will depend on technical rigor, transparent methodology, demonstrated responsiveness to data, and continuous improvement based on systems engineering principles.

CONCLUSION

San Diego County's 26 Key Performance Indicators demonstrate commitment to data-driven governance and public accountability. Strategic alignment across five initiatives provides coherent organizational direction. However, the framework exhibits fundamental systems engineering deficiencies that undermine its effectiveness:

• Critical baselines missing: Greenhouse gas emissions and justice data availability metrics lack essential baseline information preventing performance assessment.

• Verification protocols absent: No documented data quality, validation, or independent verification procedures.

• Stakeholder validation undocumented: No evidence of community engagement in metric selection.

• Definitions incomplete: All four published examples lack precision necessary for reproducible measurement.

These are not minor technical issues but fundamental failures in measurement system design. A KPI framework that cannot be verified, reproduced, or validated against stakeholder requirements cannot achieve stated objectives of measuring impact and identifying improvement opportunities.

The County should immediately address these deficiencies through comprehensive metric documentation, establishment of data quality frameworks, implementation of verification protocols, and stakeholder validation processes. Without these corrections, the initiative risks becoming performative transparency rather than genuine accountability.

As the framework matures, independent evaluation by academic researchers or government accountability organizations would validate progress and identify further enhancements. The commitment to measuring what matters is commendable; ensuring measurements meet rigorous standards will determine whether they drive genuine improvement in service to San Diego County residents.

---

VERIFIED SOURCES AND FORMAL CITATIONS

[1] County of San Diego Communications Office. "Measuring What Matters: County Launches Key Performance Indicators." County News Center, January 13, 2025. https://countynewscenter.com/measuring-what-matters-county-launches-key-performance-indicators/

[2] Project Management Institute. A Guide to the Project Management Body of Knowledge (PMBOK® Guide), 7th Edition. Newtown Square, PA: Project Management Institute, 2021.

[3] International Council on Systems Engineering (INCOSE). INCOSE Systems Engineering Handbook: A Guide for System Life Cycle Processes and Activities, 5th Edition. Hoboken, NJ: Wiley, 2023.

[4] ISO/IEC/IEEE 15288:2023. Systems and software engineering — System life cycle processes. International Organization for Standardization, 2023.

[5] Defense Acquisition University. Systems Engineering Fundamentals. Fort Belvoir, VA: Defense Acquisition University Press, 2021.

[6] Hatry, Harry P. Performance Measurement: Getting Results, 2nd Edition. Washington, DC: Urban Institute Press, 2006.

[7] IEEE Standard 29148-2018. ISO/IEC/IEEE International Standard - Systems and software engineering — Life cycle processes — Requirements engineering. Institute of Electrical and Electronics Engineers, 2018.

[8] Bovaird, Tony, and Elke Löffler. "Evaluating the Quality of Public Governance: Indicators, Models and Methodologies." International Review of Administrative Sciences 69, no. 3 (2003): 313-328.

[9] Wholey, Joseph S., Harry P. Hatry, and Kathryn E. Newcomer, eds. Handbook of Practical Program Evaluation, 4th Edition. Hoboken, NJ: Jossey-Bass, 2015.

[12] Government Finance Officers Association. "Best Practice: Establishment of Strategic Plans." GFOA, 2014. https://www.gfoa.org/materials/establishment-strategic-plans

[13] Bloomberg Philanthropies. What Works Cities Certification Framework. New York: Bloomberg Philanthropies, 2023. https://whatworkscities.bloomberg.org/certification/

[14] California Government Operations Agency. "Open Data Handbook." State of California, 2023. https://data.ca.gov/pages/open-data-handbook

[15] Behn, Robert D. "Why Measure Performance? Different Purposes Require Different Measures." Public Administration Review 63, no. 5 (2003): 586-606.

[16] Moynihan, Donald P. The Dynamics of Performance Management: Constructing Information and Reform. Washington, DC: Georgetown University Press, 2008.

[17] Kaplan, Robert S., and David P. Norton. "The Balanced Scorecard—Measures That Drive Performance." Harvard Business Review 70, no. 1 (1992): 71-79.

[18] Parmenter, David. Key Performance Indicators: Developing, Implementing, and Using Winning KPIs, 4th Edition. Hoboken, NJ: Wiley, 2020.

[23] California State Legislature. Senate Bill No. 253, Climate Corporate Data Accountability Act. Chapter 382, Statutes of 2023. https://leginfo.legislature.ca.gov/faces/billTextClient.xhtml?bill_id=202320240SB253

[24] Millard-Ball, Adam. "Do City Climate Plans Reduce Emissions?" Journal of Urban Economics 71, no. 3 (2012): 289-311.

[25] ICLEI - Local Governments for Sustainability. Greenhouse Gas Emission Inventories for Cities: Best Practices Guide. ICLEI, 2021.

[26] California State Legislature. Senate Bill No. 272, Open meetings. Chapter 534, Statutes of 2023. https://leginfo.legislature.ca.gov/faces/billNavClient.xhtml?bill_id=202320240SB272

[27] Mergel, Ines, R. Karl Rethemeyer, and Kimberley Isett. "Big Data in Public Affairs." Public Administration Review 76, no. 6 (2016): 928-937.

[28] Pew Research Center. "Public Trust in Government: 1958-2024." Pew Research Center, 2024. https://www.pewresearch.org/politics/2024/06/24/public-trust-in-government-1958-2024/

---

Author Note: This analysis applies systems engineering principles to evaluate municipal performance measurement frameworks. The author has over 20 years of systems engineering experience in defense and aerospace applications, including performance measurement system design for complex technical programs.

Document Classification: Unclassified
Distribution: Unlimited
Date: January 15, 2025