Source Inventory

docs/source-inventory.md · 38540 B · source sha256 485b87d91fde

This document is the working source inventory for urban-observatory's v0 prototype. It documents the public-data sources urban-observatory may draw on, the access posture for each, and the role each source plays in the interpretive chain. Specific dataset choices remain candidate; the inventory is not a manifest and not a claim of source sufficiency.

For the source-strategy posture (the Tier-A / Tier-B source ladder at the category level), see source-strategy.md. For the interpretive chain the inventory supports, see object-model.md. For provenance discipline, see methodology.md. For repo-local architecture context, see architecture.md.

on this page

Source inventory boundary

These stages are distinct and must not be blurred:

inventory       documents source candidates, access paths, provenance expectations,
                format, update posture, and limitations.  No data is pulled.

retrieval       pulls data from a documented source.  Produces raw artifacts.

extraction      transforms retrieved raw artifacts into chain entries
                (ImplementationSignal records, ImplementationFinding interpretations).
                Source-anchored, assumption-anchored.

interpretation  packages extraction into the Housing Element Implementation Risk Brief
                and related interpretive surfaces.

This document is the inventory stage only. An inventory entry says: "this source is reachable, structured like X, with limits Y." It does not say: "this source proves assumption Z."

Source inventory ≠ source sufficiency. Sufficiency is a finding-level question, decided per-assumption during chain extraction.

Status markers

Controlled vocabulary for v0_status:

pinned                  canonical URL or resource ID verified; structure inspected;
                        usable in v0 at the access level recorded
partial                 access path partially verified; some pattern works, others not yet
candidate               pinned at catalog level; not deep-probed for fields
needs verification      category real; specific source URL or resource ID not pinned
category-level only     recorded as a category without a specific source identified
deferred (Tier-2 case)  not pinned in v0 by default; revisited only for Tier-2 deeper
                        interpretation

Source classification

Controlled vocabulary for source_classification — what a source is in the inventory model:

raw evidence            primary source carrying the data itself
operational aggregate   curated / aggregated view over raw upstream feeds
companion source        pairs with another canonical source for a chain pair
identity anchor         provides cross-source join keys; not a signal source
enrichment layer        context layer (zoning, parcels, administrative overlays)
meta-source / discovery viewer / dashboard / catalog over raw sources
reference / lookup      controlled-vocabulary / code-lookup datasets
deferred (Tier-2 case)  category-level only; not pinned in v0
candidate               pinned at catalog level; intended classification pending deep probe

source_classification answers what a source is. v0_status answers what is verified about access to it. The two are independent fields.

Tier A — canonical sources

Each entry below belongs to the housing / planning subset of the Tier-A signal-visibility posture described in source-strategy.md. These seven pinned source categories are the sources exercised in v0; they do not exhaust the cross-system Tier-A posture. Specific dataset choices are working candidates, not v0-canonical doctrine. (A full re-tag of every source's tier field onto the revised visibility / interpretation ladder is a separate follow-on; the pinned content here is unchanged.)

A.1 — SF Housing Element site inventory / opportunity sites

Field Value
Fieldsource_id Valuesf_he_appendix_b4
Fieldsource_name ValueAppendix B.4 Sites Inventory (SF submission)
Fieldsource_owner ValueCalifornia HCD (template); SF Planning Department (submission)
Fieldsource_category ValueHousing Element site inventory / opportunity sites
Fieldsource_classification Valueraw evidence
Fieldtier ValueA
Fieldv0_status Valuepinned
Fieldurl_or_identifier Valuehttps://sfplanning.org/sites/default/files/documents/housing-for-all/Appendix_B4.xlsx
Fieldplatform Valuestatic .xlsx on SF Planning site
Fieldaccess_type Valuedirect download
Fieldformat ValueXLSX (HCD-standard multi-sheet template)
Fieldupdate_frequency Valueper Housing Element submission cycle
Fieldcoverage_period Value2023–2031 RHNA-cycle period (Dec-2022 submission)
Fieldgeographic_scope Valuewhole SF (~121,053 SF rows in Table A; 121,055 total)
Fieldprimary_keys ValueBLKLOT (hyphenated, e.g., 3704-045); Site Address; ZIP
Fieldrelevant_objects ValueOpportunitySite (anchor); Assumption (per-site capacity); Source
Fieldrelevant_chain_stage ValueAssumption
Fieldreported_category_relevance Valueopportunity-site capacity (structures the RHNA target); Site Status; Pipeline flag; Publicly-Owned flag; Identified-in-Prior-Cycle flag
Fieldknown_limitations ValueMega-Development-Agreement projects (Treasure Island, Candlestick, Parkmerced, Stonestown, Potrero Power Station, Pier 70, Mission Rock, Hope SF Sunnydale, Freedom West, 610 Brannan / Flower Mart) live in Appendix B.1, not B.4
Fielddata_quality_caveats ValueCapacity is planning-stated, not delivery-verified. ~99.6% of rows are < 5-unit ADU-strategy parcels. Only ~171 sites have capacity ≥ 20; 80% of those are already Pipeline-flagged.
Fieldprovenance_requirements ValueCite as HCD-template Appendix B.4 (SF submission); record table name and BLKLOT; note hyphenated BLKLOT format

A.2 — SF APR / Housing Element annual progress reporting

Field Value
Fieldsource_id Valuehcd_apr_portal
Fieldsource_name ValueHCD Annual Progress Reports portal
Fieldsource_owner ValueCalifornia HCD
Fieldsource_category ValueAPR / Housing Element annual progress reporting
Fieldsource_classification Valueraw evidence
Fieldtier ValueA
Fieldv0_status Valuepinned
Fieldurl_or_identifier Valuehttps://www.hcd.ca.gov/planning-and-community-development/annual-progress-reports
Fieldplatform ValueHCD web portal + California Open Data Portal
Fieldaccess_type Valuelanding page + downloadable raw datasets + APR Dashboard
Fieldformat ValueCSV (raw); interactive dashboard
Fieldupdate_frequency Valueannual reporting cycle
Fieldcoverage_period Value2018–present (raw datasets)
Fieldgeographic_scope Valuestatewide; SF filtered as one jurisdiction
Fieldprimary_keys ValueJurisdiction; Reporting Year; project-level rows in site-level tables
Fieldrelevant_objects ValueAssumption (against adopted HE targets); ImplementationSignal (production, permits, completions); Source; Confidence
Fieldrelevant_chain_stage ValueAssumption + ImplementationSignal
Fieldreported_category_relevance ValueRHNA progress — HCD's primary benchmark is issued building permits (verbatim from HCD landing)
Fieldknown_limitations ValueMethodology is distributed across HCD APR Forms, Instructions PDF, dataset metadata, and Building Blocks guidance — not consolidated in one place
Fielddata_quality_caveats ValueSF most-recent APR currency requires direct verification. The "explicit counting" layer is documented; the implementation-assumption layer is urban-observatory's interpretive contribution, not explicit in HCD source materials
Fieldprovenance_requirements ValueCite HCD APR statewide dataset (or jurisdiction-level table) with reporting year and table identifier; note retrieval date

A.3 — SF Planning development pipeline

Field Value
Fieldsource_id Valuesf_dev_pipeline_6jgi-cpb4
Fieldsource_name ValueSan Francisco Development Pipeline
Fieldsource_owner ValueSF Planning + SF DBI (combined); DataSF (publishing)
Fieldsource_category ValueSF Planning development pipeline data
Fieldsource_classification Valueoperational aggregate (over raw PPTS feeds at A.4 and raw PTS feed at A.5)
Fieldtier ValueA
Fieldv0_status Valuepinned
Fieldurl_or_identifier ValueDataSF Socrata resource ID 6jgi-cpb4
Fieldplatform ValueDataSF / Socrata (SODA API)
Fieldaccess_type ValueAPI + CSV / JSON / GeoJSON download
Fieldformat ValueSocrata-hosted tabular
Fieldupdate_frequency Valueregular
Fieldcoverage_period Valueactive pipeline (current and historic statuses)
Fieldgeographic_scope Valuewhole SF
Fieldprimary_keys Valueplanning case number; BLKLOT (concatenated, e.g., 3704045); address
Fieldrelevant_objects ValueHousingProject; ImplementationSignal; OpportunitySite ↔ HousingProject relationship statuses
Fieldrelevant_chain_stage ValueImplementationSignal
Fieldreported_category_relevance ValuePipeline status values: PL Filed, PL Approved, BP Filed, BP Approved, BP Issued, Construction. TCO/CFC exit trigger. DA (Development Agreement) field flags major multi-phased projects
Fieldknown_limitations ValueAggregate over upstream raw feeds; for Tier-2 deeper interpretation, raw PPTS (A.4) and PTS (A.5) may surface signal that Pipeline collapses
Fielddata_quality_caveats ValueVerbatim metadata: "Significant data and systems improvements are underway; data may change at any time." BLKLOT format = concatenated; normalize against B.4
Fieldprovenance_requirements ValueCite DataSF resource ID 6jgi-cpb4 + data_as_of + query / filter
A.3.b — Production / completion companion (post-TCO/CFC)
Field Value
Fieldsource_id Valuesf_housing_production_xdht-4php
Fieldsource_name ValueSan Francisco Housing Production
Fieldsource_owner ValueSF Planning Department; DataSF (publishing)
Fieldsource_category Valuepost-TCO/CFC companion to pipeline
Fieldsource_classification Valuecompanion source (post-TCO/CFC pair to A.3)
Fieldtier ValueA
Fieldv0_status Valuepinned
Fieldurl_or_identifier ValueDataSF Socrata resource ID xdht-4php
Fieldplatform ValueDataSF / Socrata (SODA API)
Fieldaccess_type ValueAPI + CSV / JSON / GeoJSON download
Fieldformat ValueSocrata-hosted tabular
Fieldupdate_frequency Valueannual (April 1)
Fieldcoverage_period Value2005–present
Fieldgeographic_scope Valuewhole SF
Fieldprimary_keys Valueplanning case number; BLKLOT (concatenated); address
Fieldrelevant_objects ValueHousingProject (completion side); Outcome (delivered units); Source
Fieldrelevant_chain_stage ValueImplementationSignal (delivery milestone)
Fieldreported_category_relevance ValueCompleted units counted at TCO/CFC. Pipeline and Production are mutually exclusive views. Affordable-unit tiers reported
Fieldknown_limitations ValueCounts at completion, not issuance — authorized-but-not-completed sits in Pipeline. Pipeline and Production are not summed
Fielddata_quality_caveats ValueShares Pipeline's data-systems caveat by adjacency. Coverage from 2005 — earlier delivered units are not included
Fieldprovenance_requirements ValueCite DataSF resource ID xdht-4php + query / filter; when pairing with Pipeline for entitlement-to-delivery interpretation, cite both resource IDs

A.4 — SF Planning applications / entitlement records (PPTS raw)

Field Value
Fieldsource_id Valuesf_planning_records_projects_qvu5-m3a2
Fieldsource_name ValuePlanning Department Records - Projects (PRJ)
Fieldsource_owner ValueSF Planning Department
Fieldsource_category ValueSF Planning applications / entitlement records
Fieldsource_classification Valueraw evidence (upstream feed for A.3 Pipeline aggregate)
Fieldtier ValueA
Fieldv0_status Valuepinned
Fieldurl_or_identifier ValueDataSF Socrata resource ID qvu5-m3a2
Fieldplatform ValueDataSF / Socrata (SODA API)
Fieldaccess_type ValueAPI + CSV / JSON download
Fieldformat ValueSocrata-hosted tabular
Fieldupdate_frequency Valuenightly
Fieldcoverage_period Valueactive Accela project records
Fieldgeographic_scope Valuewhole SF
Fieldprimary_keys ValueRECORD_ID (PRJ); BLOCK + LOT; PROJECT_ADDRESS
Fieldrelevant_objects ValueHousingProject (raw project case records); ImplementationSignal (entitlement-stage activity); OpportunitySite (via BLOCK + LOT)
Fieldrelevant_chain_stage ValueImplementationSignal
Fieldreported_category_relevance ValueRaw upstream for the Planning side of Pipeline's combined view. 103 columns of project, housing-feature, environmental-review, and unit-count fields
Fieldknown_limitations ValuePipeline (A.3) provides easier first-pass interpretation; raw PRJ records have higher column count and may include cases not yet promoted to Pipeline
Fielddata_quality_caveats ValueAccela record states (open, closed, withdrawn) require careful filtering; non-housing PRJ records will be present
Fieldprovenance_requirements ValueCite DataSF resource ID qvu5-m3a2 + RECORD_ID + query / filter
Field Value
Fieldsource_id Valuesf_planning_records_non_projects_y673-d69b
Fieldsource_name ValuePlanning Department Records - Non-Projects
Fieldsource_owner ValueSF Planning Department
Fieldsource_category ValueSF Planning applications / entitlement records (companion to qvu5-m3a2)
Fieldsource_classification Valueraw evidence (upstream feed for A.3 Pipeline aggregate)
Fieldtier ValueA
Fieldv0_status Valuepinned
Fieldurl_or_identifier ValueDataSF Socrata resource ID y673-d69b
Fieldplatform ValueDataSF / Socrata (SODA API)
Fieldaccess_type ValueAPI + CSV / JSON download
Fieldformat ValueSocrata-hosted tabular
Fieldupdate_frequency Valuenightly
Fieldgeographic_scope Valuewhole SF
Fieldprimary_keys ValueRECORD_ID; BLOCK + LOT; PROJECT_ADDRESS; PARENT_ID (links to PRJ parent)
Fieldrelevant_objects ValueImplementationSignal (supplemental application activity); child-record link to HousingProject via PARENT_ID
Fieldrelevant_chain_stage ValueImplementationSignal
Fieldreported_category_relevance ValueSupplemental applications during project review (variance, conditional use, environmental review filings); finer signal layer than Pipeline status alone
Fieldknown_limitations ValueRecords from Accela database excluding PRJ records; includes child records linked to PRJ parents
Fielddata_quality_caveats ValueHeterogeneous record types; filtering by RECORD_TYPE required; non-housing records present
Fieldprovenance_requirements ValueCite DataSF resource ID y673-d69b + RECORD_ID + PARENT_ID where applicable

A.5 — DBI building permit records (PTS raw)

Field Value
Fieldsource_id Valuedbi_building_permits_i98e-djp9
Fieldsource_name ValueDBI Building Permits
Fieldsource_owner ValueSF Department of Building Inspection; DataSF (publishing)
Fieldsource_category ValueDBI building permit records
Fieldsource_classification Valueraw evidence (upstream feed for A.3 Pipeline aggregate)
Fieldtier ValueA
Fieldv0_status Valuepinned
Fieldurl_or_identifier ValueDataSF Socrata resource ID i98e-djp9
Fieldplatform ValueDataSF / Socrata (SODA API)
Fieldaccess_type ValueAPI + CSV / JSON / GeoJSON download (bulk export warned against in Excel due to ~1M rows)
Fieldformat ValueSocrata-hosted tabular
Fieldupdate_frequency Valuenightly / multiple times per hour
Fieldcoverage_period Valuefull DBI permits history; since 2024-12-10 only new and updated records republished, identifiable via data_as_of and data_loaded_at
Fieldgeographic_scope Valuewhole SF
Fieldprimary_keys ValuePermit Number; Block + Lot (separate fields); address; Location (lat/lon)
Fieldrelevant_objects ValueImplementationSignal (permit activity, status, dates); links to OpportunitySite and HousingProject via BLKLOT
Fieldrelevant_chain_stage ValueImplementationSignal
Fieldreported_category_relevance ValueIssued building permits = HCD's primary RHNA benchmark
Fieldknown_limitations ValueBulk export not Excel-friendly; SODA queries needed for site-scoped fetches. Block and Lot are separate fields. Site-permit / fire-only / re-roofing flags must be respected when filtering for housing-relevant permits
Fielddata_quality_caveats ValueLicense: Open Data Commons Public Domain Dedication (PDDL). Housing-relevant permit filtering requires care
Fieldprovenance_requirements ValueCite DataSF resource ID i98e-djp9 + data_as_of / data_loaded_at + Permit Number + Block + Lot

A.6 — Planning Code and zoning GIS layers

Three current canonical bulk datasets, all published by SF Planning, all PDDL-licensed, all updated quarterly. All source_classification: enrichment layer. All tier: A. All v0_status: pinned.

source_id Resource ID Coverage
source_idsf_zoning_districts_3i4a-hu95 Resource ID3i4a-hu95 Coveragebase zoning districts; multipolygon; fields include the_geom, zoning_sim, districtname, gen, zoning, codesection
source_idsf_height_bulk_districts_h9wh-cg3m Resource IDh9wh-cg3m Coverageheight-and-bulk districts; multipolygon
source_idsf_special_use_districts_5yf5-ms5f Resource ID5yf5-ms5f CoverageSpecial Use District overlays per Article 2 of the SF Planning Code; multipolygon

Common fields:

  • source_owner — SF Planning Department
  • source_category — Planning Code and zoning GIS layers
  • platform / access_type / format — DataSF / Socrata (geospatial) / API + CSV / JSON / GeoJSON / multipolygon GIS layer
  • update_frequency — quarterly
  • primary_keys — the_geom (geographic primary key); district / overlay attribute fields
  • relevant_objects — Constraint; OpportunitySite enrichment
  • relevant_chain_stage — site-context enrichment
  • reported_category_relevance — defines the zoning / height-bulk / SUD constraint surface that B.4 capacity assumptions are built against
  • known_limitations — district-level; per-parcel application requires spatial join against parcels. SUD 5yf5-ms5f is overlay-only and must be combined with base Districts
  • data_quality_caveats — quarterly cadence; codesection references may lag formal Planning Code amendments
  • provenance_requirements — cite DataSF resource ID + rowsUpdatedAt + spatial filter

A.7 — Parcel / assessor / land-use GIS layers

Two parcel datasets plus one assessor records dataset.

Field Value
Fieldsource_id Valuesf_parcels_overlay_9grn-xjpx
Fieldsource_name ValueParcels with Overlay Attributes
Fieldsource_owner ValueCity and County of San Francisco (DataSF)
Fieldsource_category ValueParcel / assessor / land-use GIS
Fieldsource_classification Valueidentity anchor
Fieldtier ValueA
Fieldv0_status Valuepinned
Fieldurl_or_identifier ValueDataSF Socrata resource ID 9grn-xjpx
Fieldplatform ValueDataSF / Socrata
Fieldaccess_type ValueAPI + CSV / JSON / GeoJSON download
Fieldformat Valuetabular with geospatial fields
Fieldgeographic_scope Valuewhole SF
Fieldprimary_keys Valueparcel identifier (BLOCK + LOT); geographic boundary; pre-joined overlay fields (neighborhoods, supervisor districts, police districts, planning districts)
Fieldrelevant_objects Valueidentity anchor for cross-source joins; OpportunitySite enrichment; Constraint (administrative overlays)
Fieldrelevant_chain_stage Valueidentity anchor / site-context enrichment
Fieldreported_category_relevance ValueNot a reporting source; provides the parcel identity layer underlying other sources
Fieldknown_limitations ValuePre-joined overlays may lag if administrative boundaries change between updates
Fielddata_quality_caveats ValueDerived from base parcels (acdm-wktn); use base parcels if overlay fields are not needed
Fieldprovenance_requirements ValueCite DataSF resource ID 9grn-xjpx + BLOCK + LOT
Field Value
Fieldsource_id Valuesf_parcels_base_acdm-wktn
Fieldsource_name ValueParcels – Active and Retired
Fieldsource_owner ValueCity and County of San Francisco
Fieldsource_category ValueParcel / assessor / land-use GIS
Fieldsource_classification Valueidentity anchor
Fieldtier ValueA
Fieldv0_status Valuepinned
Fieldurl_or_identifier ValueDataSF Socrata resource ID acdm-wktn
Fieldplatform ValueDataSF / Socrata
Fieldaccess_type ValueAPI + CSV / JSON / GeoJSON download
Fieldformat Valuetabular with geospatial fields
Fieldcoverage_period Valueactive and retired parcels with date-added / date-retired
Fieldprimary_keys Valueparcel identifier; date-added; date-retired
Fieldrelevant_objects Valueidentity anchor / OpportunitySite enrichment
Fieldrelevant_chain_stage Valueidentity anchor
Fieldreported_category_relevance ValueCanonical authority for parcel existence at a point in time
Fieldknown_limitations ValueDoes not include the administrative overlays of 9grn-xjpx
Fielddata_quality_caveats ValueRetired parcels are included — filter on date-retired for current view
Fieldprovenance_requirements ValueCite DataSF resource ID acdm-wktn + BLOCK + LOT + as-of date
Field Value
Fieldsource_id Valuesf_assessor_property_rolls_wv5m-vpq2
Fieldsource_name ValueAssessor Historical Secured Property Tax Rolls
Fieldsource_owner ValueSF Office of the Assessor-Recorder
Fieldsource_category ValueParcel / assessor / land-use GIS (assessor side)
Fieldsource_classification Valueenrichment layer
Fieldtier ValueA
Fieldv0_status Valuepinned
Fieldurl_or_identifier ValueDataSF Socrata resource ID wv5m-vpq2
Fieldplatform ValueDataSF / Socrata
Fieldaccess_type ValueAPI + CSV / JSON download
Fieldformat Valuetabular
Fieldupdate_frequency Valueannual / per fiscal-year roll
Fieldcoverage_period ValueJuly 1, 2007 to June 30, 2024 (current published coverage)
Fieldgeographic_scope Valuewhole SF
Fieldprimary_keys Valueparcel identifier (BLOCK + LOT); fiscal-year roll
Fieldrelevant_objects ValueConstraint (ownership context, use-code); OpportunitySite enrichment
Fieldrelevant_chain_stage Valuesite-context enrichment
Fieldreported_category_relevance ValueProperty value, owner, last sale, square footage, use codes, classification codes
Fieldknown_limitations Value"Historical" naming: dataset covers through June 2024. Current-year rolls may require a separate / sibling dataset. LLC / trust ownership opacity remains a known limitation
Fielddata_quality_caveats ValueUse codes use Assessor-Recorder reference codes (companion reference / lookup datasets pa56-ek2h Property Class, dx7g-zwbx Neighborhood, g77e-ikb4 Exemption, 2367-5au8 Assessor Block Maps; all source_classification: reference / lookup)
Fieldprovenance_requirements ValueCite DataSF resource ID wv5m-vpq2 + fiscal-year roll + BLOCK + LOT

Meta-sources and discovery surfaces

These are navigation / interpretive entry points, not raw Tier A evidence. They surface or aggregate raw sources cataloged above. On their own they do not bear on Assumption → ImplementationSignal → ImplementationFinding interpretation; they help locate the raw sources that do.

M.1 — SF Planning Property Information Map (PIM)

Field Value
Fieldsource_id Valuesf_pim
Fieldsource_name ValueSF Planning Property Information Map
Fieldsource_owner ValueSF Planning Department
Fieldsource_classification Valuemeta-source / discovery (site-level lookup tool)
Fieldurl_or_identifier Valuehttps://sfplanninggis.org/pim/
Fieldplatform ValueArcGIS-based GIS viewer; underlying ArcGIS REST endpoints not pinned in v0; bulk substrate lives in the Tier A datasets above
Fieldaccess_type Valueinteractive web viewer (site-level by address / parcel / planning application number)
Fieldprimary_keys ValueBLKLOT; address; planning application number
Fieldchain_relevance Valuesite-level navigation only; raw chain support routes through the Tier A sources surfaced by PIM
Fieldv0_status Valuepinned (as meta-source)
Fieldknown_limitations Value"Housing Element Reused Sites and Low-Income Sites" layer is a subset / visualization; the comprehensive table is Appendix B.4
Fieldprovenance_requirements ValueWhen used as a lookup tool, cite the underlying authoritative source for any field extracted via PIM

M.2 — SF Housing Dashboard

Field Value
Fieldsource_id Valuesf_housing_dashboard
Fieldsource_name ValueSan Francisco Housing Dashboard
Fieldsource_owner ValueSF Planning Department
Fieldsource_classification Valuemeta-source / discovery (aggregated reporting + discovery hub)
Fieldurl_or_identifier Valuehttps://sfplanning.org/san-francisco-housing-dashboard
Fieldplatform ValueSF Planning web portal (interactive dashboard)
Fieldaccess_type Valueweb UI; references underlying DataSF datasets via data-source links
Fieldchain_relevance Valueaggregated reporting only; raw chain support routes through xdht-4php (A.3.b) and 6jgi-cpb4 (A.3)
Fieldv0_status Valuepinned (as meta-source)
Fieldreported_category_relevance Value"Authorized Units" = SF's primary RHNA-tracking metric (verbatim from dashboard)
Fielddata_quality_caveats ValuePipeline and Completed views are mutually exclusive
Fieldprovenance_requirements ValueWhen citing a dashboard view, also cite the underlying dataset that backs it

M.3 — MOHCD discovery surfaces

source_id URL Role
source_idmohcd_dashboards_data URLhttps://sfmohcd.org/dashboards-and-data RoleMOHCD Dashboards and Data hub
source_idmohcd_bmr_listings URLhttps://housing.sfgov.org/listings/for-sale RoleFirst Come First Served BMR Listings
source_idmohcd_plans_reports URLhttps://www.sf.gov/mohcd-plans-and-reports RoleMOHCD Plans and Reports
source_idmohcd_department_landing URLhttps://sf.gov/departments/mayors-office-housing-and-community-development RoleMOHCD main department landing

Common fields: source_classification: meta-source / discovery; chain_relevance: aggregated reporting / navigation only; v0_status: pinned (as meta-source). Raw MOHCD evidence routes through the Tier B B.3 candidate datasets below.

Tier B — candidate dependency-interpretation sources

Tier B is the dependency-interpretation layer (causal / delivery-risk / cross-system) — gated and not exercised in v0 by default (see source-strategy.md). The candidate sources below illuminate the conditions under which implementation occurs; reading them as dependency interpretation is Tier-B work, triggered by a specific finding, claim, reviewer need, or demo concept.

B.1 — Capital plan and infrastructure investment references

source_classification: deferred (Tier-2 case) · v0_status: category-level only · Chain consumption: Assumption (infrastructure-dependency assumptions per flagged site); InfrastructureDependency. v0 need: none for Tier-1; activated only when a flagged site's assumption depends on infrastructure not yet sequenced. Verification posture: no pinning needed unless Tier-2 work surfaces a specific infrastructure-dependent assumption.

B.2 — Transportation plans and transit access

source_classification: deferred (Tier-2 case) · v0_status: category-level only · Chain consumption: Assumption (TOD / transit-proximity capacity assumptions); Constraint. v0 need: none for Tier-1; PIM surfaces transit-served-area flags at site level as a partial substitute. Verification posture: no pinning needed unless Tier-2 work surfaces a transit-dependent capacity assumption.

B.3 — Affordable housing project / funding signals (MOHCD)

Three candidate-pinned datasets. v0 posture: candidate Tier B (not promoted to Tier A); deep-probe required before any signal extraction.

Field Value
Fieldsource_id Valuemohcd_affordable_housing_pipeline_aaxw-2cb8
Fieldsource_name ValueMOHCD Affordable Housing Pipeline
Fieldsource_owner ValueMayor's Office of Housing and Community Development (MOHCD) + Office of Community Investment and Infrastructure (OCII) — joint
Fieldsource_classification Valuecandidate (intended: raw evidence pending deep probe)
Fieldtier ValueB
Fieldv0_status Valuecandidate
Fieldurl_or_identifier ValueDataSF Socrata resource ID aaxw-2cb8
Fieldplatform / access_type / format ValueDataSF / Socrata / API + CSV / JSON / tabular (~70+ columns)
Fieldupdate_frequency / coverage_period Valuesnapshot / active pipeline
Fieldrelevant_objects ValueHousingProject (affordable); Assumption (affordable-pipeline assumptions); ImplementationSignal (funding-side momentum)
Fieldrelevant_chain_stage ValueImplementationSignal (funding)
Fieldreported_category_relevance ValueMOHCD/OCII affordable-housing pipeline, including units produced through the Inclusionary Affordable Housing Program; nonprofit / for-profit partnerships; AMI 20%–150%
Fieldknown_limitations ValueJoint MOHCD+OCII view; partition between the two not pinned. Field-level structure not deep-probed
Fieldprovenance_requirements ValueCite DataSF resource ID aaxw-2cb8 + snapshot date
Field Value
Fieldsource_id Valuemohcd_affordable_housing_portfolio_pyxv-n29e
Fieldsource_name ValueMOHCD Affordable Housing Portfolio
Fieldsource_owner ValueMayor's Office of Housing and Community Development (MOHCD)
Fieldsource_classification Valuecandidate
Fieldtier ValueB
Fieldv0_status Valuecandidate
Fieldurl_or_identifier ValueDataSF Socrata resource ID pyxv-n29e
Fieldplatform / access_type / format ValueDataSF / Socrata / API + CSV / JSON / tabular (34 columns)
Fieldcoverage_period Valuecompleted affordable rental and ownership developments
Fieldrelevant_objects ValueHousingProject (affordable, completed); Outcome (affordable delivered units)
Fieldrelevant_chain_stage ValueImplementationSignal (delivery, affordable side)
Fieldreported_category_relevance ValueMOHCD/OCII-financed and Planning Code §415 inclusionary 10+ unit projects, completed
Fieldprovenance_requirements ValueCite DataSF resource ID pyxv-n29e + retrieval date
Field Value
Fieldsource_id Valuemohcd_non_unit_inclusionary_alternatives_g6kh-9pnv
Fieldsource_name ValueMOHCD Non-Unit Inclusionary Housing Alternatives
Fieldsource_owner ValueMayor's Office of Housing and Community Development (MOHCD)
Fieldsource_classification Valuecandidate (intended: raw evidence)
Fieldtier ValueB
Fieldv0_status Valuecandidate
Fieldurl_or_identifier ValueDataSF Socrata resource ID g6kh-9pnv
Fieldplatform / access_type / format ValueDataSF / Socrata / API + CSV / JSON / tabular
Fieldrelevant_objects ValueImplementationSignal (alternative-compliance signals — fee payments, land dedication, off-site BMR); PolicyProgram (§415 Inclusionary)
Fieldrelevant_chain_stage ValueImplementationSignal
Fieldreported_category_relevance Value§415 Inclusionary Housing requirements fulfilled through alternatives to on-site units — fee payments, land dedication, off-site construction
Fieldprovenance_requirements ValueCite DataSF resource ID g6kh-9pnv + retrieval date

B.4 — Construction cost, interest rate, vacancy indicators

source_classification: deferred (Tier-2 case) · v0_status: category-level only · Chain consumption: Assumption (feasibility); ImplementationSignal (market-condition signals); MarketSignal candidate subtype. v0 need: none for Tier-1; public-data-only discipline rules out vendor sources. Verification posture: no pinning needed unless Tier-2 work surfaces a market-condition-dependent assumption.

source_classification: deferred (Tier-2 case) · v0_status: category-level pinned · Chain consumption: PolicyProgram; Assumption (program-level). v0 need: category-level only for Tier-1; specific documents identified case-by-case (includes SF's adopted Housing Element 2023–2031 — the policy document that sets the assumption surface Appendix B.4 derives from). Verification posture: case-by-case as a specific PolicyProgram interpretation requires.

Tier C — selective / deferred supporting sources

Tier C sources are selective for Tier-2 deeper interpretation of flagged sites. They are listed at the category level in source-strategy.md. Specific Tier C datasets are not pinned in v0 by default. All Tier C entries carry source_classification: deferred (Tier-2 case) and v0_status: deferred (Tier-2 case).

ParticipatorySignal candidate sources (public comments, meeting transcripts, comment letters, survey summaries) are reserved by default in v0. If activated, representativeness and bias documentation are required per source.

Source inventory fields / metadata model

The field set used throughout this inventory:

source_id                       short, lowercase, underscore-delimited; stable;
                                used as canonical short reference
source_name                     human-readable name
source_owner                    publishing organization (and underlying agency if different)
source_category                 category from docs/source-strategy.md
source_classification           raw evidence / operational aggregate / companion source /
                                identity anchor / enrichment layer / meta-source / discovery /
                                reference / lookup / deferred (Tier-2 case) / candidate
tier                            A / B / C
url_or_identifier               canonical URL or DataSF / ArcGIS resource ID
platform                        DataSF/Socrata, ArcGIS, HCD portal, static download, etc.
access_type                     direct download, API, web viewer, etc.
format                          CSV, JSON, GeoJSON, XLSX, PDF, GIS layer, etc.
update_frequency                nightly, quarterly, per-cycle, etc. (when known)
coverage_period                 time range covered by the data (when known)
geographic_scope                SF whole-city, neighborhood, statewide, etc.
primary_keys                    join keys; BLKLOT format variant noted where relevant
relevant_objects                which object-model primitives this source supports
relevant_chain_stage            Assumption / ImplementationSignal / ImplementationFinding /
                                InterventionCandidate
reported_category_relevance     how this source maps to APR / Pipeline / Production reporting
known_limitations               structural limits
data_quality_caveats            data-quality notes (verbatim where dataset publishes them)
provenance_requirements         what to cite when using this source
v0_status                       pinned / partial / candidate / needs verification /
                                category-level only / deferred (Tier-2 case)

Notes on field use:

  • source_classification is the role discipline — what the source is in the inventory model. v0_status is the access-verification state. Separate fields, separate questions.
  • relevant_chain_stage is restricted to the four chain stages from object-model.md: Assumption / ImplementationSignal / ImplementationFinding / InterventionCandidate. Supporting primitives (Outcome, Constraint, OpportunitySite, etc.) belong in relevant_objects, not relevant_chain_stage.
  • Meta-source entries omit fields not applicable to viewers and substitute chain_relevance (descriptive prose) for relevant_chain_stage since meta-sources do not bear on the chain directly.

This field set is documentation, not an executable schema. The inventory is Markdown-first and consistent with data-dictionary.md. Promotion to a runnable manifest (YAML, JSON, CSV) is deferred until a v0 consumer requires it.

Known caveats

The cautions below belong with the inventory itself; they are not commentary about it.

  1. Source inventory ≠ source sufficiency. Naming a source in the inventory does not certify that the source carries enough signal to weaken or support a given assumption. Sufficiency is a finding-level question, decided per-assumption during chain extraction.
  2. Metadata accessibility ≠ row-level use. A confirmed-accessible metadata endpoint says nothing about whether the rows behind it are clean, current, or interpretable.
  3. DataSF / Socrata resource IDs may change. IDs are generally stable but not guaranteed permanent. Cite IDs with retrieval / verification dates.
  4. Source dashboards are often JS-rendered. Cite the linked dataset, not the dashboard view, when source-anchoring a signal.
  5. PIM is site-level via the viewer; bulk is via the canonical Tier A datasets. Do not treat PIM as a bulk dataset.
  6. Appendix B.4 is large and statewide-format. ~99.6% of rows are < 5-unit ADU-strategy parcels. Only ~171 sites have capacity ≥ 20; 80% of those are already Pipeline-flagged. The aggregate ADU strategy means testing the chain at any individual RH-1 site does not really test SF's strategy.
  7. SF Pipeline has data-quality caveats. Verbatim metadata: "Significant data and systems improvements are underway; data may change at any time." Any signal sourced from Pipeline must carry this caveat.
  8. Public-input sources are deferred / cautious. ParticipatorySignal candidate sources are reserved by default in v0.
  9. BLKLOT formats vary. Any cross-source join requires explicit normalization.
Source BLKLOT format Example
SourceAppendix B.4 BLKLOT formathyphenated Example3704-045
SourceSF Pipeline 6jgi-cpb4 BLKLOT formatconcatenated Example3704045
SourceDBI Permits i98e-djp9 BLKLOT formatseparate Block + Lot Example3704 + 045
SourcePlanning Non-Projects y673-d69b BLKLOT formatseparate Block + Lot Example3704 + 045
SourceParcels with Overlay 9grn-xjpx BLKLOT formatparcel identifier (BLOCK + LOT) Exampleper dataset
SourceAssessor Property Rolls wv5m-vpq2 BLKLOT formatparcel identifier (BLOCK + LOT) Exampleper dataset

Lots with letter suffix (e.g., 0044-002A / 0044002A) require consistent handling.

  1. Category-level entry ≠ pinned source. Tier B abstract entries (B.1, B.2, B.4) and Tier C entries are category-level placeholders, not pinned sources. Do not let category-level entries imply more verification than has happened.
  2. Pipeline (6jgi-cpb4) is an operational aggregate, not a primary source. Pipeline curates upstream raw feeds: PPTS (Planning Accela: qvu5-m3a2 + y673-d69b) and PTS (DBI: i98e-djp9). For Tier-2 deeper interpretation, dropping back to the raw feeds may surface signal that Pipeline collapses.

Posture

This inventory is a working snapshot. Specific dataset choices remain candidate per the posture set in source-strategy.md. Entries will be revised as prototype work tests them. No data has been retrieved as part of this document; retrieval is a separate, later stage.