Source Inventory
This document is the working source inventory for urban-observatory's v0 prototype. It documents the public-data sources urban-observatory may draw on, the access posture for each, and the role each source plays in the interpretive chain. Specific dataset choices remain candidate; the inventory is not a manifest and not a claim of source sufficiency.
For the source-strategy posture (the Tier-A / Tier-B source ladder at the category level), see source-strategy.md. For the interpretive chain the inventory supports, see object-model.md. For provenance discipline, see methodology.md. For repo-local architecture context, see architecture.md.
on this page
Source inventory boundary
These stages are distinct and must not be blurred:
inventory documents source candidates, access paths, provenance expectations,
format, update posture, and limitations. No data is pulled.
retrieval pulls data from a documented source. Produces raw artifacts.
extraction transforms retrieved raw artifacts into chain entries
(ImplementationSignal records, ImplementationFinding interpretations).
Source-anchored, assumption-anchored.
interpretation packages extraction into the Housing Element Implementation Risk Brief
and related interpretive surfaces.
This document is the inventory stage only. An inventory entry says: "this source is reachable, structured like X, with limits Y." It does not say: "this source proves assumption Z."
Source inventory ≠ source sufficiency. Sufficiency is a finding-level question, decided per-assumption during chain extraction.
Status markers
Controlled vocabulary for v0_status:
pinned canonical URL or resource ID verified; structure inspected;
usable in v0 at the access level recorded
partial access path partially verified; some pattern works, others not yet
candidate pinned at catalog level; not deep-probed for fields
needs verification category real; specific source URL or resource ID not pinned
category-level only recorded as a category without a specific source identified
deferred (Tier-2 case) not pinned in v0 by default; revisited only for Tier-2 deeper
interpretation
Source classification
Controlled vocabulary for source_classification — what a source is in the inventory model:
raw evidence primary source carrying the data itself operational aggregate curated / aggregated view over raw upstream feeds companion source pairs with another canonical source for a chain pair identity anchor provides cross-source join keys; not a signal source enrichment layer context layer (zoning, parcels, administrative overlays) meta-source / discovery viewer / dashboard / catalog over raw sources reference / lookup controlled-vocabulary / code-lookup datasets deferred (Tier-2 case) category-level only; not pinned in v0 candidate pinned at catalog level; intended classification pending deep probe
source_classification answers what a source is. v0_status answers what is verified about access to it. The two are independent fields.
Tier A — canonical sources
Each entry below belongs to the housing / planning subset of the Tier-A signal-visibility posture described in source-strategy.md. These seven pinned source categories are the sources exercised in v0; they do not exhaust the cross-system Tier-A posture. Specific dataset choices are working candidates, not v0-canonical doctrine. (A full re-tag of every source's tier field onto the revised visibility / interpretation ladder is a separate follow-on; the pinned content here is unchanged.)
A.1 — SF Housing Element site inventory / opportunity sites
| Field | Value |
|---|---|
Fieldsource_id |
Valuesf_he_appendix_b4 |
Fieldsource_name |
ValueAppendix B.4 Sites Inventory (SF submission) |
Fieldsource_owner |
ValueCalifornia HCD (template); SF Planning Department (submission) |
Fieldsource_category |
ValueHousing Element site inventory / opportunity sites |
Fieldsource_classification |
Valueraw evidence |
Fieldtier |
ValueA |
Fieldv0_status |
Valuepinned |
Fieldurl_or_identifier |
Valuehttps://sfplanning.org/sites/default/files/documents/housing-for-all/Appendix_B4.xlsx |
Fieldplatform |
Valuestatic .xlsx on SF Planning site |
Fieldaccess_type |
Valuedirect download |
Fieldformat |
ValueXLSX (HCD-standard multi-sheet template) |
Fieldupdate_frequency |
Valueper Housing Element submission cycle |
Fieldcoverage_period |
Value2023–2031 RHNA-cycle period (Dec-2022 submission) |
Fieldgeographic_scope |
Valuewhole SF (~121,053 SF rows in Table A; 121,055 total) |
Fieldprimary_keys |
ValueBLKLOT (hyphenated, e.g., 3704-045); Site Address; ZIP |
Fieldrelevant_objects |
ValueOpportunitySite (anchor); Assumption (per-site capacity); Source |
Fieldrelevant_chain_stage |
ValueAssumption |
Fieldreported_category_relevance |
Valueopportunity-site capacity (structures the RHNA target); Site Status; Pipeline flag; Publicly-Owned flag; Identified-in-Prior-Cycle flag |
Fieldknown_limitations |
ValueMega-Development-Agreement projects (Treasure Island, Candlestick, Parkmerced, Stonestown, Potrero Power Station, Pier 70, Mission Rock, Hope SF Sunnydale, Freedom West, 610 Brannan / Flower Mart) live in Appendix B.1, not B.4 |
Fielddata_quality_caveats |
ValueCapacity is planning-stated, not delivery-verified. ~99.6% of rows are < 5-unit ADU-strategy parcels. Only ~171 sites have capacity ≥ 20; 80% of those are already Pipeline-flagged. |
Fieldprovenance_requirements |
ValueCite as HCD-template Appendix B.4 (SF submission); record table name and BLKLOT; note hyphenated BLKLOT format |
A.2 — SF APR / Housing Element annual progress reporting
| Field | Value |
|---|---|
Fieldsource_id |
Valuehcd_apr_portal |
Fieldsource_name |
ValueHCD Annual Progress Reports portal |
Fieldsource_owner |
ValueCalifornia HCD |
Fieldsource_category |
ValueAPR / Housing Element annual progress reporting |
Fieldsource_classification |
Valueraw evidence |
Fieldtier |
ValueA |
Fieldv0_status |
Valuepinned |
Fieldurl_or_identifier |
Valuehttps://www.hcd.ca.gov/planning-and-community-development/annual-progress-reports |
Fieldplatform |
ValueHCD web portal + California Open Data Portal |
Fieldaccess_type |
Valuelanding page + downloadable raw datasets + APR Dashboard |
Fieldformat |
ValueCSV (raw); interactive dashboard |
Fieldupdate_frequency |
Valueannual reporting cycle |
Fieldcoverage_period |
Value2018–present (raw datasets) |
Fieldgeographic_scope |
Valuestatewide; SF filtered as one jurisdiction |
Fieldprimary_keys |
ValueJurisdiction; Reporting Year; project-level rows in site-level tables |
Fieldrelevant_objects |
ValueAssumption (against adopted HE targets); ImplementationSignal (production, permits, completions); Source; Confidence |
Fieldrelevant_chain_stage |
ValueAssumption + ImplementationSignal |
Fieldreported_category_relevance |
ValueRHNA progress — HCD's primary benchmark is issued building permits (verbatim from HCD landing) |
Fieldknown_limitations |
ValueMethodology is distributed across HCD APR Forms, Instructions PDF, dataset metadata, and Building Blocks guidance — not consolidated in one place |
Fielddata_quality_caveats |
ValueSF most-recent APR currency requires direct verification. The "explicit counting" layer is documented; the implementation-assumption layer is urban-observatory's interpretive contribution, not explicit in HCD source materials |
Fieldprovenance_requirements |
ValueCite HCD APR statewide dataset (or jurisdiction-level table) with reporting year and table identifier; note retrieval date |
A.3 — SF Planning development pipeline
| Field | Value |
|---|---|
Fieldsource_id |
Valuesf_dev_pipeline_6jgi-cpb4 |
Fieldsource_name |
ValueSan Francisco Development Pipeline |
Fieldsource_owner |
ValueSF Planning + SF DBI (combined); DataSF (publishing) |
Fieldsource_category |
ValueSF Planning development pipeline data |
Fieldsource_classification |
Valueoperational aggregate (over raw PPTS feeds at A.4 and raw PTS feed at A.5) |
Fieldtier |
ValueA |
Fieldv0_status |
Valuepinned |
Fieldurl_or_identifier |
ValueDataSF Socrata resource ID 6jgi-cpb4 |
Fieldplatform |
ValueDataSF / Socrata (SODA API) |
Fieldaccess_type |
ValueAPI + CSV / JSON / GeoJSON download |
Fieldformat |
ValueSocrata-hosted tabular |
Fieldupdate_frequency |
Valueregular |
Fieldcoverage_period |
Valueactive pipeline (current and historic statuses) |
Fieldgeographic_scope |
Valuewhole SF |
Fieldprimary_keys |
Valueplanning case number; BLKLOT (concatenated, e.g., 3704045); address |
Fieldrelevant_objects |
ValueHousingProject; ImplementationSignal; OpportunitySite ↔ HousingProject relationship statuses |
Fieldrelevant_chain_stage |
ValueImplementationSignal |
Fieldreported_category_relevance |
ValuePipeline status values: PL Filed, PL Approved, BP Filed, BP Approved, BP Issued, Construction. TCO/CFC exit trigger. DA (Development Agreement) field flags major multi-phased projects |
Fieldknown_limitations |
ValueAggregate over upstream raw feeds; for Tier-2 deeper interpretation, raw PPTS (A.4) and PTS (A.5) may surface signal that Pipeline collapses |
Fielddata_quality_caveats |
ValueVerbatim metadata: "Significant data and systems improvements are underway; data may change at any time." BLKLOT format = concatenated; normalize against B.4 |
Fieldprovenance_requirements |
ValueCite DataSF resource ID 6jgi-cpb4 + data_as_of + query / filter |
A.3.b — Production / completion companion (post-TCO/CFC)
| Field | Value |
|---|---|
Fieldsource_id |
Valuesf_housing_production_xdht-4php |
Fieldsource_name |
ValueSan Francisco Housing Production |
Fieldsource_owner |
ValueSF Planning Department; DataSF (publishing) |
Fieldsource_category |
Valuepost-TCO/CFC companion to pipeline |
Fieldsource_classification |
Valuecompanion source (post-TCO/CFC pair to A.3) |
Fieldtier |
ValueA |
Fieldv0_status |
Valuepinned |
Fieldurl_or_identifier |
ValueDataSF Socrata resource ID xdht-4php |
Fieldplatform |
ValueDataSF / Socrata (SODA API) |
Fieldaccess_type |
ValueAPI + CSV / JSON / GeoJSON download |
Fieldformat |
ValueSocrata-hosted tabular |
Fieldupdate_frequency |
Valueannual (April 1) |
Fieldcoverage_period |
Value2005–present |
Fieldgeographic_scope |
Valuewhole SF |
Fieldprimary_keys |
Valueplanning case number; BLKLOT (concatenated); address |
Fieldrelevant_objects |
ValueHousingProject (completion side); Outcome (delivered units); Source |
Fieldrelevant_chain_stage |
ValueImplementationSignal (delivery milestone) |
Fieldreported_category_relevance |
ValueCompleted units counted at TCO/CFC. Pipeline and Production are mutually exclusive views. Affordable-unit tiers reported |
Fieldknown_limitations |
ValueCounts at completion, not issuance — authorized-but-not-completed sits in Pipeline. Pipeline and Production are not summed |
Fielddata_quality_caveats |
ValueShares Pipeline's data-systems caveat by adjacency. Coverage from 2005 — earlier delivered units are not included |
Fieldprovenance_requirements |
ValueCite DataSF resource ID xdht-4php + query / filter; when pairing with Pipeline for entitlement-to-delivery interpretation, cite both resource IDs |
A.4 — SF Planning applications / entitlement records (PPTS raw)
| Field | Value |
|---|---|
Fieldsource_id |
Valuesf_planning_records_projects_qvu5-m3a2 |
Fieldsource_name |
ValuePlanning Department Records - Projects (PRJ) |
Fieldsource_owner |
ValueSF Planning Department |
Fieldsource_category |
ValueSF Planning applications / entitlement records |
Fieldsource_classification |
Valueraw evidence (upstream feed for A.3 Pipeline aggregate) |
Fieldtier |
ValueA |
Fieldv0_status |
Valuepinned |
Fieldurl_or_identifier |
ValueDataSF Socrata resource ID qvu5-m3a2 |
Fieldplatform |
ValueDataSF / Socrata (SODA API) |
Fieldaccess_type |
ValueAPI + CSV / JSON download |
Fieldformat |
ValueSocrata-hosted tabular |
Fieldupdate_frequency |
Valuenightly |
Fieldcoverage_period |
Valueactive Accela project records |
Fieldgeographic_scope |
Valuewhole SF |
Fieldprimary_keys |
ValueRECORD_ID (PRJ); BLOCK + LOT; PROJECT_ADDRESS |
Fieldrelevant_objects |
ValueHousingProject (raw project case records); ImplementationSignal (entitlement-stage activity); OpportunitySite (via BLOCK + LOT) |
Fieldrelevant_chain_stage |
ValueImplementationSignal |
Fieldreported_category_relevance |
ValueRaw upstream for the Planning side of Pipeline's combined view. 103 columns of project, housing-feature, environmental-review, and unit-count fields |
Fieldknown_limitations |
ValuePipeline (A.3) provides easier first-pass interpretation; raw PRJ records have higher column count and may include cases not yet promoted to Pipeline |
Fielddata_quality_caveats |
ValueAccela record states (open, closed, withdrawn) require careful filtering; non-housing PRJ records will be present |
Fieldprovenance_requirements |
ValueCite DataSF resource ID qvu5-m3a2 + RECORD_ID + query / filter |
| Field | Value |
|---|---|
Fieldsource_id |
Valuesf_planning_records_non_projects_y673-d69b |
Fieldsource_name |
ValuePlanning Department Records - Non-Projects |
Fieldsource_owner |
ValueSF Planning Department |
Fieldsource_category |
ValueSF Planning applications / entitlement records (companion to qvu5-m3a2) |
Fieldsource_classification |
Valueraw evidence (upstream feed for A.3 Pipeline aggregate) |
Fieldtier |
ValueA |
Fieldv0_status |
Valuepinned |
Fieldurl_or_identifier |
ValueDataSF Socrata resource ID y673-d69b |
Fieldplatform |
ValueDataSF / Socrata (SODA API) |
Fieldaccess_type |
ValueAPI + CSV / JSON download |
Fieldformat |
ValueSocrata-hosted tabular |
Fieldupdate_frequency |
Valuenightly |
Fieldgeographic_scope |
Valuewhole SF |
Fieldprimary_keys |
ValueRECORD_ID; BLOCK + LOT; PROJECT_ADDRESS; PARENT_ID (links to PRJ parent) |
Fieldrelevant_objects |
ValueImplementationSignal (supplemental application activity); child-record link to HousingProject via PARENT_ID |
Fieldrelevant_chain_stage |
ValueImplementationSignal |
Fieldreported_category_relevance |
ValueSupplemental applications during project review (variance, conditional use, environmental review filings); finer signal layer than Pipeline status alone |
Fieldknown_limitations |
ValueRecords from Accela database excluding PRJ records; includes child records linked to PRJ parents |
Fielddata_quality_caveats |
ValueHeterogeneous record types; filtering by RECORD_TYPE required; non-housing records present |
Fieldprovenance_requirements |
ValueCite DataSF resource ID y673-d69b + RECORD_ID + PARENT_ID where applicable |
A.5 — DBI building permit records (PTS raw)
| Field | Value |
|---|---|
Fieldsource_id |
Valuedbi_building_permits_i98e-djp9 |
Fieldsource_name |
ValueDBI Building Permits |
Fieldsource_owner |
ValueSF Department of Building Inspection; DataSF (publishing) |
Fieldsource_category |
ValueDBI building permit records |
Fieldsource_classification |
Valueraw evidence (upstream feed for A.3 Pipeline aggregate) |
Fieldtier |
ValueA |
Fieldv0_status |
Valuepinned |
Fieldurl_or_identifier |
ValueDataSF Socrata resource ID i98e-djp9 |
Fieldplatform |
ValueDataSF / Socrata (SODA API) |
Fieldaccess_type |
ValueAPI + CSV / JSON / GeoJSON download (bulk export warned against in Excel due to ~1M rows) |
Fieldformat |
ValueSocrata-hosted tabular |
Fieldupdate_frequency |
Valuenightly / multiple times per hour |
Fieldcoverage_period |
Valuefull DBI permits history; since 2024-12-10 only new and updated records republished, identifiable via data_as_of and data_loaded_at |
Fieldgeographic_scope |
Valuewhole SF |
Fieldprimary_keys |
ValuePermit Number; Block + Lot (separate fields); address; Location (lat/lon) |
Fieldrelevant_objects |
ValueImplementationSignal (permit activity, status, dates); links to OpportunitySite and HousingProject via BLKLOT |
Fieldrelevant_chain_stage |
ValueImplementationSignal |
Fieldreported_category_relevance |
ValueIssued building permits = HCD's primary RHNA benchmark |
Fieldknown_limitations |
ValueBulk export not Excel-friendly; SODA queries needed for site-scoped fetches. Block and Lot are separate fields. Site-permit / fire-only / re-roofing flags must be respected when filtering for housing-relevant permits |
Fielddata_quality_caveats |
ValueLicense: Open Data Commons Public Domain Dedication (PDDL). Housing-relevant permit filtering requires care |
Fieldprovenance_requirements |
ValueCite DataSF resource ID i98e-djp9 + data_as_of / data_loaded_at + Permit Number + Block + Lot |
A.6 — Planning Code and zoning GIS layers
Three current canonical bulk datasets, all published by SF Planning, all PDDL-licensed, all updated quarterly. All source_classification: enrichment layer. All tier: A. All v0_status: pinned.
source_id |
Resource ID | Coverage |
|---|---|---|
source_idsf_zoning_districts_3i4a-hu95 |
Resource ID3i4a-hu95 |
Coveragebase zoning districts; multipolygon; fields include the_geom, zoning_sim, districtname, gen, zoning, codesection |
source_idsf_height_bulk_districts_h9wh-cg3m |
Resource IDh9wh-cg3m |
Coverageheight-and-bulk districts; multipolygon |
source_idsf_special_use_districts_5yf5-ms5f |
Resource ID5yf5-ms5f |
CoverageSpecial Use District overlays per Article 2 of the SF Planning Code; multipolygon |
Common fields:
source_owner— SF Planning Departmentsource_category— Planning Code and zoning GIS layersplatform/access_type/format— DataSF / Socrata (geospatial) / API + CSV / JSON / GeoJSON / multipolygon GIS layerupdate_frequency— quarterlyprimary_keys—the_geom(geographic primary key); district / overlay attribute fieldsrelevant_objects—Constraint;OpportunitySiteenrichmentrelevant_chain_stage— site-context enrichmentreported_category_relevance— defines the zoning / height-bulk / SUD constraint surface that B.4 capacity assumptions are built againstknown_limitations— district-level; per-parcel application requires spatial join against parcels. SUD5yf5-ms5fis overlay-only and must be combined with base Districtsdata_quality_caveats— quarterly cadence; codesection references may lag formal Planning Code amendmentsprovenance_requirements— cite DataSF resource ID +rowsUpdatedAt+ spatial filter
A.7 — Parcel / assessor / land-use GIS layers
Two parcel datasets plus one assessor records dataset.
| Field | Value |
|---|---|
Fieldsource_id |
Valuesf_parcels_overlay_9grn-xjpx |
Fieldsource_name |
ValueParcels with Overlay Attributes |
Fieldsource_owner |
ValueCity and County of San Francisco (DataSF) |
Fieldsource_category |
ValueParcel / assessor / land-use GIS |
Fieldsource_classification |
Valueidentity anchor |
Fieldtier |
ValueA |
Fieldv0_status |
Valuepinned |
Fieldurl_or_identifier |
ValueDataSF Socrata resource ID 9grn-xjpx |
Fieldplatform |
ValueDataSF / Socrata |
Fieldaccess_type |
ValueAPI + CSV / JSON / GeoJSON download |
Fieldformat |
Valuetabular with geospatial fields |
Fieldgeographic_scope |
Valuewhole SF |
Fieldprimary_keys |
Valueparcel identifier (BLOCK + LOT); geographic boundary; pre-joined overlay fields (neighborhoods, supervisor districts, police districts, planning districts) |
Fieldrelevant_objects |
Valueidentity anchor for cross-source joins; OpportunitySite enrichment; Constraint (administrative overlays) |
Fieldrelevant_chain_stage |
Valueidentity anchor / site-context enrichment |
Fieldreported_category_relevance |
ValueNot a reporting source; provides the parcel identity layer underlying other sources |
Fieldknown_limitations |
ValuePre-joined overlays may lag if administrative boundaries change between updates |
Fielddata_quality_caveats |
ValueDerived from base parcels (acdm-wktn); use base parcels if overlay fields are not needed |
Fieldprovenance_requirements |
ValueCite DataSF resource ID 9grn-xjpx + BLOCK + LOT |
| Field | Value |
|---|---|
Fieldsource_id |
Valuesf_parcels_base_acdm-wktn |
Fieldsource_name |
ValueParcels – Active and Retired |
Fieldsource_owner |
ValueCity and County of San Francisco |
Fieldsource_category |
ValueParcel / assessor / land-use GIS |
Fieldsource_classification |
Valueidentity anchor |
Fieldtier |
ValueA |
Fieldv0_status |
Valuepinned |
Fieldurl_or_identifier |
ValueDataSF Socrata resource ID acdm-wktn |
Fieldplatform |
ValueDataSF / Socrata |
Fieldaccess_type |
ValueAPI + CSV / JSON / GeoJSON download |
Fieldformat |
Valuetabular with geospatial fields |
Fieldcoverage_period |
Valueactive and retired parcels with date-added / date-retired |
Fieldprimary_keys |
Valueparcel identifier; date-added; date-retired |
Fieldrelevant_objects |
Valueidentity anchor / OpportunitySite enrichment |
Fieldrelevant_chain_stage |
Valueidentity anchor |
Fieldreported_category_relevance |
ValueCanonical authority for parcel existence at a point in time |
Fieldknown_limitations |
ValueDoes not include the administrative overlays of 9grn-xjpx |
Fielddata_quality_caveats |
ValueRetired parcels are included — filter on date-retired for current view |
Fieldprovenance_requirements |
ValueCite DataSF resource ID acdm-wktn + BLOCK + LOT + as-of date |
| Field | Value |
|---|---|
Fieldsource_id |
Valuesf_assessor_property_rolls_wv5m-vpq2 |
Fieldsource_name |
ValueAssessor Historical Secured Property Tax Rolls |
Fieldsource_owner |
ValueSF Office of the Assessor-Recorder |
Fieldsource_category |
ValueParcel / assessor / land-use GIS (assessor side) |
Fieldsource_classification |
Valueenrichment layer |
Fieldtier |
ValueA |
Fieldv0_status |
Valuepinned |
Fieldurl_or_identifier |
ValueDataSF Socrata resource ID wv5m-vpq2 |
Fieldplatform |
ValueDataSF / Socrata |
Fieldaccess_type |
ValueAPI + CSV / JSON download |
Fieldformat |
Valuetabular |
Fieldupdate_frequency |
Valueannual / per fiscal-year roll |
Fieldcoverage_period |
ValueJuly 1, 2007 to June 30, 2024 (current published coverage) |
Fieldgeographic_scope |
Valuewhole SF |
Fieldprimary_keys |
Valueparcel identifier (BLOCK + LOT); fiscal-year roll |
Fieldrelevant_objects |
ValueConstraint (ownership context, use-code); OpportunitySite enrichment |
Fieldrelevant_chain_stage |
Valuesite-context enrichment |
Fieldreported_category_relevance |
ValueProperty value, owner, last sale, square footage, use codes, classification codes |
Fieldknown_limitations |
Value"Historical" naming: dataset covers through June 2024. Current-year rolls may require a separate / sibling dataset. LLC / trust ownership opacity remains a known limitation |
Fielddata_quality_caveats |
ValueUse codes use Assessor-Recorder reference codes (companion reference / lookup datasets pa56-ek2h Property Class, dx7g-zwbx Neighborhood, g77e-ikb4 Exemption, 2367-5au8 Assessor Block Maps; all source_classification: reference / lookup) |
Fieldprovenance_requirements |
ValueCite DataSF resource ID wv5m-vpq2 + fiscal-year roll + BLOCK + LOT |
Meta-sources and discovery surfaces
These are navigation / interpretive entry points, not raw Tier A evidence. They surface or aggregate raw sources cataloged above. On their own they do not bear on Assumption → ImplementationSignal → ImplementationFinding interpretation; they help locate the raw sources that do.
M.1 — SF Planning Property Information Map (PIM)
| Field | Value |
|---|---|
Fieldsource_id |
Valuesf_pim |
Fieldsource_name |
ValueSF Planning Property Information Map |
Fieldsource_owner |
ValueSF Planning Department |
Fieldsource_classification |
Valuemeta-source / discovery (site-level lookup tool) |
Fieldurl_or_identifier |
Valuehttps://sfplanninggis.org/pim/ |
Fieldplatform |
ValueArcGIS-based GIS viewer; underlying ArcGIS REST endpoints not pinned in v0; bulk substrate lives in the Tier A datasets above |
Fieldaccess_type |
Valueinteractive web viewer (site-level by address / parcel / planning application number) |
Fieldprimary_keys |
ValueBLKLOT; address; planning application number |
Fieldchain_relevance |
Valuesite-level navigation only; raw chain support routes through the Tier A sources surfaced by PIM |
Fieldv0_status |
Valuepinned (as meta-source) |
Fieldknown_limitations |
Value"Housing Element Reused Sites and Low-Income Sites" layer is a subset / visualization; the comprehensive table is Appendix B.4 |
Fieldprovenance_requirements |
ValueWhen used as a lookup tool, cite the underlying authoritative source for any field extracted via PIM |
M.2 — SF Housing Dashboard
| Field | Value |
|---|---|
Fieldsource_id |
Valuesf_housing_dashboard |
Fieldsource_name |
ValueSan Francisco Housing Dashboard |
Fieldsource_owner |
ValueSF Planning Department |
Fieldsource_classification |
Valuemeta-source / discovery (aggregated reporting + discovery hub) |
Fieldurl_or_identifier |
Valuehttps://sfplanning.org/san-francisco-housing-dashboard |
Fieldplatform |
ValueSF Planning web portal (interactive dashboard) |
Fieldaccess_type |
Valueweb UI; references underlying DataSF datasets via data-source links |
Fieldchain_relevance |
Valueaggregated reporting only; raw chain support routes through xdht-4php (A.3.b) and 6jgi-cpb4 (A.3) |
Fieldv0_status |
Valuepinned (as meta-source) |
Fieldreported_category_relevance |
Value"Authorized Units" = SF's primary RHNA-tracking metric (verbatim from dashboard) |
Fielddata_quality_caveats |
ValuePipeline and Completed views are mutually exclusive |
Fieldprovenance_requirements |
ValueWhen citing a dashboard view, also cite the underlying dataset that backs it |
M.3 — MOHCD discovery surfaces
source_id |
URL | Role |
|---|---|---|
source_idmohcd_dashboards_data |
URLhttps://sfmohcd.org/dashboards-and-data |
RoleMOHCD Dashboards and Data hub |
source_idmohcd_bmr_listings |
URLhttps://housing.sfgov.org/listings/for-sale |
RoleFirst Come First Served BMR Listings |
source_idmohcd_plans_reports |
URLhttps://www.sf.gov/mohcd-plans-and-reports |
RoleMOHCD Plans and Reports |
source_idmohcd_department_landing |
URLhttps://sf.gov/departments/mayors-office-housing-and-community-development |
RoleMOHCD main department landing |
Common fields: source_classification: meta-source / discovery; chain_relevance: aggregated reporting / navigation only; v0_status: pinned (as meta-source). Raw MOHCD evidence routes through the Tier B B.3 candidate datasets below.
Tier B — candidate dependency-interpretation sources
Tier B is the dependency-interpretation layer (causal / delivery-risk / cross-system) — gated and not exercised in v0 by default (see source-strategy.md). The candidate sources below illuminate the conditions under which implementation occurs; reading them as dependency interpretation is Tier-B work, triggered by a specific finding, claim, reviewer need, or demo concept.
B.1 — Capital plan and infrastructure investment references
source_classification: deferred (Tier-2 case) · v0_status: category-level only · Chain consumption: Assumption (infrastructure-dependency assumptions per flagged site); InfrastructureDependency. v0 need: none for Tier-1; activated only when a flagged site's assumption depends on infrastructure not yet sequenced. Verification posture: no pinning needed unless Tier-2 work surfaces a specific infrastructure-dependent assumption.
B.2 — Transportation plans and transit access
source_classification: deferred (Tier-2 case) · v0_status: category-level only · Chain consumption: Assumption (TOD / transit-proximity capacity assumptions); Constraint. v0 need: none for Tier-1; PIM surfaces transit-served-area flags at site level as a partial substitute. Verification posture: no pinning needed unless Tier-2 work surfaces a transit-dependent capacity assumption.
B.3 — Affordable housing project / funding signals (MOHCD)
Three candidate-pinned datasets. v0 posture: candidate Tier B (not promoted to Tier A); deep-probe required before any signal extraction.
| Field | Value |
|---|---|
Fieldsource_id |
Valuemohcd_affordable_housing_pipeline_aaxw-2cb8 |
Fieldsource_name |
ValueMOHCD Affordable Housing Pipeline |
Fieldsource_owner |
ValueMayor's Office of Housing and Community Development (MOHCD) + Office of Community Investment and Infrastructure (OCII) — joint |
Fieldsource_classification |
Valuecandidate (intended: raw evidence pending deep probe) |
Fieldtier |
ValueB |
Fieldv0_status |
Valuecandidate |
Fieldurl_or_identifier |
ValueDataSF Socrata resource ID aaxw-2cb8 |
Fieldplatform / access_type / format |
ValueDataSF / Socrata / API + CSV / JSON / tabular (~70+ columns) |
Fieldupdate_frequency / coverage_period |
Valuesnapshot / active pipeline |
Fieldrelevant_objects |
ValueHousingProject (affordable); Assumption (affordable-pipeline assumptions); ImplementationSignal (funding-side momentum) |
Fieldrelevant_chain_stage |
ValueImplementationSignal (funding) |
Fieldreported_category_relevance |
ValueMOHCD/OCII affordable-housing pipeline, including units produced through the Inclusionary Affordable Housing Program; nonprofit / for-profit partnerships; AMI 20%–150% |
Fieldknown_limitations |
ValueJoint MOHCD+OCII view; partition between the two not pinned. Field-level structure not deep-probed |
Fieldprovenance_requirements |
ValueCite DataSF resource ID aaxw-2cb8 + snapshot date |
| Field | Value |
|---|---|
Fieldsource_id |
Valuemohcd_affordable_housing_portfolio_pyxv-n29e |
Fieldsource_name |
ValueMOHCD Affordable Housing Portfolio |
Fieldsource_owner |
ValueMayor's Office of Housing and Community Development (MOHCD) |
Fieldsource_classification |
Valuecandidate |
Fieldtier |
ValueB |
Fieldv0_status |
Valuecandidate |
Fieldurl_or_identifier |
ValueDataSF Socrata resource ID pyxv-n29e |
Fieldplatform / access_type / format |
ValueDataSF / Socrata / API + CSV / JSON / tabular (34 columns) |
Fieldcoverage_period |
Valuecompleted affordable rental and ownership developments |
Fieldrelevant_objects |
ValueHousingProject (affordable, completed); Outcome (affordable delivered units) |
Fieldrelevant_chain_stage |
ValueImplementationSignal (delivery, affordable side) |
Fieldreported_category_relevance |
ValueMOHCD/OCII-financed and Planning Code §415 inclusionary 10+ unit projects, completed |
Fieldprovenance_requirements |
ValueCite DataSF resource ID pyxv-n29e + retrieval date |
| Field | Value |
|---|---|
Fieldsource_id |
Valuemohcd_non_unit_inclusionary_alternatives_g6kh-9pnv |
Fieldsource_name |
ValueMOHCD Non-Unit Inclusionary Housing Alternatives |
Fieldsource_owner |
ValueMayor's Office of Housing and Community Development (MOHCD) |
Fieldsource_classification |
Valuecandidate (intended: raw evidence) |
Fieldtier |
ValueB |
Fieldv0_status |
Valuecandidate |
Fieldurl_or_identifier |
ValueDataSF Socrata resource ID g6kh-9pnv |
Fieldplatform / access_type / format |
ValueDataSF / Socrata / API + CSV / JSON / tabular |
Fieldrelevant_objects |
ValueImplementationSignal (alternative-compliance signals — fee payments, land dedication, off-site BMR); PolicyProgram (§415 Inclusionary) |
Fieldrelevant_chain_stage |
ValueImplementationSignal |
Fieldreported_category_relevance |
Value§415 Inclusionary Housing requirements fulfilled through alternatives to on-site units — fee payments, land dedication, off-site construction |
Fieldprovenance_requirements |
ValueCite DataSF resource ID g6kh-9pnv + retrieval date |
B.4 — Construction cost, interest rate, vacancy indicators
source_classification: deferred (Tier-2 case) · v0_status: category-level only · Chain consumption: Assumption (feasibility); ImplementationSignal (market-condition signals); MarketSignal candidate subtype. v0 need: none for Tier-1; public-data-only discipline rules out vendor sources. Verification posture: no pinning needed unless Tier-2 work surfaces a market-condition-dependent assumption.
B.5 — Policy and program documents related to HE implementation
source_classification: deferred (Tier-2 case) · v0_status: category-level pinned · Chain consumption: PolicyProgram; Assumption (program-level). v0 need: category-level only for Tier-1; specific documents identified case-by-case (includes SF's adopted Housing Element 2023–2031 — the policy document that sets the assumption surface Appendix B.4 derives from). Verification posture: case-by-case as a specific PolicyProgram interpretation requires.
Tier C — selective / deferred supporting sources
Tier C sources are selective for Tier-2 deeper interpretation of flagged sites. They are listed at the category level in source-strategy.md. Specific Tier C datasets are not pinned in v0 by default. All Tier C entries carry source_classification: deferred (Tier-2 case) and v0_status: deferred (Tier-2 case).
ParticipatorySignal candidate sources (public comments, meeting transcripts, comment letters, survey summaries) are reserved by default in v0. If activated, representativeness and bias documentation are required per source.
Source inventory fields / metadata model
The field set used throughout this inventory:
source_id short, lowercase, underscore-delimited; stable;
used as canonical short reference
source_name human-readable name
source_owner publishing organization (and underlying agency if different)
source_category category from docs/source-strategy.md
source_classification raw evidence / operational aggregate / companion source /
identity anchor / enrichment layer / meta-source / discovery /
reference / lookup / deferred (Tier-2 case) / candidate
tier A / B / C
url_or_identifier canonical URL or DataSF / ArcGIS resource ID
platform DataSF/Socrata, ArcGIS, HCD portal, static download, etc.
access_type direct download, API, web viewer, etc.
format CSV, JSON, GeoJSON, XLSX, PDF, GIS layer, etc.
update_frequency nightly, quarterly, per-cycle, etc. (when known)
coverage_period time range covered by the data (when known)
geographic_scope SF whole-city, neighborhood, statewide, etc.
primary_keys join keys; BLKLOT format variant noted where relevant
relevant_objects which object-model primitives this source supports
relevant_chain_stage Assumption / ImplementationSignal / ImplementationFinding /
InterventionCandidate
reported_category_relevance how this source maps to APR / Pipeline / Production reporting
known_limitations structural limits
data_quality_caveats data-quality notes (verbatim where dataset publishes them)
provenance_requirements what to cite when using this source
v0_status pinned / partial / candidate / needs verification /
category-level only / deferred (Tier-2 case)
Notes on field use:
source_classificationis the role discipline — what the source is in the inventory model.v0_statusis the access-verification state. Separate fields, separate questions.relevant_chain_stageis restricted to the four chain stages fromobject-model.md:Assumption / ImplementationSignal / ImplementationFinding / InterventionCandidate. Supporting primitives (Outcome,Constraint,OpportunitySite, etc.) belong inrelevant_objects, notrelevant_chain_stage.- Meta-source entries omit fields not applicable to viewers and substitute
chain_relevance(descriptive prose) forrelevant_chain_stagesince meta-sources do not bear on the chain directly.
This field set is documentation, not an executable schema. The inventory is Markdown-first and consistent with data-dictionary.md. Promotion to a runnable manifest (YAML, JSON, CSV) is deferred until a v0 consumer requires it.
Known caveats
The cautions below belong with the inventory itself; they are not commentary about it.
- Source inventory ≠ source sufficiency. Naming a source in the inventory does not certify that the source carries enough signal to weaken or support a given assumption. Sufficiency is a finding-level question, decided per-assumption during chain extraction.
- Metadata accessibility ≠ row-level use. A confirmed-accessible metadata endpoint says nothing about whether the rows behind it are clean, current, or interpretable.
- DataSF / Socrata resource IDs may change. IDs are generally stable but not guaranteed permanent. Cite IDs with retrieval / verification dates.
- Source dashboards are often JS-rendered. Cite the linked dataset, not the dashboard view, when source-anchoring a signal.
- PIM is site-level via the viewer; bulk is via the canonical Tier A datasets. Do not treat PIM as a bulk dataset.
- Appendix B.4 is large and statewide-format. ~99.6% of rows are < 5-unit ADU-strategy parcels. Only ~171 sites have capacity ≥ 20; 80% of those are already Pipeline-flagged. The aggregate ADU strategy means testing the chain at any individual RH-1 site does not really test SF's strategy.
- SF Pipeline has data-quality caveats. Verbatim metadata: "Significant data and systems improvements are underway; data may change at any time." Any signal sourced from Pipeline must carry this caveat.
- Public-input sources are deferred / cautious.
ParticipatorySignalcandidate sources are reserved by default in v0. - BLKLOT formats vary. Any cross-source join requires explicit normalization.
| Source | BLKLOT format | Example |
|---|---|---|
| SourceAppendix B.4 | BLKLOT formathyphenated | Example3704-045 |
SourceSF Pipeline 6jgi-cpb4 |
BLKLOT formatconcatenated | Example3704045 |
SourceDBI Permits i98e-djp9 |
BLKLOT formatseparate Block + Lot | Example3704 + 045 |
SourcePlanning Non-Projects y673-d69b |
BLKLOT formatseparate Block + Lot | Example3704 + 045 |
SourceParcels with Overlay 9grn-xjpx |
BLKLOT formatparcel identifier (BLOCK + LOT) | Exampleper dataset |
SourceAssessor Property Rolls wv5m-vpq2 |
BLKLOT formatparcel identifier (BLOCK + LOT) | Exampleper dataset |
Lots with letter suffix (e.g., 0044-002A / 0044002A) require consistent handling.
- Category-level entry ≠ pinned source. Tier B abstract entries (B.1, B.2, B.4) and Tier C entries are category-level placeholders, not pinned sources. Do not let category-level entries imply more verification than has happened.
- Pipeline (
6jgi-cpb4) is an operational aggregate, not a primary source. Pipeline curates upstream raw feeds: PPTS (Planning Accela:qvu5-m3a2+y673-d69b) and PTS (DBI:i98e-djp9). For Tier-2 deeper interpretation, dropping back to the raw feeds may surface signal that Pipeline collapses.
Posture
This inventory is a working snapshot. Specific dataset choices remain candidate per the posture set in source-strategy.md. Entries will be revised as prototype work tests them. No data has been retrieved as part of this document; retrieval is a separate, later stage.