Predicting property prices of Brazilian markets using machine learning on socioeconomic data.
Brazilian real estate data is fragmented, unreliable, and expensive. Registry (cartório) records are incomplete. Price per square meter varies wildly not just by location but by economic cycle, labor market composition, zoning, and urban density — factors that traditional brokers price by gut feeling.
The core insight: since reliable transaction data is scarce, prices can be normalized by location using public socioeconomic indicators as a structural baseline. This creates a multiplier for price-per-square-meter that captures inequality patterns across the city.
A recommendation system that matches buyer profiles to available listings using KNN similarity over normalized property features.
Pipeline:
- Parallel ingestion — fetches listing inventory and buyer intake forms from BigQuery concurrently via
ThreadPoolExecutor - Hard filtering — pre-filters by
Tipo_Negocio,Tipo,SubTipo,Municipio,Bairrobefore running KNN, avoiding distance pollution from type/location mismatches - Amenity normalization — maps 80+ Portuguese amenity strings (
"Churrasqueira","Sala de Ginastica","Heliponto") to a canonical vocabulary; handles multi-valued amenities and unicode normalization withunidecode - KNN with StandardScaler — scales continuous features (
Area_Construida_m2,Preco) per query; fitsNearestNeighborson filtered candidates; returns ranked similarity scores - Output — recommendations with Google Maps links written back to BigQuery (
Warehouse.Recomendacoes)
Multilevel regression framework integrating two categories of features:
Property-level features (from GeoSampa / VivaReal):
- Type, subtype, zoning class, floor-area ratio, lot area, built area
- Rooms, suites, parking, amenities (pool, gym, gourmet space, etc.)
- IPTU fiscal value, condominium fee
Macro & socioeconomic controls (monthly, scraped from public sources):
| Indicator | Source | What it captures |
|---|---|---|
| FipeZAP | FIPE | Price/m² by city, type, rooms |
| IGMI-R | BCB | Real estate market profitability by capital |
| IGP-M | FGV | Rental inflation |
| Selic | BCB | Base interest rate (financing cost) |
| IPCA | IBGE | General inflation |
| INCC | FGV | Civil construction cost index |
| IBC-BR | BCB | GDP preview (economic cycle proxy) |
| IVG-R | BCB | Residential collateral value index |
| IIE-BR | FGV | Economic uncertainty indicator |
| CubSP | SINDUSCON | Construction cost in São Paulo |
FipeZAP matching logic: exact match by city/type/rooms → fallback to total rooms → residential type → state capital → national average.
Built for São Paulo's 2023 urban densification plan, which opened zoning for higher-density construction near transit hubs — creating demand for buildable lots faster than traditional "perdigueiros" (manual scouts) could supply.
- PCA — dimensionality reduction across IPTU fiscal value, lot area, zoning class, floor-area ratio, and socioeconomic features
- KNN in PCA space — finds underutilized lots structurally similar to known high-potential parcels from 11M+ IPTU records
- Surfaces development opportunities at scale, replacing a labor-intensive manual process
End-to-end owner contact extraction from public IPTU records:
- Bulk IPTU query by neighborhood and property type
- Owner name extraction and CPF inference
- Contact lookup via SeekLoc API
- Output: owner name, phone, email, property — ready for outreach
| Source | Used for |
|---|---|
| GeoSampa (PMSP) | 11M+ IPTU records, zoning, land use, spatial boundaries |
| RAIS (MTE) | Labor market composition and wage levels by municipality |
| IPEA | Socioeconomic development indices |
| ONU / UDH | Sub-municipal human development index |
| BCB / FipeZAP / IGMI-R / IGP-M | Real estate and macro price indices |
| VivaReal (scraped) | Active listing inventory with full feature set |
Geodata enrichment: bquant/geo — GeoPipeline produces the parquet files consumed here.
Rural counterpart: solo-inteligente.