A self-improving home price prediction engine that runs experiments on itself every night, keeps what works, throws away what doesn’t, and wakes up a little smarter each morning.

This is the project I’m most excited about.

What it does

The engine predicts what a home will sell for — before it’s listed. Not a Zestimate-style AVM that updates a number on a webpage. This is a comp-based prediction system that queries a database of tens of thousands of closed sales, finds similar properties using a tiered search strategy, applies adjustments for property differences, and synthesizes a price estimate the way an experienced appraiser would — except it never gets tired, never anchors on asking price, and runs backtests against thousands of real outcomes to prove itself right or wrong.

The comp-finder uses a tiered search strategy: tight geographic radius and strict property criteria first, expanding the radius and relaxing criteria in stages if the initial pool is too small. Each comparable gets scored on similarity (0-100 scale) across eight adjustment dimensions — square footage, age, condition, lot size, pool, garage, stories, and location quality. The final prediction blends the adjusted comp values using the similarity scores as weights.

The interesting part isn’t the prediction. It’s how the engine improves.

The Karpathy Loop

Named after Andrej Karpathy’s concept of autonomous self-improvement, the engine runs a nightly loop:

  1. An AI strategist reads the full experiment history — what worked, what failed, what produced marginal gains, what hasn’t been tried — and proposes a single hypothesis to test. The strategist sees the experiment log, the latest accuracy metrics, and domain hints I’ve written, but never touches the codebase directly.
  2. An AI executor implements the experiment — modifies the engine’s JavaScript source code, runs a full backtest against 10,000 real sold properties, and evaluates the accuracy delta. The backtest processes the full property set in minutes.
  3. The system evaluates deterministically — if median absolute percentage error improved, the change is kept and committed. If not, the code is reverted automatically. Ten experiments per nightly session. No human in the loop.
  4. Guard scripts enforce integrity — a no-lookahead guard greps the codebase before every experiment to ensure the engine never peeks at the actual sale price during prediction. If the guard fails, the experiment is auto-reverted regardless of accuracy improvement. This was added after I discovered five separate contamination vectors in an earlier version that made accuracy look artificially good.

Each morning I wake up to an experiment log: KEEP or DISCARD, with the accuracy delta down to the hundredth of a percentage point. The engine has run hundreds of experiments autonomously. Most get discarded. The ones that survive compound.

What I’ve learned

The biggest accuracy gains don’t come from where you’d expect. Rate tuning and parameter optimization produce marginal improvements at best. The breakthroughs come from structural insights — and the AI strategist finds angles that a human would miss because it reads the entire experiment history before proposing:

Listing descriptions contain signals that structured data misses. Keywords like “as-is,” “foundation repair,” or “TLC” reveal condition problems invisible in the structured fields. A distress-detection layer that reads listing text improved accuracy more than weeks of rate tuning.

Market temperature affects predictions asymmetrically. In Texas, cooling markets (where the active-to-sold price ratio drops below 0.97) require a discount adjustment. But hot markets don’t reliably produce premiums — sellers price aspirationally, and bidding wars are inconsistent. The cold-arm adjustment works; the hot-arm regresses.

Small comp pools make sophisticated models unstable. When you only have 4-8 comparable sales, hedonic OLS regression is too noisy to be useful. Simple weighted approaches — boring, but stable — beat regression every time at that sample size. The AI strategist tried OLS twice, failed both times, and learned to stop proposing it.

Determinism matters for tiny gains. Before I added deterministic ordering to the comp-finder and scorer, two identical backtest runs could disagree by 0.01 percentage points due to PostgreSQL row-order instability and JavaScript sort instability. The loop was making KEEP/DISCARD decisions against its own noise floor. Adding ORDER BY close_date DESC, id DESC and a tiebreaker to the scorer made the improvement signal real.

How it’s built

The engine is Node.js: engine.js orchestrates the prediction pipeline, comp-finder.js handles tiered comp selection with PostgreSQL queries, adjuster.js applies eight adjustment types, and scorer.js produces similarity scores with deterministic tiebreaking. rates.json is the tunable surface — adjustment rates, comp selection parameters, scoring weights — that the Karpathy Loop experiments modify.

The strategist runs as a separate AI invocation with read-only access to the experiment history and a lean digest of current metrics. It returns one hypothesis as structured JSON. The executor implements exactly that hypothesis — it’s not allowed to substitute its own idea. This separation prevents the executor from second-guessing the strategist and keeps experiment provenance clean.

Backtesting is fully deterministic and parallelized. The 10,000-property benchmark set completes in minutes. A separate 5,000-property holdout set (never tuned against) validates that accuracy improvements generalize. The overfit gap after hundreds of experiments is 0.03 percentage points — the engine is genuine, not memorizing.

The system feeds into Neuhaus Realty Group’s CMA and BPO tools. The long-term goal is a unified valuation system that replaces the three independent systems currently handling appraisals, comparative market analysis, and market pricing.

Why I built it

Every real estate agent does comparative market analysis by hand. Pull comps, eyeball adjustments, arrive at a number that’s part data and part gut feel. I wanted to see if AI could do it systematically — not by replacing the agent’s judgment, but by testing thousands of judgment calls and keeping only the ones that actually predict prices more accurately.

The Karpathy Loop is the answer to “can AI get better at this without me babysitting it?” So far: yes, but slowly. The gains compound, the experiments are autonomous, and every morning the engine knows a little more about what makes Austin home prices tick.